Skip to main content
SpringerLink
Log in

BulletArm: An Open-Source Robotic Manipulation Benchmark and Learning Framework

  • Conference paper
  • First Online:
Robotics Research (ISRR 2022)

Part of the book series: Springer Proceedings in Advanced Robotics ((SPAR,volume 27))

Included in the following conference series:

Abstract

We present BulletArm, a novel benchmark and learning-environment for robotic manipulation. BulletArm is designed around two key principles: reproducibility and extensibility. We aim to encourage more direct comparisons between robotic learning methods by providing a set of standardized benchmark tasks in simulation alongside a collection of baseline algorithms. The framework consists of 31 different manipulation tasks of varying difficulty, ranging from simple reaching and picking tasks to more realistic tasks such as bin packing and pallet stacking. In addition to the provided tasks, BulletArm has been built to facilitate easy expansion and provides a suite of tools to assist users when adding new tasks to the framework. Moreover, we introduce a set of five benchmarks and evaluate them using a series of state-of-the-art baseline algorithms. By including these algorithms as part of our framework, we hope to encourage users to benchmark their work on any new tasks against these baselines.

D. Wang and C. Kohler—Equal Contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andrychowicz, O.M., et al.: Learning dexterous in-hand manipulation. Int. J. Robot. Res. 39(1), 3–20 (2020)

    Article  Google Scholar 

  2. Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents. J. Artif. Intell. Res. 47, 253–279 (2013)

    Article  Google Scholar 

  3. Biza, O., Wang, D., Platt, R., van de Meent, J.-W., Wong, L.L.: Action priors for large action spaces in robotics. In: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pp. 205–213 (2021)

    Google Scholar 

  4. Brockman, G., et al.: OpenAI Gym. arXiv preprint arXiv:1606.01540 (2016)

  5. Cohen, T.S., Welling, M.: Steerable CNNs. arXiv preprint arXiv:1612.08498 (2016)

  6. Coumans, E., Bai, Y.: Pybullet, a python module for physics simulation for games, robotics and machine learning. GitHub repository (2016)

    Google Scholar 

  7. Delhaisse, B., Rozo, L.D., Caldwell, D.G.: PyRoboLearn: a python framework for robot learning practitioners. In: CoRL (2019)

    Google Scholar 

  8. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  9. Depierre, A., Dellandréa, E., Chen, L.: Jacquard: a large scale dataset for robotic grasp detection. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3511–3516. IEEE (2018)

    Google Scholar 

  10. Devin, C., Gupta, A., Darrell, T., Abbeel, P., Levine, S.: Learning modular neural network policies for multi-task and multi-robot transfer. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 2169–2176. IEEE (2017)

    Google Scholar 

  11. Fang, H.-S., Wang, C., Gou, M., Lu, C.: GraspNet-1billion: a large-scale benchmark for general object grasping. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11444–11453 (2020)

    Google Scholar 

  12. Gu, S., Holly, E., Lillicrap, T., Levine, S.: Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3389–3396. IEEE (2017)

    Google Scholar 

  13. Haarnoja, T., et al.: Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 (2018)

  14. Hausman, K., Springenberg, J.T., Wang, Z., Heess, N., Riedmiller, M.: Learning an embedding space for transferable robot skills. In: International Conference on Learning Representations (2018)

    Google Scholar 

  15. Hester, T., et al.: Deep q-learning from demonstrations. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  16. Huber, P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 35(1), 73–101 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  17. James, S., Freese, M., Davison, A.J.: PyRep: bringing V-REP to deep robot learning. arXiv preprint arXiv:1906.11176 (2019)

  18. James, S., Ma, Z., Arrojo, D.R., Davison, A.J.: RLBench: the robot learning benchmark & learning environment. IEEE Robot. Autom. Lett. 5(2), 3019–3026 (2020)

    Article  Google Scholar 

  19. Jiang, Y., Moseson, S., Saxena, A.: Efficient grasping from RGBD images: learning using a new rectangle representation. In: 2011 IEEE International Conference on Robotics and Automation, pp. 3304–3311. IEEE (2011)

    Google Scholar 

  20. Kalashnikov, D., et al.: Scalable deep reinforcement learning for vision-based robotic manipulation. In: Conference on Robot Learning, pp. 651–673. PMLR (2018)

    Google Scholar 

  21. Kempka, M., Wydmuch, M., Runc, G., Toczek, J., Jaśkowski, W.: ViZDoom: a doom-based AI research platform for visual reinforcement learning. In: 2016 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1–8. IEEE (2016)

    Google Scholar 

  22. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  23. Kostrikov, I., Yarats, D., Fergus, R.: Image augmentation is all you need: regularizing deep reinforcement learning from pixels. arXiv preprint arXiv:2004.13649 (2020)

  24. Lakshminarayanan, A.S., Ozair, S., Bengio, Y.: Reinforcement learning with few expert demonstrations. In: NIPS Workshop on Deep Learning for Action and Interaction, vol. 2016 (2016)

    Google Scholar 

  25. Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., Srinivas, A.: Reinforcement learning with augmented data. In: Advances in Neural Information Processing Systems, vol. 33, pp. 19884–19895 (2020)

    Google Scholar 

  26. Laskin, M., Srinivas, A., Abbeel, P.: Curl: contrastive unsupervised representations for reinforcement learning. In: International Conference on Machine Learning, pp. 5639–5650. PMLR (2020)

    Google Scholar 

  27. Lee, Y., Hu, E., Yang, Z., Yin, A.C., Lim, J.J.: IKEA furniture assembly environment for long-horizon complex manipulation tasks. In: 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 6343–6349 (2021)

    Google Scholar 

  28. Li, R., Jabri, A., Darrell, T., Agrawal, P.: Towards practical multi-object manipulation using relational reinforcement learning. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 4051–4058. IEEE (2020)

    Google Scholar 

  29. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  30. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  31. Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel, P.: Overcoming exploration in reinforcement learning with demonstrations. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 6292–6299. IEEE (2018)

    Google Scholar 

  32. Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS Autodiff Workshop (2017)

    Google Scholar 

  33. Rohlfshagen, P., Lucas, S.M.: Ms Pac-Man versus ghost team CEC 2011 competition. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 70–77. IEEE (2011)

    Google Scholar 

  34. Rohmer, E., Singh, S.P., Freese, M.: V-REP: a versatile and scalable robot simulation framework. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1321–1326. IEEE (2013)

    Google Scholar 

  35. Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. arXiv preprint arXiv:1511.05952 (2015)

  36. Silver, D., et al.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  37. Tassa, Y., et al.: DeepMind control suite. arXiv preprint arXiv:1801.00690 (2018)

  38. Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033. IEEE (2012)

    Google Scholar 

  39. Togelius, J., Karakovskiy, S., Baumgarten, R.: The 2009 Mario AI competition. In: IEEE Congress on Evolutionary Computation, pp. 1–8. IEEE (2010)

    Google Scholar 

  40. Urakami, Y., Hodgkinson, A., Carlin, C., Leu, R., Rigazio, L., Abbeel, P.: DoorGym: a scalable door opening environment and baseline agent. arXiv, abs/1908.01887 (2019)

    Google Scholar 

  41. Vinyals, O., et al.: Starcraft II: a new challenge for reinforcement learning. arXiv preprint arXiv:1708.04782 (2017)

  42. Wang, D., Jia, M., Zhu, X., Walters, R., Platt, R.: On-robot policy learning with O(2)-equivariant SAC. arXiv preprint arXiv:2203.04923 (2022)

  43. Wang, D., Kohler, C., Platt, R.: Policy learning in SE(3) action spaces. In: Conference on Robot Learning, pp. 1481–1497. PMLR (2021)

    Google Scholar 

  44. Wang, D., Walters, R., Platt, R.: SO(2)-equivariant reinforcement learning. In: International Conference on Learning Representations (2022)

    Google Scholar 

  45. Wang, D., Walters, R., Zhu, X., Platt, R.: Equivariant \(Q\) learning in spatial action spaces. In: Conference on Robot Learning, pp. 1713–1723. PMLR (2022)

    Google Scholar 

  46. Weiler, M., Cesa, G.: General \(E(2)\)-equivariant steerable CNNs. arXiv preprint arXiv:1911.08251 (2019)

  47. Wohlkinger, W., Aldoma Buchaca, A., Rusu, R., Vincze, M.: 3DNet: large-scale object class recognition from CAD models. In: IEEE International Conference on Robotics and Automation (ICRA) (2012)

    Google Scholar 

  48. Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., Levine, S.: Meta-world: a benchmark and evaluation for multi-task and meta reinforcement learning. In: Conference on Robot Learning, pp. 1094–1100. PMLR (2020)

    Google Scholar 

  49. Zakka, K., Zeng, A., Lee, J., Song, S.: Form2Fit: learning shape priors for generalizable assembly from disassembly. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 9404–9410. IEEE (2020)

    Google Scholar 

  50. Zeng, A., et al.: Transporter networks: rearranging the visual world for robotic manipulation. In: Conference on Robot Learning, pp. 726–747. PMLR (2021)

    Google Scholar 

  51. Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., Funkhouser, T.: Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4238–4245. IEEE (2018)

    Google Scholar 

  52. Zeng, A., et al.: Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3750–3757. IEEE (2018)

    Google Scholar 

  53. Zhan, A., Zhao, P., Pinto, L., Abbeel, P., Laskin, M.: A framework for efficient robotic manipulation. arXiv preprint arXiv:2012.07975 (2020)

  54. Zhu, X., Wang, D., Biza, O., Su, G., Walters, R., Platt, R.: Sample efficient grasp learning using equivariant models. In: Proceedings of Robotics: Science and Systems (RSS) (2022)

    Google Scholar 

  55. Zhu, Y., Wong, J., Mandlekar, A., Mart’in-Mart’in, R.: Robosuite: a modular simulation framework and benchmark for robot learning. arXiv, abs/2009.12293 (2020)

    Google Scholar 

Download references

Acknowledgments

This work is supported in part by NSF 1724257, NSF 1724191, NSF 1763878, NSF 1750649, and NASA 80NSSC19K1474.

Author information

Authors and Affiliations

  1. Khoury College of Computer Sciences, Northeastern University, Boston, MA, 02115, USA

    Dian Wang, Colin Kohler, Xupeng Zhu, Mingxi Jia & Robert Platt

Authors
  1. Dian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Colin Kohler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xupeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingxi Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Robert Platt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dian Wang .

Editor information

Editors and Affiliations

  1. EPFL STI SMT-GE, Ecole Polytechnique Federale de Lausanne, Lausanne, Vaud, Switzerland

    Aude Billard

  2. Institute for Anthropomatics and Robotic, KIT, Karlsruhe, Baden-Württemberg, Germany

    Tamim Asfour

  3. Artificial Intelligence Laboratory, Department of Computer Science, Stanford University, Stanford, CA, USA

    Oussama Khatib

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, D., Kohler, C., Zhu, X., Jia, M., Platt, R. (2023). BulletArm: An Open-Source Robotic Manipulation Benchmark and Learning Framework. In: Billard, A., Asfour, T., Khatib, O. (eds) Robotics Research. ISRR 2022. Springer Proceedings in Advanced Robotics, vol 27. Springer, Cham. https://doi.org/10.1007/978-3-031-25555-7_23

Download citation

Publish with us

Policies and ethics

Search

Navigation