A model-based scooping grasp for the autonomous picking of unknown objects with a two-fingered gripper

https://doi.org/10.1016/j.robot.2018.04.003Get rights and content

Highlights

  • A model-based scooping grasp that is capable of picking thin objects on a flat surface.

  • An implementation of the scooping grasp using a commercial gripper.

  • Autonomous grasping using a robotic cell including a gripper, a 6-dof robot and a camera.

  • Tests results of the robotic cell involving 80 objects and 800 trials.

Abstract

Grasping objects used in daily activities is not an easy task for a robot: the diversity of shapes and volumes of objects renders specific grasping methods inefficient. In this paper, we propose a novel model-based scooping grasp for the picking of thin objects lying on a flat surface, which are typically elusive to common grippers and grasping strategies. A robotic work cell composed of a serial arm, a commercially available gripper and a 3D camera overlooking the workspace is used to demonstrate and test the algorithm. Since a commercial gripper is used, the robot is capable of grasping a large variety of objects, in addition to the targeted thin objects. An experiment based on a test set of 80 objects results in an overall grasp success rate of 84%, which demonstrates the potential of the novel scooping grasp to extend the capabilities of existing grippers.

Introduction

Some tasks such as lifting heavy payloads or repeating the same operation thousands of times are difficult for humans but easy to program on robotic manipulators. Conversely, other tasks are effortless for humans, but remain difficult for robotic manipulators. Reliably grasping unknown objects is one such task. Various methods of grasping have been reviewed in [[1], [2], [3]]. The grasping problem can be classified according to whether the object is known [4], familiar [5] or unknown [[6], [7]]. The solutions to these problems can also be classified according to their nature: the empirical approach [[8], [9]], based on the replication of the behaviour of the human hand, and the analytical approach [[10], [11]], based on the mechanical properties of the grasping action.

As described in Section 2, various research groups have succeeded in developing autonomous systems for the grasping of objects. Some works use machine learning or deep learning, either to identify objects or to synthesize grasps. Others use recognition with elementary forms or faces, or work with known objects. Approaches based on learning require large amounts of data and may not perform well in unusual situations (or objects). On the other hand, techniques based on the recognition of elementary shapes may not be well adapted to objects with complex shapes. Most existing methods are designed for non-flat objects and only a few tackle the problem of grasping thin objects.

Hence, the objective of this work is to develop a robotic cell for grasping a wide diversity of unknown objects used in the everyday life, including flat thin objects. Moreover, the robotic cell is tested on a large number of objects to ensure its reliability. In order to succeed in this task, three steps are essential, namely : (i) recognize and locate the objects in the workspace, (ii) choose the most appropriate grasping method and (iii) find a robust feasible grasp (i.e. find a grasping configuration).

In this paper, we propose a model-based scooping grasp that is capable of picking thin objects on a flat surface, in order to extend the variety of unknown objects that can be autonomously grasped by a robot in a work cell. Two different grasping methods are implemented, – namely a conventional overhead grasp and the novel scooping grasp – in order to demonstrate the effectiveness of the proposed method at increasing the variety of objects that can be grasped. In the scooping grasp, one finger of an underactuated gripper slips under the object. Thus, the gripper pinches the object from one of its sides much like a human hand grasps a sheet of paper on a flat surface. The proposed scooping maneuver is tested in a work cell consisting of a 3D vision sensor to detect and characterize the objects, an underactuated two-finger gripper and a six-degree-of-freedom (6-dof) serial robot.

This paper is organized as follows: Section 2 reviews some of the relevant prior art regarding the grasping of objects. The overhead grasping strategy is briefly recalled in Section 3 while the novel scooping grasp maneuver is described in detail in Section 4. Section 5 describes the general algorithm used to plan the grasping maneuvers. The experimental apparatus as well as the experimental tests and results are presented in Section 6. The results are then analysed in Section 7. Finally, Section 8 presents the closing discussion.

Section snippets

Related work

In the present study, related works are divided in two groups, namely those addressing the grasping of thin objects and those focusing on the grasping of unknown objects.

Overhead pinch grasp

A general strategy that can be used to grasp voluminous objects consists in picking the objects from above. When the size and thickness of the objects allow this type of grasp, it is preferred to scooping because it is simple to perform and generally stable. Overhead pinch grasp strategies have been described in many papers (see for instance [32]) and are not the focus of this paper. The first step in this type of grasp consists in choosing the grasping contact points. To this end, a set of

Scooping grasp

The second grasping strategy, which is the focus of this paper, is designed for the objects that cannot be grasped from above, for example a book or any thin object lying on a flat surface.

The scooping grasp consists in sliding a finger between the object and the base surface, lifting up slightly and closing the gripper to finish the task. This method is facilitated by the use of an underactuated gripper. Indeed, when contact with the base surface is made by the distal phalanx of a two-finger

Operational algorithm

Fig. 17 describes the algorithm used to perform the pick and place operations with the robot (picking up an object and dropping it in a bin). The structure of the algorithm is of the ‘Sense-Plan-Act’ family of algorithms, meaning that a picking cycle is sequentially performed by sensing objects with a camera, planning a grasping maneuver and then executing it. The algorithm can be decomposed into four simple steps as follows:

  • 1.

    Initialize: Establish communication and control of

Experimental apparatus

The experimental apparatus used to test the algorithm and the grasping strategies consists of a 3D vision sensor (Kinect from Microsoft), a 6-dof serial manipulator (UR5 robot from Universal Robots) and an underactuated gripper (2-finger 85 gripper from Robotiq), as shown in Fig. 19. The communication between these components is performed using the Robot Operating System (ROS).

Results

Table 3 shows the distribution of the number of successful grasps over the 800 trials. 673 trials were successful (84.1%). Four objects out of 80 (5%) were never grasped after 10 trials. 60 objects (75%) were grasped either 9 or 10 times. 50.5 objects were picked using the scooping grasp with a success rate of 80.4%. 29.5 objects were picked using the overhead grasps, with a success rate of 90.5%. The decimals in Table 3 correspond to an object that was grasped using the overhead grasp in half

Conclusion

Grasping a wide variety of objects arbitrarily placed in a workspace is a complex task that is difficult to automate. Several challenges arise including properly estimating the shape of the objects, ensuring a proper and stable grasp as well as avoiding collisions.

This paper proposes a new scooping grasp that allows the grasping of flat objects, thereby increasing the variety of objects that can be autonomously picked by a robotic system. Based on a static analysis of the grasping maneuver, the

Acknowledgement

This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) (RDCPJ461709-13), by Robotiq and by Prompt Quebec.

Bruno Sauvet received his M.Sc. in robotics from the Université Pierre et Marie CURIE (France) in 2008, and a Ph.D. degree in mechanics and robotics under the supervision of Prof. Stéphane RÉGNIER within ISIR (Institut des Systèmes Intelligents et de Robotique) in 2013. His Ph.D. thesis focused on the manipulation of thin films (like graphene) with a microrobotic approach and to characterize the physics properties of these nano/micro objects. He was then a post-doctoral fellow at Université

References (33)

  • SahbaniA. et al.

    An overview of 3d object grasp synthesis algorithms

    Robot. Auton. Syst.

    (2012)
  • AleottiJ. et al.

    Interactive teaching of task-oriented robot grasps

    Robot. Auton. Syst.

    (2010)
  • KapplerD. et al.

    Templates for pre-grasp sliding interactions

    Robot. Auton. Syst.

    (2012)
  • BohgJ. et al.

    Data-driven grasp synthesis a survey

    IEEE Trans. Robot.

    (2014)
  • RoaM.A. et al.

    Grasp quality measures: review and performance

    Auton. Robots

    (2015)
  • RoaM.A. et al.

    Power grasp planning for anthropomorphic robot hands

  • S. El Khoury, A. Sahbani, Handling objects by their handles, in: IEEE/RSJ International Conference on Intelligent...
  • KehoeB. et al.

    Toward cloud-based grasping with uncertainty in shape: Estimating lower bounds on achieving force closure with zero-slip push grasps

  • FischingerD. et al.

    Learning grasps with topographic features

    Int. J. Robot. Res.

    (2015)
  • PedroL.M. et al.

    Learning how to grasp based on neural network retraining

    Adv. Robot.

    (2013)
  • ZhengY.

    An efficient algorithm for a grasp quality measure

    IEEE Trans. Robot.

    (2013)
  • GuayF. et al.

    Measuring how well a structure supports varying external wrenches

  • KosugeK. et al.

    A novel grasping mechanism for flat-shaped objects inspired by lateral grasp

  • OdhnerL.U. et al.

    Open-loop precision grasping with underactuated hands inspired by a human manipulation strategy

    IEEE Trans. Autom. Sci. Eng.

    (2013)
  • V. Babin, D. St-Onge, C. Gosselin, Robust grasping of flat objects on hard surfaces using passive and epicyclic...
  • EppnerC. et al.

    Exploitation of environmental constraints in human and robotic grasping

    Int. J. Robot. Res.

    (2015)
  • Cited by (31)

    • Parameter estimation and object gripping based on fingertip force/torque sensors

      2021, Measurement: Journal of the International Measurement Confederation
      Citation Excerpt :

      They improved the gripping stability by optimizing fingertip design and taking advantage of the jamming effect of soft fingertips. Lévesque et al. [36] proposed a model-based gripping method for the autonomous picking of unknown objects with a two-fingered gripper. This method is more suitable for the picking of thin objects lying on the flat surface.

    • Grasp prediction and evaluation of multi-fingered dexterous hands using deep learning

      2020, Robotics and Autonomous Systems
      Citation Excerpt :

      Considering this, more researchers tend to utilize empirical methods. Empirical methods [8–19] are effective on the basis of a small amount of existing data which can be easily collected in simulators or real environments. Moreover, most empirical methods are based on traditional machine learning algorithms, e.g. Logistic Regression (LR) [8], Support Vector Machine (SVM) [9].

    • Dual-camera-based method for identification and location of scattered self-plugging rivets for robot grasping

      2019, Measurement: Journal of the International Measurement Confederation
      Citation Excerpt :

      Visual guided robot grabbing not only requires stable visual detection, but also accurately calibrated relationship between vision and robot [4,5]. Visual guided robot grasping is very effective in grasping regular objects such as typical parts [6], regular shaped cartons [7], bottles, fruits [8], etc. However, in most of the literatures, the accuracy of visual processing is not high enough, and objects need not to be measured accurately.

    View all citing articles on Scopus

    Bruno Sauvet received his M.Sc. in robotics from the Université Pierre et Marie CURIE (France) in 2008, and a Ph.D. degree in mechanics and robotics under the supervision of Prof. Stéphane RÉGNIER within ISIR (Institut des Systèmes Intelligents et de Robotique) in 2013. His Ph.D. thesis focused on the manipulation of thin films (like graphene) with a microrobotic approach and to characterize the physics properties of these nano/micro objects. He was then a post-doctoral fellow at Université laval, Québec in 2014–2017 under the supervision of Prof. Clément Gosselin and Prof. Philippe Cardou on haptics and autonomous grasping of objects. He is currently a post-doctoral fellow at FEMTO-ST (France), with a PRESTIGE post-doctoral funding (Marie Curie fellowship). His work focuses on the challenging manipulation of nano/micro objects (1–10 um).

    Philippe Cardou received the degree in mechanical engineering from Laval University, Quebec City, Canada, in 2003, and the Ph.D. degree in mechanical engineering from McGill University in 2008. Since September 2007, he has been with the Department of Mechanical Engineering, Laval University, where he is a Professor and a member of the Robotics Laboratory. His research interests are related to robotics and mechanisms:kinematics, in general, and in particular, mechanism design and cable-driven parallel robots. Dr. Cardou has published several articles on these topics in international scientific journals and conferences.

    Clément Gosselin received the B.Eng. degree in Mechanical Engineering from the Université de Sherbrooke, Québec, Canada, in 1985, and the Ph.D. degree from McGill University, Montréal, Québec, Canada in 1988. He was then a post-doctoral fellow at INRIA in Sophia-Antipolis, France in 1988–89. In 1989 he was appointed by the Department of Mechanical Engineering at Université Laval, Québec where he is a Full Professor since 1997. He is currently holding a Canada Research Chair in Robotics and Mechatronics since January 2001. He was a visiting researcher at the RWTH in Aachen, Germany in 1995, at the University of Victoria, Canada in 1996 and at the IRCCyN in Nantes, France in 1999.

    His research interests are kinematics, dynamics and control of robotic mechanical systems with a particular emphasis on the mechanics of grasping, the kinematics and dynamics of parallel manipulators and the development of human-friendly robots. His work in the aforementioned areas has been the subject of numerous publications in international journals and conferences as well as of several patents and two books. He has been directing many research initiatives, including collaborations with several Canadian and foreign high-technology companies and he has trained more than 100 graduate students. He is an Associate Editor of the IEEE Robotics and Automation Letters and of the ASME Journal of Mechanisms and Robotics.

    Dr. Gosselin received several awards including the ASME DED Mechanisms and Robotics Committee Award in 2008 and the ASME Machine Design Award in 2013. He was appointed Officer of the Order of Canada in 2010 for contributions to research in parallel mechanisms and underactuated systems. He is a fellow of the ASME, of the IEEE and of the Royal Society of Canada.

    View full text