Skip to main content
SpringerLink
Log in

Combining human guidance and structured task execution during physical human–robot collaboration

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In this work, we consider a scenario in which a human operator physically interacts with a collaborative robot (CoBot) to perform shared and structured tasks. We assume that collaborative operations are formulated as hierarchical task networks to be interactively executed exploiting the human physical guidance. In this scenario, the human interventions are continuously interpreted by the robotic system in order to infer whether the human guidance is aligned or not with respect to the planned activities. The interpreted human interventions are also exploited by the robotic system to on-line adapt its cooperative behavior during the execution of the shared plan. Depending on the estimated operator intentions, the robotic system can adjust tasks or motions, while regulating the robot compliance with respect to the co-worker physical guidance. We describe the overall framework illustrating the architecture and its components. The proposed approach is demonstrated in a testing scenario consisting of a human operator that interacts with the Kuka LBR iiwa manipulator in order to perform a collaborative task. The collected results show the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The Wilcoxon signed-rank test (Wilcoxon, 1945) with Pratt modification (Pratt, 1959) (for zero differences in Liker values) was deployed since the collected questionnaire results are paired and not normally distributed.

References

  • Broquère, X., Finzi, A., Mainprice, J., Rossi, S., Sidobre, D., & Staffa, M. (2014). An attentional approach to human-robot interactive manipulation. International Journal of Social Robotics, 6(4), 533–553.

    Article  Google Scholar 

  • Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2018). Interactive plan execution during human–robot cooperative manipulation. IFAC-PapersOnLine, 51(22), 500–505.

    Article  Google Scholar 

  • Cacace, J., Caccavale, R., Finzi, A., & Lippiello, V. (2019). Variable admittance control based on virtual fixtures for human–robot co-manipulation. In Proceedings of the IEEE international conference on systems, man and cybernetics (SMC 2019), pp. 1569–1574

  • Cacace, J., Finzi, A., & Lippiello, V. (2019). Enhancing shared control via contact force classification in human–robot cooperative task execution. In F. Ficuciello, F. Ruggiero, & A. Finzi (Eds.), Human friendly robotics (pp. 167–179). Springer International Publishing.

  • Caccavale, R., & Finzi, A. (2015). Plan execution and attentional regulations for flexible human–robot interaction. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2015), IEEE, pp. 2453–2458

  • Caccavale, R., & Finzi, A. (2017). Flexible task execution and attentional regulations in human–robot interaction. IEEE Transactions on Cognitive and Developmental Systems, 9(1), 68–79.

    Article  Google Scholar 

  • Caccavale, R., & Finzi, A. (2019). Learning attentional regulations for structured tasks execution in robotic cognitive control. Autonomous Robots, 43(8), 2229–2243.

    Article  Google Scholar 

  • Caccavale, R., & Finzi, A. (2022). A robotic cognitive control framework for collaborative task execution and learning. Topics in Cognitive Science, 14(2), 327–343.

    Article  Google Scholar 

  • Caccavale, R., Cacace, J., Fiore, M., Alami, R., & Finzi, A. (2016). Attentional supervision of human-robot collaborative plans. In Proceedings of the IEEE international symposium on robot and human interactive communication (RO-MAN 2016), pp. 867–873

  • Caccavale, R., Saveriano, M., Finzi, A., & Lee, D. (2019). Kinesthetic teaching and attentional supervision of structured tasks in human–robot interaction. Autonomous Robots, 43(6), 1291–1307.

    Article  Google Scholar 

  • Carbone, A., Finzi, A., Orlandini, A., & Pirri, F. (2008). Model-based control architecture for attentive robots in rescue scenarios. Autonomous Robots, 24(1), 87–120.

    Article  Google Scholar 

  • Chen, T. L., & Kemp, C. C. (2010). Lead me by the hand: Evaluation of a direct physical interface for nursing assistant robots. In Proceedings of the 5th ACM/IEEE international conference on human–robot interaction (HRI 2010), pp. 367–374

  • Clodic, A., Cao, H., Alili, S., Montreuil, V., Alami, R., & Chatila, R. (2008). SHARY: A supervision system adapted to human-robot interaction. ISER, Springer, Springer Tracts in Advanced Robotics, 54, 229–238.

    Article  Google Scholar 

  • Colgate, E., & Hogan, N. (1989). An analysis of contact instability in terms of passive physical equivalents. In Proceedings of the international conference on robotics and automation (ICRA 1989), Vol. 1, pp. 404–409

  • Colgate, J. E., & Hogan, N. (1988). Robust control of dynamically interacting systems. International Journal of Control, 48(1), 65–88.

    Article  Google Scholar 

  • Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activies. Cognitive Neuropsychology, 17(4), 297–338.

    Article  Google Scholar 

  • Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887–916.

    Article  Google Scholar 

  • Corrales, J. A., Garcia Gomez, G. J., Torres, F., & Perdereau, V. (2012). Cooperative tasks between humans and robots in industrial environments. International Journal of Advanced Robotic Systems, 9, 1–10.

    Article  Google Scholar 

  • De Santis, A., Siciliano, B., Luca, A., & Bicchi, A. (2007). An atlas of physical human–robot interaction. Mechanism and Machine Theory, 43(3), 253–270.

    Article  Google Scholar 

  • Grafakos, S., Dimeas, F., & Aspragathos, N. (2016). Variable admittance control in phri using emg-based arm muscles co-activation. In Proceedings of the IEEE international conference on systems, man, and cybernetics (SMC 2016), pp. 1900–1905

  • Hart, S., & Staveland, L. (1988). Development of nasa-tlx (task load index): Results of empirical and theoretical research. Human Mental Workload Advances in Psychology, 52, 139–183.

    Article  Google Scholar 

  • Hoffman, G., & Breazeal, C. (2004). Collaboration in human-robot teams. In Proceedings of the AIAA 1st intelligent systems technical conference, pp. 1–18

  • Hoffman, G., & Breazeal, C. (2007). Effects of anticipatory action on human-robot teamwork: Efficiency, fluency, and perception of team. In Proceedings of the 2nd ACM/IEEE international conference on human–robot interaction (HRI 2007), pp. 1–8

  • Johannsmeier, L., & Haddadin, S. (2017). A hierarchical human–robot interaction-planning framework for task allocation in collaborative industrial assembly processes. IEEE Robotics and Automation Letters, 2(1), 41–48.

    Article  Google Scholar 

  • Joseph, L., & Cacace, J. (2018). Mastering ROS for robotics programming: Design, build, and simulate complex robots using the robot operating system, 2nd edn. Packt Publishing

  • Karpas, E., Levine, S. J., Yu, P., & Williams, BC. (2015). Robust execution of plans for human-robot teams. In: Proceedings of the twenty-fifth international conference on international conference on automated planning and scheduling (ICAPS 2015), AAAI Press, pp. 342–346

  • Lallement, R., de Silva, L., & Alami, R. (2014). HATP: An HTN planner for robotics. In: Proceedings of the 2nd ICAPS workshop on planning and robotics (PlanRob 2014), pp. 20–27

  • Maurtua, I., Ibarguren, A., Kildal, J., Susperregi, L., & Sierra, B. (2017). Human-robot collaboration in industrial applications: Safety, interaction and trust. International Journal of Advanced Robotic Systems, 14, 1–10.

  • Nicolis, D., Zanchettin, AM., & Rocco, P. (2018). Human intention estimation based on neural networks for enhanced collaboration with robots. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 1326–1333

  • Norman, D. A., & Shallice, T. (1986). Attention to action. In: Consciousness and self-regulation, pp 1–18. Springer

  • Park, J. S., Park, C., & Manocha, D. (2019). I-planner: Intention-aware motion planning using learning-based human motion prediction. The International Journal of Robotics Research, 38(1), 23–39.

    Article  Google Scholar 

  • Peternel, L., Tsagarakis, N., Caldwell, D., & Ajoudani, A. (2016). Adaptation of robot physical behaviour to human fatigue in human–robot co-manipulation. In: Proceedings of the IEEE-RAS 16th international conference on humanoid robots (humanoids 2016), pp. 489–494

  • Pratt, J. (1959). Remarks on zeros and ties in the Wilcoxon signed rank procedures. Journal of the American Statistical Association, 54(287), 655–667.

    Article  Google Scholar 

  • Raiola, G., Lamy, X., & Stulp, F. (2015). Co-manipulation with multiple probabilistic virtual guides. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS 2015), pp. 7–13

  • Romero, D., Bernus, P., Noran, O., Stahre, J., & Fast-Berglund, Å. (2016). The operator 4.0: human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. In: Proceedings of the IFIP international conference on advances in production management systems, Springer, pp. 677–686

  • Shah, J., Wiken, J., Williams, B., & Breazeal, C. (2011). Improved human-robot team performance using chaski, a human-inspired plan execution system. In: Proceedings of the 6th ACM/IEEE international conference on human–robot interaction (HRI 2011), pp. 29–36

  • Siciliano, B., Sciavicco, L., Villani, L., & Oriolo, G. (2008). Robotics: Modelling, planning and control (1st ed.). Springer Publishing Company, Incorporated.

  • Sisbot, E. A., Marin-Urias, L. F., Alami, R., & Simeon, T. (2007). A human aware mobile robot motion planner. IEEE Transactions on Robotics, 23(5), 874–883.

    Article  Google Scholar 

  • Steinfeld, A., Fong, T., Kaber, D., Lewis, M., Scholtz, J., Schultz, A., & Goodrich, M. (2006). Common metrics for human-robot interaction. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human–robot interaction (HRI 2006). ACM, pp 33–40

  • Vernon, D., & Vincze, M. (2016). Industrial priorities for cognitive robotics. In EUCognition, CEUR-WS.org, CEUR workshop proceedings, Vol. 1855, pp. 6–9

  • Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.

    Article  Google Scholar 

  • Young, J. E., Sung, J., Voida, A., Sharlin, E., Igarashi, T., Christensen, H. I., & Grinter, R. E. (2011). Evaluating human–robot interaction. International Journal of Social Robotics, 3(1), 53–67.

    Article  Google Scholar 

Download references

Acknowledgements

The research leading to these results has been partially supported by the projects HARMONY (H2020-ICT-46-2020, grant agreement 101017008), ICOSAF (PON R &I 2014-2020) and HYFLIERS (H2020-ICT-779411).

Author information

Authors and Affiliations

  1. Dipartimento di Ingegneria Elettrica e Tecnologie dell’Informazione (DIETI), Università degli Studi di Napoli “Federico II”, via Claudio 21, 80125, Naples, Italy

    Jonathan Cacace, Riccardo Caccavale, Alberto Finzi & Riccardo Grieco

Authors
  1. Jonathan Cacace
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riccardo Caccavale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alberto Finzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Riccardo Grieco
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alberto Finzi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cacace, J., Caccavale, R., Finzi, A. et al. Combining human guidance and structured task execution during physical human–robot collaboration. J Intell Manuf 34, 3053–3067 (2023). https://doi.org/10.1007/s10845-022-01989-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-022-01989-y

Keywords

Search

Navigation