Skip to main content
SpringerLink
Log in

3D Reconstruction of deformable linear objects based on cylindrical fitting

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The manipulation of deformable linear objects (DLOs) is an important task in many fields, which raises demands for the perception of DLOs in real situation. In this paper, we propose a cylindrical fitting reconstruction method for DLOs with only one frame of point clouds captured by a depth camera. The point clouds are first processed by operation space filtering and outlier removal to eliminate the interference. To accurately segment the specific object from the complex background, the PointSIFT module is inserted into PointNet++ architecture and fine-tuned on our dataset. To reconstruct the flexible DLOs in 3D space, an improved adaptive K-means algorithm which accommodates to the unknown length and curvature is designed. The adaptive K-means algorithm distributes the point clouds into appropriate number of cylindrical clusters. To achieve the main axis of the cylinders, we construct the point clouds covariance matrix. By applying principal component analysis (PCA), three orthogonal dimensions and the PCA bounding box are obtained. Afterward, an octree-based directional constraints is designed to sort the center points of DLOs with arbitrary curvature. The proposed framework achieves an average error of less than 1 mm during a manipulation experiment in a simulation live-line maintenance site.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Availability of data and materials

The cable dataset generated during the current study is shot in private scene and involves other technology privacy. So they are not publicly available but are available from the corresponding author on reasonable request.

References

  1. Bo, G., Qingquan, L., Xianfeng, H., et al.: An improved method for power-line reconstruction from point cloud data. Remote Sens. 8(1), 36 (2016). https://doi.org/10.3390/rs8010036

    Article  Google Scholar 

  2. Borum, A., Matthews, D., Bretl, T.: State estimation and tracking of deforming planar elastic rods. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 4127–4132. IEEE (2014). https://doi.org/10.1109/ICRA.2014.6907459

  3. Bretl, T., McCarthy, Z.: Equilibrium configurations of a Kirchhoff elastic rod under quasi-static manipulation. In: Algorithmic Foundations of Robotics X, pp. 71–87. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-36279-8_5

    Chapter  Google Scholar 

  4. Caldwell, T.M., Coleman, D., Correll, N.: Optimal parameter identification for discrete mechanical systems with application to flexible object manipulation. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 898–905. IEEE (2014). https://doi.org/10.1109/IROS.2014.6942666

  5. Chen, W., Liu, J., Tang, Y., et al.: Automatic spray trajectory optimization on Bézier surface. Electronics 8(2), 168 (2019). https://doi.org/10.3390/electronics8020168

    Article  Google Scholar 

  6. Cretu, A.M., Petriu, E.M., Payeur, P., et al.: Deformable object segmentation and contour tracking in image sequences using unsupervised networks. In: 2010 Canadian Conference on Computer and Robot Vision, pp. 277–284. IEEE (2010). https://doi.org/10.1109/CRV.2010.43

  7. Dai, A., Chang, A.X., Savva, M., et al.: Scannet: richly-annotated 3d reconstructions of indoor scenes. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2432–2443. IEEE Computer Society, Los Alamitos, CA, USA (2017). https://doi.org/10.1109/CVPR.2017.261

  8. Fan, Z., Guo, L., Chen, Y., et al.: CPC: cable parameters calculation based on 3D point cloud. In: International Conference on Intelligent Robotics and Applications, pp. 264–271. Springer, Berlin (2022). https://doi.org/10.1007/978-3-031-13841-6_25

  9. Gao, H., Ye, C., Lin, W., et al.: Complex workpiece positioning system with nonrigid registration method for 6-DoFs automatic spray painting robot. IEEE Trans. Syst. Man Cybern. Syst. 51(12), 7305–7313 (2020). https://doi.org/10.1109/TSMC.2020.2980424

    Article  Google Scholar 

  10. Hu, Q., Yang, B., Xie, L., et al.: Learning semantic segmentation of large-scale point clouds with random sampling. IEEE Trans. Pattern Anal. Mach. Intell. (2021). https://doi.org/10.1109/TPAMI.2021.3083288

    Article  Google Scholar 

  11. Lee, M.J., Jeon, Y.J., Son, G.E., et al.: Grain boundary conformed volumetric mesh generation from a three-dimensional voxellated polycrystalline microstructure. Met. Mater. Int. 24(4), 845–859 (2018). https://doi.org/10.1007/s12540-018-0083-x

    Article  Google Scholar 

  12. Li, P., Wang, R., Wang, Y., et al.: Evaluation of the ICP algorithm in 3D point cloud registration. IEEE Access 8, 68030–68048 (2020). https://doi.org/10.1109/ACCESS.2020.2986470

    Article  Google Scholar 

  13. Qi, C.R., Su, H., Mo, K., et al.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017). arXiv:1612.00593

  14. Qi, C.R., Yi, L., Su, H., et al.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

  15. Rostami, M., Oussalah, M., Farrahi, V.: A novel time-aware food recommender-system based on deep learning and graph clustering. IEEE Access 10, 52508–52524 (2022). https://doi.org/10.1109/ACCESS.2022.3175317

    Article  Google Scholar 

  16. Saberi-Movahed, F., Rostami, M., Berahmand, K., et al.: Dual regularized unsupervised feature selection based on matrix factorization and minimum redundancy with application in gene selection. Knowl. Based Syst. 256, 109,884-109,900 (2022). https://doi.org/10.1016/j.knosys.2022.109884

    Article  Google Scholar 

  17. Sanchez, J., Corrales, J.A., Bouzgarrou, B.C., et al.: Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. Int. J. Robotics Res. 37(7), 688–716 (2018). https://doi.org/10.1177/0278364918779698

    Article  Google Scholar 

  18. Wei, W., Hui, Z., Yu, G., et al.: Peg-in-hole assembly in live-line maintenance based on generative mapping and searching network. Robotics Auton. Syst. 143(103), 797 (2021). https://doi.org/10.1016/j.robot.2021.103797

    Article  Google Scholar 

  19. Wu, P., Li, W., Yan, M.: 3D scene reconstruction based on improved ICP algorithm. Microprocess. Microsyst. 75(103), 064 (2020). https://doi.org/10.1016/j.micpro.2020.103064

    Article  Google Scholar 

  20. Xie, Y., Tian, J., Zhu, X.X.: Linking points with labels in 3D: a review of point cloud semantic segmentation. IEEE Geosci. Remote Sens. Mag. 8(4), 38–59 (2020). https://doi.org/10.1109/MGRS.2019.2937630

    Article  Google Scholar 

  21. Yan, M., Zhu, Y., Jin, N., et al.: Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robotics Autom. Lett. 5(2), 2372–2379 (2020). https://doi.org/10.1109/LRA.2020.2969931

    Article  Google Scholar 

  22. Zhao, H., Jiang, L., Fu, C.W., et al.: Pointweb: enhancing local neighborhood features for point cloud processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5565–5573 (2019)

Download references

Funding

This study was supported by National Natural Science Foundation of China(61973167).

Author information

Authors and Affiliations

  1. School of Automation, Nanjing University of Science and Technology, Xiaolingwei Street, Nanjing, 210094, Jiangsu, China

    Yiman Zhu, Wei Wu & Yu Guo

  2. The fiftieth Research Institute of China Electronic Technology Group Corporation, Changhe Street, Shanghai, 200000, China

    Xiao Xiao

Authors
  1. Yiman Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yiman Zhu designed and implemented the method. Xiao Xiao wrote the original draft. Wei Wu did visualization and investigation. Yu Guo was responsible for conceptualization, resources, review and editing, project administration and funding acquisition.

Corresponding author

Correspondence to Yu Guo.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, Y., Xiao, X., Wu, W. et al. 3D Reconstruction of deformable linear objects based on cylindrical fitting. SIViP 17, 2617–2625 (2023). https://doi.org/10.1007/s11760-022-02478-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02478-8

Keywords

Search

Navigation