Abstract
In this chapter, a collaborative work between stereo and a laser scanning system is presented in order to obtain an enriched matrix of rectangular coordinates of elements in a scene using SAD for stereovision and real-time dynamic triangulation in ROS as processing algorithms. Dense data volume and precise depth estimation are the complementary parameters for enrichment and the main reasons for taking these systems to work together on applications as robot navigation, structural health, and body scanning. This combination improves weak points of each separate system, thanks to the use of output data of each technique for calibration and control of the other technique, making it redundant when data confirmation is required and increasing of data volume. Systems link is done through ROIs and known relative position between cameras and ROS positioner and aperture, achieving a collaborative work even when there is displacement. For future work, sensors comparison and motor control optimization is expected for scanning time reduction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- CCD:
-
Charge-coupled device
- CMOS:
-
Complementary metal-oxide-semiconductor
- DC:
-
Direct current
- DLT:
-
Direct linear transformation
- FOV:
-
Field of view
- FPGA:
-
Field-programmable gate array
- NCC:
-
Normalized cross-correlation
- RADAR:
-
Radio detection and ranging
- ROI:
-
Region of interest
- ROS:
-
Rotational optical scanner
- SAD:
-
Sum of absolute differences
- SONAR:
-
Sound navigation and ranging
- SSD:
-
Sum of squared differences
- TOF:
-
Time of flight
References
Vongbunyong S, Chen WH (2015) Vision System. Springer, Cham, pp. 55–93
Chen X (2007) Stereo vision based motion identification. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, pp. 575–586
He Z, Ren Q, Yang T, Li J, Zhang Y (2016) Multi-object detection based on binocular stereo vision. In: Communications in Computer and Information Science. Springer Verlag, pp. 114–121
Mo H, Luo C, Liu K (2016) Robot indoor navigation based on computer vision and machine learning. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer Verlag, pp. 528–534
Panigrahi PK, Tripathy HK (2015) Analysis on intelligent based navigation and path finding of autonomous mobile robot. In: Advances in Intelligent Systems and Computing. Springer Verlag, pp. 219–232
Hart S, Mikhailova E, Post C, McMillan P, Sharp J, Bridges W (2017) Spatio-temporal analysis of flowering using LiDAR topography. J Geogr Sci 27:62–78. https://doi.org/10.1007/s11442-017-1364-x
Broggi A, Medici P, Porta PP (2007) StereoBox: A robust and efficient solution for automotive short-range obstacle detection. Eurasip J Embed Syst 2007:1–7. https://doi.org/10.1155/2007/70256
Kim S, Kim H Bin (2010) High resolution mobile robot obstacle detection using low directivity ultrasonic sensor ring. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Berlin, Heidelberg, pp. 426–433
Zarandy A, Nagy Z, Vanek B, Zsedrovits T, Kiss A, Nemeth M (2013) A five-camera vision system for UAV visual attitude calculation and collision warning. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Berlin, Heidelberg, pp. 11–20
Sergiyenko O, Tyrsa V (2020) 3D optical machine vision sensors with intelligent data management for robotic swarm navigation improvement. IEEE Sens J. https://doi.org/10.1109/JSEN.2020.3007856
Flores-Fuentes W, Rivas-Lopez M, Sergiyenko O, Rodríguez-Quiñonez JC, Hernández-Balbuena D, Rivera-Castillo J (2014) Energy center detection in light scanning sensors for structural health monitoring accuracy enhancement. IEEE Sens J 14:2355–2361. https://doi.org/10.1109/JSEN.2014.2310224
C. J, Sergiyenko O, Tyrsa V, C. L, Rivas-Lopez M, Hernndez-Balbuena D, Pea-Cabrer M (2011) 3D Body & Medical Scanners’ Technologies: Methodology and Spatial Discriminations. In: Optoelectronic Devices and Properties. InTech
Lindner L, Sergiyenko O, Rivas-López M, Hernández-Balbuena D, Flores-Fuentes W, Rodríguez-Quiñonez JC, Murrieta-Rico FN, Ivanov M, Tyrsa V, Básaca-Preciado LC (2017) Exact laser beam positioning for measurement of vegetation vitality. Ind Rob 44:532–541. https://doi.org/10.1108/IR-11-2016-0297
Chung SH, Lee SW, Lee SK, Park JH (2019) LIDAR system with electromagnetic two-axis scanning micromirror based on indirect time-of-flight method. Micro Nano Syst Lett 7:1–5. https://doi.org/10.1186/s40486-019-0082-9
Chung S-H, Lee S-W, Lee S-K, Park J-H (2019) LIDAR system with electromagnetic two-axis scanning micromirror based on indirect time-of-flight method. Micro Nano Syst Lett 2019 71 7:1–5. https://doi.org/10.1186/S40486-019-0082-9
Szeliski R (2011) Stereo correspondence. Springer, London, pp. 467–503
Vilaça JL, Fonseca JC, Pinho AM (2009) Non-contact 3D acquisition system based on stereo vision and laser triangulation. Mach Vis Appl 2008 213 21:341–350. https://doi.org/10.1007/S00138-008-0166-7
Denker K, Lehner B, Umlauf G (2010) Real-time triangulation of point streams. Eng with Comput 2010 271 27:67–80. https://doi.org/10.1007/S00366-010-0181-Y
Mikhaylichenko AA, Kleshchenkov AB (2018) Approach to Non-Contact Measurement of Geometric Parameters of Large-Sized Objects. Program Comput Softw 2018 444 44:271–277. https://doi.org/10.1134/S0361768818040096
Yoo H-S, Kim Y-S, Kwon S-W (2014) A comparative study of noise elimination algorithms for a 3D terrain model through object clustering and the differential method. KSCE J Civ Eng 2015 193 19:498–509. https://doi.org/10.1007/S12205-013-0370-5
Madeo S, Pelliccia R, Salvadori C, del Rincon JM, Nebel JC (2016) An optimized stereo vision implementation for embedded systems: application to RGB and infra-red images. In: Journal of Real-Time Image Processing. Springer Verlag, pp. 725–746
Lu R, Lai J, Xie X (2018) Asymmetric Two-Stream Networks for RGB-Disparity Based Object Detection. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 11259 LNCS:3–15. https://doi.org/10.1007/978-3-030-03341-5_1
Heng J, Xu Z, Zheng Y, Liu Y (2017) Disparity Refinement Using Merged Super-Pixels for Stereo Matching. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 10666 LNCS:295–305. https://doi.org/10.1007/978-3-319-71607-7_26
Popielski P, Wróbel Z, Koprowski R (2014) Object Detail Correspondence Problem in Stereovision. Adv Intell Syst Comput 283:209–222. https://doi.org/10.1007/978-3-319-06593-9_19
Yousfi J, Lahouar S, Ben Amara A (2017) Strategy of Image Capture and Its Impact on Correspondence Error in Reconstructed 3D-Images-Based Point. Lect Notes Mech Eng 115–126. https://doi.org/10.1007/978-3-319-66697-6_12
Lazaros N, Sirakoulis GC, Gasteratos A (2008) Review of stereo vision algorithms: From software to hardware. Int J Optomechatronics 2:435–462. https://doi.org/10.1080/15599610802438680
Hirschmüller H, Scharstein D (2007) Evaluation of cost functions for stereo matching. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. https://doi.org/10.1109/CVPR.2007.383248
Bae K ryeol, Moon B (2017) An accurate and cost-effective stereo matching algorithm and processor for real-time embedded multimedia systems. Multimed Tools Appl 76:17907–17922. https://doi.org/10.1007/s11042-016-3248-y
Patil S, Nadar JS, Gada J, Motghare S, Nair SS (2013) Comparison of Various Stereo Vision Cost Aggregation Methods. Int J Eng Innov Technol 2:222–226
Shen Y (2011) Efficient normalized cross correlation calculation method for stereo vision based robot navigation. Front Comput Sci China 5:227–235. https://doi.org/10.1007/s11704-011-9190-2
Dong Q, Feng J (2018) Adaptive disparity computation using local and non-local cost aggregations. Multimed Tools Appl 2018 7724 77:31647–31663. https://doi.org/10.1007/S11042-018-6236-6
Middlebury (2014) Adirondack. https://vision.middlebury.edu/stereo/data/scenes2014/
Hisham MB, Yaakob SN, Raof RAA, Nazren ABA, Embedded NMW (2015) Template Matching using Sum of Squared Difference and Normalized Cross Correlation. In: 2015 IEEE Student Conference on Research and Development, SCOReD 2015. Institute of Electrical and Electronics Engineers Inc., pp. 100–104
Ilmenau TU, Kuhl A (2005) Comparison of Stereo Matching Algorithms for Mobile Robots
Michalik S, Michalik S, Naghmouchi J, Berekovic M (2017) Real-Time smart stereo camera based on FPGA-SoC. In: IEEE-RAS International Conference on Humanoid Robots. IEEE Computer Society, pp. 311–317
Cambuim LFS, Oliveira LA, Barros ENS, Ferreira APA (2020) An FPGA-based real-time occlusion robust stereo vision system using semi-global matching. J Real-Time Image Process 17:1447–1468. https://doi.org/10.1007/s11554-019-00902-w
Masmoudi MBM, Jerad C, Attia R (2016) On-the-Fly Architecture Design and Implementation of a Real-Time Stereovision System. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 10016 LNCS:711–722. https://doi.org/10.1007/978-3-319-48680-2_62
Li Y, Huang K, Claesen L (2016) A Novel Hardware-Oriented Stereo Matching Algorithm and Its Architecture Design in FPGA. IFIP Adv Inf Commun Technol 508:213–232. https://doi.org/10.1007/978-3-319-67104-8_11
Jin S, Yuanzhi W, Yining S (2018) Design and implementation of wireless multimedia sensor network node based on FPGA and binocular vision. EURASIP J Wirel Commun Netw 2018 20181 2018:1–8. https://doi.org/10.1186/S13638-018-1172-8
Rodriguez-Quinonez JC, Sergiyenko O, Gonzalez-Navarro FF, Basaca-Preciado L, Tyrsa V (2013) Surface recognition improvement in 3D medical laser scanner using Levenberg-Marquardt method. Signal Processing 93:378–386. https://doi.org/10.1016/j.sigpro.2012.07.001
Sankowski W, Włodarczyk M, Kacperski D, Grabowski K (2017) Estimation of measurement uncertainty in stereo vision system. Image Vis Comput 61:70–81. https://doi.org/10.1016/j.imavis.2017.02.005
Rajeshkannan S, Korah R (2016) Improved CRC Based Disparity Estimation of Vision System Using Local Adaptive Hue Census and Mean Shift Clustering. Natl Acad Sci Lett 2016 391 39:35–38. https://doi.org/10.1007/S40009-015-0412-2
Pratt P, Bergeles C, Darzi A, Yang G-Z (2014) Practical Intraoperative Stereo Camera Calibration. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 8674 LNCS:667–675. https://doi.org/10.1007/978-3-319-10470-6_83
Yang S, Liu M, Song J, Yin S, Guo Y, Ren Y, Zhu J (2017) Projector calibration method based on stereo vision system. Opt Rev 2017 246 24:727–733. https://doi.org/10.1007/S10043-017-0370-7
Taryudi, Wang M-S (2017) Eye to hand calibration using ANFIS for stereo vision-based object manipulation system. Microsyst Technol 2017 241 24:305–317. https://doi.org/10.1007/S00542-017-3315-Y
Hyun J, Moon B (2016) A simplified rectification method and its hardware architecture for embedded multimedia systems. Multimed Tools Appl 2016 7619 76:19761–19779. https://doi.org/10.1007/S11042-016-3517-9
Hansen P, Alismail H, Rander P, Browning B (2012) Online continuous stereo extrinsic parameter estimation. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1059–1066. https://doi.org/10.1109/CVPR.2012.6247784
Taketomi T, Okada K, Yamamoto G, Miyazaki J, Kato H (2014) Camera pose estimation under dynamic intrinsic parameter change for augmented reality. Comput Graph 44:11–19. https://doi.org/10.1016/j.cag.2014.07.003
Zhao Z, Ye D, Zhang X, Chen G, Zhang B (2016) Improved Direct Linear Transformation for Parameter Decoupling in Camera Calibration. Algorithms 9:31. https://doi.org/10.3390/a9020031
Monasse P, Morel JM, Tang Z (2010) Three-step image rectification. Br Mach Vis Conf BMVC 2010 – Proc. https://doi.org/10.5244/C.24.89
Dinh VQ, Nguyen TP, Jeon JW (2019) Rectification Using Different Types of Cameras Attached to a Vehicle. IEEE Trans Image Process 28:815–826. https://doi.org/10.1109/TIP.2018.2870930
Abraham S, Förstner W (2005) Fish-eye-stereo calibration and epipolar rectification. ISPRS J Photogramm Remote Sens 59:278–288. https://doi.org/10.1016/j.isprsjprs.2005.03.001
Kumar S, Micheloni C, Piciarelli C, Foresti GL (2010) Stereo rectification of uncalibrated and heterogeneous images. Pattern Recognit Lett 31:1445–1452. https://doi.org/10.1016/j.patrec.2010.03.019
Ramírez-Hernández LR, Rodríguez-Quiñonez JC, Castro-Toscano MJ, Hernández-Balbuena D, Flores-Fuentes W, Rascón-Carmona R, Lindner L, Sergiyenko O (2020) Improve three-dimensional point localization accuracy in stereo vision systems using a novel camera calibration method. Int J Adv Robot Syst 17:172988141989671. https://doi.org/10.1177/1729881419896717
Barrena N, Sánchez JR, Ugarte RJ, Alonso AG (2018) Proving the efficiency of template matching-based markerless tracking methods which consider the camera perspective deformations. Mach Vis Appl 29:573–584. https://doi.org/10.1007/s00138-018-0914-2
Lindner L, Sergiyenko O, Rodríguez-Quiñonez JC, Rivas-Lopez M, Hernandez-Balbuena D, Flores-Fuentes W, Murrieta-Rico FN, Tyrsa V (2016) Mobile robot vision system using continuous laser scanning for industrial application. Ind Rob 43:360–369. https://doi.org/10.1108/IR-01-2016-0048
Sohn K, Jang G (2019) Ground Vehicle Driving by Full Sized Humanoid. J Intell Robot Syst 2019 992 99:407–425. https://doi.org/10.1007/S10846-019-01130-X
Everett MF (2017) Robot designed for socially acceptable navigation
Belbachir A (2017) An embedded testbed architecture to evaluate autonomous car driving. Intell Serv Robot 2017 102 10:109–119. https://doi.org/10.1007/S11370-016-0213-6
Ellery A (2016) Autonomous Navigation—Self-localization and Mapping (SLAM). Planet Rovers 331–374. https://doi.org/10.1007/978-3-642-03259-2_9
Acknowledgments
Authors have special thanks to Mexican organization CONACyT for funding of PhD students participating as co-authors of this chapter, as well as our universities who provide our group with facilities, laboratory and equipment for herein presented research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix A
Code used to send stepper motors’ sequence usable for Arduino. This code considers a serial input with format “DX SY,” where D is a symbolic reference to the direction where Y can take a value from 1 to 4, being 1–2 both directions of one stepper motor (horizontal for ROS) and 3–4 for the second stepper motor (vertical for ROS). S refers to the number of steps Y.
#define m1_1 22 #define m1_2 23 #define m1_3 24 #define m1_4 25 #define m2_1 26 #define m2_2 27 #define m2_3 28 #define m2_4 29 int dir = 0, lastdirH = 0, lastdirV = 0, steps = 0, binH = 1, binV = 1; bool enc = false; float vMotor = 0; void setup() { pinMode(m1_1,OUTPUT); pinMode(m1_2,OUTPUT); pinMode(m1_3,OUTPUT); pinMode(m1_4,OUTPUT); pinMode(m2_1,OUTPUT); pinMode(m2_2,OUTPUT); pinMode(m2_3,OUTPUT); pinMode(m2_4,OUTPUT); digitalWrite(m1_1,0); digitalWrite(m1_2,0); digitalWrite(m1_3,0); digitalWrite(m1_4,0); digitalWrite(m2_1,0); digitalWrite(m2_2,0); digitalWrite(m2_3,0); digitalWrite(m2_4,0); Serial.begin(9600); } void loop(){ if (Serial.available() > 0){ String dataSt = Serial.readString(); int str_len = dataSt.length() + 1; char data[str_len]; dataSt.toCharArray(data, str_len); sscanf(data,"D%d S%d",&dir,&steps); if (lastdirH == 0 && dir < 3){ lastdirH = dir; } if (lastdirV == 0 && dir > 2){ lastdirV = dir; } if (dir == 1 || dir == 2){ if (lastdirH != dir){ binH = -(binH-5); lastdirH = dir; } } if (dir == 3 || dir == 4){ if (lastdirV != dir){ binV = -(binV-5); lastdirV = dir; } } int i = 0; for (i = 1;i <= steps;i++){ delay(500); if(dir == 2 || dir == 4){ if (dir == 2){digitalWrite(m1_1+binH-1,0);} if (dir == 4){digitalWrite(m2_1+binV-1,0);} } if(dir == 1 || dir == 3){ if (dir == 1){digitalWrite(m1_4-binH+1,0);} if (dir == 3){digitalWrite(m2_4-binV+1,0);} } if (dir == 1 || dir == 2){ if (binH == 4){binH = 0;} binH++; }else{ if (binV == 4){binV = 0;} binV++;} if (dir == 1 || dir == 2){ if (dir == 2){digitalWrite(m1_1+binH-1,1);} if (dir == 1){digitalWrite(m1_4-binH+1,1);} } if (dir == 3 || dir == 4){ if (dir == 4){digitalWrite(m2_1+binV-1,1);} if (dir == 3){digitalWrite(m2_4-binV+1,1);} } } Serial.println((String) 5); } }
Appendix B
Code used for beta angle estimation in ROS usable for Arduino
float vmaxO = 0,vminO = 999,vrefO = 0; //Reference volages int capture = 0,c = 0,pos = 0, pos_ant = 0,change = 0; //Flags float opto = 0,foto = 0; //Data int picoO[2] = {0},picoF = 0, maxF = -9999; //Peaks int cpicosO = 0,diffO = 0,diffF = 0; //Peaks; void setup() { pinMode(A0,INPUT); pinMode(A1,INPUT); Serial.begin(9600); } void loop() { if (c>50000){ vmaxO = vminO * 2; } opto = analogRead(A0); foto = analogRead(A1); foto = - foto; if (opto < vminO){vminO = opto;} if (opto > vmaxO || vrefO > vmaxO){vmaxO = opto;} vrefO = (vminO + vmaxO) / 2; if (opto<=vrefO){capture=1;} if (capture==1){ if (foto > maxF){ maxF = foto; picoF = c; } c++; if (opto > vrefO * 1.1){ pos = 1; if (pos_ant == 0){ change = 1; if (cpicosO == 1){ picoO[0] = c; } else{ picoO[1] = c; } } pos_ant = 1; } else if (opto < vrefO * .9){ pos = 0; if (pos_ant == 1){ change = 2; if (cpicosO == 0){ cpicosO++; } else{ cpicosO=2; } } pos_ant = 0; } if (cpicosO == 2 && change == 1){ diffO=picoO[1]-picoO[0]; diffF=picoF-picoO[0]; Serial.println((String) "O" + diffO + " F" + diffF); maxF = -9999; c = 0; cpicosO = 1; picoO[0] = 0; picoO[1] = 0; } } }
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Alaniz-Plata, R. et al. (2022). ROS and Stereovision Collaborative System. In: Sergiyenko, O. (eds) Optoelectronic Devices in Robotic Systems. Springer, Cham. https://doi.org/10.1007/978-3-031-09791-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-09791-1_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09790-4
Online ISBN: 978-3-031-09791-1
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)