The Acquisition of Position and Orientation of the Conveyor Belt Workpiece Based on the Inter Frame Difference in ROI in Camera Video

Conveyor belt transfer is a widely used transportation means in industry and agriculture, with the help of the robot arms the workpiece on the belt can be picked and placed, replacing human sorters for production lines work. The position and orientation of the workpiece are important for grabbing by the robot arms. The goal of the paper was to investigate the acquisition of the position and orientation of the conveyor belt workpiece by means of the camera video overhead looking down the belt. The proposed method is the inter frame difference in nature, using the conveyor belt background as the first frame, but the other frames were not used wholly as usually, only an ROI all around the conveyor belt in the camera video was chosen, and the inter frame difference was carried out in the ROI. The ROI was of the same width as that of the belt in the video which was known in advance, while the length of the ROI was arbitrary, so one pixel in the frame was scaled to the actual length conveniently. Every read frame behind the background was computed the difference with the background in such ROI, and the four vertexes coordinates of the rectangle workpiece image on the belt were obtained when it passed the ROI, and then the distance apart from the right belt boundary was calculated due to the proportional relation between the width of workpiece and that of the ROI. Two kind workpiece orientation on the belt toward the left and right were judged using the same obtained four vertexes coordinates by means of Euclidian length, and the tilt angle was calculated by arc tangent function in favour of two narrow sides of rectangle workpiece grab. The actual test showed that the method of obtaining the position and orientation of workpiece on the belt proposed in the paper could be realized correctly.


Introduction
Conveyor belt serves as a useful transportation tool, which is applied in industry, agriculture, etc. An integrated system composed of a conveyor belt and a posts array which could simultaneously implement the functions of size separation and transportation of disk particles was discussed [1][2][3]. Advanced robots are used in the manufacturing industry, robot arms can pick and place workpiece on the conveyor belt effectively, hence robotic units are replacing human sorters for production lines work [4].Sorting of objects or materials is very essential in almost any Industry, and by doing it in a proper way the time can be saved and overall productivity is improved. That's why there is an intervention of smart robots in Industry [5,6].The multi-fingered manipulator grasps points planning facing the medical robotics was discussed. Aiming at the problem of grab point planning, a crawling planning method based on big data Gaussian process classification was proposed. and the hyper constant used in the Gaussian process to judge the probability of capture was calculated [7].To classify  [8].Machine vision technology has been used in the objects recognition aiming to duplicate the effect of human vision by electronically perceiving and understanding an image. A methodology which provided a fast inspection of defective objects and generated a real time motion trajectory for processing objects being conveyed with high speed in an industrial large-scale production was presented. The image data obtained by a multispectral imaging system was analysed within image processing algorithms using classification methods based on support vector machine [9].In [10] a method combining YOLOv3 algorithm and hand-engineered features was developed to calculate the maturity of winter-jujube. After using image distortion correction and object detection, an image processing flow for spatial positioning, size measurement and maturity calculation for winter-jujubes was designed.

System Framework
We consider the following conveyor belt transfer system shown in the figure 1, suppose both the speed moving toward and the width of the belt are known. The workpiece distance from the right belt is key parameter for the robot arm's grabbing nearby, this assumption is rational for some kinds of robot arms grabbing object in the belt, and the orientation of workpiece is also need to know when the workpiece was put askew, namely, it is not parallel to the belt boundary. In order to get the position and orientation of the workpiece a camera was set overhead looking down the conveyor belt, then the camera video was processed by the PC computer, we supposed that the frame image reversed left and right due to the camera overhead capture.

Inter Frame Difference in ROI
Inter frame difference is suitable for target detection with fixed background, camera video captured the background as the first frame, it was the same here in the paper, but it would lose the baseline for workpiece position on the belt if we used the whole background frame and other video frame to make difference directly, and the calibration step was unavoidable in that case, which meant extra computing outgoings. We proposed a method to simplify the computing process, an ROI only alone the conveyor belt boundary in the camera video was selected, the ROI was of the same width as that of the belt in the video which was known in advance, while the length of the ROI was arbitrary, so one pixel in the frame was scaled to the actual length conveniently. All the inter frame difference computation were carried out in the ROI. the width of ROT was R , the right boundary appeared in the figure 1 was X , which was to be solved, the right coordinate of workpiece was described as D . Because one pixel in the frame was scaled to the actual length, so the actual X was expressed as

Orientation Determination of the Workpieces
Considering the workpiece was put skew on the belt which was common, it was need to calculate the incline angle. Due to the fixed gauge of the robot jaw, It had better to grasp the workpiece at the two narrow edges to avoid bump against the it. There are usually two kinds incline as shown of figure 2, 3.
Otherwise the angle (angle degree) was equal to formula (4).

180
angle angle   (4) And the position X obtained above was revised correspondingly, it needed to plus px shown in the figure 5. The px was given as: * cos( ) px l l angle   (5) where l was the half length of the gauge related to the robot jaw.

Materials and Setup
A 1/6 inch CMOS lens, 1080P with USB interface HD98 Camera named GuKe was used, which was hung 190 centimetres higher overhead the conveyor belt, the frame velocity was 30 f/s with 640X480. The width of conveyor belt is 40cm, and the size of the test paper box on the belt is 17cmX12.5cmX4.0cm. The PC operation system is Windows 7.

Measure and Discuss
Label A was the place where the ROI began to selected shown in figure 6, it was determined by the linear distance between the robot jaw and workpiece. Figure 7 showed the contour of the BW image of workpiece using red rectangle box in the ROI. The measurement of position and orientation of workpiece on the belt were shown as the table 1, where the l was chosen as 6, and the conveyor belt velocity was 11.1m/s. The 0ºorientation meant the vertical entering of the workpiece into the ROI.

Conclusion
Inter frame difference method in the ROI on the camera video was proposed to acquire the position and orientation of workpiece on the conveyor belt, and a position revising algorithm was explored further, in favour of two narrow sides of rectangle workpiece to be grabbed by the robot arms. The actual test showed that the position and orientation of the workpiece on the belt could be obtained correctly according to the method proposed in the paper, and the method was obviously of simple and fast advantages.