A Procedure to Integrate a CIS Sensor in an Additive Manufacturing Machine for In-Situ Digitizing of Deposited Material Layers

A notable constraint to the industrial application of additive manufacturing processes is the lack of geometric accuracy of the parts produced. In-situ inspection systems, therefore, need to be developed to detect and compensate for any geometric deviations that appear during the process. This article proposes a noncontact system for the digitization of deposited material layers, based on a contact image sensor extracted from a low-cost flatbed paper scanner. The viability of using this sensor for such a purpose was analyzed by integrating it in a tailor-made test bench with the dual capability of fused filament fabrication and digitizing of deposited layers. The integration procedure includes analysis of the sensor's operating principle, development of the hardware and control routines required for PC operation, and image processing. To validate the integration, two specimen parts were built and digitized in the test bench. The measurements performed on the captured images showed low discrepancies with respect to those obtained on a coordinate measuring machine.

Despite the advantages offered by AM processes, factors such as the conversion of the CAD model to STL format, as well as process parameter settings, machine errors, or material shrinkage, cause deviations that affect the geometric accuracy of the resulting part compared to the CAD model. In recent years, therefore, numerous contributions have been studied and attempted to compensate for these geometric errors. Some authors have focused on the design stages prior to fabrication of the part by modifying the faces of the STL model to reduce the chordal error with respect to the CAD model [4]. Others have proposed to moderate the staircase effect of parts built using adaptive slicing techniques [5] or developing predictive models to compensate for shrinkage errors [6], [7]. Still, others have analyzed the effect of process parameters [8], [9], or AM machine positioning errors [10], [11], on the dimensional accuracy of the manufactured parts.
In most of the above-mentioned studies, compensation of errors was performed offline. However, according to the National Institute of Standards and Technology, in-situ monitoring and control could provide a significant opportunity to enhance the quality in parts produced by AM [12]. Although it is unquestionable that the part manufacturing time increases when performing inspection tasks at intermediate stages of the process, it is also true that the part precision becomes more in line with the operational requirements. In this way, final inspection phases could be eliminated, and the production of defective parts avoided what, in turn, would mean less nonquality costs and an improvement of productivity. For this reason, in-situ verification techniques are applied in many processes providing clear advantages over out-of-machine verification. However, so far, most studies have focused on in-situ identification of defects or compensation of deviations in thickness of the deposited layers [13]- [19], while research aimed at detecting geometric deviations in the contour of these layers are still scarce [20].
A laser powder deposition system using a charge-coupled device (CCD) camera to determine the cladding height and angle of the liquid/solid interface in real time is described in [14]. Using a PID controller, the laser pulse was regulated to improve the geometric characteristics of the cladding, but the system was limited to simple straight tracks. In [15], a robot with a 3-D scanner was used to control the thickness of the layers in a laser metal wire deposition process. In this case, deviations in layer height were compensated by controlling the wire feed This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ rate in the next deposited layer. In [16], a 3-D scanner was also used to digitize the layers deposited in an inkjet 3-D printing process. A predictive control algorithm for layer-to-layer inkjet 3-D printing is proposed to improve printing quality in terms of surface uniformity and layer height consistency.
Detection of surface defects in parts produced by the process known as fused deposition modeling or fused filament fabrication (FFF) is described in [17] and [18]. In the former, images were captured under different angles by a camera rotating around the part; the detected surface defects were then classified using a convolutional neural network algorithm. The latter used a 3-D laser scanner and applied different algorithms to identify defects by comparing the point cloud of the digitized surface with the ideal surface extracted from the original CAD model. A further study of the detection of surface defects in parts produced by FFF is presented in [19]. In this case, stereoscopic images of the deposited layers were captured by two cameras, and the Z differences between the digitized point cloud and the CAD model point cloud were calculated to analyze the defects.
In [20], a digital video microscope was proposed for contour detection to infer layers in an FFF process. After the fabrication of each layer, the microscope captured sequential images along the contour of the deposited material. These images were then processed to extract the edges, which were in turn compared with the reference contour of the initial model. A positive and negative threshold with respect to the reference contour helped to define the acceptance area for the detected edges.
The studies referenced the above-mentioned propose the use of different types of sensors to carry out in-situ monitoring and control tasks. The efficiency of these sensors is conditioned by factors such as the optical behavior of the digitized material, illumination, distortion of the captured images, or limitation of the digitized area, requiring repeated repositioning of the sensor to digitize a complete layer. A number of authors have, therefore, studied the capability of contact image sensors (CIS), commonly used in 2-D commercial office scanners, for digitizing layers deposited on AM machines, despite this type of sensors does not reach the image quality of CCD-type sensors. The study presented in [21] demonstrated the capacity of this technology to digitize the material layers deposited in an AM process. By taking advantage of the very shallow depth of field (DOF) of the CIS and using a focus measure operator algorithm, the authors were able to distinguish the areas of the captured image that were above or below the focal plane (out-of-focus areas) and considered defective. Here the sensor was only partially integrated since the hardware and software used were those of the commercial scanner.
A number of other studies did not integrate a CIS in the AM machine, but showed the ability of these sensors to capture images on parts manufactured by FFF. For example, in [22], scanning distortion and abnormalities concerning commercial flatbed scanners were evaluated and discussed. Two different scanner adjustment models were considered and tested by means of a linear-array certified dot artifact: one based on compensation of global image deformation, and the other on compensation of local distortions. In [23], the authors developed a calibration method for a flatbed scanner to correct position and straightness distortions. They then used the scanner to detect errors in 3-D printed parts to compensate the printer's G-code and improve the quality of the parts manufactured thereafter. Other studies demonstrated the capability of flatbed scanners for use in geometric measurement applications. For example, in [24]- [26], the authors developed methods of calibrating flatbed scanners to minimize errors generated during the digitizing process and enable accurate measurements to be performed on the captured images.
All these studies demonstrate that digitization with flatbed scanners is sufficiently accurate to enable the measurements performed on the captured images to be used to improve the geometric quality of manufactured parts. Other advantages of CIS over alternative digitizing systems, such as triangulation systems or cameras, include the following: 1) the sensor length enables a larger area to be digitized, minimizing repositioning time and errors; 2) no external illumination is required since the sensor is equipped with an internal light source; 3) the telecentric lens used by the CIS sensor minimizes distortion; 4) the resulting image is not affected by the optical properties of the digitized material; 5) the short stand-off distance allows the sensor to be installed in confined spaces; 6) ease of installation and low cost of the sensor. These characteristics suggest that this type of sensor could be used in AM machines to analyze the parts produced and improve their quality. The present study, therefore, proposes the use of a CIS sensor for the digitization of layers of deposited material. The sensor was integrated in a tailor-made test bench with the dual capability of 3-D printing by FFF and digitizing of deposited layers. The procedure for integration of the sensor includes analysis of its operating principle, development of the necessary hardware and software to be controlled from a PC, as well as processing of the captured images. Finally, a validation of the system is performed based on assessing whether the images captured by the sensor have quality enough to appreciate errors similarly to a coordinate measuring machine (CMM). Only the integration procedure is described in this article; contour recognition, geometric error detection, and compensation will be addressed in future work.

II. TEST BENCH DESCRIPTION
The test bench in which the CIS sensor was integrated consists of a general aluminum frame and two main functional subsystems: one for FFF-based 3-D printing, and the other for noncontact inspection of the layers deposited by the former. The printing head is mounted on a mobile bridge with a mobile carriage to enable the tracing of 2-D paths for the deposition of each material layer. A second movable bridge and carriage (inspection carriage) can be used to move the inspection sensor to digitize the layers (see Fig. 1). Different noncontact sensors will be tested on the test bench in the future, but this article focuses exclusively on the integration and use of a CIS-type sensor, which is mounted on the inspection carriage by means of an adapter that enables the height and orientation of the sensor to be adjusted (see Fig. 1).  The test bench is equipped with independent control boards for printing and inspection subsystems, in addition to a highlevel control application running on an external PC. This application allows both subsystems to operate in a coordinated manner by alternating the deposition and digitizing of the material layers. The top-level control application also handles the operation of the inspection sensor by interacting with its specific control routine (e.g., the CIS sensor routine). The integrated control system of the test bench is shown in Fig. 2.
An MKS RUMBA motherboard with an ATmega16U2 processor with Marlin firmware was chosen as the printer controller, while the control system chosen for the inspection head was an Arduino Mega 2560, which includes an ATmega2560 processor. The top-level control software running on an external PC was developed in Qt and is connected via serial ports to the printing and inspection system control boards. Apart from manual operation of each subsystem, the top-level control software allows the operator to import a G-code file, generated by any slicing software (e.g., Slic3r or Cura), and easily modify the code by inserting stop instructions between the layers to be inspected. When running the modified G-code file, the construction process stops at each of these layers, and the inspection head digitizes the last deposited layer while the printing head is held in a safety position. Once the layer is scanned, the inspection head is shifted to a safe position, and construction of the part continues with the printing of the next layer. The process is repeated until the part is complete.

III. SCANNER DESCRIPTION
This article used a low-cost commercial office scanner (Epson Perfection V39) operated by means of CIS technology. This technology is prevalent in office scanners, mainly due to its smaller size and better performance-price ratio compared to other technologies (e.g., CCD sensors).
In addition to its mechanical components (e.g., housing, sensor carriage, and paper digitizing glass), the selected scanner consists of a control board, CIS-type sensor, and control panel.
Before the CIS can be integrated in the test bench, it is necessary to determine the operating protocol of the scanner and the way in which the sensor interacts with the rest of the components.
The control board is responsible for coordinating all the elements that make up the system. When the scanner is connected by USB to a computer, a routine checks the system status and brings the sensor to the machine's home position. First, the user sets the parameters on the computer to capture an image. Next, the carriage on which the sensor is installed moves along the image. Finally, the image is captured line by line, according to the system resolution, which is defined by the distance between the captured lines and the number of points in each line. The distance between the captured lines is set by the user, but the number of points on each line is limited by the characteristics of the sensor.

IV. REVERSE ENGINEERING APPLIED TO THE SCANNER
To analyze the operation of the scanner during the digitizing process, the electrical signals were monitored using a Tektronix TDS 1001B oscilloscope coupled to the transmission cable via a 13-pin connector between the CIS sensor and the control board.
Pins 6, 8, 11, and 13 were identified as grounded (GND) and used as the reference point from which the remaining electrical signals would be measured. Pins 1 and 9 were identified as corresponding to the 5 and 3.3 V dc power supply, respectively.
According to its operating principle, in common with other similar scanners, the sensor was found to be equipped with an RGB LED light source. It was thus possible to assume the presence of three pins, one for each color, and pins 4, 3, and 2 were identified with the colors R, G, and B, respectively.
Taking GND as the reference, the oscilloscope also enables the clock signal (CLK) connected to pin 7 to be displayed and identified. This is a sinusoidal signal with a frequency of 3 MHz and an amplitude of between 0 and 3.3 V [see Fig. 3(a)]. This signal remains active throughout the process, irrespective of whether or not the sensor receives any information.
Pin 5 was connected to the trigger signal used to begin the capture of each line [see Fig. 3(b)] over the image being digitized. The trigger signal is a square pulsed signal, the width of which directly influences the image scanning resolution.
Having identified the pins indicated earlier, the remaining pins (10 and 12) should relate to the captured image. Using the oscilloscope set to a time scale of the clock period, a periodic response with fluctuating amplitude was found on both pins [see Fig. 3(c)], which behaved in a similar way. After analyzing the changes the signal undergoes in response to a change in the incident light on the sensor (e.g., caused by obstacles, applying additional light, etc.), it was concluded that each of the two pins provided an analog signal corresponding to one half of the sensor. In other words, the image captured by the scanner is simultaneously collected in two halves, as if by two sensors arranged one after the other (see Fig. 4).
After a simultaneous study of the set of signals, it was concluded that their operation occurs as shown in Fig. 5. The control board emits a trigger pulse each time a line is about to be captured. This is done by keeping the clock signal (CLK) active. The sensor responds with two analog signals synchronized with the clock frequency (AN_1 and AN_2) until all the pixels of the scanned line have been provided. Each line is captured three times, once for each color (R, G, B) in sequence. The process of conversion to a greyscale image is given by the weighted combination of each color component at each pixel.
It was also observed that the basic resolution image (600 dpi) provided by the sensor is repeated a number of times, depending on the desired resolution, as shown in Table I. Thus, the   construction of higher resolution images is achieved by combining and further processing of several basic resolution images.
Once the identification process has been completed, Table II lists the signals involved in the sensor's control and operation.

V. DEVELOPMENT OF THE CIS SENSOR CONTROL BOARD
The development of a specific control board that includes a microcontroller for signal processing and sensor handling is required for the integration of the sensor in the test bench.

A. Selection of Electronic Components
Sampling rate is the main parameter for selecting the microcontroller, which must match the speed at which the sensor provides the analog values for each pixel.
A PROMAX GF-232 function generator was used to determine the working frequency range of the scanner: different frequency clock signals were sent from the generator to the sensor, while a trigger signal was activated at regular intervals. An operating frequency range of between 1.1 and 3 MHz was concluded as a result.
Taking this information into account, a Teensy 4.0 microcontroller, capable of generating clock signals with frequencies compatible with the scanner's sensor, was chosen. Despite its apparent compatibility, this microcontroller includes an integrated A/D converter with a maximum sampling rate of 1 MS/s, which is insufficient for the present application. After comparing various microcontrollers, the final decision was to use an external

8-bit A/D converter with parallel interface (model AD7822)
, which was capable of providing a sampling rate of 2 MS/s. In fact, two A/D converters were required, since the sensor distributes the captured image through two channels simultaneously.
The remaining electronic components of the control board mainly consist of resistors for the sensor's RGB LED, and connectors for the wiring to the sensor.

B. Control Board Design
The control board is driven by an external PC connected directly to the microcontroller through a USB serial port, while the CIS is plugged into the control board via a 13-pin connector.
The connections between the microcontroller and the A/D converters were arranged to enable parallel reading of the 8 b of each converter. In this way, the readout speed was maximized to 50 ns per conversion. This was fast enough not to interfere with the main digitizing program, which has a clock period of 500 ns. Table III shows the pins used for the connections, and Fig. 6 the main components of the control board.

VI. DIGITIZING CONTROL ROUTINES
As shown in Fig. 2, the operation of the digitizing process with the CIS sensor is conducted from the top-level control application that interacts with the so-called "CIS sensor routine." In addition, this routine governs the operation of the microcontroller routine on the CIS control board. Both routines are described in detail in the following subsections.

A. CIS Routine
The CIS routine is responsible for configuring the sensor and moving it from an initial position, and through a series of  intervals defined by the image resolution before arriving at a final position. At each of these intervals, the routine activates the CIS to digitize a line. The set of lines captured by the sensor forms a point cloud that is stored in a CSV file and later processed to generate an image. Fig. 7 shows a flowchart of the routine.
By means of a desktop graphical user interface (GUI), the routine allows the user to send commands to the microcontroller of the CIS control board. These commands are considered as input parameters for the microcontroller routine, which will perform different actions accordingly. The following commands are possible: 1) "s": to start digitizing a line; 2) "r," "g," "b," or "w": to switch ON and configure the color of the sensor LED to red, green, blue, or white, respectively; and 3) "o": to switch OFF the LED.

B. Microcontroller Routine
The microcontroller routine is responsible for operating the CIS according to the commands set by the user in the CIS routine. If the command is "r," "g," "b," or "w," the microcontroller adjusts the light color of the sensor LED; if the command is "o," the light source is switched OFF. The process execution then returns to the CIS routine. If the command is "s," however, the microcontroller activates the CIS to digitize a line. Once the line is captured, the process is returned to the CIS routine. Fig. 8 shows the flowchart of the microcontroller routine.
The process of digitizing a line consists of capturing all the pixels at the set resolution. To do so, the routine allocates a pixel storage array of the same size as that of the line to be captured. The clock signal is enabled for both the CIS and the A/D converters to obtain accurately synchronized operation. A trigger signal is then sent to the sensor, the width of which will correspond to the set resolution for the image. As the falling edge of the trigger signal is reached, the microcontroller begins to collect the information, pixel by pixel. The pixels are stored in a 16-bit value, where the lower 8 b correspond to one of the converters and the upper 8 b to the other. Once all the pixels are collected, the clock signal is deactivated and the pixel array is returned to the CIS routine.

VII. IMAGE PROCESSING
As described earlier, each image captured by the CIS is stored line by line in a CSV file. Each line contains a 16-bit pixel array derived from two analog signals. By identifying the 8 b associated with each of the signals, it is possible to compose the two halves that generate the complete image (see Fig. 4).
Meanwhile, according to the resolution selected at the beginning of the digitizing process, the sensor captures one or more images that must then be combined to obtain the desired final resolution image (see Table I). In this study, a column-based image interlacing algorithm was used. The example in Fig. 9(a) shows a scan resolution set to 2400 dpi, with four images of 600 dpi obtained along the sensor reading direction and 2400 dpi along the scanning direction. By combining these images sequentially, column by column, the resolution increases to 2400 dpi along the reading direction, while remaining constant along the scanning direction [see Fig. 9(b)].

A. Examples of Captured Images
Two specimen parts manufactured in the test bench were subsequently digitized using the integrated CIS. A blue polylactic acid (PLA) material was used for the parts, whose dimensions are shown in Fig. 10.
The digitizing process was first carried out to obtain a 600 dpi image. In this case, the sensor captured a single image [see Fig. 11(a)]. The specimen parts were then digitized again with a resolution set to 2400 dpi. This time, the sensor took four images of 600 dpi along the sensor reading direction and 2400 dpi along the scanning direction [see Fig. 11(b)]. After applying the interlacing algorithm, these images were combined into a final 2400 × 2400 dpi image [see Fig. 11(c)].

B. Validation of the Digitized Images
To validate the reliability of the digitized images captured by CIS, the measurements obtained were compared with measurements performed using a reference measuring system. Reference measurements were obtained by continuous probing of the outer and inner contours with a CMM. The inverted angle design (10°) of the specimen parts ensured that the points obtained corresponded to the upper layer. These points were analyzed using Geomagic software by first adjusting linear and circular elements, then calculating the characteristic dimensions of the contours (see Fig. 10).
ImageJ software was used to analyze the images obtained with the CIS. Having configured the image resolution, a set of points on the contours were selected, exported to a point cloud, and processed in Geomagic in a similar manner to the points obtained with the CMM.
The measurements obtained for the parts from the CMM and from the digitized images, and the deviations between the two measurement methods, are shown in Table IV.
The results indicate that the deviations were within ±50 µm in all cases, and did not exceed 0.6% of the reference measurements. The precision of the measurement is, therefore, one magnitude order lower than the expected dimensional accuracy for FFF parts, typically measured in tenths of millimeters. Although a more rigorous procedure needs to be performed to determine the degree of measurement error, it can nevertheless be stated

VIII. CONCLUSION
This article proposes the use of a CIS sensor for in-situ digitizing of material layers deposited in a FFF machine. Compared with other technologies, this type of sensor has a number of advantages, including its ability to digitize large areas in a single pass, the fact that the sensor's internal light source can be adapted to the optical characteristics of the digitized material, ease of installation, and low cost.
Given that the CIS used was extracted from a low-cost commercial flatbed scanner, its control system (hardware and software) had to be developed from scratch. It was integrated in a tailor-made test bench with the dual capability of FFF and digitizing of deposited layers. A reverse-engineering process was initially carried out to identify the electrical signals transmitted between the sensor and the control board of the original scanner during a digitizing operation. A specific control board was then developed, using a microcontroller and two A/D converters to manage the sensor efficiently. Control routines were also developed to control the digitizing process from a PC in coordination with the overall control system of the test bench. In addition, a column-based image interlacing algorithm was implemented to process the digitized images to obtain resolutions from 600 to 4800 dpi.
Two specimen parts were built and digitized on the test bench to validate the reliability of the integrated CIS sensor. A comparison of measurements obtained by CMM on the one hand, or from images captured by CIS on the other, indicated low discrepancies. Although further analysis will need to be performed, these results may be acceptable for an initial use of the CIS sensor for in-situ geometric analysis of FFF layers.
In view of these results, further research will focus on in-situ detection and compensation of layer contour errors to improve the geometric precision of FFF parts. Although it involves higher cost, volume, and weight, the use of an industrial-type CIS sensor will also be considered, as it would simplify the integration process and would enable higher capture rates and higher native resolution.