2.1

Three-dimensional Optical Inspection

Laser triangulation^1,2 typically uses a line laser projection and triangulation to assign a height value to every observed pixel. The usage of line laser enables the system to cover one lateral dimension of the PCB with one observation. However, when inspecting only a portion of the PCB, most of the data is unused in the following classification. The main drawback of this approach is the overall slow inspection speed when using high lateral resolutions, and the observed interferences on shiny surfaces.

Phase-shift profilometry^1,3,4 seems to be the most widely spread method in the field of SMD assembled PCBs and solder-paste inspection. All well known AOI suppliers have at least one such system in their portfolio. An angled light source brings structured light onto the surface of the PCB. The geometry of the components and pins phase-modulates the structured light pattern in the observation. Based on this phase-modulation, the height of the object can be calculated. Like laser triangulation, this method reaches its limits with highly reflective surfaces.

Binocular stereo, as described in [Luhmann],¹ uses a calibrated pair of cameras to observe the object. The simultaneous observations then get processed into a disparity map, which can be triangulated into a depth map of the object using the calibrated poses of the cameras. The calculation of the disparity map relies on robust optical features, which can be identified in both observations. Such features are not present on unoxidized and high reflective solder joints. With the lack of features, binocular stereo and similar approaches are useless to inspect solder joints of THT components.

Chromatic confocal microscopy⁵ uses stacked layers of focal planes to reconstruct the three dimensional surface of the inspected object. However, in contrast to z-axis-scanning methods, this approach uses color coding through intentional chromatic aberration. Although its faster than laser confocal microscopy, it lacks the speed and measurement depth of other techniques.

Shape from shading^6–10 is a group of methods using a constrained illumination. The known position of the observing camera and illuminations is used to calculate the surface profile from shadows and reflections of the inspected object.

[Kim et al.]¹⁰ used three rings of LEDs to illuminate the surface of SMD solder joints. They extract two-dimensional features from the images taken of each illumination ring. If the classification based on the extracted features results in an ambiguous decision, additional three-dimensional features are calculated. This process is fine-tuned for SMD components and does not aim to give an overall three-dimensional visualization of the solder joints.

[Loh, Lu]¹¹ uses a sophisticated illumination setup to extract two-dimensional features of solder joints in an SMD process. Their main focus was to determine if the slant angle of the solder forms a convex or concave surface. As discussed in subsection 2.2, this cannot be used to discriminate good and bad solder joints in a THT-process.

[Kong]⁸ proposed a new model for shape from shading specialized on solder joints. The influence of highly specular reflections on the state of the art approaches is reduced to give a realistic impression of the inspected solder joints.

[Pang, Chu]⁷ uses an angled illumination setup to inspect dispensed solder paste. The segments of the illumination use different colors to extract the surface slant of the solder paste from a two-dimensional image. Different features are proposed to use the information from this approach.

[Vitoriano, Amaral]⁶ described a method to estimate heightmaps of SMD solder joints based on a shape from shading approach. Similarly to the other approaches, rings of LEDs are used to illuminate the solder joint. Three rings with three different color are used to capture the slant of the solder in one two-dimensional image. By using assumptions about the geometry of the solder joints, the height is summed up from the pad to the component. This approach only works in one lateral direction and ignores tin metal wetting on the sides of the component.

2.2

Optical Inspection of THT Solder Joints

The IPC standard “Acceptability of Electronic Assemblies”¹² is a worldwide accepted standard for quality inspection of manufactured electronics. Examples for solder joint anomalies described in the document are excess solder, missing solder, holes or cracks in the solder connection and deformations resulting from thermal processes. Depending on the product class, these anomalies can lead to a fail classification of the connection.

The IPC standard¹² describes a good solder connection as an even shiny appearance with a concave surface and an angle to the solder pad of fewer than 90 degrees. A dull surface is only acceptable if the soldering process is unable to avoid them, like slow heat dissipation or lead-free solder.

Specifically for THT solder joint inspection on the bottom side, the standard describes different quality metrics. This approach focuses on unbend pins, since they are the most common in automatic soldering processes, while bend pins are common with manual soldered connections. A quality through-hole solder connection should have a visible wetting of tin on the whole pad. The tin has to distribute around the pin evenly. Figure 1 illustrates an optimal concave through-hole solder joint.

Figure 1

(a) The illumination setup used in this approach. Shown are: (A) the camera and optics; (B) coaxial illumination; (C) ring illumination; (D) the PCB as detailed in figure 1b (b) THT solder joint. Shown are: (A) the pin; (B) the solder; (C) the metal plating of the pad and hole; (D) the non-conductive substrate of the PCB; (E) part of the soldered component.

When inspecting THT solder joints using automatic optical inspection systems, two criterions are essential. The first is the length of the pin, and the second is the quality of the electrical and mechanical connection of the pin and pad. The amount of tin contacting the pin cannot be measured or estimated using pure optical inspection. An x-ray inspection system can be used to measure the overall volume of the tin and check for voids in the hole. [Fonseka, Jayasinghe]¹³ demonstrated a comprehensive algorithm to inspect THT solder joints in a two-dimensional color image.

The pin height can give inside whether the components were placed correctly into the holes. Comparing pin heights with each other can be used to check components for coplanarity. Some manufacturers even want to inspect the correct trimming of the pins, since too long pins can contact the case of the finished product and lead to malfunctions.

3.1

Illumination

Figure 1 shows the illumination setup used in this method, consisting of four concentric rings of LEDs and one coaxial light source for the camera. The rings have individual distances to the camera. In this way, the overall appearance resembles a parabolic shape. The purpose of this setup is to illuminate the scene with discrete elevation angles. Every possible angle of the setup is observed in sequence by the camera. Since this method evaluates the reflections of the lights in shiny surfaces, the scene illumination does not have to be even.

3.2

Extraction and Transformation of Pixel Data

The first step of the reconstruction is the extraction of the relevant pixel data. Only the pixels showing the pin and solder joint are useful for the reconstruction. When ignoring bend pins, the only possible location of the pin going through the PCB is the location of the drilled hole. Similarly, the only possible location of soldered tin metal is the region of the pad. Both pad and hole, are defined through the CAD data for the inspected PCB. A window with the size of at least the diameter of the pad is used to extract the pixel data around the drilled hole. Figure 2 shows such extracted regions for all the illuminations described in subsection 3.1.

Figure 2:

Two different solder joints illuminated with led rings in different heights. The top row [2a to 2e] shows the effect of the illumination on a desoldered pin, while [2f to 2j] show a correctly soldered pin.

After extracting the regions, the algorithm transforms the images into a polar coordinate system using the rotational symmetry of the solder joints. By transforming the images, the surface angles of the sloped surface are all aligned along the same axis. This transformation breaks the following radial operations down into simpler operations along one image dimension. The transformation also affects the way the light sources are modeled in the following steps. The observed rings of LEDs now become axis-parallel strips of light sources. Since this method evaluates specular reflections, the light source model can be further simplified to use a single point light source along each polar angle.

Figure 3:

The effect of the polar transformation. The transformed images are shown rotated for better visualization. 3a and 3b show the polar transformation of a simulation image. The arrows indicate the increasing surface angles along the solder joint. In the transformed image, these arrows now are parallel. 3c and 3d demonstrate the polar transformation on the solder joint. Circular features now become linear and axis aligned.

3.3

Estimation of Surface Angles

The estimation algorithm relies on an illumination model capable of producing predictions of the observed scene depending on the surface angle, illumination parameters, and the observer position. For the experiments, the Phong illumination model¹⁴ was chosen, since it is robust and straightforward in its predictions. For each illumination angle in the list of observations, the model makes a prediction based on an assumed surface angle. The summed distance of the predictions and the observations overall illumination has to be minimized to find the surface angle matching the observation and prediction closely. This operation, as shown in the equation 1, is done for each pixel of the transformed images.

α is the assumed surface angle used to minimize the distance of the observations C_n and the predictions over all N images.

As the described operations run per pixel, the whole process can be sped up by using parallel processing. The result of this step is an image with an estimated surface angle per pixel in polar coordinates. The next step uses these angles to reconstruct a height map of the solder joint.

3.4

Reconstruction of Height and Retransformation

For each pixel the tangent of the estimated surface angle tan(α) gives a height difference Δh. As it was previously assumed that all surfaces are angled away from the pin, the actual height of each pixel can be calculated by summing up all the Δh from the outer perimeter of the pad R towards the pin. This is shown in equation 2.

After the calculation of the heights, the retransformation into the cartesian coordinate system gives a complete heightmap of the solder joint. The reconstructed heightmaps for the example solder joints are visualized in Figure 4. These heightmaps can be used to displace a mesh map for three-dimensional visualization, as shown in Subfigure 4c and 4d.

Figure 4:

The calculated height maps. The pins are added for better visualization of the complete solder joints. The height of the pins is reconstructed based on a different approach.