Design and characterisation of an additive manufacturing benchmarking artefact following a design-for-metrology approach

We present the design and characterisation of a high-speed sintering additive manufacturing benchmarking artefact following a design-for-metrology approach. In an important improvement over conventional approaches, the speci ﬁ cations and operating principles of the instruments that would be used to measure the manufactured artefact were taken into account during its design process. With the design-for-metrology methodology, we aim to improve and facilitate measurements on parts produced using additive manufacturing. The benchmarking artefact has a number of geometrical features, including sphericity, cylindricity, coaxiality and minimum feature size, all of which are measured using contact, optical and X-ray computed tomography co- ordinate measuring systems. The results highlight the di ﬀ erences between the measuring methods, and the need to establish a speci ﬁ cation standards and guidance for the dimensional assessment of additive manufacturing parts.


Introduction
Additive manufacturing (AM) is the common term for a range of manufacturing technologies that create parts using a layer-by-layer approach, as opposed to traditional subtractive or formative techniques [1]. The advantages of AM include the customisation of parts to meet bespoke requirements and the reduction of lead-time between the design and manufacturing stages. Additionally, AM enables enhanced design freedom, allowing the designer to specify geometries that may be impossible to produce using subtractive or formative techniques, and the reduction of waste at the manufacturing stage [2][3][4][5].
There is not yet an established consensus regarding AM design methodologies [5,6], although there are developments in the specification standards committees [7][8][9]. Despite this, the new opportunities that AM offers are clear [4]. AM can produce complex and customised geometries, consolidating assemblies and reducing the effect of many conventional manufacturing constraints [10]. As with all commercially manufactured parts, AM parts require quality control [11,12]. As such, metrology is a key tool for the AM industry, and will be essential in the application of new design features. However, metrology has only been superficially considered in previous design methodologies [13,14] and is often not considered at all at the design stage. To improve efficiency in implementing measurement for AM parts, new design methodologies are required, including support within AM CAD and optimisation software.
Here we define "design-for-metrology" (DM) as a methodology in which the user aims to optimise the part measurement and quality control processes without compromising the part functionality. DM also aims to facilitate fast and accurate measurements on parts produced using AM. For example, datum features could be designed into fabricated components, or extra geometries could be added to the design to better enable accurate measurement with a specific sensing technology. Through the use of DM, we want to ensure that the available measurement tools and their characteristics, limitations and operating procedures are taken into account at the component design stage.
Dimensional and geometric measurement presents a barrier to the further industrial adoption of AM, due to difficulties in adapting current measurement systems and techniques [12,15]. Quality control of AM parts commonly includes the measurement of freeform geometries, lattice or cellular structures, cavities and internal features, which challenge the currently available measurement systems [4,[16][17][18][19]. Measurements of these features can be expensive and time-consuming, and can increase the risk of damage to the fabricated part, for example, when an automated contact-based measurement system is used to assess an AM part with fine or delicate surface features.
The use of benchmarking artefacts to test AM systems has been the subject of previous research, which is reviewed elsewhere [20,21], but existing artefacts usually have complex and time-consuming measurement procedures. The National Institute of Standards and Technology test artefact, for example, was designed with lateral features not accessible by the contact coordinate measuring system probe intended for their measurement [22]. Other authors have tested AM systems with more traditional artefacts, such as the NAS 979 intended for CNC machines [23]. Previous benchmarking artefact developments do not usually provide target uncertainties for measured quantities. Some recent work has suggested simpler artefacts focussing on typical AM features [24], and a similar approach has been proposed in a draft ISO specification standard on the subject [25].
In this paper, we apply DM to the development of a benchmarking artefact for a high-speed sintering (HSS) AM process [26]. The artefact has been designed to provide meaningful data about the HSS system's fabrication capability. Data are acquired using several measurement techniques commonly employed for AM part measurement, including a contact coordinate measurement machine (CMM) [27], a close-range photogrammetry (PG) system [28] and an X-ray computed tomography (XCT) system [19,29]. The measurements have been designed according to recognised good practice guidance and associated uncertainties have been estimated [30][31][32][33].

Artefact production by high-speed sintering
Our objective with this work was to produce an AM artefact capable of testing the manufacturing capabilities of a HSS system and with DM considered as part of the design process. HSS is a powder bed fusion process in which a polymer powder is sintered using an infra-red lamp [26]. A layer of powder is spread over the built area, and an ink composed of a radiation absorbent material (RAM) is selectively jetted in areas determined from the CAD file. An infrared lamp heats the build area, providing enough energy to sinter the areas impregnated with the RAM ink, while leaving other areas unsintered (see Fig. 1). The powder bed is then lowered for a new layer of unsintered powder to be spread on top. The process is repeated until the 3D fabricated part is complete and can be removed from the unsintered powder bed. The powder bed is maintained just below sintering temperature throughout the process. The HSS system used in this work was a HP Fusion 3D. Note that support structures are not required with the HSS process used here, but would need to be considered if other AM processes were employed.

Measurement systems
The measurement systems used in this study were a Nikon MCT 225 XCT and a PG system (see [28] for details) at the University of Nottingham, and a Mitutoyo Crysta S Apex 776 contact CMM at the National Physical Laboratory (NPL) in University of Huddersfield.
Each XCT measurement requires a series of X-ray images to be recorded while the part is rotated through 360°. The XCT system uses a cone beam configuration and has a variable source to object distance; the resolution of the system will be dependent on the source size and geometric magnification [29]. XCT is also typically more time-consuming and expensive than other non-contact measurement methods, due to relatively long measurement times, resulting from the large number of projections required to perform measurements, each often lasting multiple seconds [29]. These projections are then reconstructed into a 3D representation of the part. The XCT system used here has a manufacturer-quoted maximum permissible error (MPE), of (9 + L/50) μm (where L is the test length in millimetres). MPE is defined as the limiting value of the difference between the measurand and the true value. In the case of XCT, this is specifically referred to in VDI/VDE 2630 as the measured distance between the centre of two spheres [31]. In this work, measurements were made in a temperature-controlled cabinet, set at (20 ± 0.1)°C.
PG was used due to its relatively high speed compared to the other two systems (CMM and XCT). The PG system at University of Nottingham consists of a camera mount and a rotating stage that allows images to be taken from different angles. The camera position and angle can be modified to produce the required image magnification at a range of elevation angles. The camera is a commercial Nikon D3300 with a 24 M P sensor. Using the same approach to MPE calculation as given for the XCT case above, and using relevant data from previous work [28], the PG system was determined to have an MPE of (28 + 1.66L) μm (where L is the test length in millimetres). This MPE is larger than that of high-specification CMMs, but PG is capable of measuring complex parts (such as those commonly produced using AM) that cannot be measured using CMMs due to the physical geometry of the probe. The typical surface texture of parts manufactured by the HSS system produces strong correspondence between PG images [36], allowing highdensity point clouds to be produced for geometrical analysis. Previous work showed that the PG system used is agnostic to the type of AM surface employed [36].
A contact CMM was used to provide a traceable dataset for comparison with the other two measurement systems. Contact CMMs tend to be slower than non-contact measurement systems, and they require a detailed measurement and probing strategy to be generated prior to the measurement. The contact CMM used had a maximum permissible measuring error E 0 = (1.7 + 3L/1000) μm (where L is the test length in millimetres) and a maximum permissible probing error P FTU =2.3 μm [37].

Artefact design
The AM benchmarking artefact was designed to meet the following requirements: • It should be manufacturable using the HP Fusion 3D HSS systemmaximum build volume 145 mm × 65 mm × 75 mm.
• It should test the HSS system's manufacturing capabilities with a range of geometrical features representative of typical AM designs.
• It should enable comparison of different measurements approaches both contact and non-contact.
• It should have dimensions small enough to allow full X-ray penetration for the chosen part material and avoid large plane features that can cause cone beam artefacts in XCT data [38].
• It should have direct line-of-sight access for PG measurement.
• It should be manufacturable in a relatively short timescalethis is not an absolute necessity from a metrology perspective but is included to promote uptake of the proposed artefact in the AM community.
• It should be stiff enough to avoid warping effects and, if necessary, support structures should be considered. Distances between cylinder axes or sphere centres are included in the benchmark artefact design, as they can be more accurately measured than plane-to-plane distances because they are not dependent of surface determination and provide a simple way to compare the different measured datasets. A test to understand the minimum feature size that the HSS machine can produce is included as well as a flatness test, in which the deviation of an extracted surface is compared against an ideal plane. The measurement of AM surface texture is known to be a complex issue; AM surfaces can contain overhangs, voids or un-sintered particles [39] as well as staircase effects due to the layer-by-layer fabrication process [40]. Flat surfaces with different build angles are, therefore, also included to assess texture change due to manufacturing angle. A roundness test feature (on pins and holes), plane-to-plane distance features (on several axes) and parallelism features are also included.
The final design of the artefact (see Fig. 2) is based on a hollow, four-walled cube with spheres at each vertex. The edge of the cube is 40 mm long, while the spheres have a radius of 5 mm. The sphere centres define a reference plane. Sphere centre-to-centre distances provide a low measurement uncertainty (relative to other measurements such as diameters or plane-to-plane distances) due to their independence on surface determination. Sphere-to-sphere distances can be used to compare the different measurement data. This design allows the rotation of the sample in order to take measurements with the available instruments.
One of the vertices is left without support, which enables measurement of the underside of the unsupported sphere. The face with walls and slots of different sizes (from 0.2 mm to 2 mm) is included as a pass/fail test to investigate the smallest features that the HSS system can produce. Due to its pass/fail characteristic, the walls and slots are not designed to be dimensionally measured. The other vertical face contains a sixteen-faceted hole that enables the evaluation of flatness, surface texture and angular distance produced by the system at different build angles. The top face contains at its centre a 20 mm square feature with 5 mm height and a hole of 5 mm radius. The bottom face contains at its centre a cylinder of 5 mm radius and 5 mm height.

CMMmeasurement and uncertainty estimation
The CMM selected for the measurements of the benchmarking artefacts had a 55 mm long, 1 mm diameter ball-tipped stylus. The stylus had a nominal probing deflection of 0.2 mm. The CMM measurements were performed in line with good practice presented in the National Physical Laboratory good practice guides numbers 29 and 130 [30,41], which are in turn based on the ISO standards 10360-1, 10360-2, 1101, Fig. 1. a) Schematic representation of the HSS system [26,34] and b) detailed ink deposition representation [35].  5459, 15530-3, 14253-1, 15530-4 and the ISO/IEC Guide 98-3 [37,[42][43][44][45][46][47][48]. While common industrial practice is to perform geometric dimensioning and tolerancing (GD&T) in benchmarking exercises, the aim of this work is not to test the HSS system, but to demonstrate the concept of design for metrology using HSS as a test case. As such, no GD &T protocols were performed as part of this study. In an industrial setting, however, it would be simple to apply GD&T to the artefact once a baseline for the system has been decided. The contact CMM is supplemented by a virtual (computer simulated) CMM, enabling task-specific measurement uncertainties to be estimated using a Monte Carlo approach [41]. The "virtual CMM" software used for this was PUNDIT CMM5. The use of a simulated CMM to estimate the uncertainties arises from the impracticability of using a material measure and the substitution method [49]. Task-specific uncertainty estimations for CMMs using the substitution method are not usually performed on complex and individual AM parts due to the difficulty and expense of developing a calibrated artefact to compare with the workpiece. Determination of measurement uncertainty using a virtual CMM is carried out by performing repeated simulated measurements with varying inputs (influence factors) on a simulated CMM and determining how those inputs affect the physical measurements of the measurand. The software determines the variability that will occur within the physical CMM measurements by modelling the uncertainty contributions for each point probed by the CMM. Known systematic uncertainty contributions remain constant, while unknown systematic and random contributions are varied in each simulated measurement throughout their ranges. This simulation is repeated a significant number of times until a statistical evaluation of these virtual measurements is made, and the expanded uncertainty that corresponds to the physical measurements is then reported. The following inputs were assigned in the virtual CMM software: the temperature of the laboratory where the measurements were taken (20 ± 1)°C and the linear coefficient of thermal expansion of the material (Nylon-12, α= (80 ± 1) × 10 −6 K -1 ). The peak to valley deviation of the measured surface from the nominal is an essential input for the uncertainty calculations. Here, the ISO 4287 [50] Rz parameter is used to define peak to valley deviation. Previous work by Sachdeva et al. showed that Rz values for similar surfaces lie between 31 μm and 75 μm for selectively laser sintered polyamide [51]. The up-facing surface texture of our HSS printed artefact was measured with a Talysurf Form Intra 50 contact stylus instrument equipped with a probe of radius 2 μm providing a value of (50 ± 3) μm (where this includes an expanded uncertainty at k = 2). In addition to the 50 μm Rz value, 25 μm and 75 μm were also input as possible Rz values in the software. Various values were used to gain a good understanding of the effect of surfaces texture on the magnitude of the CMM measurement uncertainties and because the surface texture will vary significantly with build angle.

X-ray computed tomography
There is a lack of standardisation regarding both measurement procedures and uncertainty analysis for XCT dimensional measurements. This lack of standardisation challenges the ability to create an uncertainty budget based on a quantification of various measurement influence factors. Therefore, in order to avoid underestimating the uncertainty, and possibly overestimating it (due the mentioned lack of other established methods), measurements from the XCT system are provided with an uncertainty analysis comprising the addition in quadrature of the MPE and the statistical experimental uncertainty contributors (which are heavily dependent on the user) [31,41,48]. It should be noted that this MPE is not all inclusive, as it is only representative of one type of measurement; sphere to sphere distances, and compromises only the manufacturers' non-traceable confidence in the measurement. This equation is expressed as: Where, u MPE:XCT is the uncertainty contribution related to the MPE (with a rectangular distribution), which represents the error associated with an individual measurement; u SD:XCT is the standard uncertainty contribution of the measurement procedure carried out on the artefact, which represents the error associated with the statistical variation of repeated measurements; and u FF:XCT is the standard uncertainty contribution of fitting the different features to the point cloud. Also, σ is the standard deviation and n the number of repetitions. Finally, the expanded uncertainty U XCT is expressed as Where, k is the coverage factor (k = 2 is the value providing a level of confidence of approximately 95 %). Measurements were performed at a geometric magnification of 4.7×, leading to a voxel size of 43 μm. The following parameters were used for XCT measurement: 3142 projections formed by averaging two frames per projection, with each frame lasting 2 s and using a detector gain of 24 dB; X-ray tube voltage was 110 kV and current was 318 μA. A detector shading correction was applied by averaging 256 reference frames (128 bright and 128 dark) and a warmup scan of approximately one hour was performed prior to a sequential batch of three repeated scans. A 0.75 mm copper pre-filter was used between the X-ray source and the artefact to attenuate lower energy X-rays. The test object was placed at an angle to reduce artefacts on the top/bottom planes. X-ray imaging and volumetric reconstruction were performed using the manufacturer's software (X-Inspect and CT-Pro, respectively). Reconstruction was performed using the manufacturer's implementation of the FDK algorithm [52] with a first-order beam hardening correction and a ramp noise filter with a cut-off at the maximum spatial frequency. Following reconstruction, XCT data were imported into Volume Graphics VGStudioMAX 3.0 [53] and surfaces were determined using the local maximum gradient algorithm over a search distance of four voxels, beginning from the ISO 50 % isosurface [54].

Photogrammetry
For the PG, images of the artefact were taken from different angles with the aid of a rotating table. Thirty images per face of the artefact were taken, spaced at 12°increments. Once the reconstructed form data was obtained, a scaling factor was applied to obtain the dimensional measurements. The scaling factor for the artefact point cloud was obtained using the distance between centres of spheres obtained with the CMM, due to its relatively low uncertainty of 0.006 mm. Results are provided with an uncertainty analysis comprising the addition in quadrature of the MPE, scaling, fitting and statistical experimental uncertainty contributors [28], expressed as Where, u MPE:PG is the uncertainty contribution related to the MPE (with a rectangular distribution), u Scaling:PG is the uncertainty contribution of the scaling (provided by the CMM measurements), u Mes:PG is the standard uncertainty contribution of the measurement procedure and u FF:PG is the standard uncertainty contribution of fitting the different features to the point cloud, with σ being the standard deviation and n the number of repetitions. Finally, the expanded uncertainty U PG is expressed as Where, k is the coverage factor (k = 2 is the value providing a level of confidence of approximately 95 %) (Fig. 3).

Results
CMM uncertainties were estimated using a virtual model (see section 2.4.1), and uncertainties were calculated for three different Rz values: the Rz of the artefact and two other values based on previously reported values for polymer sintered parts [51]. The distance uncertainties on all three directions (x, y and z) were the same on the virtual CMM calculations. For the XCT and the PG systems, their unidirectional characteristics, made axis distinction irrelevant. Surface texture was considered of particular interest due to its significance on AM parts, but unfortunately, the virtual model used for the CMM uncertainty calculations did not apply to the XCT and PG measurements. CMM measurement uncertainties for different features on the artefact are reported in Table 1 for the different Rz values. The uncertainty values in Table 1 for angular distance were calculated using the faceted hole, while the flatness values were calculated using the top face (see Fig. 2).
In Fig. 4, diameter measurements of the AM artefact spheres, cylinder and hole are shown. Table 2 shows the linear distances between sphere centres and between planes and Fig. 5 shows deviations of the different measurement systems with respect to the CAD model.

Discussion
Uncertainties for CMM measurements (given in Table 1) are dependent upon surface texture. XCT and PG systems are generally considered to have lower precision than CMMs, but when the rough surface texture of AM surfaces is taken into consideration, XCT and PG systems can potentially provide measurements with similar uncertainties to those made using the CMM. However, it should be noted that if parts were post-processed to obtain smoother surfaces (as reported elsewhere [55]), then the CMM would gain an advantage in terms of lower uncertainty.
Concerning the external diameters of the AM artefact spheres, larger values were obtained from the CMM than from the XCT and PG systems, while the opposite was true for the internal hole diameter (see Fig. 4). The discrepancy between values generated from CMM and XCT data is caused by the rough surface texture of the part [56] and a fundamental difference in the ways that these systems collect data. That is, because of the use of a relatively large physical probe, the CMM can only access the highest external points of the artefact surface, which results in a substantial mechanical filtering effect [56]. This effect causes, in turn, the reconstructed diameter of cylinders and spheres to be larger than those obtained using XCT and, to a slightly lesser extent, PG. This difference is because, while the CMM uses a large mechanical probe, the XCT and PG systems are non-contact and so can capture surface data in a manner that is less constrained by mechanical filtering effects. PG has a slight filtering effect in comparison to XCT as, while the measurement is non-contact, the 'probing' performed by the PG system is still essentially unidirectional in nature and made from above the surface. Some filtering then is likely still present in PG as a result of optical effects in small, deep surface features. By contrast, the XCT system 'probes' in an omnidirectional manner and so is not similarly limited. The data presented here follow the systematic difference between XCT and contact CMM presented elsewhere [57], where the measured external diameter of a cylinder on an XCT system is smaller than the CMM measurement by approximately Rz/2, while internal diameters are larger than CMM values by Rz/2.
Linear distances between sphere centres (see Table 2) provide more robust information that can be used to understand the dimensional accuracy of the HSS system. This dimensional precision can be investigated because of the relatively low uncertainties of these measurements, compared, for example, to the plane-to-plane distances (one order of magnitude less in the case of the CMM due the low contributions of mechanical filtering effects). The artefact shows a slight shrinkage in the horizontal plane and an expansion along the z-axis compared to the nominal dimensions. The shrinkage in the horizontal plane is approximately equal in the x and y directions, so is unlikely directly related to the RAM ink deposition process, or the travel of the printer head along the x direction. Expansion along the vertical direction could be caused by warping due to thermal effects, or errors on the displacement of the build platform that affect the layer thickness.
The artefact plane-to-plane distances (see Table 2) show a larger deviation from the CAD nominal values than the sphere-to-sphere distances. However, because the plane-to-plane measurements depend on the position of the surface, and this position is different across the measurement systems due to the effects of the rough surface texture, the different measurements should not be directly compared. Expanded uncertainties for these measurements are higher than the MPE over the measurand, due to the uncertainty contribution of the relatively rough surfaces. This difference between expanded uncertainties and MPE is higher on the CMM system.
The deviation maps on Fig. 5 show the differences in the number of points acquired between CMM and the non-contact measurement systems. Characteristics of the part, such as the lip on the hole border, are not visible with the CMM. Due to the difficulty of keeping large areas on  focus with the PG system, areas further away from the centre of the artefact show more noise than central areas. Also, the PG system has difficulties in acquiring points on the internal faces of the artefact due to the limited depth of focus of the system. As such, these points are absent in the data. Finally, regarding the practicality of the measurement processes employed here, the processing times of the two non-contact measurement systems are similar, but the acquisition time for PG images is much lower than for XCT (a few minutes versus a few hours for the full set of images in each case). The contact CMM requires less post-processing time (if we exclude uncertainty modelling), but programming the measurement strategy can be so time-consuming that it is often the slowest of the three systems. Once programmed, however, the CMM is faster in subsequent measurements. CMMs provide a straightforward set of results through the acquisition of a small number of points. Noncontact systems provide more dense point clouds that could provide more information about the measured parts. However, data post-processing would be needed in order to extract that extra information.

Conclusions
Many benchmarking artefacts have been produced with the aim of testing different AM system capabilities and to evaluate the geometric properties of parts produced. However, the design of previous benchmarking artefacts does not take into consideration the difficulties and complexities of the measurements required to correctly characterise them. Here, we use a design methodology that considers the measurement process at the artefact design stage. The resulting artefact is tailored not only for the AM system (a HSS system), but also for the measurement systems used (contact CMM, XCT and PG). Measurement results provide information about the performance of the HSS system and also demonstrate the effectiveness of the benchmarking artefact design, which validates the design approach. Results show a high dependence of CMM measurement uncertainties on surface texture. These uncertainty values are relatively large in comparison to other traditional manufacturing processes (e.g. machining). High uncertainties greatly complicate the use of AM in precision engineering applications. To reduce the measurement uncertainties, improvements either in the measurements or in the printing and finishing processes are needed. Spherical features, present in the artefact, allow for effective comparison between different measurements systems and, by using sphere-tosphere distances, provide lower uncertainties than other features, due to lesser effects from rough surface texture on such measurements. These spherical features could be added to the part design to achieve a more accurate dimensional assessment and could potentially be removed later. Results of the uncertainty calculations shown that the XCT Fig. 4. Diameters of features of the artefact measured using different measurement systems.

Table 2
Results for the plane-to-plane and sphere-to-sphere distance from the different measuring instruments. system can achieve lower uncertainties than the CMM when measuring features other than sphere-to-sphere distances, although the manufacturers MPE statement has been taken at face value in this work and further investigation in measurement uncertainty estimation for XCT is required. The contribution of the MPE to the uncertainty of the PG system means it has the highest uncertainty of the three systems used. However, lack of standards for both PG and XCT (i.e. surface determination settings) means that the use of such systems in industrial applications should be taken with caution. For the geometrical control of AM parts, we recommend the addition of features that allow centreto-centre measurements at the design stage, as such measurements result in higher precision than their plane-to-plane counterparts. Results show that CMMs may not be the most effective systems for reference measurements of AM parts, as non-contact systems (i.e. the PG and XCT systems showed here, or other non-contact technologies like structured light measurement systems), once appropriate traceability is in place, may be more appropriate due to the complex features and rough surface textures that AM parts can present. Care should be taken on the measurement of AM parts with rough surface texture, as the choice of measurement system will influence the measurement result and the accuracy of the measurement obtained.