µ DIC: An open-source toolkit for digital image correlation

We here present a Digital Image Correlation toolkit, formulated as a Python package. This package aims at providing a complete toolkit for performing DIC analysis on experimental data, performing virtual experiments, as well as a framework for further development. A suite of tools for generating synthetic speckle images, modelling of sensor artefacts and deformation of images by displacement fields, are included. The virtual experiments are used as a part of the accuracy assessment of the toolkit as well as for testing during development. B-spline elements are employed for the discretisation of the displacement fields and allow the polynomial order and degree of continuity to be controlled by the user.


Motivation and significance
Digital Image Correlation (DIC) has become a popular tool for quantification of surface deformations, due to its ease of use and ability to provide local deformation measurements even for inhomogeneous deformation fields [1,2]. DIC is performed by processing a series of images captured during the deformation of the object of interest, and a mapping between the image coordinates of an image of the undeformed (reference) object and the image coordinates of an image of the deformed object is calculated. This operation is often facilitated by applying a speckle-like pattern on the surface of the specimen. In order to obtain such a mapping, an optimisation scheme is employed, searching for the mapping which gives the best correlation between a reference image and the current image. The coordinate mapping is in turn is used to calculate the strain fields. * Corresponding author.
There are several variants of DIC, the most common being subsets-based DIC (local DIC) and Finite-Element-based DIC (global DIC). Subset-based DIC attempts to correlate independent subsets of the image, often being in the order of ten to thirty pixels in each direction. In order to cover a large portion of the specimen surface, a large number of subsets are used. FEbased DIC, on the other hand, discretises the deformation field using finite elements, approximating the inter-nodal coordinates by interpolation. FE-based DIC normally enforces displacement continuity, but by using higher order element formulations [12,13], B-splines [14] or NURBS [15], higher order continuity can be enforced. Interestingly, a performance comparison of subsetand FE-based DIC was done by Hild et al. [16] and later by Pan et al. [17] field of DIC include self-adaptive element order [18] and more computationally efficient formulations [19].
Commercial DIC software such as ISI-Sys VIC and GOMcorrelate, and open source DIC codes such as DICE [20] and nCorr [21] are now available. For scientific use, it can be appealing to use open source software, as insight into the internal architecture of the software is available. However, at the time of writing, the available open source DIC codes were either formulated in C++, calling for significant programming knowledge, bound to commercial software such as MATLAB, or formulated as scripts with very limited testing and documentation.
This project aims to provide a complete and easy to use toolkit for 2D digital image correlation, that is verified and provided with automated testing, implemented in an accessible programming language. Python was chosen for this task, aided by NumPy [22], SciPy [23] and Numba [24] for fast array operations. Visualisation is provided by Matplotlib [25].
Motivated by the improved accuracy and flexibility reported by Cheng et al. [14], B-splines are used to discretise the deformation field. B-splines allows for a very compact and flexible formulation where the user can specify the degree of the interpolation polynomials and the degree of continuity as most appropriate for their application. Linear and parabolic element formulations such as Q4 and Q8 are subsets of the B-splines and are therefore an integrated part of the package.
A modified Newton-Raphson iterative solver [12] is used to minimise the sum of squared differences between the reference and current image. During the optimisation process, the interpixel values of the image are obtained by interpolation using bi-cubic or bi-quintic splines.
An additional but important component is a toolkit for generating and deforming speckle images, including camera artefacts such as fill-factor and noise. This toolkit is based on the ideas presented by Orteu et al. [26]. The toolkit allows for assessment of element formulations, aliasing artefacts and filtering, and is an integral part of the provided suite of integration tests. Additional validation is provided by experimental data where other techniques have been used to measure deformations.
The user can implement this package in the daily workflow by adopting one of the provided example-scripts, where in most cases the only required alteration is the path to the folder containing the images to be analysed.
This toolkit was conceived as a Matlab-code during the PhD study of Dr Andersen [12] and has been used in several studies [10][11][12]. It was later rewritten in Python by the corresponding author and has been applied in [9,27].

Software description
The µDIC toolkit provides a collection of high-level functions and classes, used for importing images, generating synthetic input data, meshing, image correlation and post-processing. The API is constructed such that the functions and classes can be used as stand-alone components. Factories are included in all packages, providing default behaviour for the most common tasks. The toolkit can be used in the daily workflow when performing real experiments or used for exploring the impact of different factors such as element formulations, speckle design and filtering by performing virtual experiments. An interactive experience can be obtained by using IPython [28] or Jupyter notebooks. Testing is an integral part of this toolkit, and virtual experiments, simulating know deformation fields, are used as integration tests along with other unit tests.

Software architecture
The µDIC tookit is formulated as a set of packages: • IO: Tools for image import and export • VirtualLab: Tools for synthetic data generation • Mesher: Tools for mesh generation and manipulation • Correlator: The digital image correlation routines • PostProcessor: Tools for post-processing and visualisation The packages can be combined, providing a complete processing pipeline tailored to the user's needs. A typical pipeline and the most relevant package components are shown in Fig. 1 and run as follows: An ImageStack object is generated either by loading a set of images by using imagestack_from_folder or by constructing a VirtualExperiment. A Mesher is instantiated and a mesh object is obtained by passing the ImageStack to the mesh method. The resulting Mesh object and ImageStack is then stored in the DICInput object. The DICAnalysis factory verifies DICInput and a DICJob is constructed. The DIC analysis can now be initiated by calling the run method. The resulting DICRes object contains all relevant output from the DIC analysis, and field variables can be calculated by DICPost class and visualised by the plot function.

Software functionality
The core functionality of the main packages in the toolkit is presented in the following.

IO
The IO package provides all the tools needed for importing images and saving objects. The most typical tasks, such as making an image stack of all images in a folder with a given extension, are made easier by providing factories such as imagestack_from_ folder.

Virtual lab
The VirtualLab package provides tools for the generation of synthetic speckle images, deformation of speckle images using prescribed deformation fields and modelling of camera artefacts such as fill-factor and pixel eccentricity. The virtual lab toolkit is based on the work of Orteu et al. [26]. The main modules of the package and their intended usage are illustrated in Fig. 2 and described below: • The Speckle module contains tools for generating spraypaint-like speckles using a Perlin noise [29] based algorithm or the Rosta algorithm found in Appendix. Tools for generating speckles formed by circular dots or bi-harmonic fields are also included. The resolution, speckle size and degree of smoothing can be controlled by the user.
• The Deformer module contains tools for distorting the speckle images according to a prescribed displacement field or by a deformation gradient. Inversion of the displacement function is done by employing a Newton solver. Bi-quintic splines are used to interpolate the grey-scale values of the image.
• The Downsampler module contains tools for down-sampling of images, including sensor artefacts such as fill-factor and pixel eccentricity. The sensor is modelled as a grid where each pixel of the sensor covers a finite region of the image. The grey scale values of the image covered by a pixel of the sensor are then integrated, giving the grey scale value of the sensor pixel. Each pixel on the sensor can be offset by a random value picked from a Gaussian distribution with a prescribed standard deviation. The position of the super-sampled points covered by each pixel of the sensor is calculated, as shown in [26].  • The NoiseInjector module is used to introduce noise to the downsampled images. The user can define a noise model and the standard deviation of the noise distribution. An additive Gaussian noise model is provided with the toolkit, and the user can easily add other models.

Mesher
The Mesher package contains the tools for mesh generation and manipulation. B-splines are used to discretise the deformation field into a bi-directional grid of k x l control points. The formulation and algorithms are taken from [30]. The coordinates x(u, v) on the B-spline surface is given by: where N i,p and N j,q are the B-spline basis functions, u and v are the spline coordinates in the range [0, 1] and p and q denotes the polynomial order. m ij contains the coordinates of the control points along the bi-directional grid axes of the spline surface. By rearranging, Eq. (1) can be written on the form: where n is a vector contains all the control point positions and

A(u, v) is a matrix containing the values of the B-spline basis
functions.

Correlator
The Correlator package contains the Digital Image Correlation routines. A modified Newton scheme is used to minimise the sum of squared differences between the grey scale values of the reference image and the current image. The derivation of the correlation routine follows [31] except for the chosen correlation criterion.
The objective of the correlation routine is to find the current image coordinates for each pixel x such that: where I C (x) is the grey-scale intensities of the current image at the pixel coordinates x and I R (x 0 ) is the grey-scale intensities of the reference image at the initial pixel coordinates x 0 . In real world conditions, Eq. (3) cannot be satisfied directly, and it is therefore approximated by using the sum of squared differences (SSD) between the grey scale values of the current and reference image as correlation criterion. By using Eq. (2), the SSD correlation criterion can be written in terms of control point positions as: where n 0 denotes the initial control point position. The approximate solution of Eq. (3) is then found as: In order to find the control point positions n which satisfies Eq. (5), a Newton Raphson scheme is used: where i denotes the increment. The term ∇∇R(n i ) is simplified by assuming thatĪ C (n) ≈Ī R (n 0 ) as proposed by [31] and by using the grey scale gradients of the reference image instead of the current image, giving: Here, ∆Ī i is the difference in grey-scale value between the current and reference frame. The inter-pixel values ofĪ C (n i ) andĪ R (n 0 ) are determined by means of bi-cubic or bi-quintic spline interpolation, which has been demonstrated to minimise the bias error [32]. For details and discussion regarding the derivation of the correlation routine, see [31] and [12].

Post processing
Based on the B-spline surface x t (u, v) at frame t, the deformation gradient for each material point within the surface can be calculated as [33]: where the entries of F t are partial derivatives of the spatial coordinate component fields with respect to u and v. Based on the resulting deformation gradient F t , the right Cauchy-Green tensor is calculated: U t is found by calculating the square root of the eigenvalues of U 2 t as found by spectral decomposition. The Hencky strain tensor ε t (often referred to as true or logarithmic strain) is then determined as: Note that the eigenvectors of ε t and U 2 t are now coaxial. The Hencky strain tensor ε i is then rotated into the coordinate frame of the image, giving the Hencky strain components in image coordinates.

Example 1: Processing experimental data
Processing of experimental data is one of the routine tasks of a scientist. We here show an example where Digital Image Correlation is performed on images that were acquired during a tensile test of a flat dog-bone specimen made of Docol 600L. The output of the procedure is the longitudinal true strain field shown in Fig. 3.
The complete script used to produce these results is shown below:

Example 2: Performing a virtual experiment
When the user wants to evaluate the performance of the digital image correlation routine or assess the influence of parameters such as speckle design and element formulations, the VirtualLab module can be used. We here show an example where a speckle with a resolution of 16 Mpx is generated using the Rosta algorithm and deformed by a bi-harmonic displacement field. After deformation, the speckle is down-sampled by a factor of 8 and noise is added. DIC is then performed on the generated data set. The code for performing this is shown below: The x-component of the imposed displacement field and the deviation between the DIC results and the imposed field is shown in Fig. 4, demonstrating how the accuracy of the DIC routines can be assessed.

Impact
Digital Image Correlation has been widely adopted in several fields of science due to its practicality in real-world experimental setups as well as the amount of detail which can be obtained from the measurements. The current tool aims to provide an easy-to-use, transparent and powerful toolkit for researchers and industry. The toolkit can be applied to a broad range of research problems where quantification of surface deformations is of interest. The popularity of Python makes it an ideal language for such a tool, motivating adoption, modification and extension. By distributing the toolkit via the Python package manager along with the unit test, and providing the user with examples and documentation, the threshold for taking the toolkit into use is lowered. The authors would like to highlight the following:   • Evaluation of the internals such as the correlation routines, element formulations and interpolation routines, is here facilitated through the use of the VirtualLab module.
• Due to the use of the VirtualLab module in the integration tests, modification and extension of the toolkit is made robust, as the impact of the modification is easily evaluated.
• As the toolkit is made available as a Python package, all data processing can be done without changing to other systems. This motivates the adoption of the vast ecosystem of Python packages for tasks such as visualisation, optimisation and machine learning.
• Large sets of experimental data are easily processed by scripting, and example scripts for this task are provided along with the package.
• As the toolkit is distributed under the MIT license, commercial adoption of the package and derivatives of the package are facilitated.

Conclusions
We here present a toolkit for Digital Image Correlation and outline its capabilities. The toolkit includes tools for synthetic input generation, a key feature for validation and further development. B-spline discretisation of the deformation field is used, providing a flexible framework where the degree of the polynomials and the order continuity can be specified. Python is chosen for the implementation of the toolkit, making it accessible to a broader user group and motivating use and modification.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. σ p : the size of the proximity kernel; σ s : the size of the smoothing kernel; n: the number of repetitions t: fill fraction Output: I: speckle image make an empty image, I with size s x s ; make gaussian kernel k p with σ = σ p * s; make gaussian kernel k s with σ = σ s * s; for n layers do make random number field, f of size s; calculate g = f * k p ; scale g to unity ; find v such that a fraction of t pixels are found by g < v move values of g < v into I; end calculate I = I * k s ;