Signal Conditioning in Process of High Speed Imaging

The accuracy of cinematic analysis with camera system depends on frame rate of used camera. Specific case of cinematic analysis is in medical research focusing on microscopic objects moving with high frequencies (cilia of respiratory epithelium). The signal acquired by high speed video acquisition system has very amount of data. This paper describes hardware parts, signal condition and software, which is used for image acquiring thru digital camera, intelligent illumination dimming hardware control and ROI statistic creation. All software parts are realized as virtual instruments.


Introduction
Cilium in respiratory apparatus is beating with frequencies in range to 30 Hz.Following Shannon-Kotelnik theorem for sampling, we must use high speed imaging for proper frequency analysis of cilium motion.Generally each imaging with frame ratio bigger than 30 fps is called high speed imaging.Microscopic objects investigated by light microscopy cannot be equipped with standard cinematic sensors, so high speed camera with some powerful tools for signal processing "become non-contact test-device, analogous to oscilloscopes" [1].
The light microscopy is used in many research centers for structure and movement study of biological objects.In case of cilia movement study, there are some sophisticated and expensive methods based on fast and quality hardware components like high speed digital cameras, stroboscopes, photo multipliers and many others.These methods require long time training.
Defective mucociliary clearance is inevitably seen in the chronic lung diseases and contributes substantially to their pathologies, and increasing the ciliary motility with external stimuli is considered to be efficient to relieve it.Ciliary Beating Frequency (CBF) is a regulated measurement to evaluate the drugs by comparing their effects on the motility of cilia.Therefore, CBF estimation is crucial for the therapeutic assessment on the defective mucociliary clearance diseases, and digital high speed imaging (DHSI) analysis provides an effective tool to measure CBF [2].
Primary ciliary dyskinesia (PCD) implies cilia with dysmotility or total absence of motility, which may result in sinusitis, chronicbronchitis, bronchiectasis and male infertility.A large number of deficiencies detectable on the ultrastructural level give rise to PCD, but patients with normal cilia ultrastructure are common.An early diagnosis is very important since PCD can cause permanent lung damage.Diagnosis can be difficult and is based on an abnormal CBF and beat pattern, accompanied by specific abnormalities of the ciliary axoneme or normal ultrastructural cilia.Some present methods to determine CBF using the analysis of phase-contrast microscopy imaging acquired using digital high speed video techniques.Beat frequency measurements are made estimating cilia motion by means of an optical flow algorithm [3].
The primary cilium, a hair-like projection from the cellular membrane, is involved in fluid flow sensing.The primary cilium is modeled as a thin beam undergoing large deflection due to fluid drag forces.For the first time, the bending response is analyzed with a model combining large angle rotations with the assumption of a linear drag force along the ciliary length.The model is applied on published pictures of bent cilia.Incorporating a linear drag force and allowing for a basal tilt is an important step towards a more realistic mechanical model of the primary cilium and towards more accurate values of its flexural rigidity [4].Some methods for frequency measurement (using photodiode and photomultiplier) can't do the correct analysis of object structure pathologies.The most progressive method is high speed digital video method, which brings relatively good results in formation of mathematical and mechanical model of cilia movement (or other biological objects) [5].
Based on previously mentioned objectives the workstation was designed (Fig. 1) with light microscopy and high speed camera (Basler A504kc) for recording and analyzing main motion parameters of microscopically objects in respiratory apparatus.The first, exposition time for high speed camera is often too short and the intensity of illumination source inversely depends on sensor exposition time: where ∆T EXP is a time 1/FPS (frame rate) decreased by the time of reading the digital image from C-MOS sensor (non-integration time -dark time) (Fig. 2).
In the case of high speed imaging and light microscopy -suitable illumination and its parameters are key elements for generating and acquiring good images.

Hardware Resources
Main condition in illumination unit design for high speed imaging is to concentrate high optical flow to relative small area (part of specimen).This condition can be achieved by using LED module with collimator.Collimator helps to reduce spatial radiation angle of LED module (15 • , 30 • e.g.).LED or another illumination unit is placed into microscope condenser (Fig. 3), where the light cone is conditioned for optimal specimen lighting (intensity maximum is focused on object) [6].LED modules are called "cold light sources" for high efficiency of transforming electrical energy to light (∼90 %).This fact significantly reduces need for active cooling (against using of halogen bulbs) of condenser chamber.Some details of biological objects are visible in given spectral range (green light e.g.) and other details in another spectral range.The main advantage of LED modules is also flexibility for various wavelengths usage.These modules are made with various spectral ranges.Design of illumination unit is modular and one module can be simple replaced by another with different wavelengths.
LED modules can be used in two modes: continuous and impulse.Impulse mode enables short time diode overload during suitable duty cycle for illumination decreasing.LED diode in impulse mode can be used for fast optical flow regulation (this regulation will be detailed described).Stroboscopic measurements using LED microscope illumination unit is momentarily in the phase of solving.

Illumination Regulator
Optimal parameters of acquired images and video sequences depend on correct configuration of acquisition hardware and light conditions.In case of ultra high frame ratio of camera we can meet these essential problems: if the illumination of specimen is too low, frames in video sequence are underexposed and dark; if the illumination of specimen is too high, frames are overexposed and too bright.The main goal of regulation algorithm is to distribute image intensities around centre of histogram (128).
Automatic intensity regulation uses image features (histogram statistics) for computing optimal duty cycle of external control PWM signal with frequency 55 kHz [7], [8].This PWM signal (Fig. 4) is generated in LabVIEW I/O PCI card NI PCI-6229 (or USB NI MyDAQ card).(Internal PWM signal for manual illumination control is generated with SG3225.)In process of sequence acquisition, a ROI (Region Of Interest) placed into image extracts important image feature: average image intensity and histogram distribution.Overexposed image has its histogram concentrated to high intensity values and underexposed image to low values.Histogram distribution is used as regulation parameter for setting up the PWM dimmer.Dependency of histogram mean (µ) on duty cycle (dc) for actual frame ratio (f ps) of video system can be described by kvasi-linear characteristics (Fig. 5).

Dimming equation is computed from following equations:
A and calculates ∆dc for setting histogram µ 0 ≈ 128, what is half of grayscale range 0-255; dc 0 -duty cycle corresponding to µ 0 : Delay of the regulation algorithm depends on framing ratio of camera and calculation time.

Signal Conditioning
The parts for high speed imaging are created from some components:

Bayer's Filter
A Bayer filter mosaic is a color filter array (CFA) for arranging RGB color filters on a square grid of photo sensors.Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image.The filter pattern is 50 % green, 25 % red and 25 % blue, hence is also called RGBG, GRGB or RGGB (Fig. 6).Bayer is also known for his recursively defined matrix used in ordered dithering.
The algorithm achieves dithering by applying a threshold map on the pixels displayed, causing some of the pixels to be rendered at a different color, depending on how far in between the color is of available color entries.Bayer filter used twice as many green elements as red or blue to mimic the physiology of the human eye.The raw output of Bayer-filter cameras is referred to as a Bayer pattern image.Since each pixel is filtered to record only one of three colors, the data from each pixel cannot fully determine color on its own.To obtain a full-color image, various demosaicing algorithms can be used to interpolate a set of complete red, green, and blue values for each point.

Demosaicing
A demosaicing algorithm is a digital image process used to reconstruct a full color image from the incomplete color samples output from an image sensor overlaid with a color filter array (CFA).The demosaicing is also known as CFA interpolation or color reconstruction.
Most modern digital cameras acquire images using a single image sensor overlaid with a CFA, so demosaicing is part of the processing pipeline required to render these images into a viewable format.
The aim of a demosaicing algorithm is to reconstruct a full color image (i.e. a full set of color triples) from the spatially undersampled color channels output from the CFA.

Basler A504kc Technical Parameters
The Basler A504kc digital color camera is ideal solution for using CMOS technology for high speed imaging.The resolution of this camera is 1.3 megapixels (1280×1024) with frame rate 500 fps.The camera can be triggered via an external sync signal or run in an internally controlled "free-run" mode.Basler A504kc camera operates with a single voltage power supply and using the Camera Link standard for communication with PC.The sensor type of camera is progressive scan CMOS with pixel size (µm) 12×12.
The A504kc camera has global shutter which is ideal for dynamic motion inspection, human motion analysis and many other vision applications.
Camera Link is a communication link for visual applications in the fields of science and industry.This interface enables a maximum data transmission rate of 800 MB • s −1 depending on the configuration.Frame grabbers which collect and evaluate the data are usually used to connect the camera and the PC via the Camera Link interface.

Camera Configuration Tool Plus (CCT+)
The CCT+ is designed for changing parameters for the sprint series, the A-series and the L-series of Basler Camera Link cameras.Basler aviator and Basler ace Camera Link models should be used with the Basler pylon driver package.

StreamPix5 -Sequence Files
The sequence file format, which describes files created by StreamPix while recording and saved to a file with the .seqextension, is described in the following lines (Tab.1): A sequence file is made of a header section located in the first 1024 bytes.The header contains information pertaining to the whole sequence: image size and format, frame rate, number of images etc.Following the header, each image is stored and aligned to the disk sector size boundary.
Usually, pixels in the images are stored for top left to bottom right corner.Immediately following the image data comes 8 bytes, containing the absolute timestamp at which the image has been grabbed.The first 4 bytes are date and time and the last 2 bytes are the milliseconds.Assume, for instance, a sequence of 10 images of size 640×480 pixels in 8 bit monochrome in which the first image in the sequence file is at an offset of 1024 bytes.
Read 640×480 or 307200 bytes to get all the image pixels.Then read the next 32 bit (4 bytes) to get the timestamp in seconds, formatted according to the C standard time_t data structure.Read the next 16 bit (2 bytes) as an unsigned short to get the millisecond precision on the timestamp.Also, when using dedicated timing devices, the precision can be up to the microsecond.

Videosequence Pre-Processing
Preprocessing has two goals: to shorten analysis time and improve kinetic parameters analysis.
Time needed for analysis can be reduced by cutting off of the regions or frames (redundant time intervals) which don't contain relevant information [9].In this aspect we can talk about time or spatial reduction (selection of ROI containing isolated cell).Script is very simple and output is time or spatial (or both) reduced videosequence.
Important phase is improving intensity levels in whole image.After videosequence opening we can apply follow operators and tools: • Extraction of single color plane -this tool extracts 8-bit gray plane from color image, because many LabVIEW tools and operators work only with grayscale images.The most important is Luminance 8-bit plane (from HSL model, which is RGB derivate).• Convolution (spatial) filtering -convolution filtering belongs to linear filtering.Filter uses a kernel.Kernel moves through the image, convolution between original image and kernel array is calculated on each position and result is written on the central pixel position.Result can be divided with normalization factor N, which is the sum of kernel values or 1 [9].
• Equalization -equalization enhances pixel intensities.Equalization range is the value interval on which the transformation is done; this interval will be remapped on 0-255 interval.
• Histogram -histogram shows the dependency between intensity value and number of pixels of this gray value.Histogram gives us the complex view on the image and facilitates us the noise removal or segmentation.
The preprocessing application has many degrees of freedom for user, filters and tools can be fully combined, so application can be used in various cases.The kinetic parameters are analyzed.

Acquired Image Evaluation
Healthy cilia make continuous and synchronized moves.The frequency is often called CBF (Cilia Beat Frequency).This parameter can be measured using the intensity method.In this method, we create ROI in near surrounding of beating cilia.Passing of cilia across the ROI causes intensity variations in it (Fig. 7) [11], [12], [13], [14].Let's suppose videosequence with N frames and spatial resolution of W×H.We can divide entire frame area into smaller neighboring sub-regions (ROIs) with dimensions K×L (15×15 pixels e.g.) and mark each position with coordinates (i;j).ROIs are not overlapping.Then we can construct i max × j max regions (Eq.( 5)), where: Residual image borders smaller than ROI dimensions are neglected or added with zeroes to ROI dimensions.In each sub-region (position (i; j)), average intensity is computed for all frames in videosequence and we can build discrete waveform G i,j (intensity curve -Fig.5), compute its DC offset µ G and remove it (transformed curve Ĝi,j ): where ḡi,j (k) represents average intensity level in relevant sub-region, g a;b is a single pixel value.
This step transforms videosequence represented by 8-bit integer (U8) arrays to 3-dimensional structure of real numbers.Each intensity curve contains information about object motion in given area.Mean value of intensity curve is non-zero, Fourier spectrum (result of harmonic analysis) contains high DC component.This mean value corresponds with average intensity of whole image (frame) and amplitude of intensity curve is determined by contrast of image.For accurate analysis, it is desirable to reach as high contrast as possible.Adaptive illumination regulator (dimmer) is a part of hardware solution of workstation and due to this part we can set optimal contrast and intensity relations in picture during acquisition process [16].
We average ROI intensity (Fig. 8) in time and this periodical curve is processed with of frequency analysis.By application of FFT (Fast Fourier Transform) we can see the frequency spectrum of beating (Fig. 9).
In the phase of intensity curves we can use algorithm of FFT applied on larger segments of curves (1, 2 or more seconds) for determining dominant frequency of motion, or we can use short-time Fourier transformation (STFT) for observing frequency variations in time.STFT is applied on segments with length of hundreds of milliseconds with or without covering the segments.Before harmonic analysis steps, we can also apply low pass filtering for noise artifacts removal.
We can also display PSD graph (Power Spectral Density), which is FFT of curve autocorrelation.In Fig. 10 we can see graph of Spectrogram for 1 minute sequence and in Fig. 11 frequency result.We can also see (Fig. 10) that measured frequency is in the range of 7-15 Hz, which is the physiological value.
LabVIEW contains components working with MS Office, so measurement results can be stored in Excel table.Second parameter -trajectory (or object posi-10: Spectra of 20 FBGs.tion in the frame) is analyzed by algorithms of object detection.

Conclusion
Due to fast digital camera, system contains intelligent illumination dimming hardware automatically regulated through measurement card.Regulating parameter for dimmer (PWM duty cycle) is computed from image features, histogram distribution and intensity relations.Dimming helps system to preserve optimal acquisition light conditions for accurate image/sequence processing and eliminates abnormal heat generation in microscope condenser when using high-power lamp.Good brightness and contrast relations kept in the acquisition phase saves time for videosequence processing and conditioning for another analysis.
Designed solution for measuring object beating frequency from video sequence using tools of image analysis and spectral analysis simplifies present used methods and reduces usage of hardware devices.Using some development environment (e.g.NI LabVIEW) we can create fully automated application with interactive inputting of some parameters.
The software equipment for videosequence acquisition was tested on 60 FPS camera Marlin.The software tools are applicable for high speed camera Basler.The intelligent regulation of condenser illumination through image features extraction and histogram analysis enables fully automated approach to video sequence acquisition.

Fig. 1 :
Fig. 1: A design of high speed imaging workstation with light microscope.

c
2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING