An Efficient Eulerian Video Magnification Technique for Microbiology Applications

The micro-biology videos often contain motions of particles which are not visible to naked eye. Therefore an efficient motion magnification technique is required to magnify these motions. A time efficient Eulerian video magnification technique for micro-biological applications is proposed. The proposed technique utilizes the concept of time and spatial uniformity to reduce the computational complexity. Simulation results reveal that the proposed scheme is almost four times efficient and more accurate as compared to state of art video magnification technique.


Introduction
Micro-biology videos contain small moving microorganisms which are often not visible to naked eye.As micro-organisms (such as bacteria), can be harmful to human beings, there is a need for a technique capable of detecting and magnifying small motions of these micro-organisms.The enhancement of these minute motions (hidden information) is helpful for different applications (including density estimation and behavior analysis) [1], [3][4][5][6], [12].Traditionally, microscope is an essential tool in micro-biology to visualize micro-organisms.However, the reliability of microscopic analysis of images is dependant on equipment used for visualization [10].In literature [7], [14][15][16], visual motion magnification techniques have been proposed to highlight invisible changes in videos.
As visual motions occur at different amplitudes in spatial and temporal frequencies, it is difficult to observe small movements of micro-organisms.Previously attempts have been made to reveal subtle motions in videos [7], [14], [15].Liu et al. [7] proposed a motion magnification technique to magnify small motions in videos, by enhancing specific cluster of pixels.The technique enhances small motions only in the specified area.In [15], use of Langrangian motion estimation [2] to enhance mechanical motions in cartoon animation is presented.The technique [15] is not only computationally expensive but also produce significant amount of errors in motion estimation [16].Since Eulerian motion estimation technique [2] does not involve costly optical flow computation and is less sensitive to noise, Wu et al. [16] proposed to use it for video magnification to observe the motion of visual changes at specific location as they flow through time.Although Eulerian video magnification technique [16] provides robustness, however the quality of magnification is dependant on magnification factor.In high spatial frequencies, as magnification factor is increased, it introduces unwanted/noise artifacts.To reduce these unwanted noise produced by the magnification factor, steerable pyramids [11] and complex steerable pyramids using fourier decomposition are proposed [13].These techniques result in over-complete pyramid decomposition and are computationally expensive [14].The increased computational cost and over completeness of steerable pyramids make current phase based motion magnification inefficient.
Wavelet pyramid decomposition allows interpretation of information at different resolutions (coarse-to-fine).It analyzes the image at coarse resolution and then gradually increases image resolution [8].As wavelet decomposition utilizes information gathered from both course and fine scales for magnification, the final magnified video is reconstructed with high precision.In this paper, use of wavelet representation for pyramid decomposition in Eulerian video magnification technique is proposed.The proposed technique decomposes the video frames into spatial bands using wavelet decomposition and analyze the information content at different scales.To magnify the desired frequencies, temporal processing is performed on each spatial band.The proposed technique not only magnifies the insignificant minute visual changes but also avoids spatial aliasing and is noise free.To demonstrate the effectiveness of proposed technique, quantitative comparisons on four microbiological videos have been provided.The results demonstrate that the proposed technique is almost four times efficient and provides magnification without introducing unwanted artifacts.

Proposed Technique
Let P β t ( ū) be the intensity of color input video at location ū = [u 1 u 2 ] and at time t, where β represents red, green and blue color bands i.e. β ∈ {R, G, B}.The gray scale image P t ( ū) is To reduce the computational complexity, the P t is spatially down-sampled using Bicubic interpolation with a factor η i.e.
Pt ( ūη where value of η ∈ [0, 1]. The existing Eulerian magnification [16] decomposes the image into different levels using Laplacian and Gaussian pyramids.However, there are two major limitations i.e. over-completeness (by factor of 4/3 and 2 in one and two dimensions respectively) and non-utilization of information present in high pass images.To overcome this limitation, the proposed technique uses Discrete Wavelet Transform (DWT) based on Quadrature Mirror Filters (QMF).QMF filters are suitable for multi-resolution decomposition and pyramid coding.Orthonormal QMF filters provide perfect reconstruction without error introduction [9].Different components of DWT are where A, H, V, D represent approximation, horizontal, vertical and diagonal detail components respectively.
To extract frequency bands of interest, temporal processing is performed i.e. each subband is interpolated with a temporal band pass filter.The temporal processing not only attenuates signals out of interest band but also smooths the resulting signal.For example, if the aim is to find out pulse rate then frequencies within 0.4-4Hz might be selected.The choice of filter depends on type of application.In proposed technique, Infinite Impulse Response (IIR) filter is used as it amplifies both colour and motion.To compute the filtered subband F k t ( ūη ), we use recursive difference (4) where wavelet spatial band Pk t ( ūη ) is taken as present input signal and F k t−1 ( ūη ) as past filtered output in time domain.
The resultant magnified output signal I k t is given by where γ is magnification factor.γ can be computed using first-order taylor series expansion [16] or can be specified by the user.Taylor series approximations works well for small magnification value.But for larger magnification, it produces inaccurate results and introduces noisy artifacts.If displacement function δ(t) due to observed motion at time t in an image does not lie within the range of temporal filter passband, it is attenuated by a certain factor and modified magnification factor γ m is obtained (6) which supports large magnification for observed motion δ(t) in high spatial frequencies.
where cut-off spatial wavelength λ for moving signals is taken as 2π/ω and ω is the spatial frequency (determined by Fourier transform).
To obtain processed output signal G t , wavelet pyramid is reconstructed by upsampling using bicubic interpolation, filtering and summing the individual sub-bands i-e I A t , I H t , Let V t be the result of upsampling the processed frame by factor η (to regain the original video resolution, while preserving the vital structural information) To regenerate a coloured output frame having red, green and blue coloured bands with the magnified motion, the processed magnified signal V t ( ū) is added back into the original signal where Pβ t ( ū) is coloured frame at location ū and at time t.To enhance the efficiency, above mentioned steps are performed on every alternate frame of video.The output (magnified) video highlights the hidden details which are not visible to the naked eye.

Simulation Results
Wu et al. [16] and proposed technique are evaluated on various microbiological videos.The parameters used for videos are listed in Tab. 1.For all videos, IIR filter with passband of 0.4 Hz (Γ 1 ) to 0.005 Hz (Γ 2 ) and spatial wavelength (λ) 16 are used.Accordingly, magnification factor (Ω) 30 and 100 are chosen to achieve desired magnification in output video.performed using Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) [17] to evaluate and compare the quality of videos magnified using proposed technique with the state of the art technique [16] .
Figure 1(a-c) shows the frames of micro-organism Infusoria input video when the water pumps out of the cell by contraction.Water and food particles move inside and outside of oral groove, which causes contraction in cilia (outer layer) and contractile vacuole.This movement is not visible to the naked eye but can be made visible after applying Wu et al. [16] and proposed Eulerian magnification technique (Fig. 1(d-i)).However, the results of Wu et al. [16] introduces unnecessary artifacts in the magnified video whereas the proposed technique preserves the smoothness of video by removing noise and unwanted artifacts.
Figure 2(a-c) shows the original frames of Motile bacteria motion video.The smooth gliding of bacterial leaving a black copious slimy trail behind is not clearly visible and they appear almost static.However, using existing Wu et al. [16] technique, the motion of bacteria is magnified along with noise (see Fig. 2(d-f)).Whereas the proposed technique, amplifies the motion and increases the visibility to differentiate motion of bacteria (Fig. 2(g-i)).
Figure 3(a-c) shows the original frames of Physarum Polycephalum streaming video.Physarum Polycephalum lives in isolation in slime moulds.Its streaming is dependant on to and fro (nearly invisible) movement of plasmodium.The contraction and relaxation of cell membrane forces plasmodium to move back and forth within periphery of cell.After applying [16] (Fig. 3(d-f)), noise is also amplified along with movement.However, the proposed technique only amplifies movement of plasmodium in moulds and suppresses the noise to minimum level (Fig. 3(g-i)).
Potable drinking water video frames are shown in Fig. 4(a-c).Note that the bacteria present in water are not identifiable (as the video is nearly still).Figure 4(d-f) shows that unwanted noise artifacts are introduced after applying Eulerian video magnification technique [16].However the proposed technique not only identifies moving bacteria (black artifacts) in drinking water (which are harmful) but also denoises the video and produces smooth results (Fig. 4(g-i)).
Figure 5 shows the PSNR plots of exiting Wu et al. [16] and proposed techniques.It can be seen that the results of Wu et al. [16] and proposed techniques are comparable for infusoria and potable drinking water videos (Fig. 5(a, d)).For Motile bacteria and Physarum videos (Fig. 5(b, c)) the proposed technique provides much better results as compared to Wu et al. technique [16].The average PSNR linked to Fig. 5 is given in Tab. 2. Figure 6 shows the SSIM plots of Wu et al. [16] and proposed techniques.It can easily be concluded from the results that proposed technique yield mostly better SSIM values as compared to existing Wu et al. [16] technique.The average SSIM linked to Fig. 6 is given in Tab. 2.
In Tab. 3, computational complexity results of MAT-LAB implementations of Wu et al. [16] and proposed techniques.The proposed technique yields almost 4 times faster results than the Wu et al. [16] while preserving the smoothness of the videos.

Conclusion
A time efficient Eulerian video magnification technique for micro-biological applications is proposed.The technique is four times efficient as compared to existing technique.The technique utilizes the concept of time and spatial uniformity to reduce the computational complexity.Visual and quantitative analysis verifies the effectiveness of the proposed technique as compared to state of art video magnification technique.