Visualization of acoustic waves in air and subsequent audio recovery with a high-speed schlieren imaging system: Experimental and computational development of a schlieren microphone

doi:10.1016/j.optlaseng.2018.03.015

Optics and Lasers in Engineering

Volume 107, August 2018, Pages 182-193

https://doi.org/10.1016/j.optlaseng.2018.03.015 Get rights and content

Highlights

•
A high-speed schlieren system is used to image acoustic waves propagating in air.
•
From this footage, audio signals are recovered through image and signal processing.
•
Signals are recovered for a wide range of frequencies relevant to human hearing.
•
The contrast sensitivity function of the human visual system limits wave visibility.
•
A computational approach is not limited by the contrast sensitivity function.

Abstract

We present a high-speed single-mirror double-pass coincident schlieren system and corresponding algorithms for the visualization of acoustic waves and recovery of their associated audio signals. Schlieren systems are extensively used to visualize strong shockwaves, such as those from supersonic motion or explosions. Recently, they have also been used to visualize lower amplitude non-linear acoustic phenomena, such as the weak shockwaves arising from impact events including hand claps, belt snaps, and towel cracks. Time-invariant sounds produced by loudspeakers have also been imaged, in one case leading to frequency analysis, although these have been limited to high-frequency signals at very high sound pressure levels. The research presented here shifts the focus from sound-field visualization towards audio signal recovery. A comprehensive exploration of several parameters for imaging sound sources, including frequency, wave form, and amplitude, is presented. In addition, we address for the first time the recovery of phase information, which would be essential for speech intelligibility, and the more general case of non-contact sound field reconstruction. Through image and signal processing, it is shown that audio signals can be recovered from high-speed schlieren video whose acoustic waves appear to be below the limit of visibility, and were previously deemed unrecoverable by virtue of their frequency and sound pressure level. This includes sounds at frequencies and loudnesses relevant for human hearing, producing the first ‘schlieren microphone’.

Introduction

In this paper, a novel application of schlieren imaging is presented: a ‘schlieren microphone’, whereby the optical technique for visualizing refractive inhomogeneities is exploited to indirectly recover audio signals from sound waves. An experimental setup specifically designed for the imaging of acoustic waves relevant to human hearing is combined with software built to recover audio signals from high-speed schlieren footage. This research goes beyond previous attempts to ‘visualize’ weak shockwaves or acoustic waves: there is an emphasis on signal fidelity in processing, and an evaluation of the limitations of this approach.

Previous attempts to visualize sound with schlieren techniques have mostly been limited to violent or extreme acoustic phenomena such as shock waves, as they give sharp variations of refractive indices in air. These have often relied on significant image processing that would corrupt information contained in the schlieren images. Despite the vast amounts of data present, schlieren images of sound waves have not routinely been analyzed with digital signal processing, with the images instead being inspected and remarked upon qualitatively.

In the current research, the focus is shifted away from visualization and towards signal processing and recovery. Using high-speed photography and an inexpensive schlieren apparatus, it is shown that audio signals can be recovered from schlieren high-speed video, at frequencies and sound pressure levels (SPLs) previously considered too optically weak for schlieren applications. The limitations of this setup are explored through an extensive array of tests, validated by independent microphone measurements, as well as simulated schlieren video.

Section snippets

Theoretical principles of schlieren imaging

Schlieren imaging is a classical-optics technique for making visible the refractive inhomogeneities in transparent media. Used extensively since the nineteenth century, the technique has found numerous applications in diverse academic and industrial fields such as optics, fluid dynamics [1], aerodynamics, ballistics [2], [3], ultrasonic acoustics [4], epidemiology [5], phonetics [6], and marine biology [7]. The history, optical principles, and much of the modern research surrounding schlieren

Experimental setup

A representation of the single-mirror double-pass coincident schlieren apparatus used to collect all the data featured here is shown in Fig. 2. The system has been optimized to be both affordable and highly sensitive, using a double-pass setup first described in 1933 by Taylor and Waldram [25], for which sensitivity is double that of a single-pass schlieren method, as light is deflected by refractive inhomogeneities for both the incoming illuminator and outgoing analyzer beams.

As shown in Fig. 2

Visibility of acoustic waves and image denoising

Although the motion of acoustic waves propagating through the image is perceptible when viewing the raw schlieren images as a video, for individual frames it is not possible to clearly discern the contrast variations. There are several post-processing routes to improving visibility, each with different advantages and disadvantages.

Unsharp masking is the simplest way to improve the visibility of intensity variations. However, due to the sensitivity of the schlieren system, there are many other

Simulated schlieren video of acoustic waves

Generating simulated schlieren video from specified signals and noise can validate the analysis of experiment-derived footage. A sound field can be modeled as sinusoidal waves propagating from a point source. The amplitude of pressure, P, throughout the sound field at time t and a radial distance r from the source is given by: $\begin{matrix} P (r, t) & = & \frac{P_{0}}{r^{2}} \exp {i (κ r - ω t)} \cdot \exp {ϕ (ω)} \\ \propto & \frac{1}{r^{2}} \exp {i (κ r - ω t)} \\ where ω & = & 2 π f \end{matrix}$

For a harmonic series of sine waves, with the hth harmonic corresponding to an angular frequency ω_h, the

Results

Fig. 10 shows the overall work flow of the schlieren microphone; from high-speed schlieren footage of a sound field to the recovered audio signal. Both the point-source assumption model and the auto-correlation of signals are shown as possible routes to relate and combine each pixel’s time series data into a single signal.

Three example results are provided in Fig. 11, showing the recovered audio for both high-frequency and low-frequency waveforms, and both sine and square waveforms. For the

Discussion

The origins of sound visualization go back as far as the phonautrograph of Édouard Léon Scott de Martinville, patented in 1857. While the phonautograph produced visual outputs for the inspection of acoustic information, it was only after Thomas Edison’s invention of the phonograph that the focus of acoustic engineering shifted from visualizing sound to recovering audio information for playback and analysis. With the work presented here, a similar shift is made for a different imaging modality

Conclusion

We have found that schlieren imaging is a promising imaging modality for the study of acoustic phenomena occupying the domain of human hearing. Previous work has largely been confined to discussions of visibility, which is subject to the limitations of the human visual system, particularly for lower-frequency waveforms. By focusing on a computational and image-processing approach, we have extended the scope of acoustic phenomena for which schlieren imaging may have potential applications.

Funding

This work was supported by the Arts and Humanities Research Council grant AH/N001222/1. JSH is funded by the Andrew W. Mellon Foundation and Clarendon Fund through the Oxford interdisciplinary research center, TORCH. The equipment used was developed with funding from the Oxford University John Fell Fund grant 102/630.

Acknowledgments

The authors wish to thank the researchers of the interdisciplinary Ordered Universe Project for many stimulating discussions on sound, prompted by studying the writings of thirteenth century scientist and bishop, Robert Grosseteste (c. 1170-1253). The initial inspiration for this work came from studying Grosseteste’s pioneering text on sound and phonetics, De generatione sonorum (On the Generation of Sound). We also thank Rebekah White and Sarah Regan for their comments on an earlier version of

References (30)

R. Malkin et al.
Quantitative imaging of acoustic reflection and interference
J Phys
(2015)
Guangji Shi et al.
On the importance of phase in human speech recognition
IEEE Trans Audio Speech Lang Process
(2006)
J. Mannos et al.
The effects of a visual fidelity criterion of the encoding of images
IEEE Trans Inf Theory
(1974)
P. Freymuth
Flow visualization in fluid mechanics
Rev Sci Instrum
(1993)
O.C. Bixler
Color schlieren as a quantitative tool for acoustic measurements in ballistics
J Acoust Soc Am
(1968)
M.M. Biss et al.
High-speed digital shadowgraphy of shock waves from explosions and gunshots
Shock waves
(2009)
D. Mller et al.
Schlieren visualization of ultrasonic standing waves in mm-sized chambers for ultrasonic particle manipulation
J Nanobiotechnol
(2013)
J.W. Tang et al.
Qualitative real-time schlieren and shadowgraph imaging of human exhaled airflows: an aid to aerosol infection control
PLoS ONE
(2011)
J. Rowell et al.
Schlieren study of external airflow during the production of nasal and oral vowels in French
Can Acoust
(2016)
J.A. Westphal
Schlieren technique for studying water flow in marine animals
Science
(1965)

G.S. Settles

Schlieren and shadowgraph techniques: visualizing phenomena in transparent media

(2001)

G.S. Settles et al.

A review of recent developments in schlieren and shadowgraph techniques

Meas Sci Technol

(2017)

M.J. Hargather et al.

Schlieren imaging of loud sounds and weak shock waves in air near the limit of visibility

Shock Waves

(2010)

L.M. Weinstein et al.

Visualization and image processing of aircraft shock wave structures

1st Pacific Symposium on Flow Visualization and Image Processing, February 23–26, 1997, Honolulu, Hawaii

(1997)

N. Chitanont et al.

Visualization of sound field by means of Schlieren method with spatio-temporal filtering

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

(2015)

Cited by (7)

Visualization of supersonic free jet flow structures subjected to various temperature and pressure ratio conditions
2022, Optics and Lasers in Engineering
The standard Z-type schlieren visualization technique was used to experimentally investigate the effects of inlet boundary conditions (temperature and pressure) and nozzle geometry (convergent and convergent-divergent nozzles (C-D)) on a supersonic flow structure. The focus was on the Mach disk's length (L_m) and width (W_m). The inlet temperature (T_in) was 123 to 288 K, and the nozzle pressure ratio (NPR= $P_{in} / P_{atm}$ ) was 3 to 20. When varying the NPR, over and under-expanded flow conditions were observed in the studied nozzle configurations. The results illustrate that when increasing NPR at T_in = 288 K, L_m increases for both over and under-expanded flow conditions. However, when increasing NPR in over-expanded conditions, W_m increases to reach the saturation point and then decreases for a C-D nozzle with a larger exit diameter, while it becomes constant for a small exit diameter. In contrast, Wm increased when increasing NPR for all nozzles in the under-expanded regime. At 123 K < T_in < 173 K, the shock cells were shrunken for C-D nozzles at NPR=5, and subsequently, L_m decreased. Nevertheless, W_m increased until T_in = 153 K and then decreased until T_in = 123 K, which disturbed the Mach disks and shock cells that governed the decrease in W_m.
Visual measurement of instable sound field using common-path off-axis digital holography
2022, Optics and Lasers in Engineering
Citation Excerpt :
However, to imaging a sound field, LDV requires scanning owing to its point measurement, which restricts the applicable object to reproducible sound fields. The Schlieren method [7–8] is another simple and attractive optical method to image the information of the sound field. However, its sensitivity decreases with the decrease of the sound field frequency, resulting in an application only limited in the ultrasonic sound field measurement.
A common-path off-axis digital holography (DH) using a defocus grating is developed to detect and visualize the sound field information through a single laser beam. Setting the grating outside the Fourier plane, our approach divides the laser beam into the object and reference beams by the grating, in which one of the beams is filtered using a pinhole to create the reference beam, and then yields holograms containing the phase information modulated by the sound. After phase reconstruction, a windowed Fourier filter method is employed to extract the instable sound signal and suppress the background noise. A series of experiments are conducted to demonstrate the validity of the proposed approach.
Numerical and experimental analyses of ablation measurements in expansion wind tunnel facilities using a new plasma pre-heating technique
2020, International Journal of Thermofluids
Citation Excerpt :
The heating remained on during the flow and for about 0.5 s after the flow stopped. The Schlieren technique based on the principle of changing densities in the gas was used to identify the establishment of the bow shock and therefore the commencement of Mach 4.5 flow [22]. The high speed camera was set at a frame-rate of 2500 fps.
A new technique for preheating axisymmetric samples of heat shield material has been demonstrated through this work. Future applications for the technique are expected to be found in expansion tube facilities that can simulate the true flow energy under re-entry conditions. This new technique was applied to graphite sample ablation. A 50 mm diameter graphite disc having a thickness of 2 mm, was heated from the downstream side with a plasma to approximately 2500 K, and then exposed to a cold Mach 4.5 flow. The disc was mounted on an experimental probe which was very similar to the European standard probe, normal to the flow. The experiments show a significant spatial variation in thickness loss for the graphite test material over the disc radius though the spatial variation was still largely axisymmetric. The CFD results only included the reaction chemistry during flow over the heated disc and did not consider the mechanical actions, such as erosion and spallations. The analysis shows that numerical simulation of the oxidation rate produced results that were within 50% of the mass loss measurements. Simulations indicate that oxidation through surface reactions producing CO was the major contributor to the ablation losses. Analysis involving comparisons between the experiments and simulations suggests that mechanical actions of the ablation process are only minor contributors to mass loss. The experimental material loss included losses while the heating cycle occurred for the heated-only, and heated-with-flow runs. This pre-heating part of the heating cycle was taken into account for the comparison with simulation. Experimentally measured mass loss for the heating cycles were averaged and assumed to apply for the mass loss during the heating phase of the heated-with-flow runs. The potential for mechanical actions to explain the difference between simulation and experimental results is investigated. Presented here are the analyses of results, investigating material loss characteristics, and considers the discrepancy between experimental and CFD determination of the ablation rates.
Qualitative evaluation of the flow supplied by personalized ventilation using schlieren imaging and thermography
2020, Building and Environment
Citation Excerpt :
This refraction causes light rays to deflect from their continuous paths, creating ‘schlieren’ or ‘streaks’, which are a gradient disturbance in inhomogeneous transparent media [28]. Schlieren imaging is used in the literature to investigate a wide range of topics, such as the visualization of acoustic waves [29], gas concentrations [30], and fluid jets [31]. Furthermore, the literature reports numerous studies that implemented schlieren imaging to study the indoor environment.
Personalized ventilation (PV) is a mean of delivering conditioned outdoor air into the breathing zone of the occupants. This study aims to qualitatively investigate the personalized flows using two methods of visualization: (1) schlieren imaging using a large schlieren mirror and (2) thermography using an infrared camera. While the schlieren imaging was used to render the velocity and mass transport of the supplied flow, thermography was implemented to visualize the air temperature distribution induced by the PV. Both studies were conducted using a thermal manikin to simulate an occupant facing a PV outlet. As a reference, the flow supplied by an axial fan and a cased axial fan was visualized with the schlieren system as well and compared to the flow supplied by PV. Schlieren visualization results indicate that the steady, low-turbulence flow supplied by PV was able to penetrate the thermal convective boundary layer encasing the manikin's body, providing clean air for inhalation. Contrarily, the axial fan diffused the supplied air over a large target area with high turbulence intensity; it only disturbed the convective boundary layer rather than destroying it. The cased fan supplied a flow with a reduced target area which allowed supplying more air into the breathing zone compared to the fan. The results of thermography visualization showed that the supplied cool air from PV penetrated the corona-shaped thermal boundary layer. Furthermore, the supplied air cooled the surface temperature of the face, which indicates the large impact of PV on local thermal sensation and comfort.
Raw high-speed schlieren footage of acoustic waves in air for subsequent computational analysis and audio recovery
2018, Data in Brief
Citation Excerpt :
The dataset presented here is unprocessed. As addressed in [1] the visibility of the acoustic waves may be improved by filtering and signal processing. The audio signals may then also be recovered by recombining phase-aligned signals and subsequent integration [code yet to be published].
The dataset presented here comprises 79 raw, unprocessed video files obtained from the high-speed schlieren imaging of acoustic waves in air, each with a corresponding metadata file. The majority of the footage was recorded at 50,000 frames per second, with each condition filmed for a duration of 20 ms. The dataset includes footage corresponding to a wide range of signals, in terms of waveform, frequency, and amplitude, as well as varied imaging parameters (exposure, frame rate, spatial resolution). This Data in Brief article is to accompany the research article “Visualization of acoustic waves in air and subsequent audio recovery with a high-speed schlieren imaging system: Experimental and computational development of a schlieren microphone” [1].
Visualization of ultrasonic wave fields and demonstration of sound wave property in undergraduate physics experiment
2021, Physics Education

View all citing articles on Scopus

View full text

Visualization of acoustic waves in air and subsequent audio recovery with a high-speed schlieren imaging system: Experimental and computational development of a schlieren microphone

Highlights

Abstract

Introduction

Section snippets

Theoretical principles of schlieren imaging

Experimental setup

Visibility of acoustic waves and image denoising

Simulated schlieren video of acoustic waves

Results

Discussion

Conclusion

Funding

Acknowledgments

J Phys

IEEE Trans Audio Speech Lang Process

IEEE Trans Inf Theory

Flow visualization in fluid mechanics

Rev Sci Instrum

Color schlieren as a quantitative tool for acoustic measurements in ballistics

J Acoust Soc Am

High-speed digital shadowgraphy of shock waves from explosions and gunshots

Shock waves

Schlieren visualization of ultrasonic standing waves in mm-sized chambers for ultrasonic particle manipulation

J Nanobiotechnol

Qualitative real-time schlieren and shadowgraph imaging of human exhaled airflows: an aid to aerosol infection control

PLoS ONE

Schlieren study of external airflow during the production of nasal and oral vowels in French

Can Acoust

Schlieren technique for studying water flow in marine animals

Science