1.

INTRODUCTION

The quantitative phase image represents a per-pixel map of the optical path length across the specimen, providing intrinsic contrast from the structure of the object. By decoupling the image from the properties of the microscope, quantitative phase imaging techniques provide reproducible diagnostic markers well suited for clinical applications such as blood screening or cancer diagnosis [1-5].

In Spatial Light Interference Microscopy (SLIM [6]) the relative phase shift between scattered and transmitted light is manipulated by way of a spatial light modulator (SLM). In our implementation, each SLIM image is reconstructed from four ‘phase contrast’ frames acquired at 90 degree offsets between scattered and transmitted light, with the relative offset between the two components used to reconstruct a per-pixel map of the optical path across the specimen. Among the advantages of phase shifting interferometry[7, 8] is the efficient use of camera bandwidth when compared to techniques where the image is retrieved by Fourier demodulation[9], necessitating significantly more pixels to properly sample the modulated image.

The SLIM hardware sits at the output port of a conventional white light microscope with the SLM conjugate to phase ring of the objective. This design is particular well paired with fluorescence microscopy as the low intensity fluorescent signal moves through the hardware relatively unperturbed when compared to diffraction gratings[10], lenslet arrays [11], and multipath setups[12-14] that split the weak emitted light beyond the sensitivity of the camera.

2.

INSTRUMENTATION

The acquisition of a SLIM frame begins by writing a phase mask onto the SLM. Like a conventional liquid crystal display, the response time for a phase shifting SLM depends on the values written, and is on the order of 10 ms [15]. Nevertheless, to achieve this performance, in the case of a computer display connected SLM device, the pixels must actually switch on demand. In our system, we run a dedicated OpenGL render context with a condition variable releasing the buffer, effectively implementing a software trigger (Code 1). We note that by creating the context from scratch, we can disable the post-processing and motion blurring that is desirable for video games and word processing applications, but otherwise result in an increase per-frame latency in the phase modulating applications.

By default, there is a few millisecond latency between notifying a condition variable and returning control to the associated thread (Code 2). On a Windows platform, Multimedia Timers provide a workaround to achieve response times on the order of Windows NT thread quanta, achieving near optimal response time, reducing jitter due to multithreading to under a millisecond[16] :

Although hardware camera triggering can be used for time critical acquisitions, instead we opt to use software based camera triggering as it enables us to implicitly adjust for variable image processing time, in particular for in the live acquisition mode. We perform the software triggering and datacopy operations on separate threads, enabling the camera API to respond as data becomes available. Moreover, by decoupling triggering from datacopy we can reduce camera aperture jitter, as triggering is usually immediate while waiting for the data to arrive is subject to other factors such as the computational load.

Writing data to a hard disk at our acquisition speeds is challenging because, even with the fastest solid state drives, the archived speed can vary as a function of capacity, leading to the so-called “SSD write cliff"[17]. Our implementation addresses this problem by performing file writing asynchronously reserving a memory buffer corresponding to the computer’s storage capacity letting us briefly exceed hard disk capacity. Further performance increases come from writing the files as binary data and avoiding the operating systems internal file buffering (Code 3).

As the mechanical motion of the microscope takes a variable amount of time to complete, tasks such as opening a shutter or moving the stage require synchronous operation where control returns to the program upon completion. Nevertheless, it is desirable to do such tasks in parallel, for example modulating the SLM while moving to a new position or switch to a reflectorendering the live image in fluorescence filter cube while turning off the transmitted illumination. We group such task blocks by launching them asynchronously waiting for their completion through the `std::future` framework. Further, the destruction of purpose built objects ensures a minimal execution time effectively converting portions of the code from asynchronous to synchronous ensuring predictable computation times(Code 4). With these two techniques, the total time becomes the maximum of each step rather than sum of each step - i.e., max(slm,microscope) rather than slm+microscope - and is enforced implicitly without any a-priori knowledge of the execution time of the constituent elements.

3.

USER INTERFACE

To present the SLIM image as a unified contrast setting, in the same way as Bright Field or Phase Contrast, we use CUDA and OpenGL to perform real-time rendering on a separate drawing thread. Passive mechanisms, such as write-combined memory were considered, but not used due conflicts with OEM camera hardware. Instead, we perform the data transfer on a second thread achieving a similar effect, where the next frame loads while the current one is processed. In total, the live mode runs three threads: one for triggering and data upload, one for waiting on new data, and one for rendering the live image (Figure 1).

The acquisition process reads a list of acquisition events, with each event indexing the corresponding channel information. Information stored in each channel includes items such as exposure time, along with QPI specific parameters like the background illumination (Figure.2). To maximize code reusability, OEM SDK are hidden in a particular implementarendering the live imagetion of generic devices - i.e. OspreyCamera implements Camera. The acquisition list enables us to decouple the graphical user interface (Figure 3) from the acquisition process, greatly improving code reusability, as only the list generation function needs to be changed when modifying the user interface or adding features to the acquisition process. The principle downside of such a structure being the need to use synchronization primitives when changing the list during acquisition, although the persistence of well-defined acquisition events facilitates an error correction procedure where a failed event can be easily re-attempted.

Understanding that a clinical slide scanner designed for use by medical professionals must digitize samples whose `surface of best focus` is non-uniform, we incorporate a mechanism to set focus points across the surface of the sample (figure 4). To handle arbitrary locations we generate a Delaunay triangulation and sample it when generating our acquisition list[18]. A slide scan, as would be used to digitize a peripheral blood smear or frozen biopsy section, can use this surface `as is` or take it as a hint for the automated focus system as is addressed in our separate work [19].

4.

ARCHIVAL STORAGE

A typical acquisition results in tens of thousands of images, and presents a computational challenge to existing image assembly tools. These tools are often designed for general image alignment tasks such as merging tilings with arbitrary positions or compensating for image warping resulting in producer-consumer schemes with hard to predict threading. For example, some stitching programs are able to align a tile with a variable number of neighbors, leading to parallelism schemes where too few or too many threads are invoked. Instead, we optimize our code for regularly spaced mosaic tiles and rigid transformations[20] facilitating predictable overlap between disk reading and computation, resulting in a code that aligns images comparable to hard disk read speeds. The resulting list is converted into a format compatible with tile based online image viewers, ready for inspection by a medical professional [21, 22].

The intrinsic contrast and quantitative nature of QPI techniques make them well posited for archival storage, because phase, unlike intensity, does not depend on the properties of the instrument but rather only on the actual specimen. To this end, we demonstrate a computational workflow designed to scan a large number slides, processing them in batch and storing them for easy online access. Specifically, a k-means clustering on a down-sampled version identifies the pixels as belonging to a cores and then a second k-means clustering is used to assign the labels (Figure 5) [23].

5.

SUMMARY

Our design presents spatial light interference microscopy in a user-friendly manner, unifying a multistep interferometric contrast technique into an image well suited for use by biologists and clinical practitioners. In addition to providing a real time image, our pipeline represents a realization of cloud pathology, with images moving directly from the scanner to a web based image viewer.