Longitudinal Label-free Tracking of Cell Death Dynamics in Living Engineered Human Skin Tissue with a Multimodal Microscope References and Links

We demonstrate real-time, longitudinal, label-free tracking of apoptotic and necrotic cells in living tissue using a multimodal microscope. The integrated imaging platform combines multi-photon microscopy (MPM, based on two-photon excitation fluorescence), optical coherence microscopy (OCM), and fluorescence lifetime imaging microscopy (FLIM). Three-dimensional (3-D) co-registered images are captured that carry comprehensive information of the sample, including structural, molecular, and metabolic properties, based on light scattering, autofluorescence intensity, and autofluorescence lifetime, respectively. Different cell death processes, namely, apoptosis and necrosis, of keratinocytes from different epidermal layers are longitudinally monitored and investigated. Differentiation of the two cell death processes in a complex living tissue environment is enabled by quantitative image analysis and high-confidence classification processing based on the multidimensional, cross-validating imaging data. These results suggest that despite the limitations of each individual label-free modality, this multimodal imaging approach holds the promise for studies of different cell death processes in living tissue and in vivo organs. Apoptosis: why and how does it occur in biology? " Cell Biochem. Differentiation of apoptosis from necrosis by dynamic changes of reduced nicotinamide adenine dinucleotide fluorescence lifetime in live cells, " J. In vivo cellular optical coherence tomography imaging, " Nat.and necrosis-induced changes in light attenuation measured by optical coherence tomography, " Lasers Med. Sci. Detecting cell death with optical coherence tomography and envelope statistics, " J. Cell death detection by quantitative three-dimensional single-cell tomography, " Biomed. Optical coherence tomography of cell dynamics in three-dimensional engineered tissues, " SPIE Proc. 5861 (2005). observation of cytochrome c dynamics during apoptosis, " Proc. Intrinsic fluorescence and redox changes associated with apoptosis of primary human epithelial cells, " J. Cell death, non-invasively assessed by intrinsic fluorescence intensity of NADH, is a predictive indicator of functional differentiation of embryonic stem cells, " Biol. uptake of the apoptosis-imaging agent Tc-99m recombinant human annexin V in human tumors after one course of chemotherapy as a predictor of tumor response and patient prognosis, " Clin. Non-invasive detection of apoptosis using magnetic resonance imaging and a targeted contrast agent, " Nat. " Ultrasound imaging of apoptosis: high-resolution non-invasive monitoring of programmed cell death in vitro, in situ and in vivo, " Br. Ultrasound imaging of apoptosis in tumor response: novel preclinical monitoring of photodynamic therapy effects, " Cancer Res. " Noninvasive optical imaging of apoptosis by caspase-targeted activity-based probes, " Nat. High-resolution optical coherence microscopy for high-speed, in vivo cellular imaging, " Opt. Integrated multimodal optical …


Introduction
Cell death is a fundamental but complex process that occurs under normal physiological conditions as well as in pathological states.It is widely acknowledged that cell death can occur along two major pathways, namely apoptosis and necrosis.Apoptosis, often referred to as programmed or physiological cell death, is the general pathway for the removal of cells that show DNA damage or abnormal function, without causing inflammation [1].Research interest in apoptosis has dramatically increased in the last few decades, not only because of its crucial role in normal tissue development and homeostasis, but also because deregulation of apoptosis could lead to the development of cancer and other life-threatening diseases [2][3][4].Necrosis, the other cell death pathway, is usually triggered by external factors.This so-called accidental cell death typically causes the release of intracellular content, leading to an immune response and an inflammatory reaction.Because of the different roles and consequences of these two cell death processes, having an effective method to monitor and differentiate them under different conditions is important for investigations of such diverse areas as normal tissue development, the health of developing and grafted engineered tissues, disease evolution, therapeutic efficacy, and drug delivery [5,6].
Apoptosis and necrosis are different in many aspects, including different molecular mechanisms, dynamics of morphological changes, and the methods of removal of dead cells [1,7].Because of these distinct features, differentiation between apoptosis and necrosis is straightforward using histological analysis on fixed specimens.With well-developed biomarkers, histology has been the gold standard for recognition of apoptosis, which is easily visualized by morphological changes such as chromatin condensation, mitochondrial swelling, and membrane blebbing.In contrast to the destructive manner of histology, some nondestructive optical imaging techniques, such as optical coherence tomography (OCT) [8][9][10][11][12][13], Raman spectroscopy [14], and fluorescence microscopy [6,15,16], have been exploited recently for real-time investigation of cell death processes in cell cultures.However, noninvasive real-time imaging and differentiation of apoptosis and necrosis under more natural tissue conditions, where the roles of cell death are essentially played out, becomes difficult due to the complex tissue environment involving the extracellular matrix and multiple cell types [5].Although there are a few techniques for detecting apoptosis in vivo or in clinics, such as nuclear imaging based on annexin V derivatives [17], magnetic resonance imaging (MRI) [5,18], and high frequency ultrasound [19,20], none of them offer sufficient spatial resolution to track single cells in tissue.Optical imaging offers desirable resolution, but often lacks necessary contrast and specificity unless particular fluorescence-labeling is involved [21].
In this study, we exploit the advantages of label-free multimodal optical microscopy for the real-time investigation and differentiation of apoptosis and necrosis in tissue.This integrated microscope combines multi-photon microscopy (MPM, based on two-photon excitation fluorescence [22,23]), optical coherence microscopy (OCM, the high resolution variant of OCT) [24], and fluorescence lifetime imaging microscopy (FLIM) [25].Despite the inherent limitations of each single modality, such as the low contrast of OCT and the exclusive visualization of fluorescent structures in MPM and FLIM, this unique combination of multiple contrast mechanisms and functions was able to provide additive and complementary information, enabling real-time investigation of cell death in a living tissue environment with subcellular resolutions.The multiple dimensional data extracted from spatially co-registered time-lapse multimodal images enabled quantitative analysis on automatically segmented cells and high-confidence classification of different cell death pathways on unstained, intact living tissue samples.
In experiments, we investigated human keratinocytes in living engineered skin, where two types of cell death, i.e., apoptosis and necrosis, were induced chemically.Detection and differentiation of the cell death processes were demonstrated based on the dynamics of timelapse multimodal images, and distinct features from different epidermal layers were separated using the high depth resolution and sectioning capabilities of the system.Based on the collected data, including light scattering, fluorescence intensity, lifetime, and spectroscopy, this imaging approach has the potential to be a powerful tool for real-time investigation of cell death processes in living tissues, monitoring the health and integration of grafted engineered tissues, disease development and progression, efficacy of cancer treatments, and drug delivery.

System description and characterization
The illustrative schematic of the integrated microscope is shown in Fig. 1 and its optical design can be found elsewhere [26,27].It is based on a single femtosecond oscillator laser source (Mai Tai HP, Spectra Physics) and a shared microscope platform.The Ti:Sapphire light source outputs 100 fs (pulse width) laser pulses at 3 W (maximum average power) and at a repetition rate of 78 MHz, with central wavelengths tunable from 690 nm to 1040 nm.To optimally support both the MPM and OCM modalities, the laser output is first split into two portions with a splitting ratio of 3:7.The low-energy pulses (~0.9 W) are coupled into a 1 m long photonic crystal fiber (LMA-PM-5, NKT Photonics) to generate broadband supercontinuum (SC) for OCM imaging.This broadband SC, with a quasi-Gaussian profile and full-width-half-maximum of up to 120 nm at 800 nm center wavelength, leads to an axial resolution of 2.3 µm in air.The OCM setup is a spectral-domain system based on a free-space Michelson interferometer and a home-built spectrometer.The high energy pulses, which have a narrower bandwidth (~13 nm) but a wide wavelength tuning range, are used as the excitation source for MPM.The sample arm of the OCM interferometer and the MPM excitation beam are combined by a polarized beam splitter, and then sent to the shared homebuilt laser scanning microscope.The collinearly aligned OCM sample arm and the MPM excitation beam are scanned by a pair of galvanometers (Micromax 671, Cambridge Technology).A computer-controlled motorized translation stage is used to scan the sample for both large-area mosaic imaging and axial scanning.The fluorescence and second harmonic generation signals are coupled into either a photomultiplier tube (H7421, Hamamatsu) for generation of MPM images or into a fiber bundle connected to a 16detection-channel monochrometer for FLIM imaging (Becker & Hickle).The OCM interference signals are detected by the home-built spectrometer, which consists of a reflection grating, collimation lens, and a fast line-scan camera (Piranha2-2k, Dalsa).Although the MPM images can also be generated from the FLIM data, we note that the PMT (MPM) channel is convenient because it provides for stronger signals and higher acquisition rates (about three seconds for one frame) compared to FLIM.Fluorescence lifetime measurements and FLIM images were obtained using a commercial unit integrated into the multimodal microscope.The fluorescence signal was detected by a 16channel PMT, and then digitized and analyzed by a time-correlated single-photon counting board (SPC 150, Becker & Hickle).FLIM analyses were performed using commercial software (SPCimage, Becker & Hickle).The imaging rate of FLIM was about 4 minutes per image with either 128x128 or 256x256 pixels.This low imaging speed is apparent as a drawback of FLIM, particularly for real-time investigation of biological processes and interactions.However, as shown by the data later, this can be compensated for by combination with the faster OCM and/or MPM modalities, which in turn, allow for longitudinal tracking of cell death dynamics in living tissue.
For all the experiments reported in this paper, a microscope objective with a numerical aperture (NA) of 0.95 (XLUMP20X, water immersion, Olympus) was used.The axial and transverse resolutions for MPM and FLIM were experimentally determined by imaging fluorescent beads and were measured to be 0.8 µm and 0.5 µm, respectively.The effective NA of the OCM was intentionally reduced to ~0.7 by using a telescope to adjust the beam diameter at the back-aperture to balance the resolution and imaging depth.Adjusting the telescope also gives us the flexibility to match the focal planes of MPM (and FLIM) with OCM, which vary with depth in the tissue.
The performance of the system was first tested and characterized by imaging commercially obtained bi-layer full-thickness engineered human skin tissue samples (EpiDerm FT400, MatTek Corp.).These living tissue samples contained normal, humanderived epidermal keratinocytes, and normal, human-derived dermal fibroblasts, which formed a multi-layered, highly differentiated model of the human dermis and epidermis.The epidermis consisted mostly of keratinocytes at different physiological states, and the dermal compartment was composed of a collagen matrix containing viable human fibroblasts.Samples were cultured in an incubator with 5% CO 2 at 37°C, and nourished daily with 2.5 ml Dulbecco's Modified Eagle's Medium (pH 7.4).During imaging, the engineered skin tissue was kept in a specially modified Petri dish that was compatible with the microscope (Fig. 1).Stainless steel pins were embedded into the Petri dish to hold the tissue and limit its movement during imaging.The dish was also covered with a specially designed insert with an optical window to maintain the sterile environment of the tissue during imaging, and enable the longitudinal imaging studies.The 100 -200 µm spacing between the top surface of the tissue and the cover slip was filled with cell culture media or cell culture media containing drugs for chemical induction of cell death processes.
Representative results of multimodal imaging of the engineered skin tissue are presented in Fig. 2. Morphological features of the keratinocytes from different epidermal layers are visualized by these images, which show, for example, the stratified (atypical cellular shape and hardly resolvable) cell bodies of the stratum corneum (60 µm below the surface), flattened and granule-filled cellular structures of the stratum granulosum (70 µm and 90 µm below the surface), smaller spindle-shape cells of the stratum spinosum (110 µm below the surface), and the densely packed sheets of cell of the stratum basale (125 µm below the surface).In the OCM images (red color), while the cellular structures are less distinguishable due to the low light-scattering contrast, dark nuclei are clearly identified.Overall, both the MPM and FLIM images render the same structural information, which were generated based on the signals from two-photon excited fluorescence intensity and lifetime, respectively.However, vital metabolic information of the cells is embedded in the color-coded FLIM images, which is associated with the change of NADH (reduced nicotinamide adenine dinucleotide) fluorescence lifetime.All the FLIM images, if not specifically mentioned, were obtained by first integrating all 16 channels, which range in wavelength from 374 to 565 nm, and then fitted with the appropriate lifetime decay.The fitting model is expressed by a , 2 τ , are the amplitudes and lifetimes of free and bound NADH, respectively.The average lifetime is given by fluorescence lifetime has been demonstrated as a sensitive indicator of metabolic activities of cells [6,25].It is shown by the lifetime images that the average lifetime is getting longer from the inner basal layer to the spinous layer, which indicates the lower metabolic activities of the inner (basal) layers and the higher metabolic activities in the upper layers [28,29].
The dominance of NADH fluorescence in this case is determined by the excitation wavelength and confirmed by the measured spectra.In our experiments, an excitation wavelength of 760 nm was used to excite the two-photon fluorescence from the unstained samples.This implies that the majority of the fluorescence signals are from NADH, and the other important autofluorescent protein, i.e., FAD (flavin adenine dinucleotide), which has a two-photon excitation peak of 900 nm and an emission peak at 535 nm, was not efficiently excited in our experiments [30,31].This is also confirmed from the fluorescence spectra measured by the multi-channel FLIM system (Fig. 2(d)), which show peaks mostly centered at around 450 nm, and consistent with the fluorescence peak of NADH.A slightly red-shifted spectrum is seen from the stratum corneum layer (top layer in Fig. 2(d)), suggesting the contribution of fluorescence from keratin, which is abundantly produced by cornified keratinocytes in the stratum corneum layer and has a fluorescence spectral peak at around 480 nm [32].The appearance of a peak at ~380 nm from deeper layers shows the contribution of the second harmonic generation signal from the collagen distributed in the basal layer.
Three-dimensional (Fig. 2(e) and 2(g)) and cross-sectional OCM and MPM (Fig. 2(f) and 2(h)) images are generated by z-axis scanning, which renders the 3D structural information of the sample and confirms the spatial co-registration of the multimodality images in the axial direction.Co-registration in the lateral plane is demonstrated by the merged image of the three channels (Fig. 2(i)).

Time-lapse imaging of different cell death processes in engineered human skin
To induce apoptosis in the cells, cell culture medium containing 20 µM camptothecin and 50 µM etoposide (both from BioVision, Mountain View, CA) was applied topically to the engineered skin tissue [33].To induce necrosis in the cells, cell culture medium containing 5% peroxide bleach was used.The tissue was immediately and continuously visualized with the integrated microscope to track the dynamic process of cell death in different epidermal layers.At the beginning of the each imaging session, the OCM and MPM channels were first used for a rapid scan of a large area (up to 2 mm by 2 mm) at each depth to identify an optimal image field based on the morphologies of the imaged cells.Once selected, the initial image fields were tracked for each complete time-lapse imaging session.At each time point, the OCM and MPM channels were used to scan the sample to search for the region that was correlated and co-registered with the initial image field.After longitudinal imaging over a certain period of time, usually 18 hours, the tissue was fixed and processed for histological analysis.
Time-lapse images of human keratinocytes at the granular layer in a living engineered skin tissue sample under the conditions of control, apoptosis, and necrosis are shown in Figs.10-12 in the appendix.All the longitudinal imaging experiments were repeated three times on different samples.Qualitative reproducibility was found among these experiments.

Multimodal imaging of different cell death processes in the granular layer
Time-lapse multimodal images of cells from the granular layer (Fig. 3 and Figs.10-12 in Appendix) were recorded under three conditions, i.e., control, apoptosis, and necrosis with the multimodal microscope.As shown by the representative FLIM images (Fig. 3(a)) and the plot of mean pixel values at different time points (Fig. 3(b)), dynamic changes of the fluorescence lifetime are different for these three conditions.The fluorescence lifetime increased gradually from 1.3 ns prior to the application of the apoptosis-inducing drug to 2.1 ns after 18 hours.In contrast, a slight decrease in lifetime was observed from necrotic cells, and a slight increase was observed at early times in the control cells that were not exposed to any treatment, which does not show strong statistical significance (p>0.05).The mean pixel value at each time point was calculated by averaging all the pixels within the entire field-of-view of each corresponding image.All the statistical analyses reported in this work were performed using the imbedded functions of Matlab (MathWorks).The p-values were calculated based on unevenly sampled and double-tailed Student's t-test.An increase of fluorescence lifetime, which is primarily associated with reduced nicotinamide adenine dinucleotide (NADH), at the early stage of apoptosis (within a few hours post-induction) in cell cultures has been studied previously [6].This finding is readily explained by the elevated mitochondrial metabolism that provides energy required for the processes during apoptosis.However, the further increase of the fluorescence lifetime observed in our study is different from the decrease of the lifetime in cell cultures at the late stages of apoptosis.This discrepancy could stem from intrinsic differences between tissue and cell cultures.For example, the late increase of the lifetime originates primarily from the extracellular matrix (ECM) that is not present in cell cultures.A slight increase of fluorescence lifetime at early times in control cells could indicate increased metabolism under control condition.However, differences between apoptosis and control keratinocytes are clearly shown by the spectral changes over time (Fig. 3(c)), retrieved from the spectral plots at different time points.A peak shift toward longer wavelengths and spectral broadening are seen in control cells after 18 hours.These observations are similar to the spectral changes found when imaging from the granular layer to the stratum corneum (Fig. 2(d)), where increased keratin production is expected to be associated with cornification of keratinocytes at the upper granular layer.In contrast, there is no considerable spectral change observed from apoptotic keratinocytes, suggesting that drug-induced apoptosis does not have the same molecular changes as normal keratinization.The spectral broadening in necrotic cells is due to the relative decrease in the concentration of the NADH which significantly reduced overall fluorescence intensity (Fig. 12(b) in Appendix).The fluorescence intensity changes associated with apoptotic keratinocytes from the granular layer are shown by the two MPM images (Fig. 4(a)) acquired before and 7 hours after the induction, and the mean pixel values from MPM images at different time points under the three observed conditions (Fig. 4(b) and Figs.10-12 in Appendix).Over time, the fluorescence intensity becomes stronger during apoptosis and weaker during necrosis, compared to control cells, which is consistent with observations from reported results [15,16].In addition, typical morphological changes with apoptosis are seen in the MPM images (Fig. 4(a)).A decrease in cytoplasm area is clearly observed in several cells, showing shrinkage and disassembly of cells.Brightening and blurring of mitochondria granules are also visible over time, which is strong evidence of stimulated mitochondrial activity associated with apoptosis.Ring shaped structures seen close to the nuclei are consistent with observations reported in cell culture [16].All these observations are not seen in necrotic keratinocytes (Fig. 12 in Appendix), where the difference in fluorescence intensity between the cytoplasm and ECM shows a decrease over time.This will be quantitatively analyzed and discussed further in the following section.
Additive information of the cellular structural changes are shown by OCM (Fig. 4(c)) before and 7 hours after the treatments, and in the plots for mean pixel values of time-lapse OCM images (Fig. 4(d)).Because OCM images are based on the light scattering properties of the sample, image contrast is relatively low and the cytoplasm and ECM are not clearly distinguished.However, the nuclei are clearly visible (Fig. 4(c), white circles).To take advantage of these image features, contrast was intentionally increased by the same ratio in the OCM images in Fig. 4(c) to show the morphological changes in the nuclei.Shrinkage of nuclei is observed in apoptotic cells, but not in necrotic cells, which is a characteristic difference between apoptosis and necrosis.While no statistically significant (p>0.05)change in light scattering is seen in apoptotic and normal keratinocytes, the gradual decrease in scattering is observed in necrotic cells, which probably stems from their continuous disassembly (Fig. 4(d)).Previous OCT studies have reported enhanced back-scattering from apoptotic cells in gels and cell cultures.This discrepancy could be due to the different experimental conditions of the current work.In the previous cell culture work, the OCT signal-to-noise ratio (SNR) is much higher because of the greater difference in refractive index between cells and the culture media or gel, compared to the difference between the cells and ECM in the living tissue in this study [10,11].Therefore, the subtle change in refractive index from apoptotic cell membranes may not be strong enough to be detected by the OCM system, and the changes in ECM could decrease or eliminate the changes in light backscattering in the living tissue.
It is evident from the above analyses that while each modality gives important information about these cellular dynamics, none of them alone has sufficient capability and reliability to fully characterize and differentiate the two cell death processes from the control condition.This is summarized in Table 1.

Quantitative image analysis
To quantitatively analyze the multi-dimensional imaging data, an automatic cell segmentation algorithm was developed.The active-contour based algorithm [34,35] consists of multiple steps that take full advantage of the spatially co-registered multimodal images (see Fig. 5).
The basic principle of an active-contour model is to automatically search for an energy-minimized spline that confines the boundaries of special features in an image.The optimal contour that fits the shapes of interest in the image is determined through an iterative process.
A key prerequisite for the success of the algorithm is the identification of an initial contour that is close to the shape to be segmented.Specifically, the devised segmentation algorithm works as follows: 1) locate the centers of nuclei by thresholding the smoothed OCM images (Image smoothing is based on the median filtering function embedded in Matlab); 2) segment OCM images into regions centered at the locations of the nuclei; 3) segment the nuclei from each single region based on the (region-based) active-contour algorithm; 4) correlate the centers of the segmented nuclei with the spatially co-registered MPM (or FLIM) images; 5) segment the MPM (or FLIM) images into regions centered at the locations of the coregistered nuclei centers in the OCM images; 6) segment the cytoplasm from each single region based on the (region-based) active-contour algorithm; 7) if the preceding procedures fail to segment sufficient number of cells due to the intrinsic low contrast of the image, manually locate more cells and run the segmentation algorithm following the procedures #2-#7.During the manual selection of the cells, only initial contours are provided by the user, and the segmentation is eventually completely through the automatic interactive process of the algorithm.Multiple parameters, such as fluorescence intensities, lifetimes, and sizes/areas of the nuclei and cytoplasm were calculated based on segmented cells.Based on a number (15-20) of segmented individual cells (Fig. 6(a)), average changes of different parameters were calculated (Fig. 6(b)-6(d)).The fluorescence intensity ratios between cells and ECM decreased for all three conditions (Fig. 6(b)), indicating the leakage of cellular contents into the ECM and the loss of cellular function during the cell death processes.
At the same time, the area of cytoplasm decreased in apoptotic and control cells, but increased in necrotic cells although not statistically significant (Fig. 6(c)).The decrease in cell size suggest cell shrinkage effects associated with apoptosis, which is consistent with the fact that apoptotic cells experience consecutive phases of cell shrinkage and blebbing which eventually lead to cell dissolution.This is also confirmed by the decrease in the fluorescence intensity ratio between the cytoplasm and ECM (Fig. 6(b)).Slight increase (no statistical significance) in cell size is observed in necrotic cells, which could be due to swelling associated with the initial stages of accidental cell death.The overall decrease in nuclear area (Fig. 6(d)) agrees well with nuclear condensation in apoptotic cells.In contrast, nuclear area (retrieved from OCM images) in necrotic cells shows no significant change, consistent with the OCM data at 0 hour (before the drug application) and 7 hours after the drug application (Fig. 4(c)).Nuclear area in control cells undergoing keratinization decreases gradually, reflecting the dissolution of nuclei over time.

Responses from different epidermal layers
The high axial resolution and optical sectioning capabilities provided by all the imaging modalities enable the investigation of cells in different physiological states from different epidermal layers (Fig. 2).This is important because it was found from UV radiation treated keratinocyte studies that different apoptotic responses appear in different epidermal layers [36].The overall fluorescence lifetime increases with time during apoptosis in both the spinous and basal layers (Fig. 7(a) and 7(b)), but the magnitude of change is smaller compared to the granular layer (Fig. 3(b), blue trace).The lifetime from the spinous and basal layers shows a relatively modest increase under control conditions, which is similar to the observation in the granular layer (Fig. 3(b), black trace).Interestingly, in the case of necrosis, the pattern is reversed from a decreasing trend in the granular layer (Fig. 3(b), red trace) to a slightly increasing trend at the level of the basal membrane (Fig. 7(b)).Considering the diffusion effect of the necrosis-inducing chemical (5% peroxide bleach), it could be assumed that a gradient of concentration (i.e.lower concentration at deeper layers and higher concentration at upper layers) was created since the drug was introduced topically.It is possible that the lower concentration of bleach in deeper layers might have no effect (Fig. 7(a)) or might even induce apoptosis (Fig. 7(b)), rather than necrosis [6].No statistically significant change in the fluorescence ratio was found in control cells in the spinous and basal layers (Fig. 7(c) and 7(d)), compared to the decrease shown in the granular layer (Fig. 6(b), black trace).This supports the observation that cell disassembly is not taking place in the inner epidermal layer under control condition.The overall trend of decreasing fluorescence intensity ratio is seen in both apoptosis (not statistically significant) and necrosis in both deeper layers (Fig. 7(c) and 7(d)).However, the magnitude of the changes is smaller in deeper layers than in more superficial layers.These results show the different dynamics in different epidermal layers, which is associated with different physiological states of keratinocytes, although the effects related to drug diffusion could be considered as another possible explanation for these observations.

Histological validation
The histology results demonstrate the appearance of cells at the same time point (18 hours after the treatments; Fig. 9).Nuclear condensation is seen in apoptotic cells (Fig.

Discussion and conclusion
Because of the crucial and distinctive roles of apoptosis and necrosis, detection and differentiation of these cell death processes are important for investigations of developing tissues and organisms under pertinent physiological and pathological conditions.Studies of different cell death dynamics have been made possible with the advance of new biological technologies and methodologies.Our knowledge of the molecular mechanisms of cell death processes has increased rapidly, particularly from studies using well-controlled cell cultures and based on high-resolution imaging techniques with immunochemical or fluorescence staining.Nevertheless, our capability of real-time detection and differentiation of different cell death dynamics in more complex 3D tissue environments has been limited [5,21].Since many of the current imaging technologies used for in vivo studies (such as MRI [18] and ultrasound imaging [19]) do not offer sufficient resolution, optical imaging has been intensively studied for real-time longitudinal monitoring of apoptosis and necrosis [5,15,16,21].Nevertheless, because each individual modality has its own inherent limitations, there has been strong skepticism about the feasibility of using label-free optical imaging approaches for in vivo detection of cell death processes in tissue environments.For example, as shown by our data, fluorescence based methods (such as MPM and FLIM) can only detect specifically labeled or autofluorescent molecules or structures, thus only providing limited information, and leaving many other properties of the biological processes undetected.Likewise, the dynamic morphological transformations of cells (such as nuclear condensation) which are important hallmarks of cell death, may not be observed by MPM and FLIM.Intrinsic contrast, such as light scattering measured by OCM, can provide global information on the cell and tissue structures, but its low contrast and lack of molecular specificity make it insufficient to distinguish different cell death dynamics when used alone.
As we have shown in this study, full characterization and differentiation of these three cell death processes of keratinocytes was enabled by the combination of multiple optical imaging modalities, which provide complementary, additive, and mutually-validating information of the sample, and thus compensate for the specific deficiencies of each single modality.The quantitative image analysis and classification process, which is only made possible based on multiple dimensional imaging data, allowed us to differentiate and longitudinally monitor differences between apoptosis and necrosis in this living three-dimensional engineered tissue.Unlike standalone fluorescence based approaches which only detect exclusively targeted signaling proteins and suffer from potential problems such as complicated synthesis, slow clearance, high background signal, and non-specific binding [21], this label-free imaging method demonstrated the potential for real-time tracking of multiple parameters of cells and tissues under apoptosis or necrosis.Immediately, this imaging platform could be used for monitoring the health of engineered human skin during growth, and following grafting.It is also possible to use this platform to determine the efficacy of skin cancer therapy, or the tissue response from a wide range of dermatological disease treatments, as well as for drug delivery and tracking.Although the imaging penetration depth is limited to within a few hundreds of microns, advances in optical beam delivery systems, including fiber-optic catheters [39], endoscopes [40], needles [41] and hand-held probes [42], may make it possible to perform similar studies on internal tissues or organs, providing new and valuable information on in vivo cells and tissues undergoing these natural and induced cell death processes.

Appendix:
Figures 10, 11, and 12 show time-lapse images of human keratinocytes at the granular layer in a living engineered skin tissue sample under the conditions of control, apoptosis, and necrosis, respectively.The color-coded OCM, MPM, and FLIM images show the changes in light scattering, fluorescence intensity, and fluorescence lifetime, respectively, which exhibit the biological characteristics related to the different cell death processes.Fluorescence spectra at representative time points were retrieved from the multichannel FLIM data to reveal the molecular properties of the sample.To further analyze the imaging data, the mean pixel value of each image was calculated by averaging all the pixels within the entire field-of-view of each corresponding image.Figure 13 shows the classification results based on the MPM and FLIM channels only.

Fig. 1 .
Fig. 1.Schematic drawing of the multimodal microscope and the sample culture chamber.

Fig. 2 .
Fig. 2. Multimodal images of a living engineered human skin sample.Color-coded (a) OCM, (b) MPM, and (c) FLIM images from epidermal layers at depths of 60 µm, 70 µm, 90 µm, 110 µm, and 125 µm, respectively, measured from the surface of the sample.(d) Integrated fluorescence spectra at corresponding depths.(e and g) 3D rendering of the sample reconstructed from MPM and OCM images, respectively.(f and h) Cross-sectional images of the sample reconstructed from MPM and OCM images, respectively.(i) Overlaid image from the three modalities.Scale bars are 50 µm.The color bar for the FLIM images represents 0.2 -2.5 ns (brown-to-blue).

Fig. 3 .
Fig. 3. Time-lapse FLIM imaging of keratinocytes from the granular layer of an engineered human skin sample under conditions of control, apoptosis, and necrosis.(a) FLIM images at 0, 3, 7 and 18 hours after treatment.(b) Dynamic changes of averaged fluorescence lifetime under different conditions.Each data point in (b) are the mean values of the entire field-ofview of corresponding FLIM images at each time point.(c) Fluorescence spectra recorded at 0 and 18 hours after treatment for control (upper panel), apoptosis (middle panel), and necrosis (lower panel) cells.The spectra were retrieved from the FLIM data and integrated over all pixels composing the whole field-of-view of the corresponding image.The three asterisks (***) indicate the statistically significant difference p < 0.001, which was based on the data collected at 0 hour and 18 hours.The scale bar is 50 µm, which applies to all the images shown in this figure.

Fig. 4 .
Fig. 4. Time-lapse MPM and OCM imaging of keratinocytes from the granular layer of an engineered human skin sample under conditions of control, apoptosis, and necrosis.(a) MPM images at 0 and 7 hours after apoptosis treatment.(b) Dynamic changes of averaged fluorescence intensity under different conditions.(c) OCM images at 0 and 7 hours post treatment under three different conditions.White circles highlight three randomly selected nuclei showing their morphological differences between different conditions.(d) Dynamic changes of averaged light scattering under different conditions.Each data point in (b) and (d) are the mean values of the entire field-of-view of corresponding MPM and OCM images at each time point, respectively.The three asterisks (***) indicate the statistically significant difference p < 0.001, which was based on the data collected at 0 hour and 18 hours.The scale bar is 50 µm, which applies to all the images shown in this figure.

Fig. 5 .
Fig. 5. Active contour-based cell segmentation procedure.(a) Flow-chart of the active-contour based cell-segmentation procedure.(b) Typical OCM and MPM images illustrating the multistep segmentation procedure.#1) Typical OCM image of keratinocytes.#2) Smoothed OCM image.#3) Thresholded binary OCM image with typical centers of nuclei.#4) Four typical cropped regions centered at the four nuclei in (#3).#5) OCM image with segmented nuclei (green channel) by an active-contour based segmentation algorithm.#6) Spatially co-registered MPM image.#7) Smoothed MPM image.#8) Four typical cropped regions centered at the four nuclei in (#3).#9) MPM image with segmented cytoplasm (green channel) by an activecontour based segmentation algorithm.#10) MPM image showing segmented cells and cell regions, along with several additional manually-identified cells (red channel) that have also been processed with an active-contour based segmentation algorithm.

Fig. 6 .
Fig. 6.Quantitative analysis based on automatically segmented cells.(a) Typical OCM and MPM images with segmented nuclei and cytoplasm, respectively.Green lines: Automatically identified cells based on the spatial co-registration of the two images; red lines: manually identified but automatically segmented cells.(b) Dynamic changes of fluorescence intensity ratio between cytoplasm and extracellular matrix.(c) Area of cytoplasm region.(d) Area of nuclear region.Each data point is the average value from approximately 15-20 single segmented cells, and the error bars represent respective standard deviations.The value of fluorescence intensity was calculated by averaging the values of all pixels covered by the cell cytoplasm and the adjacent ECM, respectively.The cytoplasm and nuclear areas were determined by the number of pixels covering each of these regions in each image.The three asterisks (***) and two asterisks (**) indicate the statistically significant differences p < 0.001 and p < 0.05, respectively, which were based on the data collected at 0 hour and 7 hours.

Fig. 7 .
Fig. 7. Dynamic changes of fluorescence lifetime in control, apoptotic and necrotic cells from (a) spinous and (b) basal layers.Each data point is the average value of the regions covered by approximately 15-20 segmented single cells in the corresponding FLIM images at each time point.Standard deviations are shown by the error bars.Dynamic changes of average fluorescence intensity ratio between the cytoplasm and ECM are shown for the (c) spinous and (d) basal layers for all three conditions.The value of fluorescence intensity was calculated by averaging the values of all pixels within the cell cytoplasm and the adjacent ECM, respectively.The three asterisks (***) and two asterisks (**) indicate the statistically significant differences p < 0.001 and p < 0.05, respectively, which were based on the data collected at 0 hour and 7 hours.

A
linear classification-based algorithm was devised to enhance the differentiation of the cell death processes.The basic principle of the algorithm is a semi-automatic search of a transferring matrix of weighting factors, , , l m n W , based on the evaluation function, from all three modalities and the advantage over images obtained with only a single modality.Quantitative processing and statistical analyses were performed by considering the pixels in each respective image as samples (Figs.5(b) and 5(c)).Results show high-confidence classification of apoptosis and necrosis at 18 hours with high sensitivity and specificity.The sensitivity and specificity values were calculated based on the classification evaluation function (classperf) embedded in the Matlab software.

Fig. 9 .
Fig. 9. Histology (hematoxylin and eosin staining) of the engineered human skin showing (a) control, (b) apoptotic, and (c) necrotic keratinocytes.Upper and lower panels show lower and higher magnification, respectively, of the areas within the blue frames.The scale bar is 50 µm, and applies to all images.SC, stratum corneum; SG, stratum granulosum; SS, stratum spinosum; SB, stratum basale; DM, dermis; blue arrows, condensed (picnotic) nuclei; yellow arrows, detachment of apoptotic cells from adjacent cells.