Advances in cryo-ET data processing: meeting the demands of visual proteomics

Cryogenic electron tomography (cryo-ET), a method that enables the viewing of biomolecules in near-native environments at high resolution, is rising in accessibility and applicability. Over the past several years, once slow sample preparation and data collection procedures have seen innovations which enable rapid collection of the large datasets required for attaining high resolution structures. Increased data availability has provided a driving force for exciting improvements in cryo-ET data processing methodologies throughout the entire processing pipeline and the development of accessible graphical user interfaces (GUIs) that enable individuals inex-perienced in computational fields to convert raw tilt series into 3D structures. These advances in data processing are enabling cryo-ET to attain higher resolution and extending its applicability to more complex samples


Introduction
Cryogenic electron tomography (cryo-ET) is an imaging modality that allows the visualization of samples in three-dimensions from a set of 2D projection images taken within a limited tilt range (e.g., À60 to þ60 ).Compared to standard cryogenic electron microscopy (cryo-EM), the multiple views allow the visualization of one-of-a-kind pleomorphic objects such as the interior of cells at nanometer resolution.Higher resolution models of frequently occurring molecules within the sample can be obtained by averaging multiple copies of a target of interest, a process referred to as single-particle cryo-ET.This strategy effectively overcomes the low signal-to-noise ratios (SNR) and the missing wedge effect characteristic of cryo-ET data, allowing the extraction of high-resolution information.Previously, sample preparation and data collection for cryo-ET were tedious and time consuming, especially when compared with traditional single-particle cryo-EM.However, recent advances in sample thinning [1e5], parallel data collection [6e10], and montage tomography [11e13] have enabled the production of tilt-series in highthroughput mode, effectively moving the bottleneck to the data processing stage [14].
Cryo-ET data processing is notoriously laborious due to the complexity of the image analysis pipeline, the large storage requirements, the challenges associated with the low SNR, the missing wedge artifacts, and the intrinsic heterogeneity of native samples.However, recent years have seen innovative strategies arise to tackle some of these challenges (Figure 1, Table 1).Notably, hybrid strategies for data processing that combine principles of traditional single-particle cryo-EM with cryo-ET, such as constrained single-particle tomography [6,15e18] and 2D template-matching [19e23] have been very successful.By working directly with 2D data obtained from tomographic samples, these techniques not only achieve higher resolution, but they enable faster processing compared to traditional tomographic approaches.Improvements have also been made during data pre-processing that have resulted in better tomogram reconstructions (see Data pre-processing: from raw tilt-frames to 3D tomograms).Increased contrast in tomograms, for example, has a direct and positive impact on the success of particle picking.Advances in automated particle picking methods have served to increase accuracy and the rate at which structures can be generated by rapidly generating large particle sets (see Finding frequently occurring targets within cellular tomograms).These particle sets are then refined into increasingly high-resolution structures thanks to improvements in single-particle refinement packages (see 3D refinement using constrained single particle tomography).
Conformational heterogeneity of these structures is now able to be explored in situ by leveraging advances in discrete and continuous image classification (see Studying protein dynamics in situ using image classification).Collectively, method development efforts in cryo-ET data processing over the past two years have increased the resolution of structures and enabled tackling important biological problems that were previously intractable.

Data pre-processing: from raw tilt-frames to 3D tomograms
To perform single particle cryo-ET, raw tilts collected as movie frames must be converted into tomograms ready for downstream data processing; we refer to any steps prior to particle picking as pre-processing.This typically includes executing the steps of beam-induced motioncorrection, tilt-series alignment, CTF estimation, 3D reconstruction, and denoising (Figure 1a).The use of robust strategies for movie-frame alignment [24,25] and tilted-CTF determination [6,26e28] is now routine and these methods have been implemented in many tomography packages.The most effective and commonly used strategy for tilt-series alignment relies on the use of fiducial markers that are added to the sample before imaging [25].However, as cryo-ET moves towards imaging focused ion beam (FIB)-milled lamellae where adding gold fiducials for alignment is not possible, the use of patch-based tracking methods has become more prevalent [17,25,29].AreTomo is a recent example of a marker-free GPU-based implementation for tilt-series alignment and reconstruction that is both fast and accurate [30⦁].Availability of image-based techniques for tilt-series alignment further opens the field to explore complex sample types where fiducials cannot be added.TomoTwin is an example of a multi-target approach that detects many species at the same time.(c) After particle picking, 3D refinement is performed using either subvolumes extracted from the tomograms and subjected to sub-volume averaging (SVA), or using particle projections extracted from the tiltseries subjected to constrained single-particle tomography (CSPT).Particles are initially aligned to a low-resolution reference followed by finer downstream refinement that includes dose-weighting, per-particle CTF refinement, and movieframe alignment.nextPYP is a scalable and easy-to-use package for high-resolution refinement using CSPT.(d) The resulting structures can be assessed for heterogeneity by either classifying the particles into discrete conformations or characterizing the presence of continuous heterogeneity.TomoDRGN and cryoDRGN-ET are examples of packages for continuous heterogeneity analysis that were derived from the cryoDRGN approach for single-particle analysis.
Interpretation and denoising of tomograms are challenging due to the missing wedge artifacts caused by the limited tilt-range and the low SNR resulting from fractionating the dose across multiple images in a tilt-series.While these fundamental limitations have proven difficult to overcome, recent algorithmic advances in 3D reconstruction [31⦁⦁,32] and denoising [31⦁⦁e35] have shown potential to mitigate these effects.Of particular interest in this category is IsoNet, a deep learning-based approach that can successfully impute information in the missing wedge and effectively denoise tomograms [31⦁⦁].In general, the success of pre-processing strategies for high-resolution tomography is determined by the level of automation, robustness, and computational efficiency that are needed to analyze datasets with hundreds of tomograms.

Finding frequently occurring targets within cellular environments
Single-particle cryo-ET requires finding the location of repeating molecules within tomograms.Current particle picking strategies can be classified as being either singletarget, seeking only one class of particle per search, or multi-target, seeking multiple types of particles per search.While the majority of current strategies fall into the first category, recent efforts have increasingly focused on the development of multi-target strategies.Particle picking can be combined with geometric information obtained from tomogram segmentation to narrow the search to relevant areas of the sample, for example, to find native membrane proteins (Figures.1b and 2).
To generate the large particle sets required to achieve high-resolution, designing automated methods for particle picking is essential.Different levels of automation can be achieved depending on the amount of user input required.For single-target picking, the strategy requiring the least amount of input is size-based picking [16⦁⦁] ewhich only necessitates knowing the approximate dimensions of the targete, followed by templatematching [16⦁⦁, 27,36e39] that relies on an external 3D reference, and deep learning-based picking [16⦁⦁,40e43] which requires manually selecting a set of particles for training.The number of particles required for a training set will vary based on target size, visibility, concentration, and the supervision level of the deep learning approach.From the low end, in situ bacterial ribosomes picked using a semi-supervised approach, for example, will require w50 particles from 2 to 3 tomograms [44], while fully-supervised approaches typically require in the order of thousands of particles [40,42].Each of these single-target methods has been used to successfully identify in situ targets including ribosomes (size-based, template matching, deep learning) and the photosystem II complex (template matching, deep learning).In general, choosing the most appropriate algorithm for a particular application will depend on whether: 1) the size of the target is known, 2) an external template is available, 3) labels for training can be provided, and 4) sufficient computational resources are secured (as different algorithms have dissimilar run time requirements).Single-target strategies can emulate multi-target strategies through the use of sizebased filter banks, template libraries or multiple training sets representing different protein species.While bonafide multi-target picking approaches based on deep-learning such as TomoTwin [45⦁⦁] or DISCA [46] have the potential to identify multiple protein targets simultaneously with minimal user input, their performance on crowded cellular tomograms has yet to be demonstrated.
When a target of interest is known to exist only in a certain region of the tomogram, such as a membrane, a filament, or inside a specific organelle, geometric constraints obtained from tomogram segmentation can be used to restrict the search to relevant areas of the sample.This additional input serves to reduce false positives in a particle set, which in turn facilitates downstream refinement efforts.Segmentation-guided particle picking can be used in single-[16⦁⦁, 40,47] or multi-target [48,49] methods, and has been successfully applied to pick particles from the surface of virions [16⦁⦁] and cellular membranes [40,48].A variety of methods exist to identify filamentous structures, though not all of these are considered segmentation guided as some do not perform initial tomogram segmentation prior to identification of filaments [37,40,41,43].

3D refinement using constrained singleparticle tomography
Once particles have been located, traditional subvolume averaging (SVA) consists in extracting subvolumes from the tomograms and subjecting them to alignment and averaging to improve resolution (Figure 1c).More recently, constrained single-particle tomography (CSPT) strategies that use the raw 2D projections from the tilt-series have become more prevalent [15e18,50], as they feature several advantages over traditional SVA.Namely, CSPT allows leveraging existing strategies for single-particle cryo-EM refinement and reconstruction, for example, by minimizing the number of intermediate operations between the raw data and the final reconstruction to a single step, thus contributing to improve resolution while also being more efficient in terms of storage and speed.Several years after the introduction of CSPT in 2012 [15], emClarity implemented "tomogram constrainedparticle refinement (tomoCPR)", which like CSPT, refines the parameters of the tilt-geometry using particles as fiducial markers [50].Shortly after, EMAN2 implemented the "per-particle, per-tilt" approach which mimicked CSPT but without imposing the constraints of the tilt-geometry during particle refinement and Table 1 Taxonomy of methods for cryo-ET data processing.This table contains a list of packages that execute specific steps of the data processing pipeline along with their main features and functionality.Abbreviations: Weighted Back Projection (WBP), Simultaneous Algebraic Reconstruction Technique (SART), Simultaneous Iterative Reconstruction Technique (SIRT), Convolutional Neural Network (CNN), Content-Aware Image Restoration (CARE), Deep Learning (DL), Template Matching (TM), Uniform Manifold Approximation and Projection (UMAP), Principal Component Analysis (PCA), Multi Reference Alignment (MRA).
without refining the tilt-geometry [17].More recently, M expanded the original CSPT approach by modeling local sample deformations, performing data-driven dose weighting, and per-particle CTF and tilt-frame refinement [18].In 2023, nextPYP further extended CSPT by introducing the use of constrained classification of 2D particle projections to analyze conformational heterogeneity [16⦁⦁].Overview of strategies for particle picking from cryo-ET tomograms.Reconstructed tomograms in 3D are used as input to the particle picking method of choice.Optionally, tomogram segmentation can be performed and the segmented objects be used as geometric constraints to guide particle picking.Methods for particle picking in cryo-ET can be broadly classified into single-target or multi-target, depending on whether they detect one or multiple protein species.Single-target methods include size-based, template-matching, or deep learning strategies.These methods predict a single species at a time as shown in the output tomograms with species A, B, or C identified.Multi-target methods can detect multiple species simultaneously, as shown in the output tomogram with all species A, B, and C highlighted.
While the intrinsic complexity associated with implementing CSPT frameworks has slowed down its adoption in the cryo-ET field, the advent of modern end-to-end pipelines for tomography data analysis is beginning to make these tools more widely available [16⦁⦁,17].CSPT strategies are particularly effective at: 1) compensating for local sample deformations and distortions occurring during imaging, 2) refining the parameters of the tiltgeometry and the CTF for each particle, and 3) performing tilt-frame alignment and data-driven exposure weighting to maximize the extraction of highresolution information.Indeed, the combination of these components has allowed the structure determination of targets imaged in situ at resolutions rivaling those obtained by single-particle cryo-EM [16⦁⦁,18].When coordinates for multiple particle species can be obtained from the same set of tomograms, refinement strategies have been implemented that take advantage of the whole complement of particles to improve the resolution of all species [18].Compared to traditional single-particle cryo-EM, CSPT has considerably higher storage and compute requirements due the larger size and increased complexity of analyzing tilt-series data.In terms of storage, raw tilt-series and tomograms alone take up several TBs, and 2D particle projection stacks can roughly double that amount.Refinement operations can take weeks to run on commodity CPU hardware because each particle has multiple projections and additional operations are needed to refine the parameters of the tilt-geometry and keep track of the associated metadata.To deal with these challenges, computational frameworks with higher efficiency have been proposed that either take advantage of graphical processing units (GPUs) [17,18,51], or adopt a distributed approach to 3D refinement and reconstruction that avoids storing large particle stacks and can scale to tens of millions of particle projections which are required to reach high-resolution [16⦁⦁].As the average size of tomographic datasets continues to increase, the availability of computationally efficient frameworks for single-particle cryo-ET will be instrumental for high-resolution studies to become routine.

Studying protein dynamics in situ using image classification
A well-known advantage of cryo-EM and cryo-ET is their ability to discern the discrete and continuous heterogeneity of protein structures.Unlike single-particle cryo-EM, conformation distributions are more likely to be in line with native conditions and therefore molecular dynamic simulations and thermodynamic studies can be performed based on the cryo-ET structural data [52⦁].Before these biophysical tasks can be completed, the heterogeneity present in the sample must first be characterized.Current packages for heterogeneity analysis either aim to classify the data into a discrete set of sub-structures or create models that capture a continuous set of changes (Figure 1d).Both approaches first require building a consensus 3D structure which is subsequently evaluated for variability.Discrete classification strategies have successfully been used to characterize conformational heterogeneity from tilt-series [16⦁⦁, 29,36,50,51,53,54⦁].One advantage of these techniques is that they can update the per-particle alignments during refinement which leads to better separation and higher resolution, especially when the raw 2D data and the geometric constraints are used during classification and refinement [16⦁⦁].Recent examples of packages that study continuous heterogeneity are tomoDRGN [54⦁] and cryoDRGN-ET [53].These deep learning-based frameworks can map the set of 2D projections representing an individual sub-volume into a low-dimensional latent space representation that can then be resampled to generate continuous conformational trajectories [53e55].Despite these advances, it is clear that further method development will be needed to extend the applicability of these strategies to more challenging targets, including low-molecular weight and membrane proteins imaged inside cells.

Discussion
Cryo-ET is an emerging imaging modality that allows the visualization of molecules in their native environment at high resolution.Until recently, sub-nanometer resolution structures were rare and though cryo-ETcould be used to determine conformations previously unobserved by single-particle cryo-EM or X-ray crystallography, highresolution was not a realistic goal of the field.Advances in data collection and processing have enabled remarkable progress to the point where resolutions better than 2 A ˚are achievable for purified samples, structures surpassing 3 A ˚resolution are attainable from in situ samples, and de novo structures can be obtained from cryo-ET data combined with deep learning-based protein structure prediction methods (Figure 3).Current trends in the field suggest that these integrative modeling strategies will be useful for determining protein structures in situ [52⦁,56].For example, sequence-based structure prediction was used in conjunction with cryo-ET to build multiple conformations of the human nuclear pore complex (NPC) at unprecedented rates of completion (>90%) [52⦁].As cryo-ET data processing becomes more accessible through the use of user-friendly packages, we expect an increase in the variety and number of singleparticle tomography structures [17,57e59].Together with tools that allow mapping 3D structures back into the original tomograms [60,61], this will contribute to advancing the field of visual proteomics.Additionally, as multi-target particle picking methods begin to be incorporated into end-to-end pipelines, we envision a future where advances in single-particle cryo-ET data processing will result in entire tomograms being reconstructed at high-resolution.Recent trends in structure determination using in situ single-particle tomography.Representative set of structures determined using single-particle tomography during the last two years.Samples with increasing levels of difficulty are shown along the x-axis, starting from purified samples (orange), in vitro samples (purple), in situ bacterial samples (red), and in situ eukaryotic samples (green).Each structure is labeled with the complex name, final resolution, molecular weight and corresponding Electron Microscopy Data Bank (EMDB) identifier.As the field of cryo-ET continues to develop, the boundaries of what is achievable are beginning to move towards higher-resolution and more complex sample types.

Figure 1 Current
Figure 1

Figure 2 Current
Figure 2

Figure 3 Current
Figure 3