SpeckleNN: a unified embedding for real-time speckle pattern classification in X-ray single-particle imaging with limited labeled examples

This paper presents a neural network model called SpeckleNN for real-time classification of X-ray single-particle imaging (SPI) speckle patterns, ideal for use in high-data-rate facilities like the European XFEL and LCLS-II-HE. It possesses the capability to carry out few-shot classification on novel samples and demonstrates robust performance even in the presence of substantial missing detector area, rendering it an excellent candidate for real-time high-throughput SPI experiments.


I. INTRODUCTION
Single-particle imaging (SPI) with X-ray free-electron lasers (XFELs) is a promising method for determining the threedimensional structure of noncrystalline nanoscale particles at room temperature.In SPI experiments, femtosecond coherent X-ray beams strike biomolecules injected into the beam path, causing radiation damage-free scattering of the samples inflicted by the intense X-rays.This way of collecting scattering datasets is known as diffraction before destruction [Aquila et al., 2015, Chapman et al., 2006, Neutze et al., 2000, Reddy et al., 2017, Seibert et al., 2011].Such scattering patterns are also referred to as "speckles" due to its grainy appearance.A single particle of interest can then be reconstructed by algorithms, such as EMC [Ayyer et al., 2016, Loh andElser, 2009] and M-TIP [Chang et al., 2021, Donatelli et al., 2017], from hundreds to tens of thousands of speckle patterns.
of the data being collected during beam time and require additional time for model training which precludes real-time classification.Those models are made of largely two components: (1) Some convolutional neural networks (CNN) for spatial feature extraction; (2) Some fully connected networks (FCN) for compressing CNN features into probability distribution of possible outcomes.These models demonstrated good performance on speckle patterns of one single-particle sample, bacteriophage PR772, which is an important step towards the goal of near real-time particle classification.But when it comes to a different sample, they will have to be retrained on hundreds of labeled speckle patterns.Notably, it is not a lack of computing power that precludes these model from working in near real-time, since we can run deep learning models on supercomputers with modern graphics processing units (GPUs).The bottleneck is speckle pattern labeling.It is by no means a trivial task to label speckle patterns, especially at scale, even for experts in the field.Therefore, we need solutions that enable neural network models to effectively classify speckle patterns without excessive manual labeling.
We aim to address the problem of near real-time speckle pattern classification by converting it into the task of measuring speckle pattern similarities.To accomplish this goal, we propose to train a neural network model to learn an embedding function, capable of mapping speckle patterns into a unified embedding vector space.An important property of this vector space is that similarities can be evaluated by computing the Euclidean distance between any two points in the space.Then, we classify unknown examples by comparing them to a few labeled examples per class in the embedding vector space and assigning the label of the closest class.
A contrastive approach, known as twin neural networks, is used for training the unified embedding model.The main idea is that two identical networks will extract features from a pair of examples, either with the same label or different labels.For two examples with the same label, their Euclidean distance should be small and vice versa.Contrastive approaches based on twin neural networks have achieved early success in computer vision tasks such as signature verification [Bromley et al., 1993] and face verification [Chopra et al., 2005, Schroff et al., 2015].Moreover, two twin neural networks can work together to collectively train the underlying embedding network by minimizing the Euclidean distance between identically labeled examples, while simultaneously maximizing the Euclidean distance between differently labeled examples.This approach is also referred to as triplet networks detailed in the face verification model [Schroff et al., 2015].
In this work, we present SpeckleNN as a unified embedding network for classifying speckle patterns in real-time X-ray single particle imaging.Concretely, two classification solutions are proposed, one with offline training and one with online training: • First, we show that SpeckleNN accurately classifies speckle patterns with few labeled examples per category (e.g. 5) by learning a unified embedding function from a vast number of distinct samples (e.g.different proteins).
The model can be trained entirely offline or prior to data collection, and its classification capability generalizes to new samples.

A. Speckle pattern classification
The task of single-particle speckle pattern classification often requires expert knowledge and immense manual effort.Human-engineered feature extractor and unsupervised learning came along to tackle this challenge.For instance, spectral clustering [Yoon et al., 2011], principal component analysis (PCA) and support vector machines (SVM) [Bobkov et al., 2015] were used for single-hit classification.Geometric machine learning is a supervised learning solution based on the diffusion map framework that can output a score for how likely a speckle pattern is a single-hit [Cruz-Chú et al., 2021].More recently, artificial neural network models have become a new avenue for exploring classification solutions with the advent of capable infrastructures (GPUs, machine learning frameworks) for model training.[Shi et al., 2019] uses a CNN for feature extraction and couples its last layer with two additional fully connected (FC) layers that perform binary classification, which achieved an accuracy of 83.8% in predicting single-hits.More recently, another neural network based hit classifier is proposed by [Ignatenko et al., 2021].They repurposed YOLO (You only look once) deep learning models [Redmon andFarhadi, 2016, 2018] from detecting objects to classifying speckle patterns.In fact, these YOLO models also consist of a CNN spatial feature extractor and several FC layers to compress features into the probability of classes and location of objects.However, these models cannot be directly used to classify speckle patterns of previously unseen single-particle samples without example relabeling and model retraining.Their performance with missing detector area is also unknown.YOLO models, specifically, come with extra complexities, such as requiring bounding boxes as labels and increasing computational cost for finding the location of a speckle pattern.

B. Similarity metrics in a unified embedding vector space
To the best of our knowledge, there is currently no solution that directly maps single-particle speckle patterns of distinct samples to a unified vector space, where similarity is characterized by Euclidean distance.However, the idea of introducing similarity metrics in a unified vector space is not new.One of the early examples was signature verification using a twin neural network [Bromley et al., 1993].During training, a twin neural network works on two signatures simultaneously.During verification, only one half of the twin neural network is used to map input signatures into a vector space.The output embedding will be compared with previously stored signature embedding in this unified vector space.The stored embedding that is closer to the input embedding is considered to share the same label as the input, thereby, the label of this stored embedding becomes the predicted label of the input.Similar twin neural networks were later used to train models for face recognition/verification [Chopra et al., 2005].Then, triplet neural networks [Hoffer and Ailon, 2014] were applied to further enhance the unified embedding models [Schroff et al., 2015] by training essentially two twin-neural networks on both positive and negative examples simultaneously instead of only one of them.In addition to twin neural networks and its variants, many embedding models have been explored for fewshot classification.[Vinyals et al., 2017] introduced "matching networks" that map queries and supports to a unified vector space with two independent embedding functions.[Snell et al., 2017] proposed that a unified embedding model can be trained with "prototypical networks" so that the embedding of unseen inputs is more likely to be closer to the correct "prototype", defined as the mean of the embedded supports of the same category.

III. METHODS
Our speckle pattern classifier uses a unified embedding model to measure pattern similarity through Euclidean distance in the embedding space.This model is trained by two twin neural networks simultaneously.One twin neural network processes two matching examples that share the same label, while the other works on two opposing examples with different labels.Such dual twin neural networks can be further simplified to triplet networks when the matching pair and the opposing pair share a common example.The common example is referred to as anchor, and the matching example and the opposing example are referred to as positive and negative, respectively.The complete triplet network architecture is summarized in Fig. 1.In this section, we present the details in model training with triplet networks, including the embedding model, the loss function and the selection of triplet examples.Additionally, we outline the steps for speckle pattern classification.

A. The embedding model (the vision backbone)
Our embedding model consists of two convolutional layers that extract spatial features and two fully connected layers that compress these features into a low dimensional vector, or embedding.The detailed architecture of the embedding model is delineated in Fig. 2. The first convolutional layer uses a 5×5 single channel filter with a stride of one and no padding.The second convolutional layer employs a 32-channel 5 × 5 filter with a stride of one and no padding.A ReLU (rectified linear activation unit) activation function is applied to the outcome of each convolutional layer, which is followed by a batch normalization layer and a max-pooling operation performed by a 2×2 filter with a stride of two.Then, two fully connected layers are used to generate the final embedding with the size of 128.This way, speckle patterns are encoded into embeddings in a low dimensional vector space, where the similarity of two embeddings can be evaluated by their squared L2 distance.The channel number of each data representation is marked on the top row.
The type of spatial dimension, such as 2D tensors or 1D tensor of neurons, is also annotated for each data representation at the bottom row.Notably, the initial spatial dimension of the input may change if cropping and resizing are applied.

B. Triplet loss function
We train the embedding model in each twin neural network by using a triplet loss function described in [Schroff et al., 2015].Each input consists of a triplet of training examples, which are called anchor x a , positive x p and negative x n .Anchor has the same label as positive but not negative.Together, (x a , x p ) forms a matching pair, while (x a , x n ) forms an opposing pair.During training, three embedding models (CNN+FC) f with shared weights map each element in a triplet (x a , x p , x n ) into a unified embedding vector space, respectively.The objective of training is to separate the two embeddings in each opposing pair by at least a margin of α from the embeddings in the corresponding matching pair in the vector space.Given N triplets, the training objective can be stated as (1) Meanwhile, we enforce that every embedding has a unit length of one in a d-dimensional vector space, namely f (x) ∈ R d and f (x) 2 = 1.It means that any speckle pattern will be mapped to a single point on a d-dimensional hypersphere with the radius of one.The largest value of a possible α is 4 in the squared L2 norm sense.
To facilitate the training, the objective in Eq. ( 1) is turned into the triplet loss function in Eq. ( 2). (2) where [•] + returns zero unless the input value is positive.

C. Selection of semi-hard triplets
A triplet can be randomly selected in three steps: (1) Randomly choose a class; (2) Randomly sample two unique examples from the chosen class; (3) Randomly sample one example from any class other than the chosen class.This method, despite being easy to implement, might not deliver fast convergence when there are too many easy triplets.To explain it in detail, we consider three kinds of triplets that might exist during model training: easy, semi-hard and hard, as shown in Fig. 3(a).In an easy triplet, the negative example is already separated by at least a margin of α than the positive example.In a hard triplet, the negative example is actually closer to the anchor than the positive example.In a semihard triplet, the negative example is farther away from the anchor than the positive example with a margin smaller than α.The problem with easy triplets is that they contribute to zero in the triplet loss, and thus the model weights will be adjusted only according to other triplets, namely semi-hard and hard triplets.The problem with too many hard examples is that they mostly constitute only a small fraction of the whole population.If the optimization prioritizes separating them from their corresponding anchors, the loss function will more likely get stuck in some bad local minima.Therefore, selecting semihard triplets for training is important.This strategy does not mean to ignore hard examples at all.Instead, once a hard example is pulled into the semi-hard zone, optimization can further drive them into the easy zone.This allows majority of the negative examples to stay away from their anchor by a considerable margin α.From a practical standpoint, the selection of semi-hard triplets is done at the mini-batch level, where our model randomly selects a triplet that satisfies the following condition.
(3) However, complexity arises with semi-hard selections involving multiple single-particle samples.We choose to select random anchor x a and positive x p from the same sample with the same label, with the negative x n selected from any sample with a different label.Under this selection scheme, we lay out all scenarios for semi-hard selections when two unique singleparticle samples (Particle X and Particle Y) are present, as shown in Fig. 3(b).
On the contrary, it is considered an easy negative example when it goes outside the outer arc, where f ( Lastly, it becomes a semi-hard negative example when it resides in the area bound between the two arcs.Moreover, the loss function results in L = α and L = 0 when x n semi-hard is on the inner arc and outer arc, respectively.Our model training will pull x n semi-hard close to the outer arc as much as possible, namely minimizing the loss.(b) An illustration of possible semi-hard scenarios when two unique single-particle samples are involved.

D. Optimization
We trained our neural network models using Adam [Kingma and Ba, 2017] with a learning rate of 10 −3 .The model weights are initialized to random values from a Gaussian probability distribution with a mean of 0.0 and a standard deviation of 0.2.

E. Data augmentation
Data augmentation is widely used in many machine learning tasks to address limitations imposed by expensive humanlabeling and improve model performance.In essence, "a data-augmentation is worth a thousand samples" [Balestriero et al., 2022].We applied four data augmentation strategies to each speckle pattern in our dataset, including random inplane rotation, random masking, random zooming and random shifting in both horizontal and vertical directions.Random in-plane rotation mimics the effect of single-particle rotation.Random masking covers some area of a speckle pattern with constant-value pixel intensities to be more robust to bad pixels and parasitic scattering.Random zooming and random shifting enforce the model to learn features independent of detector distance, X-ray wavelength, and X-ray beam center.These data augmentation strategies expand the data distribution for the model without manual labeling.
An important caveat when applying data augmentation is to partition the data into training set and test set before the augmentation.Otherwise, it will lead to "data leakage" as explained in [Kapoor and Narayanan, 2022].One consequence of "data leakage" is the deceptively good model predictive performance measured on a test set that already contains data augmented or "leaked" from the training set.In other words, the "good" performance will be mostly caused by model memorization or overfitting rather than generalization.

F. Four steps in classification
Our model maps speckle patterns into a unified embedding space, without directly predicting labels.Instead, label prediction is performed in a query-against-support manner, that is, comparing inputs (queries) to labeled examples (supports).This approach is also referred to few-shot classification, often implying novel classes for queries and supports.In As mentioned, training offline model requires speckle patterns from a variety of distinct samples.To this date, successful 3D reconstructions from single particle datasets are primarily from large viruses, such as mimivirus [Seibert et al., 2011], rice dwarf virus [Hajdu et al., 2016], and bacteriophage PR772 ?A queried speckle pattern is shown in the first row that has a ground truth label of single-hit.The second and third rows represent the single-hit and multi-hit support sets, respectively.5 patterns are used in each support set.The queryto-support distance is annotated at the bottom left corner of each speckle pattern.The average query-to-support distance is denoted as d.The single-hit class has a shorter d than the multi-hit class, resulting in a single-hit label for the query pattern.Additionally, a question mark in the first row indicates a to-be-determined label of a queried particle.To illustrate a single-hit example, we use a simple cartoon representation of a 6N38 molecule in the second row.Likewise, a multi-hit example is depicted by a cartoon representation of two 6N38 molecules with distinctly colored outlines.[Li, 2020].Therefore, we elected to demonstrate the offlinetrained model on simulated data.Meanwhile, our model is designed to work on one sample of interest when trained online.We chose to use real bacteriophage PR772 data collected at LCLS for demonstration of model training and testing.
A. Offline training 1) Dataset: We randomly selected 100 Protein Data Bank (PDB) entries for model training and validation, and another 345 PDB entries for model testing.The number of atoms in those PDB entries ranges from 10 4 to 10 5 .For every PDB entry, we simulated 400 speckle patterns, with 100 per hit category, from randomly oriented particles in the form of single-hit, double-hit, triple-hit and quadruple-hit.For training purposes, we keep the single-hit label but relabel the rest as multi-hit.The beam profile employed in the simulation has a radius of 0.5 µm and a photon energy of 1.66 keV , and contains 10 12 photons per pulse.We simulated all speckle patterns using skopi [Peck et al., 2022] on a square detector with a dimension of 172 × 172 pixels.To replicate the conditions similar to real experiments, we firstly applied a 6 × 8 pixel binary mask mimicking a beam stop at the center and another 172 × 4 pixel binary mask resembling a gap dividing a detector in the middle.Then, X-ray fluence jitter and shot noise were introduced to the dataset.Specifically, during model training, we introduce fluence jitter by rescaling the intensity of speckle patterns with a multiplier sampled from an experimental photon number distribution shown in Fig. 5.We also added Gaussian noise with zero mean and 0.15 standard deviation.Each speckle pattern is also cropped at the center with a window size of 96 × 96.Data augmentation described in the method section was applied subsequently.The speckle patterns simulated from the 100 PDB entries are split into 70% for training and 30% for validation.With data augmentation, we obtained 30, 000 speckle patterns for model training and another 10, 000 for model validation.We found applying data augmentation can add significant latency to the training process, so decided to cache all speckle patterns into CPU memory.But it is possible to pack even more data for model training through better practices, such as applying data augmentation to a new batch of data while training on the previous batch is still underway.
The test set was formed by simulating speckle patterns from 345 PDB entries.For model prediction, we generated 1, 000 speckle patterns for each PDB entry through random in-plane rotation as the only data augmentation strategy.The main reason is that speckle patterns aren't subject to random masking, random shifting or random zooming as long as the experimental setup stays unchanged.We computed a confusion matrix for each PDB entry and reported accuracy and F-1 scores in the following results.
2) Performance and photon fluence: X-ray photon fluence jitter is often present in SPI speckle patterns.To illustrate how fluence jitter affects our model performance, we scanned a range of fluence scaling factors from 10 −2 to 10 2 by multiplying by 10 0.5 at each step.These scaling factors are then applied to simulated speckle patterns in the test set.Meanwhile, we also measured model performance in three unique few-shot classification scenarios, including 1-shot, 5shot and 20-shot.At the baseline photon fluence, our model achieves an average accuracy of 84.3%, 89.1% and 89.6% with 1-shot, 5-shot and 20-shot classification, respectively.The corresponding F-1 scores are 82.0%,87.4% and 87.9%, respectively.Accuracy and F-1 scores both rise in response to the increase of photon fluence, and converge at about 10 0.6 (≈ 4.0)× the baseline photon fluence.
There are two main lessons we learned from this result.Firstly, the improvement in classification diminishes quickly with increase in support size.For example, 5-shot classification has a much better performance than 1-shot classification, but it delivers a comparable performance as 20-shot classification.If we were to deploy SpeckleNN at an undergoing experiment, it would be more appealing to only label 5 examples per category rather than 1 or 20 examples.Secondly, as free electron laser technology improves in peak brightness [Li et al., 2022], the benefit of having higher photon fluence can directly improve SpeckleNN's accuracy.Interestingly, a 4× photon fluence improvement is sufficient to maximize model performance.3) Performance and particle size: PDB entries vary significantly in particle size from hundreds to millions of atoms, and their sizes are unevenly distributed.It is not a good practice to form a training dataset by randomly picking PDB entries, which might result in many entries aggregated within a small size range.Models trained on such dataset might perform well only on the highly populated size ranges but less so otherwise.We pick roughly equal number of PDB entries from 20 evenly spaced intervals from 10 4 to 10 5 atoms.The distribution of PDB entries over particle size used in our model training is visualized in Fig. 7(a).The PDB entries used in the test set were also sampled from the same size range.
We pointed out that particle size is a limiting factor in model prediction but conditioned on photon fluence.As shown in Fig. 7 (b), the average test accuracy of our model at the baseline photon fluence (1× condition) is positively correlated with particle size in the region of 1 × 10 4 to 3 × 10 4 atoms.The average test accuracy becomes stable when particle size is larger than 3 × 10 4 despite the presence of outliers.Likewise, we repeated model prediction at 100× larger photon fluence.No more size dependent correlation is observed across the whole size range, as shown in Fig. 7(c).Similar observation is also present in F-1 scores as shown in Fig. 7(d  2) Robust classification despite missing detector area: To illustrate robust classification despite missing detector area, we need to choose a baseline model as a reference point for comparison.We decided to use [Shi et al., 2019]'s model for this purpose which we will refer to as Shi19 from here on out.It consists of a CNN vision backbone and a multi-layer perceptron (MLP) that outputs probabilities of each label.It reportedly achieved 83.8% accuracy in predicting single-hits.We reimplemented Shi19 model in PyTorch for measuring its performance.It is worth noting that we need to relabel nonsample-hit and multi-hit as non-single-hit to accommodate the training of Shi19 model, as it was initially designed for binary classification.We still used the original three labels to train our model, and only relabel them when producing compatible confusion matrices.
Performance comparison between models was conducted for two scenarios: (1) 100% detector area is available; (2) 25% detector area is available.The second scenario is more commonly seen in modular detectors, where certain area of the detector needs to be masked out due to spurious noise or damaged panels.Sometimes, data from some detector panels must be completely ignored to reduce computation time and thus allow rapid data collection.If speckle pattern classification can be accurately performed on only a fraction of a detector area, it opens the door to solving the "data reduction" problem that bottlenecks high-throughput single particle imaging experiments.That is to say, it can save a considerable amount of time by eliminating the need for assembling and calibrating all detector panels for the classification process.
Altogether, we randomly selected 345 single-hit and 655 non-single-hit speckle patterns to form the test set, with nonsingle-hit made up of 331 non-sample-hit and 324 multi-hit.SpeckleNN classifies speckle patterns in a 5-shot manner, whereas Shi19 model uses a probability threshold of 0.9 for the classification task.The model accuracy and F-1 scores are summarized in Table I.It is worth noting that data augmentation enhanced the accuracy of Shi19 model significantly, from 83.8% to 98% when 100% detector area is available.Meanwhile, under the same circumstances, SpeckleNN and Shi19 model have the same accuracy and F-1 scores, respectively.But SpeckleNN outperforms the competing model by a large margin when only 25% detector area is available.Fig. 8 and Fig. 9 are demonstrations of 3-way 5-shot classification with SpeckleNN on speckle patterns with 100% and 25% detector area available, respectively.This result suggests that SpeckleNN is a more robust speckle pattern classifier and thus better suited for high-throughput single particle imaging experiments.

V. CONCLUSIONS
In this work, we have introduced SpeckleNN, a unified embedding model for real-time speckle pattern classification in X-ray single particle imaging with limited labeled examples.The embedding model, trained with twin neural networks, can directly map speckle patterns to a unified vector space,  training.

Fig. 1 .
Fig. 1.The triplet network architecture for model training.Three input examples (anchor, positive and negative) are propagated through the triplet neural network simultaneously.Anchor and positive share the same label, thus forming a matching pair.In contrast, anchor and negative do not share the same label, thus forming an opposing pair.The three CNNs and FC layers share the same weights in the triplet network.After examples are embedded to a low dimensional vector space, a triplet loss function is used to simultaneously maximize similarities between matching embeddings and minimize those between opposing embeddings.A side-by-side comparison of three embeddings in a triplet are annotated at the upper right corner.

Fig. 2 .
Fig. 2. The network architecture of the embedding model.Each shaded rectangle is a volumetric data representation in the neural network pipeline.The channel number of each data representation is marked on the top row.The type of spatial dimension, such as 2D tensors or 1D tensor of neurons, is also annotated for each data representation at the bottom row.Notably, the initial spatial dimension of the input may change if cropping and resizing are applied.

Fig. 3 .
Fig.3.An illustration of the three types of negative examples.(a) x a represents an anchor example and x p is a positive example.Two arcs in dashed lines, both centered at x a , are used to divide the embedding space into three areas.The inner arc has a radius of f (x a i )−f (x p i ) 2 2 , whereas the outer arc has a radius that is larger by a margin of α.Negative examples will possess three difficulty levels in model training based on the area where they are situated.It is considered hard negative example if it is located within the inner arc, where f (x a i ) − f (x n i ) 2 2 < f (x a i ) − f (x p i ) 2 2 .On the contrary, it is considered an easy negative example when it goes outside the outer arc, where f (x a i ) − f (x n i ) 2 2 − f (x a i ) − f (x p i ) 2 2 > α.Lastly, it becomes a semi-hard negative example when it resides in the area bound between the two arcs.Moreover, the loss function results in L = α and L = 0 when x n an N -way X-shot classification, N is the number of classes and X is the number of labeled examples per class.The classification takes four steps: (1) We embed an unknown input speckle pattern and all support examples in a unified embedding space.(2) We calculate the Euclidean distances from the input to every support example.(3) We average all distances by class.(4) We rank all classes by average distance and select the class with the shortest distance as the label of the unknown speckle pattern.Fig. 4 demonstrates an example of 2-way 5shot classification.IV.EXPERIMENTS The ultimate goal of SpeckleNN is to accurately classify speckle patterns.Our unified embedding model facilitates the conversion of the speckle pattern classification problem into a set of similarity measures.Here we demonstrate two classification solutions, one with offline training and one with online training.Offline training requires past experimental data to train the model, which is then directly applied to classification of speckle patterns in future experiments.However, online training trains the model solely on newly collected data in an ongoing experiment.Both offline and online training are important for SPI experiments in high data rate facilities.Offline training offers a ready-to-use model for a wide range of samples, while online training provides a potentially more accurate experiment-specific model for the sample of interest.

Fig. 4 .
Fig. 4.An illustration of a 2-way 5-shot classification.The speckle patterns in this figure are simulated from a type VI secretion system (PDB entry: 6N38).A queried speckle pattern is shown in the first row that has a ground truth label of single-hit.The second and third rows represent the single-hit and multi-hit support sets, respectively.5 patterns are used in each support set.The queryto-support distance is annotated at the bottom left corner of each speckle pattern.The average query-to-support distance is denoted as d.The single-hit class has a shorter d than the multi-hit class, resulting in a single-hit label for the query pattern.Additionally, a question mark in the first row indicates a to-be-determined label of a queried particle.To illustrate a single-hit example, we use a simple cartoon representation of a 6N38 molecule in the second row.Likewise, a multi-hit example is depicted by a cartoon representation of two 6N38 molecules with distinctly colored outlines.
Fig.6.2-way X-shot (X = 1, 5, 20) classification performance of our model, as measured by accuracy and F-1 scores under a range of fluence conditions.The baseline fluence is 10 12 photons per X-ray pulse.
: We obtained speckle data from an LCLS experiment of bacteriophage PR772 collected at the AMO instrument (Experiment ID: amo06156, Run numbers: 90, 91,

Fig. 7 .
Fig. 7. (a) The distribution of particle size, as characterized by the number of atoms, among PDB entries used in training the model.(b-e) 2-way 20-shot classification accuracy (b,c) and F-1 scores (d,e) plotted against number of atoms in each PDB entry under two respective fluence conditions, 1 × the baseline fluence and 100 × the baseline fluence.These conditions are labeled on the y-axes.

•
Second, we demonstrate that SpeckleNN achieves accurate and robust speckle pattern classification in the presence of missing detector area (e.g.25% of a pnCCD detector).The model is designed to be trained online or during data collection on a relatively small number of labeled examples per category (around 60).Unlike the offline solution, the online training utilized only the speckle patterns from the sample of interest as opposed to generalize embedding of multiple samples.

TABLE I MODEL
ACCURACY (ACC) AND F-1 SCORES IN TWO SCENARIOS: (1) 100% DETECTOR AREA IS AVAILABLE; (2) 25% DETECTOR AREA IS AVAILABLE.THE PERCENT DETECTOR AREA VISIBILITY IS ANNOTATED AS A SUBSCRIPT IN THETABLE.IN ADDITION, WE PERFORMED 5-SHOT CLASSIFICATION USING SPECKLENN, AND THE PROBABILITY THRESHOLD USED IN SHI19 MODEL IS 0.9.