Performance Metrics for Liquid Chromatography-Tandem Mass Spectrometry Systems in Proteomics Analyses*

A major unmet need in LC-MS/MS-based proteomics analyses is a set of tools for quantitative assessment of system performance and evaluation of technical variability. Here we describe 46 system performance metrics for monitoring chromatographic performance, electrospray source stability, MS1 and MS2 signals, dynamic sampling of ions for MS/MS, and peptide identification. Applied to data sets from replicate LC-MS/MS analyses, these metrics displayed consistent, reasonable responses to controlled perturbations. The metrics typically displayed variations less than 10% and thus can reveal even subtle differences in performance of system components. Analyses of data from interlaboratory studies conducted under a common standard operating procedure identified outlier data and provided clues to specific causes. Moreover, interlaboratory variation reflected by the metrics indicates which system components vary the most between laboratories. Application of these metrics enables rational, quantitative quality assessment for proteomics and other LC-MS/MS analytical applications.


Digestion and characterization of the yeast protein lysate
To prepare the material for LC-MS analysis, the solubilized lysate was digested with trypsin using the procedure of Piening et al., (J. Proteome. Res. 2006, 5, 1527. In brief, the yeast protein (approximately 1 mg/ml in 50 mM ammonium bicarbonate, 8 M urea at pH 8.0), was reduced and denatured at 60 °C for 1 h following the addition of dithiothreitol at a concentration of 15 mM. Next, the proteins were alkylated by adding iodoacetamide to a concentration of 25 mM. After quenching the reaction by the addition of dithiothreitol (DTT) to a concentration of 50 mM, the solution was diluted 8-fold with 50 mM aqueous ammonium bicarbonate. Digestion with trypsin was carried out at 37 °C for approximately 18 h. The digestion was halted by the addition of acetic acid to a concentration of 1% and the sample was stored at -80 °C prior to analysis. All reagents used, including trypsin, were from Sigma and used without further purification.
For LC-MS analysis, varying amounts of the trypsin digest, corresponding to approximately 5, 25 and 50 pmol of total yeast protein, were injected onto an Atlantis dC18 column (3 μm particle size, 300 μm x 150 mm; Waters, Milford, MA). Elution was accomplished with a 2 h gradient from 3 % to 40 % acetonitrile in an aqueous solution of 0.1 % formic acid using a flow rate of 15 μl/min. For the LC-MS analysis, a Waters nanoAcquity LC was coupled to a Waters QTof Premier mass spectrometer. Mass spectra were obtained from m/z 400 to m/z 1600 with a scan time of 1 s. A 300 μM solution of glu-fibrinopeptide in a 50 % acetonitrile, 50 % water was sprayed at 2 μl/min, orthogonally to the LC effluent spray, for lock-mass calibration.
Prior to data analysis, the Waters RAW file was converted to mzXML format using massWolf (http://tools.proteomecenter.org/MassWolf.php). For feature detection/analysis of the LC-MS TIC, msInspect (build # 4603, http://proteomics.fhcrc.org/CPL/msinspect.html) was used. To count the total number of features, the output file generated by msInspect was imported into Excel and the total number of entries was used to determine the total number of features detected.
LC-MS/MS analysis of the digestion was performed on a Thermo LTQ using the original 95 min gradient as described in the SOP optimization. The results were searched using the MyriMatch algorithm and the results run through the IDPicker algorithm to determine the total confidently identified spectra, peptides, proteins and most parsimonious protein groups. The CPTAC Unbiased Discovery Working Group was tasked to design a renewable, standardized set of reference samples for benchmarking LC-MS/MS platforms with respect to performance characteristics relevant to shotgun proteomics-based biomarker discovery. The first CPTAC-wide experiment of this working group was to test a yeast lysate spiked with a human protein mix. Due to complications with the human protein mix that we were to use, we have modified this experiment to include only a single protein, bovine serum albumin (BSA). This initial spiked sample will provide a means for comparing performance of LC/MS/MS platforms amongst the CPTAC participants. In the future, our goal is to develop a performance standard that will provide a means for comparing performance of LC/MS/MS platforms i) as a quality control over time, ii) after the addition of new technologies to evaluate their effectiveness compared to historic data, or iii) between laboratories to inform optimization and troubleshooting.
The experiment has been designed to meet two primary goals. First, the study will provide an assessment of run-to-run and lab-to-lab variability for a complex mixture of proteins. Also, the study will provide a measure of the sensitivity of a mass-spec platform in a yeast lysate. This will be achieved by calculating the power required to detect a two-fold difference for the overall sensitivity of the platform for a given decade of protein abundance.
This proposed standard operating procedure (SOP) was developed after initial testing of several yeast lysate samples at Vanderbilt University. After an initial "dress rehearsal" run, the SOP was re-evaluated and refined to achieve better compliance as well as standardize new parameters that may have led to variance in the initial dress rehearsal runs.

Study Samples:
The yeast lysate and BSA was digested using RapiGest as denaturant and iodoacetamide as alkylating reagent. A 20 h digest was performed at 37 °C with proteomics-grade trypsin. After digestion, both samples were lyophilized and resuspended in 0.1% formic acid in water. The yeast sample has been diluted to a total protein concentration of 60 ng/μL. Therefore, the yeast samples are ready to be analyzed without additional dilution. Both yeast samples (Sample 3-A and 3-B) contains approximately 100 μL. Table 1 contains descriptive and concentration information on the study samples: Sample 3-A unspiked digested yeast 60 ng/μL 0 Sample 3-B BSA digest in digested yeast 60 ng/μL 10 fmol/μL Sample 1-B digested NCI-20 protein mix 0 0 For Study 5, 1 vial of digested, unspiked yeast (Sample 3-A) and 1 vial of digested yeast spiked with a BSA digest (Sample 3-B) was sent to study participants along with 1 vial of digested NCI-20 (Sample 1B), which will be used as a QC sample for the study. Prior to analysis, Sample 1B should be diluted 150-fold with 0.1% formic acid in water.
Below are checklists that outline the HPLC and mass spectrometry parameters of this SOP for both a Thermo LTQ and Thermo Orbitrap. The run order of samples follow the checklists. Participants will adhere as closely to these guidelines as possible with the understanding that not all labs are able to utilize a precolumn and, depending upon their nanospray set up, these chromatographic dimensions (length only) may change somewhat. Derivations from the protocol in this regard will be recorded. All participants should use the Jupiter C18 resin sent to their respective labs.
If possible, participants should pack their columns at 1000 PSI in methanol and then finish packing with the HPLC at 50:50 0.1% formic:acetonitrile on the HPLC for 30 min prior to cutting back the columns to their final dimensions. Please note any discrepancies from this procedure. The column should also be conditioned prior to use with an injection of a concentrated protein digest (such as a BSA digest) followed by several injections of a less concentrated digest. Several blank runs should be performed after conditioning to eliminate carryover of the conditioning digest into the first blank of the study.
It is recommended that each lab run a preliminary test run with the SOP methods and check the parameters recorded in the data file against the check lists provided. This will help to find errors in tune file parameters that can sometimes be tricky to spot. *If using a precolumn, the flow should be directed to waste.

Study 5 LTQ SOP Checklist
Upload Instructions for Study5: Unbiased Discovery, Yeast + BSA Experiment 1.) Browse to https://chemdata.nist.gov/cptac/ (ignore the security warning and click OK. A signed third party certificate is in the process of being purchased to prevent these warnings in the future). 2.) Enter your 'log-in' name and password. (Sent to each contact in a separate e-mail.) 3.) Select 'Study5' from the selector at the top of the navigation bar. (Do not use the Study3 section for these data files). 4.) Find your site and the instrument for which you are uploading data on the navigation bar. Click on 'Yeast + BSA Experiment' under that instrument. 5.) Follow the instructions in the box for Step 3. All data files and any annotation forms should be first copied to a single directory for upload. Please include the completed SOP checklist (above) in that directory. Uploading multiple directories or replicate experiments is allowed. 6.) Enter the name of your SOP contact in the box at Step 4. Click 'Sign.' The 'Upload' hyperlink at Step 5  Incubate at 60 ۫ C for 45 minutes and vortex repeatedly. You'll see solid material "stuck" at the bottom of the vial so I simply vortex to try to get as much of this into solution.
3. Cool to room temperature and alkylate with 10 uL of 100 mM IAM (also prepared in water) and incubate in the dark for 20 minutes.
4. Dilute with enough volume of Ambic pH 8 (360 uL) so the final concentration of TFE is about ≤10 % prior to adding the trypsin.
5. Add trypsin (Promega) in 1:50 ratio. Our trypsin concentration is 0.01 ug/uL in 100 mM Ambic, and based on the average protein concentration of a Sigma 48 vial (around 6.5 ug) I added 20 uL of the 0.01 ug/uL trypsin stock.
Digest at 37۫ ۫ C overnight, dry down and reconstitute in 0.1 % formic acid for LC/MS/MS analysis. I reconstituted in 50 uL = 100 fmol/uL (Note: DTT and IAM were from Sigma.) For Study 6, 1 vial of digested Sigma48 (Sample 6-QC1), 1 vial of digested, unspiked yeast (Sample 6-QC2), 1 vial of digested NCI 20 protein mix (Sample 1B) and 5 vials with varying concentrations of digested Sigma48 spiked into digested yeast (Samples 6A-6E) will be sent to each laboratory. Prior to analysis, Sample 1B should be diluted 150-fold with 0.1% formic acid in water. All other samples do not need be diluted.
Below are checklists that outline the HPLC and mass spectrometry parameters of this SOP for both a Thermo LTQ and Thermo Orbitrap. The run order for the samples follow the checklists.
If possible, participants should pack their columns at 1000 PSI in methanol and then finish packing with the HPLC at 50:50 0.1% formic:acetonitrile on the HPLC for 30 min prior to cutting back the columns to their final dimensions. The column should also be conditioned prior to use with an injection of a concentrated protein digest (such as a BSA digest) followed by several injections of a less concentrated digest. Several blank runs should be performed after conditioning to eliminate carryover of the conditioning digest into the first blank of the study.