Performance evaluation of the SMG HHV-6 Q Real-Time PCR Kit for quantitative detection and differentiation of human herpesvirus 6A and 6B

ABSTRACT The aim of this study was to compare the performance of the newly developed SMG HHV-6 Q Real-Time PCR Kit (SMG assay) with the RealStar HHV-6 PCR Kit (RealStar assay). The analytical sensitivity and specificity, linearity, and precision of the SMG assay were evaluated. The clinical performance of the SMG assay was assessed and compared with that of the RealStar assay using 207 clinical specimens (HHV-6A positive, n = 51; HHV-6B positive, n = 64; HHV-6A/B negative, n = 92). The limit of detection of the SMG assay was 2.92 log10 copies/mL for HHV-6A DNA and 2.88 log10 copies/mL for HHV-6B DNA. The linear range was determined to be 3.40–9.00 log10 copies/mL for both viruses. Intra- and inter-assay variability were below 5% at concentrations ranging from 4 to 9 log10 copies/mL. No cross-reactivity was observed with the 25 microorganisms included in the specificity panel. The clinical sensitivity and specificity of the SMG and RealStar assays compared to in-house polymerase chain reaction and sequencing were as follows: SMG assay, 98.0% and 100% for HHV-6A DNA, respectively, and 96.9% and 100% for HHV-6B DNA, respectively; RealStar assay, 98.0% and 100% for HHV-6A DNA, respectively, and 90.6% and 100% for HHV-6B DNA, respectively. The correlation coefficients between viral loads measured by the two assays were 0.948 and 0.975, with mean differences of 0.62 and 0.32 log10 copies/mL for HHV-6A and HHV-6B DNA, respectively. These results demonstrate that the SMG assay is a sensitive and reliable tool for the quantitative detection and differentiation of HHV-6A and HHV-6B DNA. IMPORTANCE Quantitative real-time PCR (qPCR) that can distinguish between HHV-6A and HHV-6B DNA is recommended for diagnosis of active infection. The SMG HHV-6 Q Real-Time PCR Kit (SMG assay) is a newly developed qPCR assay that can differentiate between HHV-6A and HHV-6B DNA; however, little is known about its performance. In this study, we assessed the performance of the SMG assay and compared it with that of a commercially available qPCR assay, the RealStar HHV-6 PCR Kit (RealStar assay). The SMG assay demonstrated excellent analytical sensitivity and specificity, precision, and linearity. Furthermore, the viral loads measured by the SMG assay were highly correlated with those measured by the RealStar assay. Our results suggest that the SMG assay is a useful diagnostic tool for quantitative detection and differentiation of HHV-6A and HHV-6B DNA.

HHV-6 is divided into two subgroups, HHV-6A and HHV-6B, with an overall nucleotide sequence identity of 90% (20).Initially considered as variants of the same species, HHV-6, HHV-6A, and HHV-6B, respectively, have recently been recognized as distinct species by the International Committee on Taxonomy of Viruses, based on differences in genetic, immunological, biological, and epidemiological characteristics (21,22).It is well known that HHV-6B is more commonly associated with human diseases than HHV-6A.Primary infection with HHV-6B is the main cause of exanthem subitum, a common childhood disease characterized by high fever and skin rash (1).Reactivation of HHV-6B frequently occurs in HSCT and SOT recipients and is associated with skin rash, cytomegalovirus (CMV) reactivation, hemorrhagic cystitis, acute graft-versus-host disease, and encephali tis (13,14,17,18).In contrast, HHV-6A has not yet been definitively associated with any human disease, and its natural history is largely unknown.
The SMG HHV-6 Q Real-Time PCR Kit (SMG assay; Hwainmedics, Seoul, South Korea) is a newly developed qPCR assay that can differentiate between HHV-6A and HHV-6B DNA.While this assay is now commercially available, it has not yet obtained Food and Drug Administration or Conformité Européenne in vitro Diagnostics (CE-IVD) approval as of the time of writing, and little is known about its performance.In this study, we assessed the performance of the SMG assay and compared it with that of the RealStar HHV-6 PCR Kit (RealStar assay; Altona Diagnostics, Hamburg, Germany), which is a commercially available qPCR assay routinely used in diagnostic laboratories.

Clinical specimens
Whole blood specimens were submitted to the Molecular Microbiology Laboratory at Samsung Medical Center, Seoul, South Korea, for routine HHV-6 testing.DNA was extracted using MagNA Pure 96 (Roche Diagnostics, Rotkreuz, Switzerland), with an input volume of 200 µL and an eluate volume of 100 µL, or KingFisher Flex (Thermo Fisher Scientific, Waltham, MA, USA), with an input volume of 200 µL and an eluate volume of 50 µL, and was subjected to in-house PCR targeting the HHV-6 DNA polymerase gene.The limit of detection (LOD) of the in-house PCR was 3.05 log 10 copies/mL [95% confidence interval (CI): 2.91-3.31log 10 copies/mL] for HHV-6A DNA and 3.10 log 10 copies/mL (95% CI: 2.96-3.33log 10 copies/mL) for HHV-6B DNA.The following primer pair described by Reddy and Manna (28) was used: 5′-CTGACAGACATAAAGATGCTATCC GT-3′ (forward) and 5′-CGGGTTATTGCCGTGTGT-3′ (reverse).For PCR amplification, 1 µL of DNA template and 1 µM of each primer were added to AccuPower HotStart PCR Premix (Bioneer, Daejeon, South Korea).The cycling conditions used were as follows: initial denaturation at 95°C for 5 min, then 40 cycles of 95°C for 20 s, 60°C for 30 s, and 72°C for 30 s, followed by a final extension step at 72°C for 5 min.The PCR products were separated by agarose gel electrophoresis and visualized under ultraviolet light after staining with ethidium bromide.The PCR products showing the expected band of 207 bp on agarose gel were considered positive for HHV-6 and were further subjected to Sanger sequencing using the same primer pair as for PCR amplification and BigDye Terminator (v.3.1)Cycle Sequencing Kit (Thermo Fisher Scientific).The resulting sequences were used to differentiate between HHV-6A and HHV-6B.Residual DNA specimens remaining after routine HHV-6 testing were deidentified and stored at -70°C until use in this study.Two hundred and seven DNA specimens (51 HHV-6A-positive, 64 HHV-6B-positive, and 92 HHV-6A/B-negative specimens) obtained from whole blood of 121 patients [male: 81, female: 40; median age: 48 (range: 0.2-80) years] between September 2017 and May 2023 were thawed and tested in parallel using the SMG and RealStar assays.

Quantitative real-time PCR
Both the SMG and RealStar assays were performed according to the manufacturers' instructions.Briefly, for both assays, 10 µL of sample DNA was added to 21 µL of PCR master mix, giving a total reaction volume of 31 µL.PCR was performed on the CFX96 system (Bio-Rad, Hercules, CA, USA) using the following cycling conditions: initial denaturation at 95°C for 5 min, followed by 45 cycles of 95°C for 15 s and 60°C for 30 s (SMG assay); initial denaturation at 95°C for 10 min, followed by 45 cycles of 95°C for 15 s and 58°C for 60 s (RealStar assay).In each run, four quantification standards provided in the kit (SMG assay: 2, 3, 4, and 5 log 10 copies/µL for both viruses; RealStar assay: 1, 2, 3, and 4 log 10 IU/µL for both viruses) were assayed to construct standard curves.HHV-6A and HHV-6B DNA loads were calculated from the standard curves and expressed as copies per milliliter (SMG assay) or international units per milliliter (RealStar assay) of whole blood.Values given as international units per milliliter were converted to copies per milliliter using the manufacturer's conversion factors (HHV-6A: 1 IU/mL = 0.35 copies/mL, HHV-6B: 1 IU/mL = 0.96 copies/mL).

Analytical and clinical performance evaluation
The analytical performance of the SMG assay was evaluated via determination of analytical sensitivity [LOD and lower limit of quantitation (LLOQ)], analytical specific ity, linearity, and precision.The LOD and LLOQ of the SMG assay were assessed using heat-inactivated HHV-6A culture fluid (catalog no.0810529CFHI) purchased from Zeptometrix (Buffalo, NY, USA) and the 1st World Health Organization (WHO) Interna tional Standard for HHV-6B DNA (NIBSC code: 15/266).These materials were serially diluted in HHV-6A/B-negative whole blood from a healthy volunteer, and 24 replicates per dilution level were tested.The analytical specificity of the SMG assay was assessed using 25 microorganisms (Table S1).Plasmids containing the HHV-6A and HHV-6B target sequences used for determination of linearity and precision of the SMG assay were obtained from Hwainmedics.Linearity was determined by analyzing a 10-fold dilution series of plasmids spiked in HHV-6A/B-negative whole blood, ranging from 3 to 9 log 10 copies/mL, and nine replicates were tested for each concentration.Linear, second-and third-order polynomial regression fits were assessed.The precision of the SMG assay, i.e., intra-and inter-assay variabilities, was determined by analyzing the same 10-fold dilution series used in the linearity study.Intra-assay variability was determined by triplicate testing within the same run, while inter-assay variability was determined from three independent runs on different days.The clinical performance of the SMG assay was assessed and compared with that of the RealStar assay using 207 clinical specimens.

Statistical analysis
The LOD was determined using probit regression analysis.The LLOQ was determined as the lowest concentration equal to or greater than the LOD and meeting the requirement for total analytical error (TAE), calculated as bias +2 × standard deviation, which is ≤0.5 log 10 copies/mL.Simple linear regression was used to assess the relationship between nominal and measured viral loads, and linearity was considered acceptable if the coefficient of determination (R 2 ) was greater than 0.98.Intra-and inter-assay variabilities were assessed by calculating the coefficient of variation (CV).The clinical sensitivity and specificity of the SMG and RealStar assays were calculated using in-house PCR and sequencing as a reference standard.The level of agreement between quali tative results provided by the two assays was determined by positive percent agree ment (PPA), negative percent agreement (NPA), and Cohen's kappa coefficient (κ).The level of agreement between viral loads measured by the two assays was analyzed using Bland-Altman plots, and mean differences and 95% limits of agreement (LOAs) were calculated.The correlation between viral loads measured by the two assays was analyzed using Passing-Bablok regression, and Spearman correlation coefficients (ρ) were calculated.Statistical analysis was conducted using R software (v.4.3.0), and data visualization was carried out using ggplot2 package in R software.

Analytical performance
The LOD of the SMG assay was 2.92 log 10 copies/mL (95% CI: 2.84-3.04log 10 copies/mL) for HHV-6A DNA and 2.88 log 10 copies/mL (95% CI: 2.80-3.00log 10 copies/mL) for HHV-6B DNA (Table 1).The LLOQ of the SMG assay was 3.40 log 10 copies/mL for both HHV-6A and HHV-6B DNA.This was due to the TAE at this concentration being less than 0.5 log 10 copies/mL (0.46 log 10 copies/mL for HHV-6A DNA and 0.49 log 10 copies/mL HHV-6B DNA).In the regression analysis, the firstorder equation provided the best fit.Linearity was acceptable within the concentration range evaluated (3.00-9.00log 10 copies/mL) with an R 2 of 0.999 for both HHV-6A and HHV-6B DNA.Based on the LLOQ (3.40 log 10 copies/mL), the linear range of the SMG assay was determined to be 3.40-9.00log 10 copies/mL (Fig. 1).Intra-and inter-assay variabilities expressed as % CV were below 5% at concentrations ranging from 4 to 9 log 10 copies/mL for both HHV-6A and HHV-6B DNA.However, at 3 log 10 copies/mL, which is the concentration between the LOD and LLOQ of the SMG assay, intra-and inter-assay variabilities were greater than 5% for both viruses (Table 2).In the analytical specificity evaluation, the SMG assay did not exhibit cross-reactivity with any of the 25 microorganisms included in the specificity panel (Table S1).

Clinical performance
When using in-house PCR and sequencing as a reference standard, the clinical sensi tivity and specificity of the SMG and RealStar assays are shown in Table 3.The SMG and RealStar assays yielded sensitivities of ≥96.9% and ≥90.6% for both HHV-6A and HHV-6B DNA, respectively.The specificities of the two assays were all 100% for both HHV-6A and HHV-6B DNA.The agreement between qualitative results obtained with the SMG and RealStar assays is shown in Table 4.The two assays exhibited almost perfect agreement with kappa coefficients of 1.00 (95% CI: 1.00-1.00)and 0.95 (95% CI: 0.91-1.00)for HHV-6A and HHV-6B DNA, respectively.Among the 51 specimens confirmed as HHV-6A positive by in-house PCR and sequencing, 1 was negative by both the SMG and RealStar assays.Among the 64 specimens confirmed as HHV-6B positive by in-house PCR and sequencing, 6 were negative by the RealStar assay, 4 of which were detected at concentrations lower than the LLOQ by the SMG assay (Tables 3 and 4).To rule out DNA degradation, these specimens were retested by in-house PCR and sequencing, which gave the same results as were obtained from routine HHV-6 testing using this method.A total of 79 clinical specimens within the linear measurement range by both the SMG and RealStar assays were used for Passing-Bablok regression and Bland-Altman analyses.The concentrations of HHV-6A and HHV-6B DNA ranged from 3.86 to 7.15 log 10 copies/mL (median 6.37 log 10 copies/mL) and from 3.45 to 7.11 log 10 copies/mL (median 4.64 log 10 copies/mL), respectively, as measured by the SMG assay.Passing-Bablok regression analysis yielded correlation coefficients of 0.948 and 0.975 for HHV-6A and HHV-6B DNA, respectively.Bland-Altman analysis gave mean differences of 0.62 log 10 copies/mL (95% LOA: 0.29-0.94log 10 copies/mL) and 0.32 log 10 copies/mL (95% LOA: 0.02-0.61log 10 copies/mL) for HHV-6A and HHV-6B DNA, respectively (Fig. 2).

DISCUSSION
With the increasing number of patients undergoing HSCT or SOT, the need for quantita tive detection and differentiation of HHV-6A and HHV-6B DNA has become increasingly evident.In recent decades, several qPCR assays that can differentiate between HHV-6A and HHV-6B DNA, including the RealStar assay, have been developed and are routinely used in diagnostic laboratories (28,(31)(32)(33).In the present study, we evaluated the performance of the SMG assay, a newly developed qPCR assay, and compared the results to those of the RealStar assay.Our findings demonstrate that the SMG assay is highly sensitive and accurate and produces results comparable to those obtained using the RealStar assay.
In this study, the SMG assay showed high analytical sensitivity, with LODs of 2.92 and 2.88 log 10 copies/mL for HHV-6A and HHV-6B DNA, respectively.According to the manufacturer's package insert, the LODs of the RealStar assay were 3.17 and 3.13 log 10 copies/mL elute for HHV-6A and HHV-6B DNA, respectively, suggesting that the analyti cal sensitivity of the SMG assay is comparable to that of the RealStar assay.The analytical specificity of the SMG assay was excellent, as no cross-reactivity was observed with 25 non-target microorganisms, including 7 viruses belonging to the Herpesviridae family (CMV, Epstein-Barr virus, human herpesvirus 7, human herpesvirus 8, herpes simplex viruses 1 and 2, and varicella zoster virus).In addition, the SMG assay exhibited a wide linear range from 3.40 to 9.00 log 10 copies/mL for both HHV-6A and HHV-6B DNA.In individuals with iciHHV-6, HHV-6 DNA load in whole blood typically exceeds 6 log 10  copies/mL, which is much higher than the values observed in patients with active infection (3-5 log 10 copies/mL) (1,36,37).The wide linear range of the SMG assay enables reliable differentiation between active infection and iciHHV-6, preventing individuals with iciHHV-6 from being misdiagnosed with active infection and receiving unnecessary treatment.However, individuals with iciHHV-6 may have an HHV-6 DNA load of <5.5 log 10 copies/mL in whole blood when the white blood cell count is significantly decreased (1,37).Individuals without iciHHV-6 may also transiently have an HHV-6 DNA load of >5.5 log 10 copies/mL in whole blood, albeit rarely (37).For these reasons, determination of the ratio of HHV-6 DNA to cellular DNA using qPCR or droplet digital PCR or detection of HHV-6 DNA integrated into human chromosomes using fluorescence in situ hybridization should be used as confirmatory testing (1,24,37).In this study, the SMG assay showed good precision, with intra-and inter-assay variabilities of less than 5% at concentrations ranging from 4 to 9 log 10 copies/mL for both HHV-6A and HHV-6B DNA.However, at 3 log 10 copies/mL, which is the concentration between LOD and LLOQ of the SMG assay, intra-and inter-assay variabilities of greater than 5% were noted.As HHV-6 DNA load of 3 log 10 copies/mL in whole blood is considered as a tentative threshold to distinguish between latent and active infections (1,38), the low precision of the SMG assay at this concentration can lead to misdiagnosis and inappro priate treatment decisions.When using in-house PCR and sequencing as a reference standard, the SMG and RealStar assays showed high sensitivity and specificity in detecting HHV-6A and HHV-6B DNA in whole blood.There were seven specimens positive by in-house PCR and sequencing but not detected or detected at concentrations lower than the LLOQ by the SMG or RealStar assays.A possible explanation for this discrepancy might be the presence of mutations in the primer/probe binding sites of the SMG and RealStar assays, as these mutations may result in underquantification or false-negative results for HHV-6 DNA.Since the clinical performance of the SMG and RealStar assays may vary, depending on the prevalence of HHV-6 strains harboring these mutations in a local population, local validation studies are required before use in diagnostic laboratories.Although the SMG assay, unlike the RealStar assay, was not calibrated against the 1st WHO International Standard for HHV-6B DNA, the quantitative agreement between the two assays was good.The correlation coefficients between viral loads measured by the two assays were 0.948 and 0.977, with mean differences of 0.62 and 0.32 log 10 copies/mL for HHV-6A and HHV-6B DNA, respectively.The WHO International Standard for HHV-6A DNA has yet to be established, and it remains a major barrier to the standardization of qPCR assays for the quantitative detection and differentiation of HHV-6A and HHV-6B DNA.
A major limitation of our study is that the performance characteristics of the SMG and RealStar assays were not established for specimen types other than whole blood.In addition to whole blood, various specimen types, such as plasma, serum, cerebrospinal fluid, and bronchoalveolar lavage, can be used for diagnosis of active infection (1).The performance characteristics of the SMG and RealStar assays for these specimen types should be established in further studies.Another limitation of our study is that archived DNA specimens were used to assess the clinical performance of the SMG and RealStar assays.Degradation of viral DNA may have occurred during long-term storage or the freeze-thaw process, which can be a factor in underestimating the clinical performance of the SMG and RealStar assays.Despite the use of archived DNA specimens, the SMG and RealStar assays showed high clinical sensitivity and specificity, and the viral loads obtained with the two assays were well correlated; however, further prospective studies are needed to accurately assess their clinical performance.Lastly, while in our clinical evaluation, we employed two extraction methods: MagNA Pure 96 and KingFisher Flex.We did not investigate how each extraction method influenced the performance of the SMG and RealStar assays.This will be the focus of future studies.
In conclusion, the SMG assay demonstrated excellent analytical sensitivity and specificity, precision, and linearity.Furthermore, the viral loads measured by the SMG assay were highly correlated with those measured by the RealStar assay routinely used in diagnostic laboratories.Taken together, the SMG assay is a useful diagnostic tool for quantitative detection and differentiation of HHV-6A and HHV-6B DNA in whole blood.

FIG 1
FIG 1 Linear ranges of the SMG assay for HHV-6A DNA (A) and HHV-6B DNA (B).The assay is linear over a range of 3-9 log 10 copies/mL.Based on the LLOQ (3.40 log 10 copies/mL), the linear range is 3.40-9.00log 10 copies/mL.The solid lines indicate linear regression lines.

FIG 2
FIG 2 Comparison of viral loads measured by the SMG and RealStar assays.Passing-Bablok regression and Bland-Altman analyses of HHV-6A (A) and HHV-6B (B) viral loads measured by the two assays.In scatter plots, solid lines indicate Passing-Bablok regression lines.In Bland-Altman plots, solid and dashed lines indicate mean differences and 95% limits of agreement (mean differences ± 1.96 × standard deviation), respectively.

TABLE 2
Intra-and inter-assay variability of the SMG assay

TABLE 3
Clinical sensitivity and specificity of the SMG and RealStar assays compared to in-house PCR and sequencing a CI, confidence interval.

TABLE 4
Agreement between qualitative results obtained with the SMG and RealStar assays a a CI, confidence interval; NPA, negative percent agreement; PPA, positive percent agreement.b Four specimens negative for HHV-6B by the RealStar assay were detected at concentrations lower than the LLOQ by the SMG assay.