1. Background
Respiratory syncytial virus (RSV) is the most common cause of viral acute lower respiratory tract illness in pediatric population younger than 5 years (1-4).
Due to the identical clinical symptoms of RSV and other viral respiratory infections, an early, rapid and accurate diagnosis of RSV infection in susceptibile population is of great important (5, 6). On the other hand, the rapid diagnosis of RSV infection can exclude bacterial infection, minimizing inappropriate uses of antibiotics (7). There are a variety of approaches for detecting RSV infection, including antigen detection testing, cell culture, and PCR based methods worldwide (8-11). Cell culture and PCR are considered as the gold standard, but there are time-consuming, technically demanding, laborious, and expensive.
In Hangzhou, China, most hospitals are likely to screen RSV infection in outpatients by using colloidal gold assay which works by detecting the antigen of RSV and is characterized as quick detection, while the real-time RT-PCR assay usually functions as the confirming test combined with clinical symptoms.
2. Objectives
The objective of this study was to evaluate the diagnostic accuracies of the colloidal-gold-Genesis, (CFDA: 20143400267) and the newly developed real-time RT-PCR-IngeniGen in a head-to-head comparison with an admitted real-time RT-PCR-HuaRuiAn (CFDA: 20163400961) as the gold standard.
3. Methods
3.1. Study Population and Specimens
A prospective study was conducted from July 2017 to December 2017 in the Children’s Hospital, Zhejiang University School of Medicine. We collected the nasopharyngeal specimens regarding following conditions: (1) patients younger than 14 years old, who were suspected of having RSV infection with symptoms like cough, dyspnea, and wheezing (12), (2) the specimens were divided into RSV positive or negative group detected by colloidal gold assay, (3) then all three RSV detection methods were performed on the specimens by performers blinded to the former test results.
This study was approved by the medical ethics committee of the Children’s Hospital of Zhejiang University School of Medicine (No. 2016-IEC-013, date: 2016.11.30) and written consent was obtained from parents before enrollment.
3.2. Colloidal-Gold- Genesis for RSV Detection
Colloidal gold plate (Genesis company, Hangzhou, China) contained three major parts: (1) sample well, (2) quality control part, (3) RSV test part. Specimens were tested according to the manufacturer’s instruction. Briefly, 3 drops (about 100 µL) of the sample were added into a sample well and incubated for 15 minutes. Samples with lines occurred in the quality control part and RSV test part were determined as RSV positive; samples with lines occurred in quality control part while the RSV test part showed nothing were determined as RSV negative. Samples showing nothing in quality control part indicated an invalid test, and a second test plate or other test assay was necessary till the result was either positive or negative.
3.3. Real-Time RT-PCR Testing
Total nucleic acid extraction kit by magnetic beads (IngeniGen, China) was utilized to extract the nucleic acid from all specimens. For the commercial RSV nucleic acid detection assay (HuaRuiAn Biology and IngeniGen, China), positive control, negative control, and quality control were included in each run. Real-time RT-PCR was performed in the ABI7500 system according to each brand manufacturer’s instruction (Table 1) (HuaRuiAn Biology: http://www.huayinbio.net/product-4-92-158.html; IngeniGen: http://www.ingenigenbiotech.com/col.jsp?id=106). Cycle threshold (CT) value of each sample was collected for future analysis.
Table 1.The Thermocycle Condition of Real Time RT-PCR-HuaRuiAn and Real Time RT-PCR- IngeniGen
| Real Time RT-PCR-HuaRuiAn | Real Time RT-PCR- IngeniGen |
|---|---|
| 1- 37°C for 25 min, 94°C for 2 min | 45°C for 10 min, 94°C for 10 min |
| 2- 95°C for 15 sec, 55°C for 15 sec, 72°C for 20 sec, 5 cycles | 95°C for 15 sec, 60°C for 45 sec, 45 cycles |
| 3- 94°C for 5 sec, 55°C for 35 sec, 32 cycles |
3.4. Statistical Analysis
The results were analyzed by kappa test (SPSS software, version 24.0). Kappa value greater than 0.8 was considered as a very good agreement (13, 14). The average, median, minimum and maximum CT values were determined. The comparison of CT value was analyzed by independent or paired t-test. A two-tailed P value of less than 0.05 was considered to be statistically significant. All P values were rounded up to 3 decimal places.
4. Results
During the study period, 258 children were enrolled. Among 258 specimens, RSV nucleic acid detection assay (real-time RT-PCR-HuaRuiAn) detected 166 positive samples and worked as the gold standard. 158 specimens were accurately identified as RSV positive by colloidal-gold-Genesis, 13 specimens were incorrectly identified as RSV positive, and 87 specimens identified as negative by colloidal-gold-Genesis while 8 of them were identified as positive specimens by real-time RT-PCR-HuaRuiAn (Table 2). The overall sensitivity and specificity of colloidal-gold-Genesis for RSV detection in comparison to real-time RT-PCR-HuaRuiAn were found to be 95.2% and 85.9%, respectively. And a high statistical consistency was found between the two RSV detection assays (kappa value = 0.82). The positive and negative likelihood ratios were 6.75 and 0.06 respectively.
Table 2.Comparison of Colloidal Gold Assay and Real-Time RT-PCR-IngeniGen Results with Reference to the Real-Time RT-PCR-HuaRuiAn for RSV Detection
| Test Assay | True Positives | False Positives | True Negatives | False Negatives | Sensitivity, % | Specificity, % | LR+ | LR- | Kappa |
|---|---|---|---|---|---|---|---|---|---|
| Colloidal gold | 158 | 13 | 79 | 8 | 95.2 | 85.9 | 6.75 | 0.06 | 0.82 |
| IngeniGen | 166 | 2 | 90 | 0 | 100 | 97.8 | 45.45 | 0.00 | 0.983 |
Abbreviations: LR+, positive likelihood ratio; LR-, negative likelihood ratio.
5. Discussion
BD VeritorTM System RSV and Quidel® Sofia® RSV FIA are the widely used lateral flow digital immunoassay (DIA) tests for RSV viral antigens throughout the world. Neena Kanwar et al. compared the diagnostic accuracy of the two DIA tests with the real-time RT-PCR assay results as a reference, showing a sensitivity of 71.15% for BD VeritorTM System and 80.77% for Quidel® Sofia® RSV FIA system, and the specificity of 100% for both systems (15). The sensitivity and specificity of the nCounter (NanoString Technologies), which is a multiplex digital method of RNA quantification, in comparison to real-time RT-PCR were found to be 74.3% and 98.4% for RSV-A, 77.6% and 97.8% for RSV-B, respectively (16). Compared with the results of those studies, the colloidal-gold-Genesis showed higher sensitivity and lower specificity with a higher Youden index. Thus, the colloidal-gold-Genesis is a good alternative for the clinical quick screening of RSV infection for its high specificity, sensitivity and time-saving. Furthermore, the median CT value for the specimens in the colloidal positive group was significantly lower than the negative group. The higher the CT value, the lower is the RSV titer, thus the result tested by colloidal-gold-Genesis is not credible enough, especially when the RSV titer is low, which needs another test for confirmation, that’s why colloidal-gold-Genesis is a fast screening assay rather than a confirmation assay.
The RSV nucleic acid detection assay (real-time RT-PCR-IngeniGen) detected a total of 168 positive samples with 2 of them determined incorrectly, and 90 identified correctly negative samples. The sensitivity and specificity were found to be 100% and 97.8% respectively (Table 1). The positive and negative likelihood ratio was 45.45 and 0.00 respectively. A higher statistical consistency was found between the real-time RT-PCR-HuaRuiAn and real-time RT-PCR-IngeniGen (kappa value = 0.983).
As shown in Figure 1A, the CT value of colloidal gold assay positive group was significantly lower than that of the negative group (P ≤ 0.000). The median PCR CT value for the positive group was 24.18 (range 17.83 - 28.8), and for the negative group 27.45 (22.32 - 30.14). To determine the difference between the two real-time RT-PCR assays, the CT value was detected and analyzed. The median CT value of the 166 RSV positive samples which were detected by real-time RT-PCR-IngeniGen was 30.76 (range 25.72 - 37.42), while median CT value of real-time RT-PCR-HuaRuiAn was significantly lower (24.21, range 17.83 - 30.14; P ≤ 0.000) (Figure 1B). Coming to the 2 incorrectly identified positive samples by real-time RT-PCR-IngeniGen, the CT value was 38.05 and 39.13 respectively. The 2 incorrectly identified positive samples by real-time RT-PCR-IngeniGen showed that the result of a positive sample may be inaccurate as the CT value higher than 38, suggesting that the outcome criteria of CT value should be more precise.
Figure 1.
Boxplot illustrating CT value of specimens tested by the three assays: colloidal-gold-Genesis, real-time RT-PCR-IngeniGen, and real-time RT-PCR-HuaRuiAn. A, the comparison of real-time RT-PCR-HuaRuiAn CT value between the colloidal gold assay positive group and negative group. B, the comparison between the real-time RT-PCR-HuaRuiAn Biology CT value and the real-time RT-PCR-IngeniGen CT value for the 166 positive samples.
In future, the detection kit for RSV infection will develop towards fast, automatic, high sensitivity and specificity, as well as point-of-care testing (POCT). The colloidal-gold-Genesis is effective for early screening of RSV infection, while the newly developed RSV nucleic acid detection assay (real-time RT-PCR-IngeniGen) which is performed to confirm RSV infection should be more accurate in the definition of positive samples.
