This study was approved by the Institutional Review Board of the Hallym University Dongtan Sacred Heart Hospital (IRB number 2015-475), Korea. This study utilized stool samples collected and stored in Dongtan Sacred Heart Hospital from December 2013 to September 2016. The sample collection period was different for each virus because the positive rates for norovirus GI, adenovirus, astrovirus, and sapovirus were very low; thus, the collection period was extended to obtain an adequate number of positive samples. The number of samples and collection periods were as follows: 1) 301 leftover samples after testing for rotavirus, norovirus, adenovirus, and astrovirus from November 2015 to December 2015; 2) 56 rotavirus-positive samples from December 2015 to March 2016; 3) 43 adenovirus-positive samples from August 2015 to June 2016; and 4) 46 astrovirus-positive samples from December 2013 to March 2016. Samples with inadequate volume for subsequent testing were excluded. During the above mentioned periods, the leftover samples after testing were collected and stored at −70℃ until further analysis.

Nucleic acids were extracted from the samples, using the QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) and a QIAcube platform (Qiagen) after the stored samples were thawed.

For the Allplex assay, 5 µL of RNA extract was mixed with 20 µL of master mix and qRT-PCR was performed using a CFX96 system (Bio-Rad, Hercules, CA, USA) under the following conditions: reverse transcription at 50℃ for 20 minutes, denaturation at 95℃ for 15 minutes, and 45 cycles of PCR (10 seconds at 95℃, 1 minute at 60℃, and 30 seconds at 72℃). All procedures were performed according to the manufacturer's instructions.

The Seeplex Diarrhea-V ACE Detection assay (Seegene) is a multiplex PCR system for detecting astrovirus, rotavirus, enteric adenovirus, and norovirus GI and GII. For this assay, following cDNA synthesis, 3 µL of cDNA was mixed with 17 µL of master mix and PCR was performed using SimpliAmp Thermal Cycler (Thermo Fisher Scientific, Waltham, MA, USA) under the following conditions: 15 minutes at 94℃; followed by 40 cycles of 94℃ for 30 seconds, 60℃ for 90 seconds, and 72℃ for 90 seconds; and a final single incubation at 72℃ for 10 minutes. All procedures were performed according to the manufacturer's instructions.

Genotyping targeting the capsid genes was conducted by previously described PCR and sequencing methods [78]. Briefly, rotavirus G (VP7) and P (VP4) genotyping was performed according to the WHO manual, using specific primer sets [9], with some modifications. Norovirus capsid genotyping was performed using specific primer sets [8], and adenovirus capsid hexon genotyping was performed by PCR and sequencing using a specified primer set (ADHEX1F/AD2) or a different primer set (AD1/AD2) [10]. Additionally, astrovirus and sapovirus genotyping was performed by PCR and sequencing techniques using specific capsid primer sets [1112]. The PCR products were visualized by electrophoresis on an agarose gel and analyzed by DNA sequencing. The nucleotide sequences were analyzed using ABI Prism BigDye Terminator version 3.1 (Applied Biosystems, Foster City, CA, USA), and the genotypes were confirmed using the NCBI BLAST server of the GenBank database.

To determine the analytical sensitivity (the lower limit of detection) for norovirus GI and GII and adenovirus, commercialized virus culture fluids were purchased (Catalog #0810086CF, norovirus GI [recombinant]; #0810087CF, norovirus GII [recombinant]; and #0810085CF, adenovirus type 41; ZeptoMetrix Corporation, Buffalo, NY, USA). Viral nucleic acids were extracted using the QIAamp DSP DNA Mini Kit (Qiagen). To determine the lower limit of detection for rotavirus astrovirus, and sapovirus, in vitro transcribed RNAs were prepared using the MEGAscript T7 Kit (Life Technologies, Carlsbad, CA, USA). The prepared nucleic acids were serially diluted, and the Allplex assay was carried out using a CFX96 system (Bio-Rad). The lower limit of detection was defined as the lowest concentration that was detected in ≥95% of the replicates [13].

The analytical specificity (cross-reactivity) of the Allplex assay was assessed using 169 different pathogens (43 viruses and 126 bacteria), including 24 target viruses (see Supplemental Data Table S1). Each pathogen was tested thrice, using the same procedures for sample processing.

Inter-rater agreement statistics (Cohen's kappa) were used to compare the results of the Allplex assay, Seeplex assay, and genotyping. The kappa value was interpreted as follows: <0.20, poor agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, good agreement; and 0.81–1.00, very good agreement [14]. MedCalc version 15 (MedCalc Software, Mariakerke, Belgium) was used for all statistical analyses.

A total of 446 stool samples were tested, and the results of the three assays are summarized in Table 1. The overall agreement rates between the three assays for rotavirus, norovirus GI, norovirus GII, adenovirus, and astrovirus were 98.7%, 99.1%, 93.0%, 89.2%, and 97.8%, respectively. The overall agreement rates between the Allplex and Seeplex assays were 98.7% for rotavirus, 99.1% for norovirus GI, 93.3% for norovirus GII, 98.0% for adenovirus, and 99.6% for astrovirus. Furthermore, the overall agreement rates between the Allplex assay and genotyping were 99.1% for rotavirus, 99.1% for norovirus GI, 98.7% for norovirus GII, 89.7% for adenovirus, 98.2% for astrovirus, and 99.8% for sapovirus.

As shown in Table 2, the positive agreement rate of the Allplex assay with the Seeplex assay ranged from 94.8% to 100%, while the negative agreement rate ranged from 92.9% to 100%. The kappa correlation ranged from 0.663 to 0.976 and showed good to very good agreement. The positive agreement rate between the Allplex assay and genotyping was 100% for rotavirus, norovirus GI, astrovirus, and sapovirus; 96.6% for norovirus GII; and 26.9% for adenovirus. Moreover, the negative agreement rate ranged from 98.0% to 99.8%. The kappa correlation ranged from 0.663 to 0.965 and showed good to very good agreement, except for adenovirus (with a kappa correlation of 0.332, indicating fair agreement).Detailed results for the three methods for each virus are shown in see Supplemental Data Tables S2, S3, S4, S5, S6, S7, S8.

For rotavirus, two cases (0.4%) were negative only in the Seeplex assay, and four cases (0.9%) were positive only in the Allplex assay (see Supplemental Data Tables S2, S8). All cases had high threshold cycle (C_T) values of >30 in the Allplex assay. They were interpreted as discrepancies near the cutoff with low rotavirus concentration.

For norovirus GI, four cases (0.9%) were detected by the Allplex assay only. These cases had very high C_T values (>35) and were thus interpreted as discrepancies near the cutoff with very low norovirus concentration (see Supplemental Data Tables S3, S8).

For norovirus GII, 24 cases (5.4%) were negative only in the Seeplex assay, one case (0.2%) was negative only in genotyping, one case (0.2%) was positive only in the Allplex assay, four cases (0.9%) were negative only in the Allplex assay, and one case (0.2%) was positive only in the Seeplex assay (see Supplemental Data Tables S4, S8). Among the 24 cases that were negative only in the Seeplex assay, 22 had C_T values >30 and were considered to have a low norovirus concentration. However, the C_T values of the remaining two cases were 23.79 and 25.09, and they were identified as type GII.3. The one case that was negative only in genotyping had a C_T value of 31.09 in the Allplex assay and showed weak positivity in the Seeplex assay; this case was considered to have a low norovirus concentration. The one case that was positive only in the Allplex assay had a high C_T value of 35.35. The four cases that were negative only in the Allplex assay were all identified as type GII.4 by genotyping; it was estimated that the Allplex assay could not detect these cases. Moreover, the one case that was positive only in the Seeplex assay was negative by ELISA antigen test and was thus suspected to be a false positive result.

For adenovirus, two cases (0.4%) were negative only in the Seeplex assay, one case (0.2%) was negative only in genotyping, seven cases (1.6%) were positive only in the Allplex assay, and 38 cases (8.5%) were positive only in genotyping (see Supplemental Data Tables S5, S8). The two cases that were negative only in the Seeplex assay had very high C_T values of 36.47 and 36.58, and the one case that was negative only in genotyping had a very high C_T value of 38.97. The seven cases that were positive only in the Allplex assay had very high C_T values of >35. These 10 discrepant cases were considered as low-concentration samples.

For astrovirus, eight cases (1.8%) were negative only in genotyping, and two cases (0.4%) were positive only in the Seeplex assay (see Supplemental Data Tables S6, S8). The eight cases that were negative only in genotyping consisted of genotypes that could not be detected by the genotyping method performed in our laboratory. The two cases that were positive only in the Seeplex assay showed weak bands and were interpreted as a discrepancy near the cutoff with low concentration.

For sapovirus, one discrepant case was positive in the Allplex assay, but was negative in genotyping (see Supplemental Data Tables S7, S8).

Eight rotavirus genotypes (G1P[8], G2P[4], G3P[8], G4P[6], G8P[8], G9P[4], G9P[8], and G8P[4]), three norovirus GI genotypes (GI.1, GI.5, and GI.6), four norovirus genotypes GII (GII.3, GII.4, GII.6, and GII.17), eight adenovirus genotypes (C1, C2, B3, C5, C6, A12, A31, and F41), two astrovirus genotypes (types 1 and 5), and two sapovirus genotypes (GII and GIV) were detected (see Supplemental Data Tables S2, S3, S4, S5, S6, S7, S8).

The lower limits of detection for norovirus GI (recombinant), norovirus GII (recombinant), and adenovirus using commercialized virus culture fluids were 75, 5, and 0.5 TCID₅₀ (50% tissue culture infective dose)/mL, respectively. The lower limits of detection for rotavirus, astrovirus, and sapovirus using in vitro transcribed RNA were 5×10¹, 5×10², and 5×10³ copies/reaction, respectively. The analytical specificity showed 100% negative signals for the 145 non-target pathogens and positive signals for the 24 target pathogens (see Supplemental Data Table S1).