We searched the MEDLINE, Embase, and Cochrane Library databases for articles published in English up to December 31, 2018. The search strategies were developed using combinations of free-text keywords and medical subject headings (MeSH), and we used similar search syntax across the other databases (Supplementary Table 1).

Two reviewers, who screened the titles and abstracts, evaluated every search hit for eligibility, and all potentially relevant articles were independently reviewed in full. In the case of a discrepancy in study selection, a consensus was reached through discussion. If no agreement could be reached, a third reviewer was consulted.

Studies were included if all of the following inclusion criteria were met: 1) paired studies, cohort studies, or randomized controlled trials (RCTs) comparing SFM and DM; 2) asymptomatic women aged 40 years or older who were enrolled in BC screening programs or for whom mammography was recommended; and 3) availability of sufficient data to construct 2 × 2 contingency tables for each mammography method. Studies were excluded if they assessed populations that are at a high risk of breast and/or ovarian cancer. Disagreements between the two reviewers were resolved through a consensus meeting with a third reviewer. If needed, we contacted the authors of the published studies for additional information.

Two separate observers independently evaluated the risk of bias of the included studies, using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) instrument [14]. Disagreements were resolved via a joint review of the manuscript to reach consensus. When multiple articles for a single study were found, we used data from the most recent publication. Each item was scored as “yes,” “no,” or “unclear” if insufficient information was available to make an accurate judgment. Disagreements were resolved by the third reviewer. We used RevMan 5.3 software (The Cochrane Collaboration, Copenhagen, Denmark) to graphically display the quality assessment results.

To perform accuracy analyses, we extracted the relevant data from each study, including the first author's name, publication year, study location, sample size, description of the study population (age), study design (prospective, retrospective, or unknown), and consideration of the learning-curve effect. For each study, the values for true positive, false positive, false negative, true negative, sensitivity, and specificity results for the detection of BC were extracted.

We estimated the sensitivity and specificity, with 95% confidence intervals (CIs), of each selected study. The overall pooled sensitivity, specificity, positive likelihood (LR+), negative likelihood (LR−), diagnostic odds ratio (DOR), and area under the summary receiver-operating characteristic (SROC) curve (AUC) of DM and SFM were calculated using a bivariate random-effects model [15]. We also generated a hierarchical summary receiver-operating characteristic (HSROC) curve that plotted the individual and summary estimates of sensitivities and specificities, along with 95% CI and prediction region, and calculated the area under the HSROC curve [16].

Heterogeneity was assessed using the Cochran Q test and Higgins I² heterogeneity index. We applied the random-effects model by using recently introduced bivariate mixed effects for diagnostic meta-analysis [17]. Several factors that could affect the diagnostic accuracy and cause heterogeneity were incorporated in the bivariate model to explore their influence on sensitivity and specificity. The following factors were evaluated: type of DM system (only full-field digital mammography [FFDM] or all DM, including computed radiography systems), enrollment of women younger than 50 years of age (yes or no), consideration of the learning-curve effect, initial year the digital mammogram was acquired, study design (prospective or retrospective), and study location (European countries or North America). We performed subgroup analyses to identify factors that influenced the diagnostic accuracy if four or more studies were included. To determine whether studies with short follow-up periods or negative mammography results were causing heterogeneity in the summary estimates, we performed sensitivity analyses by removing studies with a 1-year follow-up period and by estimating the I² statistic for the remaining group. Furthermore, to ensure that no single study influenced the overall results, we conducted “leave-one-out” sensitivity analyses by repeating the analyses with the following exclusion of each study. Finally, we performed meta-regression analyses based on linear regression models for assessing the influence of the heterogeneity of study populations on the meta-analysis results. We performed regression analyses when heterogeneity was significant and when a reasonable number of data points were available (≥ 4).

The statistical computations were performed using STATA 14.0 software (STATA Corp., College, USA) for all statistical analyses. We considered 2-sided p-values < 0.05 to be statistically significant.

Publication bias was assessed visually by using funnel plots.

Figure 1 is a flowchart illustrating study selection. Our initial search yielded 7,774 entries. After screening the titles and abstracts, we identified 53 articles for full-text review; of these, 13 items met all the inclusion criteria and were included in our analysis [10181920212223242526272829].

The main characteristics of the included studies are shown in Table 1. All included studies were published between 2002 and 2018. Thirteen articles described one RCT [21], three paired studies (in which the participants underwent both types of mammography) [181920], and nine cohort studies [102223242526272829] of screening mammography sensitivity and specificity for BC in average-risk asymptomatic women; the sample sizes ranged from 7,366 to 7,343,327. Six studies enrolled women aged < 50 years old [181921222528]; 8 studies were conducted in European countries [1020212324252729], and 5 were conducted in North America [1819222628].

Four studies collected digital mammograms from the year 2004 [23252627], while the initial year of digital mammogram enrollment in the other 9 studies was after 2004 [101819202122242829]. Nine of the 13 studies used the Breast Imaging Reporting and Data System (BI-RADS) for mammogram interpretation [101819222324272829]. Four studies used screening mammograms with longitudinal follow-up of women with negative mammography results through cancer registries, medical records, or national insurance claims approximately 1 year later, as the reference standard [19222528]; 9 studies performed a follow-up for 2 years or more for negative screening results [101820212324262729] (Table 2).

The QUADAS-2 instrument suggested that the highest risk of bias occurred in the “flow and timing” category (Supplementary Figure 1), which is likely attributed to a study that acquired BC screening data from a single health care delivery organization, using a cancer registry as the reference standard [28]. This study only reported the final number of patients included in the data analysis, and the flow of study population selection was not adequately described in the article.

The overall pooled sensitivity, specificity, AUC, LR+, LR−, and DOR of BC screening conducted using the DM systems were 0.76 (95% CI, 0.70–0.81), 0.96 (95% CI, 0.94–0.97), 0.94 (95% CI, 0.92–0.96), 18.0 (95% CI, 12.2–26.7), 0.25 (95% CI, 0.20–0.32), and 72 (95% CI, 44–118), respectively. The overall pooled sensitivity, specificity, AUC, LR+, LR−, and DOR for the SFM systems were 0.76 (95% CI 0.70–0.81), 0.97 (95% CI, 0.94–0.98), 0.92 (95% CI, 0.89–0.94), 22.5 (95% CI, 13.1–38.8), 0.25 (95% CI, 0.20–0.31), and 91 (95% CI, 52–157), respectively (Table 3, Supplementary Figure 2, and Supplementary Table 2).

As shown in Table 3 and Figure 2, we identified substantial heterogeneity between studies when calculating the pooled sensitivity (DM, I² = 99.3%; SFM, I² = 99.8%), specificity (DM, I² = 100.0%; SFM, I² = 100.0%), and AUC (DM, I² = 100.0%; SFM, I² = 100.0%) of screening mammograms for BC in each mammography system by using the bivariate model.

Excluding the studies with 1-year follow-up periods did not reduce the heterogeneity for overall sensitivity (DM, I² = 99.6%; SFM, I² = 99.8%), specificity (DM, I² = 100.0%; SFM, I² = 100.0%), or AUC (DM, I² = 100.0%; SFM, I² = 100.0%). However, removing studies with 1-year follow-up periods decreased the pooled AUCs of DM and SFM (DM, 0.92 [95% CI, 0.89–0.94; SFM, 0.84 [95% CI, 0.81–0.87]).

To evaluate the robustness of the summarized diagnostic accuracy results, we performed a leave-one-out sensitivity analysis by iteratively removing one study at a time and recalculating the summary sensitivities and specificities (Supplementary Figure 3). Leave-one-out analyses showed no marked difference in results, indicating that no single study had any impact on them.

There was high heterogeneity among studies in both subgroups on the SFM basis and the DM basis. To assess the influence of the heterogeneity of the study population and the data on meta-analysis results, we conducted meta-regression analyses.

Meta-regression analyses showed that the following factors led to a high heterogeneity in the sensitivity and specificity of the SFM screening results (Supplementary Figure 4A and B): enrollment of women younger than 50 years, study design, and study location. However, when we performed subgroup analyses in these subgroups, the pooled estimates remained highly heterogeneous (Table 3). In the DM screening results, on the other hand, the following factors were found to be related to the high heterogeneity (Supplementary Figure 4C and D): type of DM system, study design, and study location. According to the subgroup analysis results, however, the heterogeneity remained high except to the area under the summary receiver-operating characteristic curve of the prospective design group (Table 3).

We conducted the publication bias analysis for the included studies. However, as shown in Supplementary Figure 5, the funnel plots were asymmetric, and the p-value of Deeks' test was 0.08 for DM and 0.06 for SFM, indicating that a publication bias existed in the meta-analysis for both mammography systems.