Journal List > Clin Endosc > v.50(6) > 1151507

TOOLS
Altman: False Reassurance Based on Non-Significant Results
Letter to the editor

Clin Endosc 2017;50(6):617-617.

Published online: 18 August 2017

DOI: https://doi.org/10.5946/ce.2017.091

False Reassurance Based on Non-Significant Results

Douglas G. Altman

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK

Correspondence: Douglas G. Altman Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, OX2 6UD, Oxford, UK E-mail: doug.altman@csm.ox.ac.uk

Received 7 June 2017       Accepted 3 July 2017

Copyright © 2017 Korean Society of Gastrointestinal Endoscopy

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
A journal article with a title that begins with “There is no …” immediately suggests that there may have been a misinterpretation of a non-significant result from a small study. Non-significant results, expressed either as a p-value greater than 0.05 or a confidence interval (CI) that includes the value representing no effect, are often falsely interpreted as evidence of equivalence [1]. What matters far more than the p-value is whether the CI includes values of clinical importance.
My attention was drawn to the meta-analysis of Amin et al. [2] because of its title. Those authors conducted a systematic review and found 9 studies that examined the outcomes of patients receiving different interventions for transmural endoscopic drainage of pancreatic fluid. Three drainage methods were considered, but, for no obvious reason, only two methods were analyzed: transmural drainage (TM) alone or in combination with transpapillary drainage.
Even if the studies included in the meta-analysis were of high quality, the wide CIs indicated that the results are compatible with a large difference in outcomes. For example, the odds ratio for recurrence was 1.49 in favor of TM, but the 95% CI was 0.53 to 4.21, which is consistent with halving or quadrupling of the risk. Only 6 studies contributed to this analysis, with a total of 34 events.
Further, 6 of the 9 observational studies were found to be of poor quality regarding the comparability of the groups, one of the most fundamental aspects of study quality. Other problems were also identified. The authors used a 9-item version of the Newcastle-Ottawa scale, as against the 7-item version they cited.
I noted that these authors also conducted the largest study included in their meta-analysis [3]. Despite being based on even less data, that study too concluded that transpapillary drainage has no benefit on treatment outcomes when performed in addition to TM.
The authors sensibly call for better, larger studies. Meanwhile, these two strategies for drainage have certainly not been demonstrated to be clinically or technically equivalent, as is wrongly claimed in the titles of both articles.

Notes

Conflicts of Interest: The author has no financial conflicts of interest.

REFERENCES

1. Stang A, Poole C, Kuss O. The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol. 2010; 25:225–230.
crossref
2. Amin S, Yang DJ, Lucas AL, Gonzalez S, DiMaio CJ. There is no advantage to transpapillary pancreatic duct stenting for the transmural endoscopic drainage of pancreatic fluid collections: a meta-analysis. Clin Endosc. 2017; Apr. 17. [Epub]. https://doi.org/10.5946/ce.2016.091.
crossref
3. Yang D, Amin S, Gonzalez S, et al. Transpapillary drainage has no added benefit on treatment outcomes in patients undergoing EUS-guided transmural drainage of pancreatic pseudocysts: a large multicenter study. Gastrointest Endosc. 2016; 83:720–729.
TOOLS
Similar articles