Journal List > Urogenit Tract Infect > v.14(2) > 1135040

Kang, Jung, Kim, Ku, Jung, Kim, Hwang, and Guideline Development Committee in the Korean Association of Urogenital Tract Infection and Inflammation: Korean Translation of the GRADE Series Published in the BMJ, ‘GRADE: What Is “Quality of Evidence” and Why Is It Important to Clinicians?’ (A Secondary Publication)

Abstract

This article is second translation of a GRADE series published in the BMJ to create a highly structured, transparent, and informative system for rating quality of evidence for developing recommendations. The process to develop a guideline, we should formulate a clear question with specification of all outcomes of importance to patients. Grading of Recommendations, Assessment, Development and Evaluation (GRADE) offers four levels of evidence quality: high, moderate, low, and very low for these patient-important outcomes. Randomized trials begin as high quality evidence and observational studies as low quality evidence. Although randomized trials begin as high quality evidence, quality may be downgraded as a result of study limitations (risk of bias), inconsistency (variability in results), indirectness, imprecision (wide confidence intervals), or publication bias. While the quality of evidence derived from observational studies starts at ‘low’ but may be upgraded based on a very large magnitude of effect, a dose-response gradient, and if all plausible biases would reduce an apparent treatment effect.

REFERENCES

1. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004; 328:1490.
crossref
2. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. GRADE Working Group. What is “quality of evidence” and why is it important to clinicians? BMJ. 2008; 336:995–998.
3. Oxman AD, Guyatt GH. Guidelines for reading literature reviews. CMAJ. 1988; 138:697–703.
4. Schunemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 10. integrating values and consumer involvement. Health Res Policy Syst 2006;4:22.
5. Fletcher SW, Spitzer WO. Approach of the Canadian task force to the periodic health examination. Ann Intern Med. 1980; 92:253–254.
crossref
6. Schunemann HJ, Jaeschke R, Cook DJ, Bria WF, El-Solh AA, Ernst A, et al. ATS Documents Development and Implementation Committee. An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. Am J Respir Crit Care Med. 2006; 174:605–614.
7. Guyatt G, Cook D, Devereaux PJ, Meade M, Straus S. Therapy. In: Guyatt G, Rennie D, editors. Users\' guides to the medical literature: a manual for evidence-based clinical practice. Chicago: American Medical Association; 2002.
8. Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, et al. Randomized trials stopped early for benefit: a systematic review. JAMA. 2005; 294:2203–2209.
9. Karanicolas PJ, Davies E, Kunz R, Briel M, Koka HP, Payne DM, et al. The pylorus: take it or leave it? Systematic review and meta-analysis of pylorus-preserving versus standard whipple pancreaticoduodenectomy for pancreatic or periampullary cancer. Ann Surg Oncol. 2007; 14:1825–1834.
crossref
10. Glasziou P, Chalmers I, Rawlins M, McCulloch P. When are randomised trials unnecessary? Picking signal from noise. BMJ. 2007; 334:349–351.
crossref
11. Thompson DC, Rivara FP, Thompson R. Helmets for preventing head and facial injuries in bicyclists. Cochrane Database Syst Rev 2000;(2):CD001855.
12. Cannegieter SC, Rosendaal FR, Briet E. Thromboembolic and bleeding complications in patients with mechanical heart valve prostheses. Circulation. 1994; 89:635–641.
crossref

Fig. 1.
Hierarchy of outcomes according to importance to patients to assess effect of phosphate lowering drugs in patients with renal failure and hyperphosphataemia. Adapted from the article of Guyatt et al. BMJ 2008;336:995-8 [2].
uti-14-64f1.tif
Fig. 2.
Effect on delayed gastric emptying of pylorus preserving pancreaticoduodenectomy compared with standard Whipple procedure for pancreatic adenocarcinoma. CI: confidence interval. Adapted from the article of Guyatt et al. BMJ 2008;336:995-8 [2].
uti-14-64f2.tif
Table 1.
Factors in deciding on quality of evidence
Factors that might decrease quality of evidence
• Study limitations
• Inconsistency of results
• Indirectness of evidence
• Imprecision
• Publicationbias
Factors that might increase quality of evidence
• Large magnitude of effect
• Plausible confounding, which would reduce a demonstrated effect
• Dose-response gradient

Adapted from the article of Guyatt et al. BMJ 2008;336:995-8 [2].

See Appendix 1 (complete translate in Korean).

Table 2.
GRADE evidence profile for impact of surgical alternatives for pancreatic cancer from systematic review and meta-analysis of randomised controlled trials in inpatient hospitals of pylorus preserving versus standard Whipple pancreaticoduodenectomy for pancreatic or periampullary cancer by Karanicolas et al. [9]
No of studies (no. of participants) Quality assessment
Summary of findings
Study limitationsa) Consistency Directness Precision Publication bias Relative effect (95% CI)b) Best estimate of Whipple group risk Absolute effect (95% CI) Quality
Five year mortality:
 3 (229) Serious limitations (-1) No important inconsistency Direct No important imprecision Unlikely 0.98 (0.87 to 1.11) 82.5% 20 less/1,000; 120 less to 80 more +++, moderate
In-hospital mortality:
 6 (490) Serious limitations (-1) No important inconsistency Direct Imprecision (-1)c) Unlikely 0.40 (0.14 to 1.13) 4.9% 20 less/1,000; (50 less to 10 more) ++, low
Blood transfusions (unit):
 5 (320) Serious limitations (-1) No important inconsistency Direct No important imprecision Unlikely - 2.45 units -0.66 (-1.06 to -0.25); favours pylorus preservation +++; moderate
Biliary leaks:
 3 (268) Serious limitations (-1) No important inconsistency Direct Imprecision (-1)c) Unlikely 4.77 (0.23 to 97.96) 0 20 more/1,000 20 less to 50 more ++, low
Hospital stay (d):
 5 (446) Serious limitations (-1) No important inconsistency Direct Imprecision (-1)c) Unlikely - 19.17 days -1.45 (-3.28 to 0.38); favours pylorus preservation ++, low
Delayed gastric emptying:
 5 (442) Serious limitations (-1) Unexplained heterogeneity (-1)d) Direct Imprecision (-1)c) Unlikely 1.52 (0.74 to 3.14) 25.5% 110 more/1,000; 80 less to 290 more +, very low

a) Unclear allocation concealment in all studies, patients blinded in only one study, outcome assessors not blinded in any study, >20% loss to follow-up in three studies, not analysed using intention to treat in one study.

b) Relative risks (95% confidence intervals) are based on random effect models.

c) Confidence interval (CI) includes possible benefit from both surgical approaches.

d) I2=72.6%, p=0.006.

Adapted from the article of Guyatt et al. BMJ 2008;336:995-8 [2].

See Appendix 2 (complete translate in Korean).

Table 3.
Quality of evidence is weaker if comparisons in trials are indirect
Question of interest Source of indirectness
Relative effectiveness of alendronate and risedronate in osteoporosis Indirect comparison: randomized trials have compared alendronate with placebo and risedronate with placebo, but trials comparing alendronate with risedronate are unavailable
Oseltamivir for prophylaxis of avian flu caused by influenza A (H5N1) virus Differences in population: randomised trials of oseltamivir are available for seasonal influenza, but not for avian flu
Sigmoidoscopic screening for prevention of mortality from colon cancer Differences in intervention: randomized trials of faecal occult blood screening provide indirect evidence, bearing on potential effectiveness of sigmoidoscopy
Choice of drug for schizophrenia Differences in comparator: series of trials comparing newer generation neuroleptic agents with fixed doses of haloperidol 20 mg provide indirect evidence of how newer agents would compare with lower, flexible doses of haloperidol that clinicians typically use
Rosiglitazone for prevention of diabetic complications in patients at high risk of diabetes Differences in outcome: randomized trial shows delay in development of biochemical diabetes with rosiglitazone but was underpowered to tackle diabetic complications

Adapted from the article of Guyatt et al. BMJ 2008;336:995-8 [2].

See Appendix 3 (complete translate in Korean).

Appendices

Appendix 1.

근거수준을 결정하는 요소

근거수준을 낮출 수 있는 요소
• 비뚤림 위험(study limitation; risk of bias)
• 비일관성(inconsistency)
• 비직접성(indirectness)
• 비정밀성(imprecision)
• 출판비뚤림(publication bias)
근거수준을 높일 수 있는 요소
• 효과크기가 클 때(large magnitude of effect)
• 교란변수가 추정 효과의 크기를 낮출 때(plausible confounding, which would reduce a demonstrated effect)
• 양-반응 관계(dose-response gradient)
Appendix 2.

췌장 또는 팽대부주위 암에 대한 표준 Whipple 이자샘창자절제술과 유문 보존 이자샘창자절제술의 체계적 고찰, 메타분석을 통해 췌장암 환자의 수술적 대안의 영향에 대한 GRADE 근거요약표

연구수 (대상자수) 근거수준 평가
결과요약
비뚤림 위험a) 일관성 직접성 정밀성 출판비뚤림 상대효과 (95% CI)b) 표준 Whipple 이자샘창자절 제술 추정치 절대효과 (95% CI) 수준
5년 사망률:
3 (229) 심각함(-1) 비일관성 없음 직접적 비정밀성 없음 가능성 적음 0.98 (0.87 to 1.11) 82.5% 1,000명당 20명 적음; 120명 적음에서 80명 많음 +++, 중등도
병원내 사망률:
6 (490) 심각함(-1) 비일관성 없음 직접적 부정확(-1)c) 가능성 적음 0.40 (0.14 to 1.13) 4.9% 1,000명당 20명 적음; 50명 적음에서 10명 많음 ++, 낮음
수혈(유닛):
5 (320) 심각함(-1) 비일관성 없음 직접적 비정밀성 없음 가능성 적음 - 2.45 유닛 -0.66 (-1.06 to -0.25); 유문 보존술식 선호 +++; 중등도
담즙액 누출:
3 (268) 심각함(-1) 비일관성 없음 직접적 부정확(-1)c) 가능성 적음 4.77 (0.23 to 97.96) 0 1,000명당 20명 많음; 20명 적음에서 50명 많음 ++, 낮음
입원기간(일):
5 (446) 심각함(-1) 비일관성 없음 직접적 부정확(-1)c) 가능성 적음 - 19.17일 -1.45 (-3.28 to 0.38); 유문보존술식 선호 ++, 낮음
위 배출 지연:
5 (442) 심각함(-1) 설명되지 않는 이질성(-1)d) 직접적 부정확(-1)c) 가능성 적음 1.52 (0.74 to 3.14) 25.5% 1,000명당 110명 많음; 80명 적음에서 290명 많음 +, 매우 낮음

CI: confidence interval.

a) 모든 연구에서 불명확한 무작위 배정은폐, 한 연구에서만 환자 눈가림이 이루어짐, 결과평가자 눈가림 이루어지지 않음, 세 연구에서 20% 이상의 추적관찰 실패율, 한 연구에서는 치료의향에 따른 분석 원칙이 지켜지지 않음.

b) 상대 위험 (95 % 신뢰 구간)은 변량효과모델을 기반으로 함.

c) 신뢰 구간에는 두 가지 수술 방법의 추정 효과를 포함.

d) I2=72.6%, p=0.006.

Appendix 3.

비직접성의 유형(연구의 비교가 간접적 인 경우 증거의 질이 약하다)

핵심 질문 비직접성의 원인
골다공증에서 alendronate와 risedronate의 상대적 효과 간접 비교: 무작위 연구는 alendronate와 위약을 사용한 군 및 risedronate와 위약을 사용한 군을 비교했지만 alendronate와 risedronate를 직접 비교하는 연구는 수행되지 않았다.
인플루엔자 A (H5N1) 바이러스에 의한 조류 인플루엔자 예방을 위한 Oseltamivir 인구 집단의 차이: Oseltamivir의 무작위 연구는 계절성 인플루엔자에서 수행되었지만 조류 독감을 대상으로 이루어지지 않았다.
결장암으로 인한 사망율 감소를 위한 S자 결장경 선별검사 중재법의 차이: 대변 잠혈 검사의 무작위 연구는 S자 결장경 검사의 잠재적 효과에 대한 간접적인 증거를 제공한다.
정신 분열증 치료제 선택 대조군의 차이: 새롭게 개발된 항정신병제제 투여군과 haloperidol 20 mg의 지속 투여한 군을 비교하는 일련의 임상 시험은 임상적으로 사용되는 용량의 haloperidol (20 mg 미만)에 비하여 새로운 항정신병약제의 효용성에 대하여 간접적인 증거를 제공한다.
고위험 당뇨병 환자에서 당뇨병 합병증 예방을 위한 Rosiglitazone 건강 결과의 차이: 무작위 임상 시험에서 Rosiglitazone은 생화학적 당뇨병 발병을 지연 시켰으나 당뇨병 합병증을 줄이지 못하였다.
TOOLS
Similar articles