Among the many designs for clinical trials, the randomized controlled trial (RCT) shows the highest level of evidence, which can reduce the bias most effectively. The results of RCTs are typically used as the basis for clinical application.1 Moreover, according to the increasing importance of evidence-based medicine, the importance of RCTs has drawn even more attention.2 However, even RCTs cannot eliminate all bias. Bias can occur at any of the steps in conducting trials and can lead to incorrect results.3 These incorrect results can harm patients and cause unnecessary medical expenses.4 There are some methods for preventing the use of incorrect information in clinical applications. Representatively, peer review is one of the methods by which articles are assessed before publication.5 Another method involves the methodological quality assessment of articles, in which the design, conduct, and analysis of clinical trials is evaluated.6 Assessment of the methodological quality of articles represents the overall quality of clinical trials.7 Methodological quality assessment of clinical trials can include the use of scales, individual markers, and checklists.2 Scales address randomization, double blinding, and dropout rates. The Jadad Quality Assessment Scale (Jadad scale) is the assessment tool that can assess the scales.8 Allocation concealment, as an individual marker, is an index that can assess the randomized allocation sequence. Allocation concealment is essential to avoid selection bias.9 The Jadad scale has been widely used because of its simplicity. However, the Jadad scale cannot evaluate individual markers. On the other hand, the van Tulder scale (VTS) and Cochrane Collaboration Risk of Bias Tool (CCRBT) include indices of individual markers.

Erectile dysfunction (ED) is an inability to achieve an erection sufficient to enable sexual intercourse. Many factors are associated with ED, including vascular, psychological, hormonal, and neurological factors.10 ED affects 150 million males worldwide, and the prevalence of ED is expected to more than double within 10 years due to the increasing lifespan.11 In Korea, some clinical trials about ED have been performed. In the present study, to suggest a direction for clinical trials about ED, we performed a methodological quality assessment of RCTs about ED.

A total of 20 RCTs were selected in an electronic search. All of the selected RCTs were conducted to evaluate drugs. Among the 20 RCTs, 19 RCTs were double-blind studies, and there were only 2 RCTs which described the randomization method adequately. Only 1 RCT presented allocation concealment. Eighteen RCTs mentioned review by an IRB and 15 RCTs were funded studies. Sixteen RCTs had been published in Science Citation Index (SCI) or Science Citation Index Expanded (SCIE) journals (Table 1).

The largest number of articles had been conducted to evaluate mirodenafil (n=6), and 5 articles evaluated udenafil (Table 2).

Among the 20 articles, only 1 article was assessed as having a low risk of bias, 3 articles were assessed as having a moderate risk of bias, and 16 articles were evaluated as having a high risk of bias by the CCRBT. In the Jadad scale, there were 17 high quality articles, while 19 articles were assessed as high quality articles by the VTS. In the Jadad scale and VTS, there was no statistical improvement according to the publication year. However, in the CCRBT, there was a statistical difference in quality according to the publication year (p=0.036) (Table 3).

Sixteen RCTs had been conducted at Seoul National University. Chonbuk National University and Yonsei University had each published three RCTs. Kyung Hee University and Pusan National University had conducted two RCTs each (Fig. 1).

Three RCTs had been published in The Korean Journal of Urology, in Asian Journal of Andrology and in British Journal of Urology International. And, 2 RCTs had published in Journal of Urology.

In the present study, the quality of published RCTs about ED conducted in Korean medical institutions was assessed. In the Medline database, about 900 RCTs that dealt with ED were found. However, there were only 20 RCTs about ED that had been conducted in Korea. Only one RCT was assessed as having a low risk of bias according to the CCRBT. On the Jadad scale and the VTS, there was no significant difference in the quality of RCTs according to the publication year.

ED is defined as the inability to attain or maintain a penile erection sufficient for successful vaginal intercourse.13 In a landmark population-based study, the crude incidence of ED was 26 per 1,000 man-years.14 Many factors such as vascular, psychological, neurological and hormonal abnormalities are associated with ED.15 Many therapeutic strategies have been introduced, such as psychological therapy, lifestyle modification, phosphodiesterase type 5 inhibitors (PDE5-Is), testosterone, and penile prostheses.16 Notably, PDE5-Is have been used as a first line of treatment for ED since sildenafil was introduced in 1998.17,18 Even though many treatment options have been proposed for ED, there are many limitations in the management of ED.16 Because of the high incidence of ED and therapeutic limitations, many studies have been conducted about ED. However, none have assessed the quality of RCTs about ED.

Some previous analyses have been performed on the quality of urological RCTs. Lee et al19 analyzed the quality of RCTs that had been published in the Korean Journal of Urology by the Jadad scale. Twenty-eight RCTs have appeared since 1991 in the Korean Journal of Urology. The mean Jadad scale score of those RCTs was 1.75. In addition, 8 RCTs were assessed as high quality. Moreover, Lee et al19 showed that only one article had adequate allocation concealment. In the present study, compared to Lee et al,19 the mean Jadad scale score (2.95±0.76) was higher. However, in the present study, only 1 RCT had adequate allocation concealment. According to Schulz and Grimes,20 the absence of concealment of allocation can damage the randomization in the process of the study, even if the randomization is well conducted. This may distort the results by up to 40%. Hewitt et al21 reported that about 46% of RCTs published in four journals (British Medical Journal, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) described inappropriate allocation concealment or failed to describe it clearly. Appropriate study design and presentation of allocation concealment should improve the quality of RCTs.

Chung et al2 assessed the quality of RCTs published in the Journal of Korean Medical Science. They reported that the Jadad scale score was 2.30±1.39 and the VTS score was 4.98±2.18. According to the CCRBT, 68.18% of the RCTs were assessed as having a high risk of bias, 20.45% of the RCTs were assessed as having a moderate risk of bias, and 11.36% of the RCTs were assessed as having a low risk of bias. Compared to the previous two studies, the quality of RCTs in the present study seemed to be higher according to the Jadad scale and VTS. This may be because almost all of the selected RCTs in the present study were drug studies. In studies that use a drug as the intervention, double blinding is easier than in non-drug studies because of to the possibility of using a placebo.

The present study showed that there were differences in the quality analysis of RCTs according to the three different tools. There is no consensus on what are considered highly accurate and valid quality assessment tools.2 The Jadad scale has simple quality assessment questions, but it does not include assessment of individual markers. Therefore, additional quality analyses were performed using the VTS and CCRBT to supplement in this regard.

The review from an IRB could serve as an acknowledgement of the feasibility of a study. IRB review should be an important factor in the improvement of the quality of RCTs.2 Funding of clinical trials should enable the establishment of a well-designed study and performance of well-organized research, resulting in many high quality articles.2,6,19 In the present study, almost all of the studies were funded study and had been reviewed by an IRB. This may be an one of the factors that improved the quality of the RCTs. The relatively small numbers of selected RCTs enabled the comparison of quality between funded studies and non-funded studies. It also enabled the comparison of quality according to whether the IRB reviewed the study or not.

The limitation of the present study is the small number of RCTs included. In addition, the assessment of quality was conducted by one researcher. This could have introduced bias because the selection of RCTs and quality assessment depends on the subjective judgment of the researcher.

The assessed number of RCTs about ED that have been conducted in Korea is small. Even though the quality of RCTs in the present study was superior to that of other previous studies, considering that almost all of the RCTs included in the present study were double blinded, we cannot conclude that the RCTs in the present study are superior to others. In any case, our results emphasize that urological researchers should focus on performing high quality studies that have been reviewed by an IRB, performing adequate randomization, and describing the appropriate concealment of allocation.