Questionnaires represent an integral part in everyday medical practice and leading urological guidelines recommend their use in various sub-specialties of urology. They facilitate a time saving and structured assessment of current complaints and allow a systematic assessment of changes during a longer-term course. Many clinical trials are based on validated questionnaires as they represent the most reliable way to assess patient reported outcomes. Thus, a PubMed search reveals over 5,200 hits for the International Prostate Symptom Score (IPSS), over 3,300 for the International Index of Erectile Function (IIEF) and its short form IIEF-5 and more than 700 hits for the Chronic Prostatitis Symptom Index (CPSI).

Readability of health-related content represents a fundamental component of comprehensibility but is often neglected, which has especially been shown for patient education material [12]. Therefore, the USA National Institutes of Health (NIH) and the American Medical Association (AMA) recommend a 7th to 8th grade or 5th to 6th grade reading level for written health materials respectively [34].

Previously, Bergman et al. [5] analyzed the readability of 76 different health-related quality of life instruments and reported on a median 6.5th grade reading level. Like similar studies in different fields of medicine [67], they reported on generally satisfactory results. However, all of these studies evaluated the average readability of the complete questionnaires. This approach provides no information on potential variations in the readability of single-items. Thus, very difficult single-items can be covered up by easier ones and compromise a meaningful completion of a questionnaire and the informative value of its results [8].

The aim of our study was to assess readability of the most commonly used questionnaires in urology including a separate analysis of their single-items to identify questions that might be especially demanding for patients and, therefore, deserve particular attention when interpreting the results of such surveys.

We reviewed the guidelines published by the European Association of Urology (EAU) for recommended urological questionnaires. In addition, the 36-Item Short Form Health Survey version 2 (SF-36v2) questionnaire [9] was included into the analysis as a benchmark. This survey represents the most widely used generic measure of health-related quality of life, has already undergone a revision to improve comprehensibility [10] and is also recommended by the EAU guidelines.

The English versions of the questionnaires were assessed by copying each single-item into a Microsoft Word document (Microsoft Corp., Redmond, WA, USA). Several items within the surveys (e.g., sentence fragments with subsequent question choices) are not recognized by the readability-test programs as a complete sentence or a question. Therefore, incomplete phrases were combined with potential question choices to form and test only complete sentences as recommended elsewhere [11]. Response options which were single words and therefore are likely to score as very easy to read, were not assessed. Copyright notices, disclaimers, acknowledgments, author information, citations, and references were excluded from the analysis. Each extracted text was analyzed for readability using the software package Readability Studio Professional Edition version 2015 for Mac (Oleander Software, Ltd., Vandalia, OH, USA). Readability assessment tools used for the analyses are described in Table 1 [212131415].

Descriptive statistics using median and range were performed for all tests corresponding to a grade level (i.e., Flesch-Kincaid grade level [FKGL], Simple Measure of Gobbledygook grade level [SMOG], Coleman-Liau Index [CLI], Gunning-Fog Index [GFI]). The Flesch Reading Ease (FRE) formula is charted and results in a score between 0 and 100.

Totally 13 questionnaires were included to the analysis (Table 2) [916171819202122232425]. The median length of the questionnaires was 238 words with a range of 57 words (International Cystitis Symptom Index, ICSI) to 403 words (International Consultation on Incontinence Questionnaires-Male Lower Urinary Tract Symptoms long form, ICIQ-MLUTS-LF) for specific urological questionnaires. The SF-36v2 has a length of 546 words. Analysis of word complexity (3 or more syllables) of the questionnaires showed a range of 4% (ICIQ-Female Lower Urinary Tract Symptoms short form, ICIQ-FLUTS-SF) to 28% (IIEF-5). Analysis of word length showed 17.9% (IPSS) to 38% (Short form-Qualiveen, SF-Qualiveen) of words with six or more characters.

Calculation of grade levels (i.e., FKGL, SMOG, CLI, GFI) showed readability scores of 2.7th to 16.7th grade (Table 2). Easiest readability as calculated by median (range) grade level of FKGL, SMOG, CLI, and GFI was found for the ICIQ-FLUTS-SF (4.4 [2.7–6.4]), while the IIEF-5 showed the hardest readability (13.95 [11.2–16.7]). These results were confirmed by the FRE, where scores of 90 points (corresponding to ‘very easy’ to ‘easy’ readability) were found for the short and long form of the ICIQ-FLUTS. With a score of 42 points (corresponding to ‘difficult’ readability) the IIEF-5 proved to be the most difficult questionnaire (Fig. 1) [12]. Using the SF-36v2 as a benchmark, seven of the urological questionnaires performed better, while five showed a harder readability.

Analyses of single-items showed a wide range of readability scores for all of the different questionnaires. Fig. 2 shows the results of the three most commonly cited surveys according to a PubMed search (i.e., IPSS, IIEF, CPSI) using FKGL, SMOG, CLI, and GFI. Single-item assessments for the other questionnaires are presented in Fig. 3.

Assessing the single-items of all questionnaires by only using the FKGL, which has been used most frequently and is considered as the gold standard of calculation of reading levels [2627], readability scores still showed a wide range (Fig. 4). The widest range was found for the SF-36v2, incontinence quality of life (I-QOL), and Qualiveen-questionnaires. Based on the FKGL, a readability level above the recommended 8th grade level was found for the following numbers and proportions of single-items of the questionnaires: IIEF-5: 4 (80%), CPSI: 8 (61%), IIEF: 7 (47%), SF-36v2: 9 (25%), Qualiveen: 6 (20%), IPSS and SF-Qualiveen: 1 (13%), I-QOL: 2 (9%), MLUTS-SF: 2 (8%), FLUTS-LF: 2 (6%), MLUTS-LF: 1 (2%), ICSI and FLUTS-SF: 0%.

This study assessed the readability of questionnaires that are used in everyday urological practice and are recommended by leading guidelines.

A wide range of readability was found for the 13 questionnaires assessed, including results that clearly exceed the 7th to 8th grade or 5th to 6th grade reading level recommended by NIH or AMA respectively [34]. While questionnaires assessing male and FLUTS as well as incontinence symptoms (i.e., ICIQ-FLUTS-SF and -LF, ICIQ-MLUTS-SF and -LF, I-QOL, Qualiveen, SF-Qualiveen) are generally written very easy or easy, most of the other questionnaires (i.e., SF-36v2, IPSS, ICSI) did at least not clearly exceed recommended reading levels (Table 2). In contrast, the CPSI and the IIEF, including its short form, have a readability level that has to be considered as clearly too difficult according to all tests that were applied. These findings are largely consistent with assessments of questionnaires in other medical fields [56711].

As items exceeding patients' reading skills often induce them to give an invalid response or simply skip the item, readability analysis of questionnaires should not only report on mean results of the summarized text, but also assess each single-item separately [811]. Remarkably, even questionnaires with good average readability showed single-items clearly exceeding acceptable difficulty levels in this study (Figs. 3, 4).

Thus, based on the FKGL, the questionnaires IPSS, IIEF, and CPSI that are used most frequently in the literature have 13%, 47%, and 61% of question-items that are too difficult, respectively (Fig. 4). These numbers were even higher if other tests were applied (Fig. 2).

As reflected by the underlying formula of established assessment tools (Table 1), complexity of the sentence structure and the words used represent the main causes of poor readability. Koo and Yap [1] and Mossanen et al. [28] previously described that simplification of content is often accompanied by creation of longer sentences and words, and therefore, improved readability is not necessarily achieved. Dalziel et al. [29] could show, that the readability of patient education material could be improved by an average 3.1 grade level by simple substitutions of multisyllabic words. Further improvement has been shown to be achievable by adaption of sentence structure. Though such simplifications are not easily applicable to standardized and validated questionnaires, they might be considered if such questionnaires undergo revisions like it has been performed for the SF-36 (version 1) in the past [10] and more attention should be paid to readability if novel questionnaires are developed. Moreover, in the case of inconsistent or implausible results of questionnaires in clinical practice or trials, special attention should be paid to single-items that clearly exceed recommended readability levels (Figs. 2, 3).

The study has limitations that have to be addressed. As there is no consensus which readability formulas are most suitable for assessing questionnaires, we decided to use a combination of validated and well-established tests. Moreover, readability tests do not provide information about the complexity of the content and factors other than text quality (e.g., appealing layout, font types, images) also affect comprehensibility [30]. Only English versions of the questionnaires were assessed in the study and, therefore, the results are not transferable to the multitude of translations that are available.

The questionnaires that are used most frequently in urology mainly show a satisfactory overall readability. Inadequate readability levels were, however, not only found for individual questionnaires but also for single-items of all assessed questionnaires. This requires consideration for the interpretation of results and when developing novel health-related surveys.