PDF
ePub
Citation
Print
I. Introduction
Given that obesity is not only a major healthcare issue, but also a risk factor for many chronic illnesses, it is important to manage this condition for disease prevention and health promotion. A proper diet and regular exercise, which play an important role in obesity management, need to be managed continuously. Thus, Web-based nursing interventions with less time and fewer location restrictions have proved to be highly effective for obesity management [1].
The increasing popularity and availability of small wireless devices, such as notebooks, PDAs, and smartphones, have led to their introduction in the field of healthcare [2]. According to a review by Mosa et al. [3], many smartphone applications for healthcare professionals, medical or nursing students, and patients have been developed and evaluated. They concluded that the smartphone can play a very important role in disease self-management as well as the education and remote monitoring of patients.
According to a survey carried out by the mobile market research firm Research2guidance, approximately 17,000 mobile healthcare applications were registered over 2 years preceding 2010 [4]. Liu et al. [5] conducted an analysis of iOS-based mobile healthcare applications, and found that 1,056 applications had been registered in the medical category by January 2011, with 1,004 in the healthcare and fitness category.
The smartphone can be used as a handheld computer to provide information, just like the Internet. As with Web-based healthcare information, the most important issue related to smartphone healthcare-management applications is the provision of accurate, valid, reliable, and complete information [6]. According to Gan and Allman-Farinelli [7], only 8 out of 92 obesity-management applications provided quality health information, suggesting that an evaluation of the quality of information provided by smartphone healthcare applications is urgently required.
To the best of our knowledge, there has been no previous attempt to evaluate the information quality of Korean healthcare applications. With this background, this study analyzed the smartphone obesity-management applications developed and available in Korea, and the quality of the information that they provide.
II. Methods
A search was undertaken for Korean iOS-based obesity-management smartphone applications. These were categorized according to their main purpose, intervention used, price, type of developer, user rating, and the number of users who rated the application, and the quality of the health information that each application provides was evaluated using a Silberg score [8] modified by Griffiths and Christensen [9]. We limit our search to the iOS platform since Apple provides a reliable search application programming interface (API) for iOS-based applications.
1. Selection of Applications
Search API software (provided by Apple) was used to find iOS-based applications for obesity management between September 23 and September 27, 2013. The search was limited to applications developed in Korea and written in Korean. Seven keywords were combined in seven different ways, as follows: 'obesity + management,' 'weight + management,' 'weight + loss,' 'weight + exercise,' 'weight + diet,' 'weight + calories,' and 'diet.' An example query is "http://itunes.apple.com/search?term='obesity'+'management'&country=kr&media=software".
The search yielded a total of 269 applications. After reviewing the descriptions and applications, 80 redundant applications were excluded, as were 9 applications not related to obesity, 7 applications dealing with comorbidities, such as hypertension and diabetes in addition to obesity management, and 3 applications not for sale. Another 22 applications were excluded after comparing the intervention methods used in the applications with the three obesity-management interventions (diet, exercise, and behavioral modification) recommended in the clinical guidelines developed by the National Heart, Lung, and Blood Institute (NHLBI) [10] and the National Institute for Health and Clinical Excellence (NICE) [11]. In total, 148 applications were downloaded and analyzed for classification. Furthermore, 44 applications that did not provide health information were also excluded; the quality of health information in the remaining 104 applications was evaluated (Figure 1).
2. Analysis of Obesity-Management Applications
The main purpose of the applications was stratified into four categories, as follows (the first three categories were developed in a previous study [12]):
Providing health information on obesity management-provides Web-based healthcare information via mobile devices.
Measuring and monitoring with embedded sensors-measures and collects physical information using sensor devices embedded in smartphones, and helps the user to monitor their physical information by themselves.
Managing obesity by professional hospital healthcare providers-transmits physical information collected using sensors to healthcare facilities, and seeks advice from healthcare providers.
Documenting and displaying physical information-collects physical information by manual input and displays the information using statistics and graphs without any interpretation.
The type of intervention was categorized according to the clinical guidelines published by the NHBLI and NICE into 1) providing diet information, 2) providing exercise information, 3) providing both diet and exercise information, 4) monitoring weight, 5) providing diet information and monitoring, 6) providing exercise information and monitoring, or 7) providing both diet and exercise information and monitoring. The apps are classified as 'free' or 'charged for' and the price of the charged-for apps is analyzed separately. The type of developer was classified as 1) an individual, 2) a vendor, 3) a research institute, or 4) unknown. The user rating and the number of users who rated the application that was provided by iTunes were collected.
3. Evaluation of the Quality of Health Information
1) Evaluation scale
Since there is currently no smartphone health-information quality assessment tool available, the 9-point Silberg scale, adapted by Griffiths and Christensen [9], was used. Systematic review of Eysenbach et al. [13] assessing the quality of health information available for consumers on the Web divides information quality criteria into five classes: technical criteria, design, readability, accuracy, and completeness/comprehensiveness/coverage/scope. The Silberg scale assesses the most commonly used technical criteria for evaluating information quality [14], defined as general, domain-independent criteria, with reference to the question of how the information is presented or what meta-information is provided [13].
The Silberg scale assesses authorship (i.e., whether the authors, their affiliations and their credentials are clearly identified), attribution (i.e., whether the sources and references are mentioned), disclosure (i.e., whether the ownership of the site and sponsorship are disclosed), and currency (i.e., whether the application had been modified in the previous month and whether the date created or modified is specified). One point is awarded for the presence in an application of each of these items, with a maximum score of 9 points (Table 1).
2) Evaluation methods
Four nurse informatists who majored in nursing informatics in graduate school and had experience of developing healthcare mobile applications were trained until an inter-rater reliability of 1.0 was reached. They used the Silberg scale to evaluate the 104 applications (30 applications per nurse).
The scores for each item on the Silberg scale were analyzed using descriptive statistics, such as frequencies, means, and standard deviations. A t-test was used to compare the quality scores relative to price. The Pearson correlation coefficient was used to study the correlation between user rating and health-information quality.
III. Results
1. Classification of Obesity-Management Applications
The primary use of most of the obesity-management applications (n = 146; 98.65%) yielded in the search was health and fitness. As seen in Table 2, out of the original 148 applications, the main purpose of 104 (70.27%) was classified as 'providing health information on obesity management', that of 31 (20.95%) was labeled as 'documenting and displaying physical information', and that of 12 (8.11%) was classified as 'measuring and monitoring with embedded sensors'. Only one of the applications was classified as 'managing obesity by professional healthcare providers of hospitals'.
The 104 applications classified as 'providing health information on obesity management' were further stratified according to the type of intervention. The most frequent type was 'providing exercise information' (n = 36; 34.62%), followed by 'providing both diet and exercise information and monitoring' (n = 22; 21.15%), 'providing exercise information and monitoring' (n = 21; 20.19%), 'providing diet information and monitoring' (n = 8; 7.69%), 'providing both diet and exercise information' (n = 8; 7.69%), 'monitoring weight' (n = 7; 6.73%), and 'providing diet information' (n = 2; 1.92%). Sixty-one (41.22%) of the applications were charged for, and 87 (58.78%) were free of charge. The mean price of the charged applications was US$1.88. Regarding the developers, 108 applications (72.97%) were developed by the vendors, and one (0.68%) had unidentifiable developer information, such that they were linked to either websites without developer information or invalid or inaccessible websites.
Regarding the user rating, 93 applications (62.84%) had a rating score, and 55 (37.16%) did not. The mean rating for the 93 applications with a rating score was 3.68, with the number of users who participated in that rating ranging from 5 to 1,747 (mean ± standard deviation, 124.27 ± 245.84). In 13 applications (13.98%), fewer than 10 users participated in the rating process.
2. Information Quality Evaluation
The Silberg score of the 104 applications evaluated by the four participating nurse informatists was 4.55 ± 1.10 (range, 2-8 out of a possible 9 points). Only three of the applications (2.88%) had a Silberg score greater than or equal to 7 points. The mean Silberg score by item ranged from 0.22 out of 2 for attribution to 1.25 out of 2 for currency (Table 3), while those for authorship and disclosure were 1.79 out of 3 and 1.29 out of 2, respectively. The Silberg score for attribution was very low because few of the applications provided information sources (n = 20; 19.23%), and only two (1.92%) applications provided references.
The 'creation/last-modification date' item under the currency category scored the highest because every application is automatically assigned creation and modification dates when it is registered and updated.
The mean Silberg scores of the 41 charged for applications and the 63 free applications were 4.20 and 4.75, respectively; the difference was not statistically significant (t = 2.62, p = 0.12).
The Pearson correlation coefficient between information quality and the user rating was 0.09 (p = 0.47).
IV. Discussion
Of the obesity-management applications yielded by the API search, 146 (98.65%) were categorized as 'health and fitness' in terms of their primary use. This could be due to users considering obesity management as belonging to the beauty and health-promotion domains rather than the medical domain. The applications were classified according to their main purpose; most belonged to the 'providing health information on obesity management' category. Only 12 out of 148 applications could be classified under the 'measuring and monitoring with embedded sensors' category classified by Lee et al. [12]. This can be attributed to the fact that embedded sensors are not used as often to make physiological measurements for obesity management as they are for other conditions, such as hypertension, for which heart rate and blood pressure are measured [15]. Furthermore, only one application could be classified as 'managing obesity by professional healthcare providers of hospitals.' Professional healthcare providers may consider obesity management without any comorbidity as lifestyle management that should be handled at home rather than healthcare to be provided in a hospital. However, since obesity is a risk factor for many chronic illnesses, it is important not only for the client to manage his/her lifestyle at home, but also for him/her to receive feedback and supervision from healthcare professionals to prevent other obesity-related health problems. Thus, it is important for obesity-management applications to have a component allowing the users to access healthcare professionals if necessary.
Obesity-management applications were classified by the type of intervention; most belonged to the 'exercise' category. According to the obesity-management guidelines, the combination of diet, exercise, and self-monitoring of diet and exercise is important to maximize the likelihood of success from the process [10]. However, only 22 (21.15%) of the applications provided all of these three components of obesity management. Thus obesity-management applications should provide information about all of these components.
The mean Silberg score of the 104 smartphone obesity-management applications belonging to the 'providing health information on obesity management' category that were evaluated in this study was 4.55 out of 9. This is much lower than the scores that have been reported previously. For example, the mean Silberg score was 6.47 out of 9 in the study of Zermatten et al. [16] on the quality of Web-based information on depression, and 4.80 out of 9 in that of Morel et al. [17] on the quality of Web-based information on bipolar disorder.
The authorship of the smartphone applications was evaluated using developer information. Although there were 103 mobile applications (99.04%) with developer information, using the links provided by the developers we were able to identify their affiliations in only 78 applications (75.00%) and their credentials in only 5 (4.81%). A systematic review of the quality of consumer health information on the Web by Eysenbach et al. [13] reported authorship, affiliation, and credential disclosure rates of 43.2%, 58.5%, and 29.4%, respectively. The present study found a much higher disclosure rate for authorship and a much lower disclosure rate for credentials. This could be due to the developers of the applications being regarded as the authors in this study. However, the authors of the websites may differ from the developers of the websites; they are the domain experts with credentials, but the developers of the applications are not necessarily domain experts. One way to improve the reliability of smartphone health-application authorship would be to make it mandatory to provide author affiliations and credentials when registering them.
Regarding the application attribution, only two applications (1.92%) provided references with hyperlinks to the source. The present study had very low scores for attribution criteria compared to these other Web-based programs. References were provided in 31.1% of the websites reviewed in a systematic study by Eysenbach et al. [13]. One explanation for this discrepancy is that contrary to the case for Internet websites, smartphone applications do not have sufficient space to provide information sources and references. Again, the quality of smartphone health applications could be improved by requesting that information sources or references be provided when registering them.
The 31 applications that disclosed their sponsorship (29.81%) were developed by a private company to promote the sales of its weight-control products. The disclosure rate (6.6%) of the websites' sponsorship according to the study of Eysenbach et al. [13] was lower than the rate of the present study. To create context and to avoid misunderstanding of the applications, it is recommended that the sponsorship of the applications be disclosed.
Regarding currency, the creation or last-modification dates were specified in all of the smartphone applications-these two items are mandatory for mobile applications. According to the study of Eysenbach et al. [13], only 36.3% of the websites disclosed the date of creation or date of last update. Although 89 applications (85.58%) in the present study had been updated more than once since their creation, only 26 had been modified within the previous month, which is the currency criterion for the Silberg scale as modified by Griffiths and Christensen [9].
The quality of health information did not differ between the charged for and free applications. Moreover, there was no significant correlation between the users' rating and the quality of health information. This finding is similar to that of Demidowich et al. [18], who explored the usability scores and user rankings of Android smartphone applications on diabetes self-management applications. According to Demidowich et al. [18], this could be due to existing ranking systems being inadequate or unreliable for determining the merits of an application.
The Korean smartphone obesity-management applications were evaluated with respect to the quality of information that they provide. To the best of our knowledge, this is the first study to have evaluated smartphone obesity-management applications in Korea; however, its findings are subject to several limitations. First, we evaluated the quality of smartphone health information using the Silberg scale, a tool that was developed to assess the quality of medical information on the Web [9]. We recommend that these applications be further evaluated with other quality criteria and methods used to evaluate health information on the Web, such as assessment based on their design, readability, accuracy, and completeness/comprehensiveness/coverage/scope [14]. We also recommend the development of a new, specific tool for evaluating smartphone health information, given the similarities and differences between Web-based and smartphone health information. Second, we limited our search for smartphone obesity-management applications to those available on the iPhone platform. Further study should include applications that are available on other platforms. Finally, the applications were divided between only four nurse informatists for evaluation, although they were trained until an inter-rater reliability of 1.0 was reached.
Despite the limitations we noted, this study is the first research focusing on Korean obesity-management smartphone applications. The findings of this study suggest that the quality of information provided by smartphone applications in the healthcare domain urgently needs to be evaluated to prevent smartphone users being misinformed by those applications.
XML Download