Abstract
Background:
The development of mobile apps in healthcare facilities helps users perform self-care effectively.Objectives:
This study aimed to evaluate the quality, functionality, and features of Persian language diabetes mobile apps.Methods:
Persian language diabetes self-management mobile apps were searched in the Android and iOS App Stores using relevant keywords such as "diabetes" and "blood sugar." Twelve apps met the inclusion criteria. Nine evaluators assessed the apps' quality based on the Mobile App Rating Scale (MARS), their functionality according to the IMS institute for healthcare informatics app functionality scoring system, and their features and content using a checklist.Results:
The mean MARS score of the apps was 3.02 out of 5. Among the four dimensions of engagement, aesthetics, functionality, and information quality, functionality had the highest mean ± SD score (3.64 ± 1.13). Most of the apps used 5 - 7 out of the 11 defined functionalities. All self-management apps included the functionality of monitoring blood glucose levels, and the majority had health status monitoring features.Conclusions:
The mobile apps had several drawbacks, including a lack of engagement strategies, insufficient evidence-based information for patients, limited guideline-based self-management functionalities, failure to evaluate clinical effectiveness in trials, and an absence of mutual communication with healthcare providers.Keywords
1. Background
Diabetes is one of the most common chronic diseases, with an increasing prevalence in most parts of the world, especially in developing countries (1). The STEPs survey in Iran in 2016 showed a prevalence rate of 11% for diabetes among Iranians over 30 years old. Recent studies have indicated a rapid rise in the prevalence of diabetes in Iran. This increased prevalence imposes a heavy financial burden on patients, families, society, and the healthcare system (2). Effective self-management of diabetes requires the active participation of diabetic patients in the treatment and management of their condition. This includes patient education, maintaining a healthy diet, physical activity, medication management, tracking and monitoring health data to adjust treatment and prevent complications, and managing psychosocial problems (3).
Mobile health (mHealth), a branch of eHealth, is defined as the provision of health services through mobile communication platforms and apps to achieve health goals. The widespread adoption of smartphones and the popularity of health apps have triggered the development of mHealth initiatives (4). The development and use of mobile apps in healthcare have increased substantially in recent years. Diabetes-related apps comprise 16% of all health apps and offer various functionalities, including tracking blood glucose measurements, nutrition databases, carbohydrate consumption tracking, physical activity and weight monitoring, data sharing with physicians and peers, social support, messaging, and reminders (5). These functionalities improve diabetes control (6).
A wide variety of mobile apps have been designed and used in the healthcare domain. Since patients and healthcare providers may make important decisions based on the information presented by these apps, the information must be accurate and reliable (7, 8). Previous studies have reported that the quality and functionality of non-Persian-language diabetes self-management apps were lower than optimal. There are also reports of the very low quality and validity of Persian apps for the management of chronic diseases (5, 6, 9-11).
2. Objectives
However, no study has assessed the quality and functionality of Persian diabetes management apps. Therefore, this study was conducted to evaluate the quality, functionality, and features of Persian-language diabetes self-management apps using the Mobile App Rating Scale (MARS). Additionally, the study aimed to describe the functionalities and features of these apps.
3. Methods
3.1. Search Strategy
The Android and iOS App Stores were searched for Persian diabetes self-management apps between December 25, 2022, and January 25, 2023. Four Persian app stores, including Cafe Bazaar, Sib, Sibche, and Myket, were selected for the search. The search was conducted in each app store's search bar using Persian and English keywords such as diabetes, blood glucose, blood sugar, diabetes prevention, diabetes control, blood sugar treatment, and diabetes treatment. Two research team members independently searched the app stores and removed duplicate apps. Next, the apps were evaluated based on their titles and descriptions in the app store for primary screening.
3.2. Inclusion/Exclusion Criteria
Diabetes self-management apps in Persian that focused on at least one of the following subjects: Blood glucose management, nutrition, physical activity management, drug management, or control of diabetes comorbidities were included in the study. Apps exclusively related to health education or those targeting healthcare providers were excluded. The final list of apps was then prepared for evaluation.
3.3. Data Extraction and Evaluation
Three groups of raters, including internists working in diabetes centers, diabetic patients, and medical informatics and health information management specialists with experience in the design and assessment of mobile apps, evaluated the applications. These raters downloaded the apps and assessed them independently in terms of quality, functionality, and features. Training sessions were held for the rater groups to familiarize them with the research instruments and the installation and use of mobile apps. Before the evaluation, a few apps were piloted. To ensure the quality and consistency of the assessments, an electronic educational manual was provided to guide the evaluators and introduce the research instruments. Each rater downloaded the apps onto their cell phone and evaluated them using an electronic data collection form.
3.4. Data Collection Form
3.4.1. Part One: General Information
General information about the apps was extracted from their descriptions in the app store. This information included the platform, developer, version, last update date, cost, number of downloads, user ratings, and the number of users who rated the app (if available).
3.4.2. Part Two: Quality
The quality of the apps was assessed using the MARS. This scale is a simple, reliable, and objective tool for evaluating the quality of mobile apps from different aspects. Two English language experts translated the scale from English to Persian. The research team members agreed upon the translated version in a meeting. Then, this version was presented to an expert panel to measure its face and content validity. The panel members were asked to state their opinion about each question in terms of necessity, clarity, simplicity, and relevance. The answers were coded as 3 = completely, 2 = relatively, and 1 = not at all. Additionally, according to the guidelines, the panel members were asked to provide corrective comments for each question. The panel comprised three health informatics experts and three health information management experts with experience in questionnaire content validity assessment. The comments of the panel members regarding the necessity of the items were evaluated using the content validity ratio (CVR).
3.4.3. Content Validity Index
The Content Validity Index (CVI) is the mean CVR of all items of a validated tool and indicates the comprehensiveness of judgments related to the validity or functionality of the final instrument. The higher the final content validity, the closer the CVI is to 0.99, and vice versa.
To determine face validity, the final version of the MARS was evaluated in terms of relevance, simplicity, and clarity separately using a four-point scale as follows: (1) completely relevant/simple/clear; (2) relevant/simple/clear but requires minor revision; (3) requires major revision; and (4) not at all relevant/simple/clear. A question was confirmed if 50% of the respondents chose the first answer or 70% chose the first two answers. All 19 items were confirmed after CVR calculation, resulting in a CVI value of 0.96.
The test-retest method was used to evaluate the reliability of the scale. For this purpose, the evaluators randomly selected 20% of the apps and evaluated them twice at a two-week interval using the final version of the scale, and the scores were compared. The test-retest ICC was 0.96. Cronbach’s alpha was used to evaluate the internal consistency of the scale, which was 0.97.
The MARS consists of 19 items covering 4 dimensions: Engagement (entertainment, interest, customization, interactivity, and target group), functionality (performance, ease of use, navigation, and gestural design), aesthetics (layout, graphics, and visual appeal), and information quality (accuracy of app description, goals, quality of information, quantity of information, quality of visual information, credibility, and evidence base). Each item is assessed on a 5-point Likert scale from 1 (inadequate) to 5 (excellent). In addition to the objective assessment, four items are used to assess the subjective quality of the app, and five items are used to assess the perceived impact of the app (knowledge, attitudes, intention to change, help-seeking, and behavioral change).
3.5. Functionality
The functionality of the apps was assessed using the IMS criteria. These criteria focus on seven domains: Inform, instruct, record, display, guide, remind or alert, and communicate, and four subdomains: Collect data, share data, evaluate data, and intervene (12).
3.6. Features and Content
To evaluate the features and content required for diabetes self-management apps, a checklist was prepared based on previous studies (3, 5, 10). This checklist included health indexes, behavior monitoring, reminders, health education, communication with healthcare providers and other patients, and data sharing.
3.7. Statistical Analysis
To measure the quality of the apps, the mean score of each domain and the overall scale were calculated. The mean of the scores given by all evaluators was used to assess the overall mean score. Objective quality items were calculated separately. For the functionality of the apps, the scores were ranked from 0 to 11. Mean, standard deviation, and frequency were used to summarize the scores of MARS domains as well as the functionality and feature scores of each app. The Kruskal-Wallis test was used to compare the scores of engagement, aesthetics, functionality, and information quality among the three rater groups.
4. Results
After removing duplicates and apps in languages other than Persian, 121 out of the 140 apps identified in the initial search remained and were screened. Two researchers downloaded these apps independently. After an initial assessment, 104 apps were excluded because they did not record blood glucose levels. Of the 17 apps that met the inclusion criteria, four apps (Shekarban, Smart Medico, Diapad, and Diana App) were not accessible due to technical problems. Finally, 13 apps were evaluated. Figure 1 presents the process of search and screening.
Selection and screening process based on the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines
4.1. Specifications of the Apps
All 13 apps had Android versions. The developers of these apps were either not mentioned or were individuals without university affiliations. Only four apps were updated within the past year. Two apps, Idea and Sinacare, had the largest number of active installations (about 20,000). Only two apps needed to be purchased from the app stores; the others were free. Four apps required initial registration, and one app needed a password for login.
4.2. Behavior Change Techniques
Among the 19 behavior change techniques, three were used in the evaluated apps: Monitoring/tracking in 13 apps, information/education in two apps, and feedback in one app.
Figure 2 presents the mean scores of all MARS items.
The mean scores of all Mobile App Rating Scale (MARS) items
4.3. Quality of Apps
Three rater teams, each including three members (three physicians, three diabetic patients, and three health informatics experts), assessed each of the 13 apps. The Kruskal-Wallis test showed significant differences among the three groups in scores for aesthetics (P = 0.37), functionality (P = 0.44), and information quality (P = 0.001). However, no significant difference was found in the median scores for engagement among the three groups (P = 0.08).
The mean MARS score of all apps was 3.05. Overall, the mean scores of six apps were below three, indicating that these apps did not have acceptable quality. Among the four dimensions of engagement, aesthetics, functionality, and information quality, functionality had the highest mean ± SD score (3.64 ± 1.13), followed by aesthetics (3.17 ± 1.21), engagement (3.11 ± 1.32), and information quality (2.27 ± 1.36). In general, the apps received higher scores in performance, navigation, and gestural design and lower scores in evidence base, credibility, and information quantity.
The results of the subjective quality assessment of the apps are shown in Table 1. None of the apps were interesting enough for the raters to use them more than 50 times over the next 12 months. Although most of the apps were free, the raters were willing to pay only for two apps. Five apps received three stars. Regarding the perceived effect of these apps on behavior change, the raters agreed or completely agreed that about 40% of the apps could improve users' awareness, attitudes, intentions to change, and help-seeking behaviors.
Apps Subjective Quality Assessment
| Statement App Subjective Quality | App (n) |
|---|---|
| Recommending the app to people who might benefit from it: | |
| Not at all | 1 |
| A few people | 5 |
| Maybe | 3 |
| Many people | 4 |
| Definitely | 0 |
| Times you would use this app in the next 12 months if it was relevant: | |
| None | 1 |
| 1 - 2 | 2 |
| 3 - 10 | 6 |
| 10 - 50 | 4 |
| > 50 | > 0 |
| Eager to pay for apps | |
| No | 7 |
| Maybe | 4 |
| Yes | 2 |
| Overall star rating of the app | |
| One of the worst apps I’ve used ★ | 1 |
| Poor ★★ | 3 |
| Average ★★★ | 5 |
| Good ★★★★ | 4 |
| One of the best apps I've used ★★★★★ | 0 |
| These apps can improve the: | |
| Awareness | 1 |
| Knowledge | 1 |
| Attitudes | 2 |
| Intention to change | 2 |
| Help-seeking | 3 |
| Behavior change | 4 |
4.4. Functionality of Apps
On average, the evaluated apps supported 5 - 7 out of the 11 functionalities. None of the apps had all 11 functionalities. The only functionality present in all apps was "data collection." None of the apps provided guidance. Twelve apps could record data, and ten apps could display data graphically. Only seven apps offered the possibility of evaluating data entered by users, ten apps supported data sharing, and five apps could send reminders and establish indirect communication with healthcare providers. Figure 3 presents the IMS criteria scores of the apps.
IMS criteria scores of the apps
4.5. Features of Apps
The apps evaluated in this study had several features. Most offered health status monitoring, and blood glucose monitoring was a feature in all apps. Regarding health status monitoring, only three apps had features to record physical activity, food intake, and BMI. Less than half of the apps presented health information to users; most of the information was related to the definition of diabetes, its complications, and control methods, while the least information was about blood pressure and stress management. Setting personal reminders for self-management activities was another key feature, with reminders for drug use (7/13, 53%) and blood sugar tests (5/13, 38%) being the most common. Three apps provided the opportunity for online consultation with physicians. More than half of the apps allowed data sharing with healthcare professionals, family members, and friends. The frequency of the app features is presented in Table 2.
The Frequency of the Features of the Evaluated Apps
| Feature | Number |
|---|---|
| Health status monitoring | |
| Blood glucose level | 13 |
| Blood pressure | 6 |
| HbA1C level | 7 |
| Physical activity level | 3 |
| Food recording | 3 |
| Weight recording | 6 |
| BMI recording | 3 |
| Drug recording | 4 |
| Health information | |
| Information on diabetes | 4 |
| Blood sugar control methods | 4 |
| Diabetic complications | 4 |
| Types of diabetes and ant diabetic dug classes | 3 |
| Blood pressure control methods | 1 |
| Effective physical activity | 3 |
| Healthy diet | 4 |
| Weight control | 4 |
| Stress control | 1 |
| Setting the reminder | |
| Blood sugar measurement | 5 |
| Blood pressure measurement | 3 |
| Weight calculation | 3 |
| Physical activity | 3 |
| Drug use | 7 |
| Healthy diet | 4 |
| Appointment with healthcare provider | 3 |
| Possibility to book an appointment with a specialist | |
| Online consultation | 3 |
| Data sharing with | |
| healthcare professionals | 7 |
| Family members and friends | 7 |
| Communication through chat rooms or weblogs | 1 |
| Assessing the risk of diabetic complications | 2 |
| Selling drugs and medical equipment | 2 |
| Displaying available clinics and drug stores | 3 |
5. Discussion
The present study evaluated the quality, functionality, and features of Persian-language diabetes self-management apps. The findings indicated that the overall quality of these apps is relatively low. Although these apps offer several features, many lack crucial elements such as data credibility, evidence base, adherence to guidelines, and scientific evidence for clinical effectiveness.
Similar to other studies investigating the quality of diabetes-related apps (5, 13), our results showed that while these apps perform well in terms of functionality and aesthetics, they are weak in terms of information and engagement. Engagement is a key factor in attracting users and promoting their health outcomes (11). Most evaluated apps lacked effective strategies to enhance user experience and provide entertainment. For instance, only one app allowed users to enter photographs of food instead of manual data entry. Manual data entry can increase user workload, cause data entry errors, and result in poor engagement, reduced adherence, and eventual abandonment of the app.
In addition to engagement, the quality and evidence base of the information in these apps need improvement. Consistent with previous studies on the quality and credibility of mobile apps (5, 6, 14), our results showed inadequate educational content in the evaluated apps. The sources of information for some apps were unclear, and where sources were clear, they were not evidence-based. The lack of evidence-based information may mislead users and increase the risk of negative health outcomes (6, 11).
Another important finding was that few apps offered features to customize reminders or deliver personalized information to users. Despite the well-proven advantages of mHealth in providing information at any time (7, 15), most apps failed to deliver necessary information and health training to diabetic patients.
The majority of the apps included only some of the features recommended by guidelines. Less than half of the apps supported the recording and management of diabetes complications and comorbidities. For example, most apps did not support activities such as recording blood pressure, food intake, weight follow-up, and physical activities. According to several studies, features like stress and emotion management, cholesterol level tracking, and even ophthalmic examinations are necessary (5, 14, 15). Compared to apps studied in previous research, Persian-language apps offer fewer features.
Regarding communication with healthcare professionals, the results of this study showed that only one app provided the opportunity for online consultation. Data sharing in the apps was possible through PDF or Excel files, which users could forward to family members or healthcare professionals via social media or email. None of the apps featured chat rooms or weblogs for communication with healthcare professionals. However, similar studies have emphasized the importance of such features (5, 11, 15).
Considering the increasing number of diabetic patients in Iran, telemedicine functionalities like teleconsultation and tele-visits provide an opportunity to offer high-quality, accessible, and practical services over a distance. The results indicated a need for incorporating these services into the apps. Some studies have discussed the necessity of connecting mobile apps with other health information systems (4, 5).
According to the evaluators, the effectiveness of these apps in changing users' behavior and improving health outcomes was less than 45%. Although some apps in previous studies utilized several behavior change techniques, these techniques were used in a limited number of apps in the current study. Consistent with previous studies (14, 16), the apps evaluated in this study mostly used data sharing, training, and assessment.
None of the apps examined in this study were evaluated for effectiveness in clinical trials. In other studies, the effectiveness of a few apps has been confirmed in clinical trials (5, 14), indicating that the effectiveness of Iranian diabetes self-management apps needs to be assessed. Previous studies (5, 11) emphasized the collaboration of health providers with the app development team. Similarly, our results suggest that experts in designing mobile apps should collaborate with healthcare professionals in the field of diabetes to design a diabetes self-management app using standard app design processes.
This study was the first comprehensive evaluation of Persian diabetes-related mobile apps. Other similar studies conducted in Iran only evaluated English-language apps (11). All diabetes-related apps that provided blood sugar recording features were evaluated in this study. Internet app stores, including the Android and iOS App Stores, were searched, and the quality, functionality, and features of the apps were evaluated using the MARS.
The apps evaluated in this study had shortcomings such as poor engagement strategies, lack of evidence-based information, limited guideline-based self-management functionalities, unproven effectiveness in clinical trials, and lack of interaction between healthcare providers and health information systems. Future studies should investigate the effectiveness of Persian mobile apps in improving health outcomes in patients with diabetes.
In this study, it was not possible to involve the main target group, i.e., diabetic patients, who may have different characteristics such as low eHealth literacy and older age. Instead, the apps were evaluated by raters using the MARS tool after short-term use. Further studies are required to evaluate these apps with the participation of a diverse population of diabetic patients. Concerning other important aspects of mobile apps, such as data confidentiality and security, only password-protected entry was evaluated in this study; therefore, more studies need to be carried out in this regard.
References
-
1.
Saeedi P, Petersohn I, Salpea P, Malanda B, Karuranga S, Unwin N, et al. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: Results from the International Diabetes Federation Diabetes Atlas, 9(th) edition. Diabetes Res Clin Pract. 2019;157:107843. [PubMed ID: 31518657]. https://doi.org/10.1016/j.diabres.2019.107843.
-
2.
Mirzaei M, Rahmaninan M, Mirzaei M, Nadjarzadeh A, Dehghani Tafti AA. Epidemiology of diabetes mellitus, pre-diabetes, undiagnosed and uncontrolled diabetes in Central Iran: results from Yazd health study. BMC Public Health. 2020;20(1):166. [PubMed ID: 32013917]. [PubMed Central ID: PMC6998152]. https://doi.org/10.1186/s12889-020-8267-y.
-
3.
Nie L, Xie B, Yang Y, Shan YM. Characteristics of Chinese m-Health Applications for Diabetes Self-Management. Telemed J E Health. 2016;22(7):614-9. [PubMed ID: 27171016]. [PubMed Central ID: PMC5824655]. https://doi.org/10.1089/tmj.2015.0184.
-
4.
Weinstein RS, Lopez AM, Joseph BA, Erps KA, Holcomb M, Barker GP, et al. Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers. Am J Med. 2014;127(3):183-7. [PubMed ID: 24384059]. https://doi.org/10.1016/j.amjmed.2013.09.032.
-
5.
Gong E, Zhang Z, Jin X, Liu Y, Zhong L, Wu Y, et al. Quality, Functionality, and Features of Chinese Mobile Apps for Diabetes Self-Management: Systematic Search and Evaluation of Mobile Apps. JMIR Mhealth Uhealth. 2020;8(4). e14836. [PubMed ID: 32255432]. [PubMed Central ID: PMC7175187]. https://doi.org/10.2196/14836.
-
6.
Ghazi-Saeedi M, Rostamniakan-Kalhori S, Yasini M, Nouri R. [Mobile Health Apps: References Reliability Analysis]. Health Inf Manage. 2017;14(1):44-8. Persian.
-
7.
Abbasi R, Nabovati E, Raeesi A, Ostadmohammadi F. [Investigating the Quality of Persian Mobile Applications Related to Patients with Chronic Diseases]. J Health Biomed Inform. 2020;7(3):273-81. Persian.
-
8.
Veazie S, Winchell K, Gilbert J, Paynter R, Ivlev I, Eden KB, et al. Rapid Evidence Review of Mobile Applications for Self-management of Diabetes. J Gen Intern Med. 2018;33(7):1167-76. [PubMed ID: 29740786]. [PubMed Central ID: PMC6025680]. https://doi.org/10.1007/s11606-018-4410-1.
-
9.
Rangrazejeddi F, Farrahi SR, Anvari S, Sharif RS. The Most Popular Iranian Smartphone Applications for Traditional Medicine: A Quality Assessment. Shiraz E-Med J. 2018;19(Suppl). e66306.
-
10.
Salari R, Niakan Kalhori SR, Fatehi F, Ghazisaeedi M, Nazari M. Determining minimum set of features for diabetes mobile apps. J Diabetes Metab Disord. 2019;18(2):333-40. [PubMed ID: 31890658]. [PubMed Central ID: PMC6914742]. https://doi.org/10.1007/s40200-019-00417-y.
-
11.
Kalhori SRN, Hemmat M, Noori T, Heydarian S, Katigari MR. Quality Evaluation of English Mobile Applications for Gestational Diabetes: App Review using Mobile Application Rating Scale (MARS). Curr Diabetes Rev. 2021;17(2):161-8. [PubMed ID: 32619173]. https://doi.org/10.2174/1573399816666200703181438.
-
12.
Masterson Creber RM, Maurer MS, Reading M, Hiraldo G, Hickey KT, Iribarren S. Review and Analysis of Existing Mobile Phone Apps to Support Heart Failure Symptom Monitoring and Self-Care Management Using the Mobile Application Rating Scale (MARS). JMIR Mhealth Uhealth. 2016;4(2). e74. [PubMed ID: 27302310]. [PubMed Central ID: PMC4925936]. https://doi.org/10.2196/mhealth.5882.
-
13.
Chavez S, Fedele D, Guo Y, Bernier A, Smith M, Warnick J, et al. Mobile Apps for the Management of Diabetes. Diabetes Care. 2017;40(10):e145-6. [PubMed ID: 28774944]. [PubMed Central ID: PMC5864023]. https://doi.org/10.2337/dc17-0853.
-
14.
Bardus M, van Beurden SB, Smith JR, Abraham C. A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. Int J Behav Nutr Phys Act. 2016;13:35. [PubMed ID: 26964880]. [PubMed Central ID: PMC4785735]. https://doi.org/10.1186/s12966-016-0359-9.
-
15.
Martinez M, Park SB, Maison I, Mody V, Soh LS, Parihar HS. iOS Appstore-Based Phone Apps for Diabetes Management: Potential for Use in Medication Adherence. JMIR Diabetes. 2017;2(2). e12. [PubMed ID: 30291096]. [PubMed Central ID: PMC6238890]. https://doi.org/10.2196/diabetes.6468.
-
16.
Cui M, Wu X, Mao J, Wang X, Nie M. T2DM Self-Management via Smartphone Applications: A Systematic Review and Meta-Analysis. PLoS One. 2016;11(11). e0166718. [PubMed ID: 27861583]. [PubMed Central ID: PMC5115794]. https://doi.org/10.1371/journal.pone.0166718.