Abstract
Background:
Mobile health (mHealth) apps play an emerging role in managing chronic conditions. This study aimed to design, develop, and evaluate the usability of an exercise therapy mHealth app in individuals with patellofemoral pain syndrome (PFPS).Objectives:
This paper presents the design, development, and evaluation procedures of the conceptual mHelath app, called Vitoapp, using the progressive web application (PWA) approach.Methods:
A mixed-method approach using both qualitative interviews (n = 8 adults with PFPS) and quantitative usability evaluations (n = 30, including 15 adults with PFPS and 15 physical therapists as experts) was adopted.Results:
The results showed high levels of system usability (75.1 out of 100) and the mHealth App usability (127.2/147).Conclusions:
This study can contribute to the design and development of the physical therapy mHealth app to provide evidence-based therapeutic exercise and remotely monitor adults with chronic pain conditions, including PFPS.Keywords
mHealth Mobile Application Pain Syndrome Patellofemoral Tele-rehabilitation Usability
1. Background
The global burden of disease introduced musculoskeletal disorders as the second cause of chronic pain, leading to global morbidity (1). Among such disorders, patellofemoral pain syndrome (PFPS) has a prevalence rate of 25% throughout individuals’ lifespan (2). Approximately 50 - 56% of adolescents experience chronic knee pain two years after their initial diagnosis (3), which can have a significant negative impact on the quality of their lives and decrease their involvement in the activities of daily living (ADL), thereby arousing pain-related confusion as well as fear and concerns for the future (4).
The onset of symptoms can be gradual, or they can be acutely developed with increased pain while stair climbing and descending, prolonged sitting, squatting, jumping, or running, especially with hills (5). Many pathoanatomic associated with symptoms are reflected in the numerous therapeutic management (6). Moreover, physical therapy is the core discipline where interventions are tailored to individuals’ needs and clinical manifestations (7). Among physical therapy modalities, exercise therapy is one of the valuable interventions highly recommended for managing PFPS (5, 8). Despite such recommendations, various factors such as accessibility, mobility limitations, time constraints, transportation difficulties, and costs limit access and adherence to physical therapy interventions (9). Furthermore, barriers are caused not only by the above-mentioned factors but also by social distancing policies to reduce COVID-19 transmission, leading to the rapid promotion of Tele-rehabilitation (TR). TR is defined as ''the use of information and communication technologies as a medium for the provision of rehabilitation services'' (10).
Tele-rehabilitation (TR) is an emerging system for delivering rehabilitation using communication technologies to assist clinicians, patients, and health services in overcoming distance, time, and cost challenges (11). Tele-rehabilitation (TR) can facilitate access to health services in an individual's home. Besides, it can facilitate more frequent therapist evaluations and more objective data, thereby preventing data loss (12, 13). Increasing evidence has documented the use of TR for managing musculoskeletal dysfunctions (14). Studies have reported that TR can provide positive outcomes similar to face-to-face interventions under different conditions, including knee arthroplasty (15), low back pain (16), multiple sclerosis (17), stroke (18), and Parkinson's disease (19). Previous studies have also reported high adherence to TR-delivered interventions (20).
One approach to delivering TR services is using mobile health (mHealth) apps, which have provided promising results in managing chronic conditions (21-23). As with any new service, an assessment should is required to ensure whether the mHealth app serves its intended purpose and how well the app functions (24). The usability assessment is essential to ensure that apps are easy to use, efficient, and acceptable by the users (25); hence, it is recommended before implementing the apps (26).
2. Objectives
This study aimed to examine the usability of an application designed and developed specifically for PFPS called the Vito app.
3. Methods
The study procedure involved two phases: (1) designing and developing the Vitoapp, and (2) evaluating the usability of the Vitoapp using mixed methods (interview & useability evaluation by field testing) (Figure 1). A prospective cohort design was applied for this usability study (individuals with PFPS as users and physical therapists as experts). The field testing method was selected from available usability evaluation methods, and the query technique was applied (25, 26). This method allows observation of how participants use the app in real life.
Research flow diagram
3.1. Study 1: Design and Development of the mHealth Vitoapp
3.1.1. Participants and Sampling
In this phase, eight individuals with PFPS achieved the theoretical saturation limit, which was between 5 and 50 participants regarding the research topic (27) (Table 1).
Demographic Features of Individuals with PFPS (n = 15)
| Variables | Number | Mean ± SD |
|---|---|---|
| Gender | ||
| Male | 7 | - |
| Female | 8 | - |
| Age (18 - 55 y) | - | 31.75 ± 6.04 |
| Marital status | ||
| Single | 6 | |
| Married | 9 | |
| Level of education | ||
| High school diploma | 4 | - |
| Bachelor's degree | 7 | - |
| Master's degree | 4 | - |
| Doctoral degree | - | - |
| Place of residence | ||
| Rural | - | - |
| Suburban | - | - |
| Urban | 15 | - |
| Years of using mobile devices (y) | 11.50 ± 1.77 | |
| > 10 | 15 | |
| < 10 | - | |
| Experience of using telehealth | ||
| No experience | 15 | - |
| < 3 months | - | - |
| 3 - 6 months | - | - |
| 6 months - 1 year | - | - |
| > 1 year | - | - |
| Experience of using mobile apps | ||
| None | - | - |
| Low | - | - |
| Medium | 7 | - |
| Acceptable | 5 | - |
| High | 3 | - |
3.1.2. Procedure for Vitoapp Design and Development
Face-to-face interviews were conducted to detect desirable features to be included in the app. The interview took place in a physical therapy clinic, and one of the authors guided the one-hour discussion meetings in a friendly atmosphere for approximately in two days.
This semi-structured interview contained four general open-ended questions:
(1) What was your experience of doing exercises at physiotherapy clinics?
(2) What problems do face-to-face treatment have?
(3) What were your perceptions, experiences, and preferences for distance physical therapy?
(4) We are working on an application to provide remote treatment for patients with patellar pain. Which features would you like to observe in the remote exercise app?
Interviews were audio-recorded; transcriptions were transcribed verbatim to identify emerging themes and subthemes using a thematic analysis approach (Table 2).
Summary of Themes and Subthemes Emerged from First Phase of Interviews
| Themes | Subthemes | Codes |
|---|---|---|
| Treatment | Exercise therapy program | Not considering exercises in the treatment plan, not receiving feedback on performance during exercise, and not having a clear picture of the long-term treatment plan. |
| Exercising | Not following the exercise program due to forgetfulness, forgetting how to do exercises, or not doing exercises correctly. | |
| Consultation | Training and guidance | Lack of sufficient information about impairment and required changes in lifestyle, limited access to a physiotherapist when having a question or a problem during his/her exercise at home. |
| Features of remote exercise app | Content | Providing exercises in the form of short videos, clarity, comprehensibility of training videos, and explaining exercises using a simple language |
| Access | Access to the application on a computer |
Themes emerged from the interviews show that participants appreciate the following options: Exercise videos with high quality, clear step-by-step exercise instructions explanation, opportunity to communicate with their physical therapists, high-quality content (evidence-based information and treatment), symptom and performance recording, receiving feedback from their physical therapists, and information about their status. We have considered these suggested options in the content development of Vitoapp and the design of its user interface (UI).
3.1.3. Vitoapp Design and Development
The Vitoapp intervention aimed to create an evidence-based tele-exercise program compatible with Windows, Android, or IOS operating systems (phones or tablets). The progressive web application (PWA) approach was selected due to the ease of content updating, accessibility on different platforms, simplicity, and ease of use. Moreover, the PWA approach can guarantee advantages to user experience optimization, application performance, development team efficiency, and accessibility compared to native and cross-platform approaches (28).
The researchers produced exercise videos and educational articles to be delivered throughout a six-week program. The exercise videos included stretching, strengthening, and balance exercises derived from Patellofemoral Pain Clinical Practice Guidelines developed in 2019 by the academy of orthopaedic physical therapy of the American physical therapy association (APTA) (5).
The Vitoapp design was informed by the interview findings in Phase 1 and included the following features:
• Information and advice: Having contents on social cognitive theory, this app aimed to enhance self-regulated ADL. This section briefly defines PFPS, common symptoms, treatment options, medications, and physical therapy. It also includes instructions about safe transition and mobility, such as sitting, standing up, stair climbing, and walking (Figures 2 - 5).
From left to right: Role identification, physical therapist’s home page, practitioner selection, and patient’s home page
Exercise plan
Exercise plan calendar (left), list of patients (right)
Exercise session evaluation (left), message (right)
• Exercise plan: Exercise videos with detailed narrative instructions focus on a range of motions, muscular strength, functional strength, and balance. At the end of each session, the users report their pain level using a Visual Analogue Scale (VAS). If the patient's pain level is between 1 and 3, they will be instructed to continue their exercises. When their pain level is between 4 and 6, the number of sets and repetitions will be automatically decreased to 2 sets and 7 repetitions, and if their pain level is between 7 and 10, they will be asked to stop exercising immediately and visit their physician.
• Assessment: In this section, the Kujula questionnaire, which is a responsive outcome measure for monitoring the user's progress and effectiveness of interventions (29), is completed. The app recommends conducting a functional assessment in the first exercise session and at the end of the 2nd, 4th and 6th weeks of exercise therapy. The answers to the Kujula questionnaire and the resulting scores are recorded in the patient's personal calendar, which is accessible to the patients and their practitioners.
• Calendar: The overview screen shows a monthly calendar. A green check icon indicates that the session is carried out, and the orange circle icon implies that exercises are not completed. The functional assessment results, brief information about pain level, and patient's performance on their exercises are visualized (Figure 4).
• Message: The users can report any trouble or discomfort to their physical therapists via instant messaging. Physical therapists can answer these questions via messaging and adapt the existing exercise based on feedback, for example, by modifying the frequency and/or repetition of each exercise or removing the exercise from the program (Figure 3).
• Reminder: The patient is informed of the time specified for answering the Kujula questionnaire in this section.
• About us (Info): This section explains the main functions of the app and gives contact information for the research team who developed the app and also supervised the present study.
Along with these options, the patients have their personal information, including gender, age, weight, phone number, and others as well as a brief history of their impairment recorded on their profile, which is available for their selected physical therapist in the "patients" section on their home page
3.2. Study 2: Evaluation of the mHealth App
3.2.1. Participants and Sampling
The study was conducted at the musculoskeletal rehabilitation research center affiliated with the Jundishapur university of medical sciences (AJUMS). The ethics committee approved all procedures of AJUMS (Code: IR.AJUMS.REC.1400.052). The participants were selected via phone calls using a convenient sampling method. First, written informed consent was obtained from all participants. A total of 30 participants, 15 individuals with PFPS and 15 physical therapists, participated in the study (Tables 3 and 4). The sample size was determined according to Dworkin's study (27), who argued that a small number of participants (10 ± 2) are sufficient for identifying the usability issues. Moreover, Maramba et al. showed that the minimum and maximum sample sizes in studies using a mixed (multi) method are 4 and 450 subjects. Accordingly, a convenient sample size for this study was 38 (30).
Physical Therapists’ Demographic Features (n = 15)
| Variables | Number | Mean ± SD |
|---|---|---|
| Gender | ||
| Male | 7 | - |
| Female | 8 | - |
| Age (18 - 55 y) | - | 34.00 ± 6.12 |
| Marital status | ||
| Single | 8 | - |
| Married | 7 | - |
| Level of education | ||
| Bachelor's | 4 | - |
| Master's | 4 | - |
| Ph.D. degree | 7 | - |
| Place of residence | ||
| Rural | 0 | - |
| Suburban | 0 | - |
| Urban | 15 | - |
| Clinical work experience | 15 | 12.40 ± 7.33 |
| Preferential system | ||
| Smartphone | 15 | - |
| Tablet | 0 | - |
| Laptop | 0 | - |
| Computer | 0 | - |
| Years of using mobile devices (y) | 10.20 ± 1.09 | |
| > 10 | 13 | |
| < 10 | 2 | |
| Experience of using telehealth | ||
| No experience | 6 | - |
| < 3 months | 0 | - |
| 3 - 6 months | 3 | - |
| 6 months -1 year | 3 | - |
| > 1 year | 3 | - |
| Experience of using mobile apps | ||
| None | 0 | - |
| Low | 0 | - |
| Medium | 7 | - |
| Acceptable | 8 | - |
| High | 0 | - |
Patients’ Demographic Features (n = 15)
| Variables | Number | Mean ± SD |
|---|---|---|
| Gender | ||
| Male | 5 | - |
| Female | 10 | - |
| Age (18 - 55 y) | - | 31.55 ± 6.64 |
| Marital status | ||
| Single | 7 | - |
| Married | 8 | - |
| Level of education | ||
| High school diploma | 4 | - |
| Bachelor's | 4 | - |
| Master's | 5 | - |
| Ph.D. | 2 | - |
| Place of residence | ||
| Rural | 0 | - |
| Suburban | 0 | - |
| Urban | 15 | - |
| Employment | ||
| Not employed | 8 | - |
| Employed | 7 | - |
| Retired or disabled | 0 | - |
| Knee pain duration | 15 | 8.88 ± 10.67 |
| Preferential system | ||
| Smartphone | 15 | - |
| Tablet | 0 | - |
| Laptop | 0 | - |
| Computer | 0 | - |
| Years of using mobile devices (y) | 11.11 ± 2.71 | |
| > 10 | 9 | |
| < 10 | 6 | |
| Experience of using telehealth | ||
| No experience | 14 | - |
| < 3 months | 1 | - |
| 3 - 6 months | 0 | - |
| 6 months - 1 year | 0 | - |
| < 1 year | 0 | - |
| Experience of using mobile apps | ||
| None | 0 | - |
| Low | 0 | - |
| Medium | 3 | - |
| Acceptable | 10 | - |
| High | 2 | - |
Inclusion criteria for individuals with PFPS were the diagnosis of PFPS by their distinct physical therapist and age range of 18 - 55 years. They were excluded if they had the conservative or surgical treatment of the affected knee in the past six months or if they had any previous injury or pain in the ankle or hip joints. Inclusion criteria for physical therapists were having at least five years of experience in knee rehabilitation.
3.2.2. Usability Evaluation Procedure
Two cohorts were created for the usability testing of the Vitoapp over a three- week period. The first cohort (named experts) encompassed physical therapists using the app to monitor and manage the exercise therapy sessions of individuals with PFPS, who were invited to participate via phone calls and in-person meetings from different cities of Iran (Ahwaz, Tehran, and Shiraz). The second cohort (named users) encompassed individuals with PFPS who used the app to perform the exercise therapy. The term "participants" refers to both experts and users. After obtaining informed consent, the participants’ demographic characteristics were recorded. The participants were then instructed to log in to the Vitoapp; however, they could not sign up if their accounts were not confirmed by their selected physical therapist (for the users) or the Vito support team (for the experts). After login, all contents were made available to the participants.
The users were encouraged to use the app on their cell phones or tablets for 3 - 5 days per week during 50 minutes exercise sessions for three weeks. This included a total of 20 videos classified as stretching, strengthening, and balance exercises. The users received step-by-step narrative instructions on each video describing how to do the exercise. When the user is prepared enough, he/she clicks "start" and exercise videos are presented when all exercises are completed. Moreover, the user clicks "Done" and answers post-exercise questions on pain level and discomfort (Figure 5). The obtained results were sent to a physical therapist via instant massaging. After three weeks, usability questionnaires were completed by the participants.
3.2.3. Outcome Measures
The ISO 9241-11 standard specifies three dimensions of usability: effectiveness, efficiency, and satisfaction. Moreover, the following questionnaires are used to assess the three aspects of usability.
3.2.4. System Usability Scale (SUS)
The System Usability Scale (SUS) is a widely used scale to quantify satisfaction with a wide range of products. This questionnaire contains 10 items scored on a five-point Likert scale, with 1 for strongly disagree and 5 for strongly agree (31). The total score ranged from 10 - 100, with higher scores representing higher usability. The test-retest reliability and the validity of the Persian version were also approved (ICC = 0.96) (32).
3.2.5. mHealth App Usability Questionnaire (MAUQ)
The mHealth App Usability Questionnaire (MAUQ) was developed to evaluate the usability of mHealth for both patients and providers. Since the Vitoapp is an interactive service, the interactive versions of MAUQ for patients and providers were used. Both versions have 21 items scored on a seven-point Likert scale, with 1 for strongly disagree and 7 for strongly agree. To evaluate the app's usability, the total score was calculated, and the mean score of all items was determined. The higher the total mean score, the higher the app usability (33).
3.2.6. Statistical Analysis
The collected data was analyzed with SPSS software version 22. The Shapiro-Wilk test verified the normal distribution of variables. The descriptive analysis was reported as mean (± SD) and a confidence interval (CI 95%) for variables.
4. Results
Thirty participants, including 15 individuals with PFPS and 15 physical therapists, participated in this usability study. Tables 1 and 2 shows the participants’ demographic information.
4.1. System Usability Scale (SUS)
The Vitoapp usability was assessed by the two groups of users participating in the study using the SUS (Table 3). The mean scores of the SUS were74.83 out of 100 (SD = 13.18) in the user group and 71.50 out of 100 (SD = 7.41) in the physiotherapist group, indicating the high usability of the system
4.2. mHealth App Usability Questionnaire (MAUQ)
The usability evaluation based on the MAUQ results showed that the mean scores of this questionnaire were 125.60 out of 147 (SD = 11.19) in the expert group and 127.44 out of 147 (SD = 12.62) in the user group. The three factors corresponding to the three subscales on the MAUQ are ease of use and satisfaction (8 items), system information arrangement (6 items), and usefulness (7 items). The results of the MAUQ questionnaire subscales are summarized in Table 5.
| Test/Subscales | Range | Physical Therapists (n = 15) | Patients (n = 15) |
|---|---|---|---|
| SUS | 10 - 100 | 71.50 ± 7.41 | 74.83 ± 13.18 |
| MAUQ | 21 - 147 | 125.60 ± 11.19 | 127.44 ± 12.62 |
| Ease of use and satisfaction | 8 - 56 | 49.6 ± 4.27 | 49.00 ± 4.89 |
| System information arrangement | 6 - 42 | 34.8 ± 4.43 | 36.77 ± 3.49 |
| Usefulness | 7 - 49 | 41.2 ± 3.19 | 41.77 ± 4.71 |
In the physical therapist group: SUS (Max. score = 80 and Min. score = 60), MAUQ (Max. score = 137 and Min. = 107). In patient group: SUS (Max. score = 95.5 and Min. score = 60), MAUQ (Max. score = 146 and Min. = 109).
5. Discussion
The present study aimed to develop a PWA and assess its usability for both adults with PFPS and physical therapists. To promote TR, users must understand its technology and be allowed to adjust themselves to this app. Technology abandonment can occur when users feel that learning the TR technology is too complicated or that it requires specific equipment. This is why both adults with PFPS and physical therapists were included in this study. High SUS and MAUQ scores obtained for both patients and experts indicated the high usability of this app, confirming the hypothesis of having potential therapeutic use at home.
Considering the quantitative data, the app was useful and satisfactory via the evidence-based exercise program and instruction, exercise diary, and clinical outcome measures. The findings of this study are in agreement with some other studies in medical and rehabilitation literature. Some disorders were MS rehabilitation (17) and Parkinson's disease (19), and the present study focused on musculoskeletal disorders such as low back pain (34), Frozen shoulder (22), and RA (35). However, to date, there is no exercise app developed for individuals with PFPS. Our usability evaluation results were reliable due to the heterogeneity of users involved in the study in terms of their age groups, gender, level of education, and different levels of experience of using tele-rehabilitation, as well as the homogeneity of experts in charge of training and supporting the users.
The usability study was based on a three-week evaluation, during which the users were asked to use the app in their daily lives at home rather than in a controlled clinical setting. Accordingly, it seems to be close to the real needs of patients using tele-rehabilitation services. Moreover, in contrast to other studies (22, 36, 37), considering this three-week trial, assessing the remote monitoring of patients by a physiotherapist was possible; hence, all the users were given a convenient time to test the application’s aspects thoroughly, and the usability evaluation was performed by both physical therapists and adults with PFPS.
5.1. Limitations
This study had a few limitations. First, although a sample size of 30 may be sufficient for detecting usability issues, the generalizability of the findings is limited to individuals with similar conditions and impairments. Second, all participants preferred to use the application on their mobile phones among other available options; hence, it was not possible to assess the application usability on the Windows operating system. Third, controlling the correctness of exercise performance was not possible since sending files in video format was not allowed by the application. This option is to be added to the updated version of Vitoapp in the future.
Further studies to determine the effectiveness of Vitoapp in comparison to face-to-face interventions in a randomized controlled trial are also recommended.
5.2. Conclusions
In this study, an evidence-based exercise therapy app was developed for adults with PFPS, and its usability was confirmed by users’ high usability scores. This app can be used by physical therapists to provide evidence-based therapeutic exercise and remote monitoring for adults with PFPS.
5.3. Summary Table
5.3.1. What Was Known
• Tele-rehabilitation (TR) improves access to health services and can facilitate more frequent therapist evaluation, provide more objective data, and prevent data loss.
• There is increasing evidence investigating the use of TR for managing musculoskeletal dysfunctions.
• Considering the rapidly growing use of mHealth applications, developing based on the latest evidence and having usability confirmation is crucial for being effective.
5.3.2. What This Study Adds
• An evidence-based application (Vitoapp) was developed in this study to allow physical therapists deliver care and monitor patients remotely. A usability evaluation was also conducted as the final step before publicly launching the app.
• Vitoapp can be used by physical therapists and adults with PFPS.
Acknowledgements
References
-
1.
G. B. D. Disease, Injury I, Prevalence C. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. 2018;392(10159):1789-858. [PubMed ID: 30496104]. [PubMed Central ID: PMC6227754]. https://doi.org/10.1016/S0140-6736(18)32279-7.
-
2.
Callaghan MJ, Selfe J. Has the incidence or prevalence of patellofemoral pain in the general population in the United Kingdom been properly evaluated? Phys Ther Sport. 2007;8(1):37-43. https://doi.org/10.1016/j.ptsp.2006.07.001.
-
3.
Rathleff CR, Olesen JL, Roos EM, Rasmussen S, Rathleff MS. Half of 12-15-year-olds with knee pain still have pain after one year. Dan Med J. 2013;60(11). A4725. [PubMed ID: 24192242].
-
4.
Smith BE, Moffatt F, Hendrick P, Bateman M, Rathleff MS, Selfe J, et al. The experience of living with patellofemoral pain-loss, confusion and fear-avoidance: a UK qualitative study. BMJ Open. 2018;8(1). e018624. [PubMed ID: 29362256]. [PubMed Central ID: PMC5786111]. https://doi.org/10.1136/bmjopen-2017-018624.
-
5.
Willy RW, Hoglund LT, Barton CJ, Bolgla LA, Scalzitti DA, Logerstedt DS, et al. Patellofemoral Pain. J Orthop Sports Phys Ther. 2019;49(9):CPG1-CPG95. [PubMed ID: 31475628]. https://doi.org/10.2519/jospt.2019.0302.
-
6.
Drew BT, Redmond AC, Smith TO, Penny F, Conaghan PG. Which patellofemoral joint imaging features are associated with patellofemoral pain? Systematic review and meta-analysis. Osteoarthritis Cartilage. 2016;24(2):224-36. [PubMed ID: 26471209]. https://doi.org/10.1016/j.joca.2015.09.004.
-
7.
Crossley K, Bennell K, Green S, McConnell J. A systematic review of physical interventions for patellofemoral pain syndrome. Clin J Sport Med. 2001;11(2):103-10. [PubMed ID: 11403109]. https://doi.org/10.1097/00042752-200104000-00007.
-
8.
Albornoz-Cabello M, Barrios-Quinta CJ, Barrios-Quinta AM, Escobio-Prieto I, Cardero-Duran MLA, Espejo-Antunez L. Effectiveness of Tele-Prescription of Therapeutic Physical Exercise in Patellofemoral Pain Syndrome during the COVID-19 Pandemic. Int J Environ Res Public Health. 2021;18(3). [PubMed ID: 33504042]. [PubMed Central ID: PMC7908506]. https://doi.org/10.3390/ijerph18031048.
-
9.
Cottrell MA, Russell TG. Telehealth for musculoskeletal physiotherapy. Musculoskelet Sci Pract. 2020;48:102193. [PubMed ID: 32560876]. [PubMed Central ID: PMC7261082]. https://doi.org/10.1016/j.msksp.2020.102193.
-
10.
Rogante M, Grigioni M, Cordella D, Giacomozzi C. Ten years of telerehabilitation: A literature overview of technologies and clinical applications. NeuroRehabilitation. 2010;27(4):287-304. [PubMed ID: 21160118]. https://doi.org/10.3233/NRE-2010-0612.
-
11.
Seron P, Oliveros MJ, Gutierrez-Arias R, Fuentes-Aspe R, Torres-Castro RC, Merino-Osorio C, et al. Effectiveness of Telerehabilitation in Physical Therapy: A Rapid Overview. Phys Ther. 2021;101(6). [PubMed ID: 33561280]. [PubMed Central ID: PMC7928601]. https://doi.org/10.1093/ptj/pzab053.
-
12.
Dorsey ER, Yvonne Chan YF, McConnell MV, Shaw SY, Trister AD, Friend SH. The Use of Smartphones for Health Research. Acad Med. 2017;92(2):157-60. [PubMed ID: 27119325]. https://doi.org/10.1097/ACM.0000000000001205.
-
13.
Burdette SD, Herchline TE, Oehler R. Surfing the web: practicing medicine in a technological age: using smartphones in clinical practice. Clin Infect Dis. 2008;47(1):117-22. [PubMed ID: 18491969]. https://doi.org/10.1086/588788.
-
14.
Cottrell MA, Galea OA, O'Leary SP, Hill AJ, Russell TG. Real-time telerehabilitation for the treatment of musculoskeletal conditions is effective and comparable to standard practice: a systematic review and meta-analysis. Clin Rehabil. 2017;31(5):625-38. [PubMed ID: 27141087]. https://doi.org/10.1177/0269215516645148.
-
15.
Jiang S, Xiang J, Gao X, Guo K, Liu B. The comparison of telerehabilitation and face-to-face rehabilitation after total knee arthroplasty: A systematic review and meta-analysis. J Telemed Telecare. 2018;24(4):257-62. [PubMed ID: 28027679]. https://doi.org/10.1177/1357633X16686748.
-
16.
Dario AB, Moreti Cabral A, Almeida L, Ferreira ML, Refshauge K, Simic M, et al. Effectiveness of telehealth-based interventions in the management of non-specific low back pain: a systematic review with meta-analysis. Spine J. 2017;17(9):1342-51. [PubMed ID: 28412562]. https://doi.org/10.1016/j.spinee.2017.04.008.
-
17.
Thirumalai M, Rimmer JH, Johnson G, Wilroy J, Young HJ, Mehta T, et al. TEAMS (Tele-Exercise and Multiple Sclerosis), a Tailored Telerehabilitation mHealth App: Participant-Centered Development and Usability Study. JMIR Mhealth Uhealth. 2018;6(5). e10181. [PubMed ID: 29798832]. [PubMed Central ID: PMC5992455]. https://doi.org/10.2196/10181.
-
18.
Johansson T, Wild C. Telerehabilitation in stroke care--a systematic review. J Telemed Telecare. 2011;17(1):1-6. [PubMed ID: 21097560]. https://doi.org/10.1258/jtt.2010.100105.
-
19.
Landers MR, Ellis TD. A Mobile App Specifically Designed to Facilitate Exercise in Parkinson Disease: Single-Cohort Pilot Study on Feasibility, Safety, and Signal of Efficacy. JMIR Mhealth Uhealth. 2020;8(10). e18985. [PubMed ID: 33016887]. [PubMed Central ID: PMC7573700]. https://doi.org/10.2196/18985.
-
20.
Bennell KL, Marshall CJ, Dobson F, Kasza J, Lonsdale C, Hinman RS. Does a Web-Based Exercise Programming System Improve Home Exercise Adherence for People With Musculoskeletal Conditions?: A Randomized Controlled Trial. Am J Phys Med Rehabil. 2019;98(10):850-8. [PubMed ID: 31021823]. https://doi.org/10.1097/PHM.0000000000001204.
-
21.
Lai B, Wilroy J, Young HJ, Howell J, Rimmer JH, Mehta T, et al. A Mobile App to Promote Adapted Exercise and Social Networking for People With Physical Disabilities: Usability Study. JMIR Form Res. 2019;3(1). e11689. [PubMed ID: 30888325]. [PubMed Central ID: PMC6444218]. https://doi.org/10.2196/11689.
-
22.
Stutz T, Emsenhuber G, Huber D, Domhardt M, Tiefengrabner M, Oostingh GJ, et al. Mobile Phone-Supported Physiotherapy for Frozen Shoulder: Feasibility Assessment Based on a Usability Study. JMIR Rehabil Assist Technol. 2017;4(2). e6. [PubMed ID: 28729234]. [PubMed Central ID: PMC5544899]. https://doi.org/10.2196/rehab.7085.
-
23.
Kairy D, Mostafavi MA, Blanchette-Dallaire C, Belanger E, Corbeil A, Kandiah M, et al. A Mobile App to Optimize Social Participation for Individuals with Physical Disabilities: Content Validation and Usability Testing. Int J Environ Res Public Health. 2021;18(4). [PubMed ID: 33670222]. [PubMed Central ID: PMC7916924]. https://doi.org/10.3390/ijerph18041753.
-
24.
Jake-Schoffman DE, Silfee VJ, Waring ME, Boudreaux ED, Sadasivam RS, Mullen SP, et al. Methods for Evaluating the Content, Usability, and Efficacy of Commercial Mobile Health Apps. JMIR Mhealth Uhealth. 2017;5(12). e190. [PubMed ID: 29254914]. [PubMed Central ID: PMC5748471]. https://doi.org/10.2196/mhealth.8758.
-
25.
Zhang D, Adipat B. Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications. Int J Hum-Comput Interact. 2005;18(3):293-308. https://doi.org/10.1207/s15327590ijhc1803_3.
-
26.
Moumane K, Idri A, Abran A. Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards. Springerplus. 2016;5:548. [PubMed ID: 27190747]. [PubMed Central ID: PMC4851667]. https://doi.org/10.1186/s40064-016-2171-z.
-
27.
Dworkin SL. Sample size policy for qualitative studies using in-depth interviews. Arch Sex Behav. 2012;41(6):1319-20. [PubMed ID: 22968493]. https://doi.org/10.1007/s10508-012-0016-6.
-
28.
Rêgo F, Portela F, Santos MF. Towards PWA in Healthcare. Procedia Comput Sci. 2019;160:678-83. https://doi.org/10.1016/j.procs.2019.11.028.
-
29.
Negahban H, Mostafaee N, Sohani SM, Hessam M, Tabesh H, Montazeri A. Responsiveness and minimally important differences for selected Persian-version of outcome measures used in patients with patellofemoral pain syndrome. Disabil Rehabil. 2015;37(14):1285-90. [PubMed ID: 25243764]. https://doi.org/10.3109/09638288.2014.962107.
-
30.
Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: A scoping review. Int J Med Inform. 2019;126:95-104. [PubMed ID: 31029270]. https://doi.org/10.1016/j.ijmedinf.2019.03.018.
-
31.
Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. Int J Hum-Comput Interact. 2008;24(6):574-94. https://doi.org/10.1080/10447310802205776.
-
32.
Dianat I, Ghanbari Z, AsghariJafarabadi M. Psychometric properties of the persian language version of the system usability scale. Health Promot Perspect. 2014;4(1):82-9. [PubMed ID: 25097841]. [PubMed Central ID: PMC4122034]. https://doi.org/10.5681/hpp.2014.011.
-
33.
Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study. JMIR Mhealth Uhealth. 2019;7(4). e11500. [PubMed ID: 30973342]. [PubMed Central ID: PMC6482399]. https://doi.org/10.2196/11500.
-
34.
Machado GC, Pinheiro MB, Lee H, Ahmed OH, Hendrick P, Williams C, et al. Smartphone apps for the self-management of low back pain: A systematic review. Best Pract Res Clin Rheumatol. 2016;30(6):1098-109. [PubMed ID: 29103552]. https://doi.org/10.1016/j.berh.2017.04.002.
-
35.
Tonga E, Williamson E, Srikesavan C, Ozen T, Saritas F, Lamb SE. A hand exercise mobile app for people with rheumatoid arthritis in Turkey: design, development and usability study. Rheumatol Int. 2021;41(6):1151-60. [PubMed ID: 33870452]. https://doi.org/10.1007/s00296-021-04860-0.
-
36.
Hsieh KL, Fanning JT, Rogers WA, Wood TA, Sosnoff JJ. A Fall Risk mHealth App for Older Adults: Development and Usability Study. JMIR Aging. 2018;1(2). e11569. [PubMed ID: 31518234]. [PubMed Central ID: PMC6716481]. https://doi.org/10.2196/11569.
-
37.
Setiawan IMA, Zhou L, Alfikri Z, Saptono A, Fairman AD, Dicianno BE, et al. An Adaptive Mobile Health System to Support Self-Management for Persons With Chronic Conditions and Disabilities: Usability and Feasibility Studies. JMIR Form Res. 2019;3(2). e12982. [PubMed ID: 31021324]. [PubMed Central ID: PMC6658284]. https://doi.org/10.2196/12982.