Validation of the Mobile Application Rating Scale (MARS)

Yannik Terhorst; Paula Philippi; Lasse B Sander; Dana Schultchen; Sarah Paganini; Marco Bardus; Karla Santo; Johannes Knitza; Gustavo C Machado; Stephanie Schoeppe; Natalie Bauereiß; Alexandra Portenhauser; Matthias Domhardt; Benjamin Walter; Martin Krusche; Harald Baumeister; Eva-Maria Messner

doi:10.1371/journal.pone.0241480

Validation of the Mobile Application Rating Scale (MARS)

PLoS One. 2020 Nov 2;15(11):e0241480. doi: 10.1371/journal.pone.0241480. eCollection 2020.

Authors

Yannik Terhorst^{1

2}, Paula Philippi², Lasse B Sander³, Dana Schultchen⁴, Sarah Paganini⁵, Marco Bardus⁶, Karla Santo^{7

8

9}, Johannes Knitza¹⁰, Gustavo C Machado^{11

12}, Stephanie Schoeppe¹³, Natalie Bauereiß², Alexandra Portenhauser², Matthias Domhardt², Benjamin Walter¹⁴, Martin Krusche¹⁵, Harald Baumeister², Eva-Maria Messner²

Affiliations

¹ Department of Research Methods, Institute of Psychology and Education, University Ulm, Ulm, Germany.
² Department of Clinical Psychology and Psychotherapy, Institute of Psychology and Education, University Ulm, Ulm, Germany.
³ Department of Rehabilitation Psychology and Psychotherapy, Institute of Psychology, Albert-Ludwigs-University Freiburg, Freiburg im Breisgau, Germany.
⁴ Department of Clinical and Health Psychology, Institute of Psychology and Education, University Ulm, Ulm, Germany.
⁵ Department of Sport Psychology, Institute of Sports and Sport Science, University of Freiburg, Freiburg, Germany.
⁶ Department of Health Promotion and Community Health, Faculty of Health Sciences, American University of Beirut, Beirut, Lebanon.
⁷ Academic Research Organization, Hospital Israelita Albert Einstein, São Paulo, Brazil.
⁸ Westmead Applied Research Centre, Westmead Clinical School, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia.
⁹ Cardiovascular Division, The George Institute for Global Health, Sydney, Australia.
¹⁰ Department of Internal Medicine 3 - Rheumatology and Immunology, University Hospital Erlangen, Friedrich-Alexander University Erlangen-Nuremberg, Erlangen, Germany.
¹¹ Institute for Musculoskeletal Health, Sydney, New South Wales, Australia.
¹² Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, New South Wales, Australia.
¹³ School of Health, Medical and Applied Sciences, Appleton Institute, Physical Activity Research Group, Central Queensland University, Rockhampton, Queensland, Australia.
¹⁴ Department of Internal Medicine I, Gastroenterology, University Hospital Ulm, Ulm, Germany.
¹⁵ Department of Rheumatology and Clinical Immunology, Charité - Universitätsmedizin Berlin, Berlin, Germany.

Abstract

Background: Mobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity.

Objective: This study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS.

Methods: Data was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation.

Results: In total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05).

Conclusion: The metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.

Publication types

Validation Study

MeSH terms

Factor Analysis, Statistical
Humans
Mobile Applications / standards*
Models, Theoretical
Reproducibility of Results
Telemedicine

Grants and funding

The author(s) received no specific funding for this work.