Unintentional Bias May Impede Effectiveness of Health Apps: CDT Report Examines mHealth Ecosystem & Provides Recommendations for Developers

The mobile health (mHealth) industry holds significant potential to improve health and wellness, but unintended bias in the algorithms and data that power mHealth apps might limit their effectiveness and uptake, especially for traditionally marginalized communities, according to Heal-gorithms, a new report from the Center for Democracy & Technology (CDT).

“Mobile health apps have the potential to improve access to quality health information and care, and to support positive behavioral changes. But hidden biases in the data and algorithms that power these apps could inhibit under-represented groups from enjoying these benefits. Developers and policymakers need to understand historical inequities in healthcare and research and ensure that those inequities are not replicated and exacerbated by new data-driven and personalized tools. We hope this report will help empower developers and researchers to create more inclusive mHealth apps that can improve outcomes for everyone,” said Natasha Duarte, CDT Privacy & Data Policy Analyst.

The report, which was made possible by a grant from the Robert Wood Johnson Foundation, explores how the lightly regulated mHealth ecosystem and the data generated by its narrow user base creates numerous opportunities for unintentional bias to impact outcomes. These include over- or under-representation of certain groups in the data sets used to power the app, the type of machine learning model used to make decisions, and the design of the app’s user interface and experience, among others. It also suggests that this unintentional bias could be a factor in the limited uptake of mHealth apps in certain markets.

“The Heal-gorithms report shows that the benefits of mHealth tools may not be reaching all communities,” said Paul Tarini, senior program officer, the Robert Wood Johnson Foundation. “The guidance it provides the industry could produce a win-win—more tools for people in marginalized communities to improve health; and, increased market share for companies.”

The report also offers recommendations for health researchers and policymakers, and it provides a series of questions — derived from CDT’s Digital Decisions tool — for developers to ask throughout the development process to help mitigate bias, including:

• Are health interventions in the app evidence-based?
• What type of user are you designing for and how will they access the app?
• Is your team diverse, and if not, how will you represent your intended users?
• Is your app accessible to users who have low vision, blindness, hearing impairments, cognitive impairments, or motor impairments?
• Where did your model “training” data come from and who does it represent?
• How much error is acceptable in your app?

The researchers identified a number of areas requiring further investigation to improve mHealth effectiveness, including how to standardize health metrics across apps, how to build more open datasets that are evidence-based, and how to create a framework around health data that goes beyond privacy to address fairness and equality.