mHealth Working Group Monthly Meeting
April 26thfrom 9:00am-11:30am EDT
Association of Reproductive Health Professionals
1901 L Street, NW, Suite 300, Washington DC
Agenda
1. Where Are They Now? Past Presenters and Lessons Learned
a. CommCare by Neil Lesh, Dimagi
b. m4RH by Heather Vahdat, FHI360
c. m4QI by Pam Riley, Abt Associates and James Bon Tempo, Jhpiego
d. CycleTel by Meredith Puleio, IRH/Georgetown
2. Frameworks for Lessons Learned in mHealth
3. Landscape Analysis of mHealth Projects
Thanks to ARHP for hosting the April 2012 meeting. The presentations that accompany these notes, as well as those from previous mHealth Working Group meetings, are available on the mHealth Toolkit at: http://www.k4health.org/toolkits/mhealth/mhealth-working-group-0. For questions or comments, please contact Kelly Keisling () or Laura Raney (), co-chairs of the mHealth Working Group.
The next meeting of the mHealth Working Group will be hosted by Save the Children to discuss organizational strategy for mHealth.The meeting will be held on Tuesday May 22nd, 9:30 am -11:30 am EDT at2000 L Street NW, Suite 500. The meeting is available remotely: in USor via Skype:1-866-386-4210, international: 1-433-863-6601, code: 6348796591. Discussion is invited on participant experiences with organizational strategy in e-mHealth.
Introduction to Past Presenters and Lessons Learned- Where Are They Now?
As mHealth continues to evolve, we review with past presenters how these mHealth projects have evolved with the field. Presenters discuss their ongoing progress and lessons learned in mHealth. We then discuss lessons learned within frameworks for issues in mHealth.
1a. CommCare by Neil Lesh, Dimagi
Since its September 2009 presentation to the Working Group, Dimagi has shifted from a focus on single projects to a product orientation. (Dimagi’s 2009 presentation is available at http://archive.k4health.org/toolkits/mhealth/September-9-2009.) The CommCare product has expanded in reach and features. It is used by 1700+ community health workers (CHW) in 12+ countries, including 10+ states in India. CommCare now includes multimedia (audio, images, video). Adoption is supported by standard pricing, implementation plans, and a total cost of ownership model. Deployment is facilitated by authoring tools allowing non-programmers to deploy CommCare. Four studies will also be published in 2012 demonstrating intermediate outcomes in access, quality, experience, and accountability of care. These will be accompanied by two larger studies to be completed in 2013.
Lessons Learned and Discussion
The product approach has accelerated scale up of CommCare. Dimagi has developed standardized tools and approaches which allow organizations to more quickly and easily start using CommCare. Dimagi can more easily support its users because it has only one code base to support. Finally, organizations can deploy CommCare on their own without software programmers. It has also become clear that greater emphasis is needed on making data actionable. This is because most organizations are not used to getting real-time data, and thus are not prepared to utilize it. Technology amplifies human dimensions such as interaction with supervisors. Thus, systems work if they are driven by someone, rather than merely by automation. Likewise, human capacity is needed to use technology capacity. For example, audio messages needed to be supported by CHW training on message topics. Underlying these issues is the need to provide organizational capacity building in mHealth. This includes capacity for data management, program planning, logistics, and project management. It also includes technology capacity and computer literacy, though these are not critical gaps and CommCare is designed to reduce dependence on these.
1b. m4RH by Heather Vahdat, FHI360
Mobile for Reproductive Health (m4RH) was launched in Kenya in 2009 and replicated in Tanzania in 2010. It supports information-seeking for family planning methods and contraceptive adoption through features such as a shortcode query of names and locations of clinics. The research pilot was completed in June of 2011 and is currently under analysis. (FHI360 discussed evaluation techniques for m4RH in October 2011 at http://archive.k4health.org/toolkits/mhealth/october-20-2011.) Initial results demonstrate m4RH’s reach to women, men, and couples, including those in rural areas. Open-ended questions on how m4RH impacted family planning were texted to m4RH users and received a 12-18% response rate.
The project is still in operation in Kenya and Tanzania as part of the USAID-funded Program Research for Strengthening Services (PROGRESS) project. M4RH is currently being promoted as part of the Johns Hopkins Center for Communication Programs (CCP) in Tanzania Jiamini campaign, which was initiated on April 2, 2012. This partnership provides wide-scale promotion of m4RH and has resulted in a substantial increase in m4RH usage from approximately 30,000 hits during the pilot research period (August 2010-June 2011) to approximately 50,000 additional hits in the first three weeks of the campaign. Currently, the m4RH team is working on adapting m4RH for a youth-focused intervention in Rwanda and on efforts in Tanzania to expand m4RH content to include messages on contraceptive continuation. The m4RH program was selected as one of Women Deliver’s 50 most inspiring ideas and solutions that are delivering for girls and women for 2012 in the area of Technologies and Innovations
Lessons Learned and Discussion
Partnerships are critical for project success. These include programmatic partners such as CCP and Pathfinder, as well as ICT partners such as Text to Change and D-Tree. (FHI360 discussed health-ICT partnerships for m4RH in December 2010 at http://archive.k4health.org/toolkits/mhealth/december-13-2010.) Promotion is also a crucial factor in scale up. A question was raised about emergency contraception, clinic services and mobile phone ownership all being skewed toward higher-educated populations, but FHI360’s preceding research in Ghana found that education bias was not great for emergency contraception. Interactive voice response (IVR) was raised as an alternative format for illiterate users. M4RH can reach illiterate users through literate peers that read SMS for users, but this could create challenges for confidentiality. FHI 360 is planning to investigate options for IVR as part of future efforts but IVR is expensive, particularly for programs in Africa and we feel confident that m4RH is able to serve a large portion of their target populations. A point was made that a future concern may be that competing spam can distract from SMS and related legislation can limit its use in some countries. m4RH may be less vulnerable to these challenges since it is a “pull” system, currently using non-digital promotion to encourage user registration. m4RH has had steady SMS use and spikes after promotions. It also has repeat users, as indicated by the telephone numbers that form unique IDs. There are also reports of users sharing m4RH messages with peers by forwarding SMS. m4RH is now focused on sustainability, including costs, and significantly, content management to ensure quality and overall project coordination. Meeting participants also suggested the opportunity for advertising messages to provide revenue for financial sustainability.
Mobile network operators (MNO) may be an important partner for sustainability, but their expectations for traffic levels are very high. MNO investments in mHealth projects are often short-term, creating an unclear precedent for long-term sustainability. FHI 360 has explored both of these options and will continue to do so as part of their larger sustainability discussions, which are currently on-going.
1c. m4QI by Pamela Riley, Abt Associates and James Bon Tempo, Jhpiego
Preparations for m4QI were presented a year ago to discuss SMS assessments and reminders as a cost-effective means of reinforcing training of field staff. (March 2011 discussion available at http://archive.k4health.org/toolkits/mhealth/March-23-2011.) As part of the USAID-funded SHOPS project, the m4QI pilot has since used a version of FrontlineSMS to send daily messages for eight weeks on four topics to 34 family planning providers at six locations of Marie Stopes International in Uganda. A qualitative process evaluation found that 86% of messages were successfully delivered. The pilot was able to use assessment questions via SMS to identify knowledge gaps with actionable data, as well as self-reported behavior change for all indicators. Users said they received instant and clear feedback for wrong responses. They also said the project was a constant motivator of hand washing, a project goal. Marie Stopes Uganda is planning a national roll out of m4QI to staff, social franchisees, and outreach partners. m4QI code is being revised for alpha testing of FrontlineSMS:Learn, which is targeted for public launch in May 2012.
Lessons Learned and Discussion
The pilot experienced technical problems as intermittent system failures. A low response rate (19%) was attributed to duplicate messages sent but only answered once. Formatting errors on true/false questions also caused rejections of other answers. The program also required the right kind of modem, and messages stopped when payments stopped for the modem. Compatibility
of FrontlineSMS with Windows was also a challenge. No one was available onsite to troubleshoot these issues, requiring calls to US-based staff at two in the morning. A local IT provider is thus sought to support future efforts.
The local capacity needs, platform, processes and partnerships have evolved with the pilot. A new version of FrontlineSMS core platform is in development. The pilot also needs to incorporate new staff, processes, and policies for quality assurance. Unintended results of the m4QI pilot include mobiles being banned from the procedure room. Partners also evolve during a pilot, as FrontlineSMS transitioned from a flexible open source group to an established business with requirements for brand preservation and quality assurance review. M4qi started before change, much heavier review by FrontlineSMS team, what released on what had their brand. Resources are also needed for incorporating and supporting Learn within the FrontlineSMS family of applications. FrontlineSMS was mentioned as a good learning tool and easy to use for small, local NGOs, since the NGO is empowered to use the technology how they wish. However, it would face challenges in national rollout. FrontlineSMS is an open source platform supported by an IT community. Consequently, the principle cost is not software development, but instead program planning and supervision.
1d. CycleTel™ by Meredith Puleio, IRH/Georgetown
CycleTel is a mobile health service in India that uses text messaging to facilitate use of the Standard Days Method of family planning. In January of 2011, IRH presented on the formative research that contributed to the design of CycleTel. (See http://archive.k4health.org/toolkits/mhealth/january-19-2011.) Since then, CycleTel has progressed through proof of concept, automated testing, and business planning, as it progresses toward partnership development and scale up in 2012. Pilot results among 653 exit interviews found the mHealth service was timely (87.4%), private (79.5%), and easy to use (78.7%).
Lessons Learned and Discussion
CycleTel’s development has been a step-wise systematic and iterative process. The technology platform has evolved from initially using FrontlineSMS to building out an automated service that relies on a local SMS gateway. Partnerships will become more complex as IRH implements a business plan for the sustainability of CycleTel. After evidence was determined that the service works, IRH developed a business model with support from the Boston Consulting Group in India. This includes identification of market segments and size, industry analysis, go-to-market strategy, financial modeling, and business scenarios (high and low achievement cases depending on the level of donor investment). In the India context, it is more attractive to work with telco aggregators than mobile network operators (MNO) for this particular service. It would be difficult to compete with hundreds of local content providers for MNO attention. Conversely, there are several capable aggregators in India with a mHealth focus that work across MNOs and are keen to partner with IRH. (IRH contributed to a report coauthored by the mHealth Working Group on public private partnership in mHealth at http://archive.k4health.org/toolkits/mhealth/december-7-2011. ) To reach a critical mass (100-200K users), CycleTel would first require anchor funding for broad-based promotion via a mix of channels, including high-engagement approaches (face to face) to attract customers. CycleTel uses a direct-to-consumer approach, though there are alternative examples of mHealth projects using partner organizations in family planning for promotion. It is important to note that CycleTel is promoting access to the Standard Days Method of family planning, which does not necessarily require relying on family planning services through partner organizations. CycleTel also provides a call center to answer user questions about the service as well as to provide information about other family planning options. However, note that the call center is one of the most costly components of the operation.
2. Frameworks for Lessons Learned in mHealth
The update from past presenters shares a range of lessons learned in mHealth. A clear framework for issues would help synthesize lessons from the growing and evolving body of mHealth projects. A variety of frameworks have been developed for health information technology, in general, and mobile technology in developing countries, specifically.
A number of practical implementation issues are framed in “Unintended Consequences of Computerized Provider Order Entry”, including additional work, workflow issues, new kinds of errors, persistence of paper forms, changes in communication patterns, user emotions, changes in the power structure, continuing technology demands, and overdependence on technology.
Relevant organizational issues are also categorized by Managing Technological Change- Organizational Aspects of Health Informatics. These include organizational readiness, design issues, project planning and management, technology acceptance, personal stress, organizational leadership and politics, and evaluation.
Challenges and factors for ICT success were described in “Capturing Technology for Development: An Evaluation of World Bank Group Activities in Information and Communication Technologies”. These include quality of design of ICT, breadth of technological approach to ICT, implementation according to plan and modifications, coordination of IT initiatives and investment, strong government implementation capacity, government commitment to ICT, implementation capacity, strength and effectiveness of management, change management, and ICT procurement.
Meeting participants suggested additional considerations for an issues framework to support planning in mHealth. These include defining the prerequisites for a successful mHealth program, as well as which partners need to be involved. A clear explanation of how mHealth contributes to health outcomes would be helpful. Knowledge management is also useful for supporting these efforts. mHealth should be part of a national strategy and should be integrated with existing information systems. Business models are important, particularly when working with mobile network operators (MNO). Participants said guidance is needed on how to design and manage mHealth projects. IT organizations can provide counsel on related IT issues, though broader and impartial guidance is needed on the strategy, design and management of mHealth projects.