Summary:

Fidelity of Implementation of Prospera Digital: Evaluation of a Multi-Site mHealth Intervention Aimed at Improving Maternal Health Outcomes in Mexico

Background: Infrastructure and human capital limitations motivate the design of mHealth programs, but their large-scale implementation may be challenging in a development context. Prospera Digital (PD) is a pilot mHealth intervention aiming to improve maternal and child health and nutrition designed as a randomized controlled trial with 3 treatment arms. It was implemented during 2015–2017 in 326 treatment clinics located in 5 states in Mexico.

Objective: Assess, with an external evaluation, PD’s fidelity of implementation using 6 dimensions: adherence, quality, responsiveness, intervention complexity, facilitation strategies, and program differentiation.

Methods: Benchmark for implementation was first established by interviewing PD’s developers. Extensive fieldwork in the 5 states was then conducted to assess its fidelity in heterogeneous contexts. The evaluation team visited 39 health clinics to assess the initial sign-up events and conduct a follow-up. Overall, the team made 28 closed observations; conducted 17 focus groups; and interviewed 74 health providers, 10 community leaders, and 92 beneficiaries. Field notes from the implementation team on all clinics were also examined.

Results: Co-ordination between the Health and Social Development ministries was adequate, although some health providers were not informed about PD. Program developers added useful implementation strategies during roll-out to reinforce sign-up events. Key quality facilitators were the clarity and relevance of the messages from the short messages service. Beneficiaries expressed high satisfaction with PD. In contrast, implementation barriers to adherence in some localities might reduce the potential impact of PD. Program differentiation was low between the 3 treatment arms.

Conclusions: PD is a promising strategy to contribute to the promotion of early childhood development in Mexico. Implementation science evaluation can help improve the quality of large-scale mHealth interventions by anticipating barriers and providing insights on how to increase performance. This is especially relevant to inform impact evaluation in development contexts. The trial was registered at the American Economic Association’s registry for randomized controlled trials with trial registry number ‘AEARCTR-0001035’.

Authors: Gaitán-Rossi, Pablo; Lobato, Selene De la Cerda; Navarro, Alma Cecilia Pérez; Esteva,  Arturo Aguilar; Garcia, Manett Rocío Vargas; Vilar-Compte, Mireya