Electronic Supplemental Content
Appendix: Calculations
This section describes in full detail the calculations presented in the Results section of the paper.
A.Malawi
The program was implemented in Chikwawa District in southern Malawi, which in 2011 had an estimated population of 356,682. The selected comparison site, Mulanje District, is in the same region in southern Malawi and has an estimated population of 428,322. Chikwawa and Mulanje are not contiguous districts, which reduces the likelihood of their residents crossing between them to access services. UNICEF country representatives selected Mulanje because of the similar population size and HIV prevalence to that in Chikwawa. However, after a revision of the tools used in the UNICEF project, integration activities commenced nationwide; it is estimated that by 2013 these activities covered 40% of the Mulanje population.
Interviews with program officers identified three key components in Malawi’s integrated program: the use of SMS technology; training and deployment of Male Motivators; and the creation and implementation of Child Health Passports. We separate our analysis of lives saved, infections averted and undernutrition cases cured according to these three components.
A.1 SMS technology
The three SMS technology systems that we focus on, following our review of Malawi’s program protocols, and as determined by availability of specific indicators to assess impact, are:
-Results160: relays EID test results back to the facility, reducing turnaround time
-RemindMi: a texting service between clinic staff, especially between clinic workers and community health workers, to assist communication between clinics and their communities and to provide reminders to CHWs regarding tasks.
-AnthroWatch: tracks infant nutrition status and sends text messages reminding caregivers to attend follow-up visits.
These technologies purportedly impact attendance and adherence in PMTCT and CMAM (through reminder messages). As such, of the available indicators in the program,we selected the number of children who receive virological testing, HIV cases identified, and SAM cases identified for analysis.The results are delineated below by HIV and SAM impact.
A.1.1 HIV
To estimate the effect of the SMS technology program on adherence to PMTCT protocols, we use three of the available indicators:percent of children receiving once daily nevirapine (NVP) (see 1.1.1), co-trimoxazole (CTX) (see 1.1.2), and virological testing (see 1.1.3).
A.1.1.1 Once daily NVP
In order to calculate this indicator, we use the number of HIV-exposed children enrolled on once daily NVP in PMTCT, divided by the number of HIV-exposed children (utilizing the number of live births, at home or in a facility, to mothers with a known HIV positive status as a proxy).We use single dose NVP (sdNVP) as our counterfactual, as in the absence of Option B+, sdNVP is the default, rather than no treatment. The literature indicates that sdNVP results in a 11.9% risk of HIV transmission while breastfeeding, while the transition to once daily NVP reduces the risk to 2.5% of transmission.(1, 2)
We estimate that in Chikwawa district, 37% of children exposed children received once daily NVP in 2011, 92% in 2012, and 96% in 2013, while Mulanje district saw an estimated decrease from 93% in 2012 to 90% in 2013.
Number of HIV-exposed children receiving once daily NVP (estimate)2011 / 2012 / 2013
Chikwawa / 37% (766/2,070) / 92% (1,459/1,579) / 96% (757/787)
Mulanje (comparison site) / No data available / 93% (1,071/1,156) / 90% (1,776/1,968)
Comparing changes in Chikwawato the 2011 baseline, we estimate that 82[1] infections were averted in 2012 and 44[2] infections were averted in 2013, for a total of 126 infections. Applying the difference-in-difference (DiD) method, we subtract out Mulanje’s (negative) 3[3] percentage point change in uptake of NVP in 2012 and 6 in 2013 (this makes the assumption that the 2011 Mulanje coverage was 96%, projecting the trend backwards). This increases the estimated treatment effect; as such, we estimate that in Chikwawa an additional 4 infections would be averted in 2012 and 4 infections would be averted in 2013, for a total of 134 infections. Fasawe et al. estimate that 22.5 DALYs(3) are averted for each HIV infection averted in an infant in Malawi, meaning our estimates imply a range of 2,835-3,015 DALYs averted over the two years of the program.
A.1.1.2 CTX
To calculate this indicator, we used the number of HIV-exposed infants receiving CTX divided by the sum of HIV-exposed infants and infants with unknown status (note we include children with unknown status since the Mulanje data indicates many were given CTX). In Chikwawa, 9% of potentially exposed children received CTX in 2012, which increased to 17% in 2013. In Mulanje, 17% of children received CTX in 2012 and 108% in 2013,[4] resulting in a 91-percentage point increase in the comparison site.
Children receiving once daily CTX (estimate)2011 / 2012 / 2013
Chikwawa / N/A / 9% (336/3,762) / 17% (701/4,021)
Mulanje (comparison site) / N/A / 17% (332/1,994) / 108% (3,319/3,080)
The 8-percentage point increase in Chikwawa from 2012-2013 indicatesthat an additional 322 HIV-exposed infants were given CTX in 2013. Since we do not have 2011 data, we generate a plausible range by firstusing the 2012 rate as the baseline 2011 rate (the more conservative estimate), as well asutilizingthe same 8-percentage point increase from 2011-2012. This generates a treatment effect range of 0-301 infants for 2012 and a range of 322-643 for 2013, for a total of 322-944 infants. Next, we apply the HIV positivity rate in Chikwawa in 2013 (4%, as determined by the number of PMTCT enrolled in Pediatric ART), resulting in an estimated 0-12 HIV positive infants among those who received CTX in 2012, and 13-38 in 2013. As demonstrated by a study in Uganda, CTX is associated with a 46% reduction in mortality among HIV-positive individuals.(4) Assuming that 46% of these children are saved by the CTX, we estimate6-23 lives saved by the increase in CTX coverage. Mathers et al. (2007) calculate that preventing a death in infancy averts 33 DALYs;(5) therefore, we estimate that 198-759 DALYs were averted due to the integrated program. We do not calculate lives saved using the DiD approach since Mulanje’s rate of increase exceeded Chikwawa’s, and therefore assign a treatment effect of zero.
A.1.1.3 Virological testing
This indicator was calculated by using the number of HIV-exposed infants receiving virological testing divided by the sum of HIV-exposed infants and infants of unknown HIV status. In Chikwawa, 39% of HIV-exposed infants received virological testing in 2012, while 30% were tested in 2013; this implies a decrease of 9 percentage points. In the absence of 2011 data, we again generate a plausible range byusing the 2012 rate as the baseline 2011 rate as well as assuming the same 9-percentage point decrease from 2011-2012 to generate a range between 39-48% of children being tested in 2011. Through the pre-post approach we record no program impact. In Mulanje in 2012, 122% of children were tested (despite having more children tested in the data than were HIV-exposed or of unknown status, we maintain this percentage for methodological consistency) and in 2013 68% were tested, resulting in a 54-percentage point decrease.
Number of Children exposed to HIV receiving virological testing (estimate)2011 / 2012 / 2013
Chikwawa / N/A / 39% (1,478/3,762) / 30% (1,192/4,021)
Mulanje (comparison site) / N/A / 122% (2,442/1,994) / 68% (2,105/3,080)
Again we estimate a range for 2011 between 122-176% of children were tested. Using DiD, in 2012 we estimate a 0-45[5] percentage point increase, and in 2013, a 45-90[6] point increase in Chikwawa when compared to Mulanje; therefore, we estimate that in 2012 between 0-1,478 additional children were tested and in 2013 1,192 additional children were tested as a result of the integrated activities. Again applying the 2013 HIV positivity rate (4%), we estimate that between 48-107 of these children were HIV positive and therefore saved due to the integrated program, assuming that all positive individuals were placed on treatment. This results in 1,517-3,381 DALYs averted (using 31.6 DALYs for each child put on HIV treatment(6)).
A.1.1.4 SMS HIV Summary
The use of SMS technology is estimated to have prevented both infections and future deaths in the integrated program district, Chikwawa. We calculated through the pre-post method that the use of once daily NVP prevented 126 infections resulting in 2,835 DALYs averted, while the DiD method yields an estimate of 134 infections or 3,015 DALYs. CTX is estimated to have saved 6-23 lives and averted 198-759 DALYs using pre-post, and no effect using DiD. Virological testing is estimated to have saved zero lives using pre-post indicators, while the DiD method yields an effect of 48-107 lives saved and 1,517-3,381 DALYs averted. We therefore estimate that through the pre-post method 6-23 lives were saved, 126 infection averted, and 3,033-3,594DALYs averted and that through the DiD approach 48-107 lives were saved, 134 infections averted, and4,532-6,396 DALYs were averted.
A.1.2 Severe Acute Malnutrition (SAM)
It was anticipated that with reminder messages to attend Outpatient Treatment Program (OTP) visits, fewer children would be lost to follow-up. Additionally, we expected reminder messages to assist participants in treatment adherence improving recovery. We therefore use the default rate among OTP patients to assess the impact of SMS technology on SAM. The text messages sent prompted caregivers to bring children in for follow-up visits; this was only applicable to the OTP category and not to the Nutrition Rehabilitation Unit (NRU).
A.1.2.1 OTP Default Rate
The OTP default rate measures the number of patients who are lost to follow up (i.e., they are not known to have recovered or died). In Chikwawa in 2011, 3.5% of patients were lost to follow-up; in 2012, 5.4%; and in 2013, 7.3%. This results in a 3.8 percentage point increase in the default rate between baseline (2011) and 2013. In comparison, in Mulanje, in 2011, 3.6% of SAM children were lost to follow-up; in 2012, 7.9%; and in 2013, 3%. This is a 0.6 percentage point decrease in the same timeframe. For the purposes of calculating a programmatic impact, we will assume no effect using either pre-post measures or DiD.
OTP Default Rate (estimate)2011 / 2012 / 2013
Chikwawa / 3.5% (21/608) / 5.4% (41/763) / 7.3% (90/1,227)
Mulanje (comparison site) / 3.6% (65/1,795) / 7.9% (163/2,058) / 3% (61/2,037)
A.1.3 SMS Summary
The use of SMS technology in Chikwawa is estimated prevented both HIV infections and infant deaths. We estimate through the pre-post method that 6-23 lives were saved, 126 infections averted, and 3,033-3,594 DALYs averted and that through the DiD approach,48-107 lives were saved, 134 infections averted, and4,532-6,396 DALYs were averted.
A.2 Male Motivators
We identified male motivators (MMs) as the second key component to the integrated program in Malawi. MMs conducted door-to-door visits in villages to educate other men to better understand their roles in child health, HIV, and nutrition issues. MMs also encouraged participation in the continuum of care including couples counseling and HIV testing, support of early antenatal booking, as well asregular antenatal care (ANC), institutional delivery, and postnatal care (including IYCF and PMTCT). Of the available indicators, we hypothesize that MMs impacted the number of IYCF feeding practices, SAM cases enrolled in treatment, ANC attendance rates, and the percent of pregnant women tested for HIV. However, we recognize that due to the limitation of the available data that not all the effects of MM activities are captured in this analysis.
A.2.1 SAM
A.2.1.1 IYCF feeding “early initiation of breastfeeding”
MMs educated other men on the importance of IYCF; to estimatethe impact of this advocacy, we assessed the number of infants initiated on breastfeeding within 1 hour of birth. In Chikwawa, 85% of infants were initiated on breastfeeding within an hour of birth in 2012, while 96% were initiated in 2013 – an 11 percentage point increase. In the absence of 2011 data, we create a range for the baseline, assuming the conservative estimate of either no change, or the same 11% increase, corresponding to 74% initiation within 1 hour in 2011. During the same timeframe in Mulanje, 87% and 92% infants were initiated on breastfeeding within an hour of birth in 2012 and 2013, respectively. Again, in the absence of data for 2011 we use the same method to estimate a range of 82-87% initiation. Therefore, in comparison to Mulanje, Chikwawa increased breastfeeding initiation by 6-12 percentage points during 2011-2013 utilizing the DiD technique.
IYCF feeding “early initiation of breastfeeding” (estimate)2011 / 2012 / 2013
Chikwawa / N/A / 85% (8,766/10,285) / 96% (13,903/14,555)
Mulanje (comparison site) / N/A / 87% (7,896/9,029) / 92% (14,369/15,636)
In order to quantify the impact of early initiation of breastfeeding, we use Malawi’s national neonatal mortality rate in 2012 of24 deaths per 1,000 live births and that early initiation of breastfeeding rates in Malawi are high (94%). Further, drawing from evidence in the literaturethat 22% of neonatal deaths can be prevented if breastfeeding is initiated within an hour of birth,(7)we increase the national morality rate by 22% to 29 per 1,000 live births to approximate the mortality rate for infants not initiated on breastfeeding within one hour of birth. Looking only at changes within Chikwawa, we estimate that between 0-33 lives were saved in 2012, and that 46-93[7] lives were saved in 2013 due to early initiation of breastfeeding. Applying the DiD modeling approach, 18 lives were saved in 2012 while 25-51[8] lives were saved in 2013 due to early initiation of breastfeeding.The overall estimate of lives saved due to the program over the two years is 46-126through the pre-post approach and 43-69 through the DiD approach. Mathers et al. (2007) calculate that preventing a death in infancy averts 33 DALYs;(5) we therefore assign 1,518-4,158 DALYs (pre-post)and 1,419-2,277 DALYs (DiD) to the lives saved via IYCF due to the integrated program.
A.2.1.2 Proportion of SAM cases enrolled in treatment
The attendance rate is calculated by usingthe number of NRU+OTP patients enrolled in the CMAM program as compared to the overall SAM burden (measured by using the national malnutrition rate). In Chikwawa in 2011, 40% of children with SAM were admitted into either NRU or OTP; in 2012 the number was 46% and in 2013 it was 66%. In Mulanje, 85% of children with SAM were admitted into either NRU or OTP in 2011; 93% in 2012; and 91% in 2013. This suggests a 26 percentage point increase in Chikwawa over the two years, compared to a 6 percentage point increase in Mulanje.
Proportion of SAM cases enrolled in treatment (estimate)2011 / 2012 / 2013
Chikwawa / 40% (869/2,186) / 46% (1,038/2,250) / 66% (1,533/2,316)
Mulanje (comparison site) / 85% (2,123/2,501) / 93% (2,352/2,533) / 91% (2,323/2,566)
The increase in attendance rates in Chikwawa lets us estimate lives saved by taking the difference in the attendance rate at base year (2011) from the attendance rate in 2012 and 2013 and multiplying this by the cure rate for the respective years (89% and 89%). In 2012, 120 more children were enrolled in treatment, and in 2013, 536 extra children were enrolled in treatment in Chikwawa; for a total of 656 children. Applying the DiD approach, we estimated 0 additional children enrolled in treatment in 2012 Chikwawa (the progress was faster in Mulanje than Chikwawa), while an additional 412 were enrolled in treatment in 2013. Therefore, we estimate 656 extra children were enrolled in treatment as estimated through the pre-post approach and 412 children as estimated by the DiD approach. Using average SAM case fatality of 30-50%,(8) we estimate through the pre-post approach that 197-328 lives were saved, while the DiD approach suggest that 124-206 lives were saved. This corresponds to averting 6,501-10,824 and 4,092-6,798 DALYs respectively, where DALYS were calculated using the same 33 DALYs per infant life saved as above. Moreover, we also estimated the DALYs averted for the 50-70% of SAM children who would not have died in the absence of the program, but who would also have not been cured. We calculated DALYs through life expectancy at birth and the disability adjusted weight for severe anemia(9) and found that for every SAM case cured, 4.8[9] DALYs were averted. As such, through the pre-post approach we estimate that 1,574-2,203[10] DALYs were averted, and through the DiD approach, 989-1,382[11] DALYs were averted. Therefore, a total of 8,075-13,027 DALYs through the pre-post and 5,081-8,180 DALYs through DiD were averted during the two-year program.
A.2.1.3 Proportion of MAM cases enrolled in treatment
We calculated MAM attendance rate as the number of SFP (supplemental feeding program) patients in the CMAM program as compared to the overall burden, measured by using the estimated MAM burden in the under-5 population. In Chikwawa in 2012, 52% of children with MAM were enrolled into SFP and in 2013, 98%; lacking 2011 SFP data, we create a range through assuming either no change, 52% enrollment in 2011 (2,273/4,372), or byprojecting a similar trend estimating that6% (262/4,372) of children were enrolled. In Mulanje, in 2011, 34% of children with MAM were admitted into SFP; 32% in 2012; and 33% in 2013. This suggests between a 46-92 percentage point increase in Chikwawa over the two years, compared to a 1 percentage point decrease in Mulanje.
Proportion of MAM cases enrolled in treatment (estimate)2011 / 2012 / 2013
Chikwawa / N/A / 52% (2,326/4,500) / 98% (4,551/4,632)
Mulanje (comparison site) / 34% (1,701/5,002) / 32% (1,638/5,067) / 33% (1,693/5,132)
We estimate lives saved in Chikwawa by taking the difference in the attendance rate in 2011 from the attendance rate in 2012 and 2013 and multiplying this by the cure rate for the respective years (91% and 84%). Through the pre-post approach we estimate that in 2012, 0-1,884 lives were saved, and in 2013, 1,790-3,579 lives were saved in Chikwawa; for a total of 1,790-5,463 lives saved. Applying the DiD approach, we estimated 82-1,993 additional lives were saved in 2012, while an additional 1,829-3,619 were saved in 2013. The estimate for lives saved is therefore 1,790-5,463 using the pre-post approach, and 1,911-5,611 through the DiD approach.
However, we also estimate that MAM case fatality is 10-17%,(8, 10) as risk of death is 3 times smaller in magnitude for children with MAM than children with SAM (SAM case fatality is 30-50%(8)). As such, we estimate through the pre-post approach that 183-615 lives were saved and through DiD that 191-954 lives were saved as a result of the integrated program. This results in 6,039-20,295 and 6,303-31,482 DALYs averted, respectively. Moreover, we also estimated the DALYs for the 83-90% of MAM children who would not have died in the absence of the program, but were cured. We estimated DALYs through life expectancy at birth and the disability adjusted weight for malnutrition(9) and calculated that for every MAM case averted we avert 2.8[12] DALYs. As such, through the pre-post approach we estimate that 4,250-9,120[13] DALYs were averted, and through the DiD approach, 4,628-14,143[14]. Therefore a total of 10,289-29,415 and 10,931-45,625 DALYs were averted during the two year program.
A.2.2 Prenatal Care
A.2.2.1 ANC attendance
Male motivators promoted the importance of ANC attendance to males in the community. We used ANC attendance at four visits as recorded in the MOH HIV database to estimate the change as observed during program years. In Chikwawa, 890 women attended four ANC visits in 2012, which increased to 2,696 in 2013; a 203% increase. We did not have access to 2011 data. Despite the increase in four ANC visits observed in Chikwawa, little evidence supports four-visit ANC effectiveness on preventing maternal deaths.(11) The few impact evaluations that have been conducted test reduction of ANC visits from numbers greater than 5, but none have assessed the impact of no or very few ANC visits to 4. One study finds that reduction of ANC visits from 8 to 5 in four developing countries has no statistical effect on clinical outcomes, except that a 56% higher pre-eclampsia/eclampsia rate could not be ruled out.(12) If we assume the ANC attendance increase resulted in halving maternal deaths due to eclampsia, and use the sub-Saharan estimate of 12% of maternal deaths due to hypertensive disorders,(13) then the effect of ANC attendance would result in no discernable effect on maternal mortality for the population size of our study areas.