APPRECIATIVE INQUIRY EXPERIMENTAL STUDY AT SNEHA

Background

This document forms part of an ongoing valuation of the Appreciative Inquiry (AI) intervention under the City Initiative for Newborn Health program at Sneha.

SNEHA was founded in 1999 by a group of doctors and social workers to create better outreach in terms of health, education and nutrition for women and children in urban slums. The core mission is to value every mother and child in Mumbai. SNEHA works in partnership with public hospitals, academic and research institutions, professional bodies, NGO’s, and communities on a wide range of initiatives such as a crisis center for women, a mobile health service for street children, nutrition education, and crèche services. SNEHA played a significant role in the 2005 floods in setting up medical care for the displaced and shelter less. In consolidating its experiences over the years in 2004, SNEHA teamed up with University College London, the Municipal Corporation of Greater Mumbai (MCGM), and the Social Initiatives Group of ICICI Bank, to create the City Initiative for Newborn Health. Based on the consensus that reducing neonatal mortality is a key component of improving child survival rates in Mumbai, as well as model development in meeting the Child Survival Millennium Development Goals, theproject addresses the causes of neonatal mortality by impacting the supply and demand aspects of care for Mumbai’s most vulnerable populations.

Appreciative inquiry is the methodology used to strengthen and define the shared visions and is one of the most important components of the project, used across all components within CINH. This methodology is increasingly becoming used in development as a extended measure to not only enable groups to seek empowerment within their own systems, but also as a way to value themselves and the system. The project believes that to bring about change in the public health system of Mumbai, one cannot only focus on upgrading skills and knowledge to give better patient care, but we also need to focus on the human aspect of change. This philosophy trains and works with people working in the system to partner and commit to value their work, which then creates a framework for change. More than a structured way of bringing change to the way health staff interacts and values their work; appreciative inquiry is a philosophy of auto-inspection which engenders empowerment. The aim is to create environments for the public health personnel, to value their work and their interaction with the community. In measures of this behavior inquiry, this has translated to several positive changes, including supportive supervision, feedback forms, remodeling structures of these health posts, a commitment to communication, and a higher level of trust by the public served.

Objective of the study

To assess the impact of the AI intervention in the form of Change Management interventions at levels of all employees in maternity home facilities under the Municipal Corporation of Greater Mumbai (MCGM).

DESIGN AND METHODOLOGY

The study was designedalong three dimensions. First, as a case-control study where maternity home staff were studied before and after the AI intervention. Second, viafacility observationsby AI trainers, and third, by patient exit interviews conducted by the AI team.

The duration of the study was 3 months.

Sample

The target group was Staff across all levels from 18 maternity homes. These maternity homes offered fairly standardized services for normal pregnancy and labor.

9 maternity homeseach were randomly assigned to intervention and control groupsto receive AI or no intervention respectively.

In the facility survey and the facility observations,respondents from each maternity home were studied in two groups:

  1. Doctors and nurses
  2. Junior staff (Ayahbais, ward boys, sweepers, etc.)

The total sample size from each category was as follows:

GROUP / Doctors and Nurses / Other staff / Observation of Sr. staff / Observation of Jr. staff / Patient Exit interview / TOTAL
Intervention / 90 / 130 / 36 / 35 / 77 / 368
Control / 82 / 125 / 30 / 33 / 77 / 347

The sample size varies slightly between the intervention and control groups and between the two time periods owing to variation in the availability and attendance of staff. The above figures are the average for each group across the two time periods.

Survey instruments

Three instruments were designed for the maternity home staff, the observers, and the patients respectively along the following themes[1]:

  • Respect
  • Concern and encouragement
  • Communication with others
  • Attitude to work
  • Supportive supervision

Questionnaires for facility staff were designed keeping in mind the socio-demographic characteristics of the respondents. Emphasis was placed on psychometric robustness, internal validity, and consistency. This is true even of the translated versions of the instrument. Owing to the lower levels of literacy among junior staff, the scale and format were kept simple and easy to understand across all levels of respondents. The scale was designed as a straight line continuum from 0 (“Disagree”) to 10 (“Agree”).

The observation tool used by the AI team scored participants across the same themes of appreciative and non-appreciative behavior as the facility questionnaires, to make the findings comparable.

The patient exit interviews used a more conventional format of multiple-choice questions to assess patient experiences across the responses of “Never”, “Sometimes, “Frequently”, and “Always”.

Data was coded and analyzed using SPSS.

FINDINGS AND RESULTS

Data from the three instruments was analyzed and compared across the time periods using one-way ANOVA.

Maternity home staff:For doctors and nurses, there was no significant difference in the groups along any dimension. For the junior level staff, there was a significant improvement within the intervention group with regard to attitude to work and the workplace. However, there was a significant deterioration along the dimension of respect. The control group also displayed a significant improvement along the dimensions of attitude to work, concern and encouragement.

AI team observation of facility staff: There was no significant change in the intervention group. The control group showed a significant improvement with regard to concern for patient’s privacy, andwillingness to clarify patient’s doubts.

Patient Exit Interviews: The intervention group showed significant improvement along the following dimensions:

  • Attentiveness to patient needs by the doctor
  • Attentiveness to patient needs by other staff
  • Doctor’s ability to clarify patient’s doubts
  • Overall attitude of the doctor
  • Overall attitude of other staff

Results of one-way ANOVA comparing intervention and control groups at the end of the intervention

Dimension / F-test / Significance
Attentiveness of doctor / 12.254 / .001*
Attentiveness of other staff / 12.855 / .000*
Doctor’s ability to clarify doubt / 7.814 / .006*
Problem faced by patient / 2.180 / .142
Overall attitude of doctor / 14.159 / .000*
Overall attitude of other staff / 16.145 / .000*
Patient need for additional information / 5.766 / .017*
Overall patient satisfaction / 1.617 / .205

*Significant at F<0.05

DISCUSSION

The results, at a glance, give a mixed picture of the impact of the AI intervention in maternity homes. The facility survey showsan improvement in both the control and intervention groups. This puts intoquestion the role of the interventionper se in affecting behavior change. Further, the facility observation displays improvement but only in the control group. The patient exit interview results, however, show a clear division between the two groups. There is a marked improvement in the intervention group, while the control group shows no change.

A critical assessment brings to light limitations in the facility staff survey design. The scale used, while simple and easy to understand, leaves scope for a lot of missing information and misinterpretation of responses. The staff observation tool is susceptible to interview bias given that the interviewers were also trainers.

Further investigation of individual maternity homes reveals a parallel intervention of system upgrade and change. The contribution of these to behavior and motivation levels and patient satisfaction confounds the results of this study.

However, the patient exit tool uses a different scale, with none of the above limitations. The respondents are unbiased direct recipients of the facility service. Of the three sets of results, the results from the patient exit interviews thus give us the most complete picture. While a limitation of is that the group of patients across the two surveys were not the same, it is this feature itself that lends a true unbiased picture of the situation.

Results of the patient study are presented below.

The following graphs show the percentage improvement in the intervention group as shown by the patient interviews.

1. Treatment satisfaction

While 89% of patients were satisfied with the treatment they received before the AI intervention, this number rose to 96% after the intervention, which was statistically significant. There was no change in the status of the control group.

  1. Attentiveness of maternity home staff

The intervention group showed a 28% improvement after the AI intervention, while the control group showed no change.

  1. Treatment by staff

The intervention group showed a 20% improvement after the AI intervention, while the control group showed no change.

CONCLUSION

The experimental study while limited in some aspects of its design and implementation provides conclusive evidence of the impact of the AI intervention in maternity homes. This is best displayed in the results of the patient survey. There is substantial qualitative data and evidence to support these results. Together, the case for the need and success of an AI intervention in this setting is strong. Further, since quantitative studies on AI are rare, this study builds on the existing body of knowledge.

Some interesting Qualitative evidence:

Discussion forums and follow-up meetings with the maternity home staff after the AI intervention has provided anecdotal evidence of a positive impact.

At the facility end, the staff has repeatedly reported improved communication within their facilities and with their patients. They feel AI has helped them resolve internal conflicts amicably. It has motivated them, and given them a shared understanding and learning of the limitations of their workplace as well as the potential for change amid a constrained environment.

A number of maternity homes have discussed an increase in respectand empathy between staff across levels, and of the importance of role models for junior staff. A sense of ownership around facilities as well as an ownership of challenges and problems has been a recurring theme.

Introspection, before affecting change in others and in the system, has been flagged as an outcome that has helped staff shift focus from blaming others for their problems.

The following quotes exemplify some of the above themes and learnings:

“Earlier, I felt change in the system was impossible, but now I see it as a possibility. I can think in terms of a vision for my maternity home. I know that if Itake charge of the problem, I can make change happen in spite of limitations within the BMC structure. Its up to me to make the change.”

“My seniors never listened to me or my problems. After the training, I feel heard. They are sensitive and understanding in a way they have never been before.”

“We are all human beings first. If we recognize that we need to respect ourselves and each other, we’ll be able to do the same for our patients.”

“AI has completely changed my attitude towards my work and my co-workers. I feel motivated to perform better.”

“I would encourage my own family members and relatives to use maternity home facilities. This is not something I would have said before I experienced AI.”

At the patient end, there are a number of stories, but here is an interesting story of a patient who delivered her baby at one of the maternity homes under the AI project. Subsequently, she met a non-AI team of SNEHA working at a different hospital. She was pleased to see them there and commented, “I am so happy to see SNEHA members here. I was a patient at maternity home X, and had a very pleasant experience thanks to the presence of SNEHA. I am sure I’ll be well looked after here too.”

“The maternity home staff was very understanding and caring and worked together. They paid a lot of attention to my needs through labor and delivery.”

-----

1

[1] Behavior patterns were categorized across these themes through extensive formative research at the outset of the project.