< BACK TO E&L HOMEPAGE
The Impact of Electronic Integrated Management of Childhood Illness (e-IMCI) Program in KwaZulu-Natal as Compared to Paper-Based IMCI
June 1, 2022
June Lee
Director, Evidence & Learning
The ELMA Philanthropies
Photo: A health worker assesses a patient in the KwaZulu-Natal e-IMCI pilot district.
Evaluation Partner: Centre for Rural Health at the University of KwaZulu-Natal
This evaluation compares the effectiveness of electronic integrated management of childhood illness (e-IMCI) to traditional, paper-based IMCI to improve care for sick children under five in primary health clinics in iLembe district.
Background
Using standardized algorithms, IMCI guides healthcare workers to assess, treat, and manage common childhood illnesses. E-IMCI automates this process and is intended to be faster, more complete, and more accurate. As part of ELMA’s “Unfinished Business” suite of investments in pediatric HIV, e-IMCI helps ensure children with HIV are properly diagnosed and treated.
Findings
Although e-IMCI was well-received by health clinic staff, implementation tracking found that e-IMCI use was low overall and minimal in some periods. The highest usage observed was during the first full month of data collection when 23% of children benefitted from e-IMCI.
ELMA continued to support e-IMCI implementation because it performed better in correctly classifying HIV, TB, malnutrition, and anemia. However, e-IMCI did not perform better than paper-based IMCI in most outcomes of interest. Evaluators found that IMCI implementation was fragmented and inconsistent regardless of paper or electronic format.
e-IMCI and paper-IMCI performed similarly in:
Comprehensiveness of assessment of main symptoms (e.g., cough, diarrhea, fever). Neither e-IMCI or paper-IMCI were comprehensive.
Ensuring that children received all the medication they needed
Ensuring that children received the correct medications for specific conditions
e-IMCI performed better than paper-IMCI in:
Correctly classifying HIV, TB, malnutrition, and anemia
e-IMCI performed worse than paper-IMCI in:
Duration of consultation (e-IMCI took three minutes longer on average)
Correct classification of main symptoms
Overprescribing medications (i.e., children in the e-IMCI group were more likely to receive unnecessary medication)
It remains unclear whether the promise of better assessment and management of sick children could have been fulfilled if e-IMCI were implemented fully and regularly as intended. The evaluation revealed key reasons underlying poor e-IMCI uptake despite positive reception by clinic staff, including:
Technical challenges: Lack of computer skills among some staff presented persistent challenges despite extensive technical support. When IT problems arose, it was easier for clinic staff to default to paper-IMCI. E-IMCI was loaded onto one clinic computer in a specific room, which was sometimes unavailable. Clinic staff felt that using a desktop computer for assessments interrupted the flow of consultation.
Shortcomings in design of e-IMCI software program and printouts
Poor integration with other concurrent programs that had different patient record requirements
Health system challenges: With high workloads, staff shortages, and limited staff trained in e-IMCI, it was not used when the clinic was busy, preventing clinic staff from gaining fluency and confidence in the process
Conclusion
The evaluation does not support scaling of e-IMCI as currently implemented. However, the results point to useful adaptations that can be considered for both IMCI in general as well as e-IMCI, such as:
Re-allocating components of IMCI assessment to other health cadres so that one health worker is not tasked with the entire long IMCI assessment, or adopting a more differentiated model of care that distinguishes between lower- and higher-risk children, the latter of whom should be assessed more comprehensively
Focusing on more complex elements of IMCI that benefit from automation or digitalization (such as HIV and malnutrition screening)
Potentially informing the design of digital community health worker health assessment tools by avoiding the design pitfalls that were revealed and documented through the e-IMCI implementation
The evaluation underscores the usefulness of understanding implementation in the real-world context, and that positive reception is necessary but not sufficient for a successful intervention. Understanding implementation allows us to better unpack whether the absence of impact is due to flaws in the theorized pathways to impact (i.e., a program was carried out as intended but did not have the desired effect) or flaws in implementation, thus better informing ELMA’s investments.
