MERL Tech DC 2019

Using technology in remote, rural areas is advancing global health, but still comes with challenges. During this session Medic Mobile is teaming up with Vitamin Angels to discuss examples of community level mobile data collection tools and how to make data relevant, timely and useful for community health workers and rural organizations.

First, Medic Mobile will discuss how the use of a customized app by Community Health workers (CHWs) has played a significant role on how they deliver services in the community. Technology, especially mobile technology has been seen as a key towards real time data gathering to support timely decision making. The Medic Mobile (MM) App supports CHWs daily decision making through tasks, reminders, and the ability to access information on the go, regardless of connectivity. MM believes data can improve quality of service for CHWs. However, currently, some of this data is known to be unusable due to various issues. On-the-ground limitations impact how often CHW’s data is uploaded, with up to 20% of CHWs not submitting their data on a monthly basis. Although a CHW’s individual data may be current, supervisors and programme managers can’t access that information until synchronized. The result is that they may make programmatic decisions based on incomplete or outdated information, hampering the ability to respond to challenges. Thus, data timeliness presents a unique challenge to others who may wish to do work in similar contexts.

Then, Vitamin Angels (VA) will share how the use of technology has allowed the expansion of their monitoring system in 20 countries, with 30 monitors routinely sending data on partner organizations serving hard-to-reach beneficiaries. VA utilizes ONA plus ODK Collect tools to collect and store data on mobile devices; and R to manage and analyze data. Although these technologies have allowed VA to greatly expand its own use of the data, making results accessible to its partners in a timely and meaningful way has proved difficult. VA will discuss different mechanisms that have been piloted for sharing real-time data and tools for performance improvement, including the use of ONA to create basic result summaries and a job aid.

Technology abounds for conducting MERL with developing communities and markets using mobile channels (SMS, IVR, USSD, WhatsApp etc.). As one provider, Echo Mobile is often asked, “what sort of response rate can I expect for my SMS survey?” or, “what percentage of my audience will engage my IVR initiative?” The unsatisfying answer is always the same: it depends. And financial incentives don't always make a difference.

Whatever your mobile engagement technology, achieving MERL goals often rests on the design and psychology behind your mobile engagement strategy—the content, tone, language, and timing of your communications and the underlying incentives of the audience. Technology is easy, but people are hard, and these are psychological and strategic considerations more than technological ones.

We’ll raise them for deeper discussion. Instead of “it depends”, we’ll dig into what engagement depends _on_. We’ll present two cases (5-10mins each) of organizations that use the Echo platform, and which have increased mobile engagement by experimenting and tweaking their approach rather than having us tweak the technology:

1. A UN agency that nearly doubled SMS engagement rates from 40,000 Kenyan teachers by dropping financial incentives and tweaking the structure, tone and content of their messaging.

2. A BBOXX team in Rwanda that sought to understand the welfare impact of solar consumption amongst their customers via SMS surveys. They first ran a set of small experiments, modifying survey financial incentives, timing, length, and language to see which moved the needle on response rates and compare the results to what customers told them in focus groups. Only one of these tweaks made a difference, and despite what customers had said in the focus groups, it wasn’t the financial incentives! But it nearly doubled response rates.

These two examples will not be the focus of the discussion, but will prompt an interactive discussion to draw out the experience of other participant that reinforce or challenge our takeaways about how to grow and sustain mobile engagement to achieve MERL goals.

Join us for the First MERL Tech Dashboard Contest! Leading MERL Tech experts will showcase their dashboards in fast, fun five-minute lightning talks that will tell us:

1. the goal of their dashboard 
2. how they organized the underlying data 
3. why they choose that design 
4. the impact it had in decision-making.

Three distinguished judges (Amanda Makulec, Charles Guedenet, and Christopher Robert) will rate the dashboards and we’ll declare three winners based on their votes. A fourth winner will be declared via a special People’s Choice award chosen by conference participants. Each contest participant will receive a prize and the top four dashboards will have a special award signifying their excellence.

Creating useful software for development and humanitarian professionals is an imperfect science, and we don’t always get it right. In this session, panelists with very different approaches to providing MERL software will talk about how the sausage gets made. For the first half, we’ll share why we often end up disappointing clients or telling users ‘no’ from our experiences at companies past and present. We’ll also provide insight on different business models and delivery mechanisms (open/closed source, products/services, SaaS/custom, etc.) to equip the buyers/users of software to make the very best decisions and establish fruitful partnerships with their vendors. During the second half, our moderator will field questions from the audience and facilitate an open dialogue about frustrations and tough decisions from both sides. There will be no presentations, no pitching, no self-promotion. Any anecdotes or success stories will be leavened with failures, lost clients, self-deprecation, etc. Panelists will try to humbly and truthfully describe where their business models and the needs of prospective users may conflict, and how to manage that tension. Participants in this session will leave with a fuller understanding of how software companies operate, how various business models work in practice, and why finding the 'perfect' MERL tech solution is sometimes so dang frustrating.

Participants will understand how a simple data collection tool can be re-purposed for community/citizen led monitoring efforts. We will walk through examples of farmers using mobile phones to monitor agricultural policies in Rwanda, similarly farmers in Zimbabwe monitoring local weather patterns and also Trócaire's experience exploring tech solution for citizen led monitoring of human rights issues. Through these examples, we will highlight lessons learned and discuss security and privacy considerations as well. We not only wish to share our experience but learn from the expertise in the room!

Traditional tools to monitor programmatic activities and manage the resulting data can be expensive, complex and require extensive IT support. Pact Colombia and Seed Global Health have harnessed low-cost, user-friendly Google tools for collecting, collating, displaying, and managing data. This session will demonstrate how each organization is using Google tools for MERL. Presenters will display key features, share implementation, ethics and data privacy considerations, and facilitate a discussion on good practices and user-friendly tech solutions for MERL. Attendees will leave this session with a better understanding of how Google Sites, Forms, Sheets and Data Studio can be used to display, capture and showcase a variety of information, and how it can be integrated with their work.

Session format:
First part (20 minutes): Through the Pilares project (funded by USDOL), Pact has created a customized user friendly monitoring system using Google Sites that provides access to project information, key documents and forms for data collection. Project partners submit data through Google Forms and Google Sheets and use Google Drive to store updated files, reports, images and other information, thus capturing disaggregated data on project activities and their outputs and outcomes in real time. The system was so well received by project partners that it has been re-created for partner CSOs to use as well. This part of the session includes: 1) Quick overview of Pilares project; 2) Challenges and needs for data collection in the Project; 3) Short demonstration of the google site; 4) Best practices and lessons learned from implementing

Second Part (20 minutes): Seed Global Health harnessed the power of Google’s Data Studio, a free dashboarding and data visualization tool mostly used in the marketing and advertising fields, to create interactive and customizable data reports, showcase information, and form a repository of knowledge within the organization. Google Data Studio allows Seed to make programmatic data and information easily accessible to its stakeholders in a user-friendly way, which allows them to continuously interact with the data and use it to inform day-to-day program and organizational operations. This part of the session includes: 1) Quick overview of Seed Global Health; 2) Background information on challenges and needs for management of Monitoring, Evaluation and Learning data and information in the organization; 3) Short demonstration of the Google Data Studio dashboard and its capabilities; 4) Best practices and lessons learned from creating and implementing this tool

Third Part (20 minutes) General considerations and Q&A - May include: Quick Presentation of child labor monitoring by civil society organizations that includes all the Google tools mentioned before; Possibly short video of staff/CSOs discussing their experience with the Google tools; Ethical and data considerations in both organizations; Security concerns; Additional open questions

This session will deliver a panel discussion of USAID initiatives to advance management of development data across a lifecycle, in partnership with the broader development data community. The discussion will highlight the importance of planning for and managing different types of USAID-funded development data (e.g., monitoring, evaluation, geospatial, etc.) across a lifecycle to enhance both data usability and responsible sharing and publication. It will focus on data management practices reflected in policy updates and revisions (e.g, ADS 579, Activity Location Data Mandatory Reference), partner perspectives, current governance and legal efforts and risk management and responsible data frameworks that respond to new Federal data mandates.The panel will emphasize how USAID and the development data community can work together in ways that advance a collective goal of using data to improve development outcomes.

Technology, it is assumed, enables practitioners to conduct traditional kinds of monitoring, evaluation, research, and learning (MERL) in better, faster, more efficient and higher quality ways at greater scale than pre-technology days. Yet to date there has been no attempt to synthesize and distill the emerging evidence base for this swirl of MERL activity. Over the past year, we conducted a systematic evidence synthesis using bibliographic databases that sought to identify gaps in the knowledge base, clarify key concepts, report on the types of evidence that inform practice, and lay the foundation for synthesizing conclusions and lessons learned. At this session we will present back our initial findings and invite participants to review the findings and add their thoughts.

Nepal Health Sector Strategy (NHSS) 2015-2020 aspires to leverage mHealth and eHealth to improve health outcomes for Nepalese citizens. At present, there is a paucity in evidence on the mHealth and eHealth interventions that have been implemented in Nepal and their effectiveness. Our study mapped technology-enabled projects in Nepal using Arksey and O’Malley’s scoping design framework and assessed the projects using WHO Building Blocks of a Health System framework. In total, 20 technology-enabled health solutions implemented between 1993 to 2016 were identified. The momentum for technology-enabled projects in Nepal is sporadic but continuous and largely donor driven. Technology-enabled health solutions in Nepal are limited in scope, focus areas, target audiences, and sustainability potential. Prevalent target audience for identified projects were front-line workers in rural areas, women of reproductive age, and adolescents. Evaluation or sustainability were not within the scope of most projects. Technology-enabled solutions are frayed, issue and organization centric, and primarily driven by donor or non-governmental organizations. Engaging the private sector, especially telecommunications companies, is an underutilized strategy to address barriers related to evaluating, scaling and integrating effective solutions in the health system. Existing pioneers in this space (government, national and international NGOs) must engage in strategic planning on how to increase collaborative partnerships with the private sector or incentivize independent commercial health technology ventures. Our session will shed light on opportunities and challenges for national and international governing bodies and health systems to chart a course for technology-enabled health solutions to move beyond “pilotitis” in low-resource settings.

Every1Mobile’s U Afya programme is a digital community for young mothers and prospective mothers in low-income areas of Nairobi to learn, share and connect on a variety of topics related to health, nutrition, hygiene and motherhood.

In an effort to leverage the technology that our target audience possesses, Every1Mobile created the U Afya 10-Day Handwashing Challenge - a mobile habit-tracking solution optimised for the most basic web-enabled feature phone.

The technology leveraged self-reported actions, where users were required to perform a daily activity, then complete a single-question survey to log their challenge as completed. The challenges varied but were all rooted in the WHO “5 Moments” model.

The habit-tracker adopted the behaviour change approach employed by Unilever and Lifebouy in their “Way of Life” campaign, namely Awareness, Commitment, Reinforcement and Reward.

This lighting talk will share specific advice and guidance for audience members who have experienced tech clutter, untidy data, or too many integrations. Our M&E house is tidy because we have a few tools and we know how to use them well. Through trial and much error, we've learned that a lean tech approach to MERL is incredibly powerful, and can create the time and space we need for data use. The lean tech approach gives us room to innovate with new tech like drones, blockchain, machine learning. I'll share what we've learned about when to keep it simple and when to invest in something new.

In this talk, we will present Integrity's work to implement ‘Aqoonta’, a web-based platform for DFID’s Learning and Monitoring Program – Somalia (LAMPS, 2016-2020). The LAMPS program verifies DFID Somalia’s Governance, Health, and Economic Development portfolios to improve DFID’s understanding of how the activities it funds are performing, the challenges faced, and ways to identify, mitigate and learn from risks affecting programming. Integrity and partners developed Aqoonta (Somali for “knowledge”) – as a space where DFID and partners interact with concise, visual LAMPS reports, photographs, management dashboards and learning resources. Aqoonta is a vehicle through which DFID and its implementing partners can learn from and improve their delivery. This talk will outline lessons learned through all stages of Aqoonta’s life, challenges during the process. Participants will gain an understanding of how even in challenging contexts such as Somalia, digital solutions can support efficient processes in monitoring, evaluation, and learning provided they follow an agile methodology and are grounded in a strong understanding of users, context, and needs.

PREP WORK
If you are attending this session please prepare your computers before this session and ensure you have the following applications installed:

Tableau (free trial)
Power BI (free)
Google Account

Data
Click here to to download the data that will be used during this session.


MERL practitioners live in a world of an over-abundance of information and are expected to be knowledge brokers between data producers and data consumers. This two-hour hands-on workshop will give participants the ability to walk through the data lifecycle and build informative dashboards using readily-available tools that turn information into insights in order to drive data-driven decision-making.
In this hands-on workshop, facilitators will walk through each phase of the data lifecycle and demonstrate tools, techniques, best practices, and lessons learned working with NGOs, federal government clients, and local organizations.

By the end of the session, participants will be able to plan for data collection, collect data, analyze data, and distill and disseminate findings through an interactive or static dashboard. Participants will be able to describe and use readily-available tools including (but not limited to): Excel, Power BI, QGIS, Tableau, and GitHub.

Participants are expected to bring their own computers and follow along with the facilitators. Participants will also be asked to present their dashboard and describe their experience using the tools.