“C/Can’s Monitoring, Evaluation and Learning Framework anchors its commitment to an evidence-based approach in which decision-making and interventions are based on accurate and reliable evidence, and are always aimed at achieving results.” Jade Chakowa, Senior Manager, Monitoring, Evaluation and Learning, City Cancer Challenge
Monitoring, Evaluation and Learning (MEL) is a process of systematically assessing progress towards short- and long-term changes. In the City Cancer Challenge (C/Can) context, MEL is a powerful tool for change at both the city and global levels. It provides a framework to collect data and lessons learnt that provide valuable insights into what is working well and could be adapted and scaled to other cities, and what adjustments may be needed to drive improvements in access to quality cancer care in cities.
What does good look like?
The first step in defining how C/Can measures success has been to develop a Theory of Change, articulating what changes we aim to advance in the short, medium and long term, and how.
1.In the short-term, during C’Can’s initial 2-year commitment in a city we expect to see cities progressing through the entire City Engagement Process. Using a set of common indicators we track the achievement of key milestones and deliverables, such as the creation of a City Activity plan, as well as quality of engagement throughout the process. For example, we use indicators on the number of healthcare professionals, civil society organisations and patients participating in the City Needs Assessment as a measure of the level of multisectoral engagement.
2. In the medium-term, cities progress in developing localised solutions for cancer care identified through the C/Can city engagement process. These solutions, such as the creation of standardised guidelines, quality control programmes or training of healthcare professionals, are important contributions to strengthening the health system for cancer care. We work with a group of technical experts in each city to develop tailored metrics to measure development and implementation of these solutions.
3. In the long-term, our ambition is to measure the impact of city solutions on access to quality cancer care, and ultimately how this contributes to improving patient outcomes. We will use a combination of qualitative and quantitative techniques, including comparing hallmarks of quality cancer care such as standardisation of care, availability of key healthcare workers and effective patient navigation, before and after solutions are implemented. Consulting with C/Can city stakeholders, partners and experts in the fields of health information, impact evaluation and data science among others, we are defining an approach to determine the potential and actual impact on improving patient outcomes, using available data.
4. The City Multiplier Effect also presents a unique opportunity for C/Can to go beyond tracking its direct activities to look at how cities leverage C/Can’s catalytic support to generate value in strengthening the health system for cancer care. In Asuncion, Paraguay for example, C/Can’s ability to convene key decision-makers across disciplines and sectors, was critical in passing a national cancer law.
C/Can’s approach to MEL: agile, collaborative, and data-driven
C/Can’s monitoring, evaluation and learning framework (MELF) is regularly reviewed and updated to integrate key learnings from the first cities, and is closely aligned with C/Can’s guiding principles:
Solutions created by and for people affected by cancer
It is critical to ensure that patients remain at the heart of C/Can activities. This means measuring how patients perceive quality and access to cancer care services so that we can determine whether the changes observed in individual health institutions and at the health system levels are translating into improvements in the lived experience of patients. Through periodic patient surveys and by including local civil society organisations in the development and roll-out of data collection processes, C/Can’s MELF allows us to regularly assess the extent to which activities are supporting the ultimate goal of improved outcomes for patients.
Cities driving change
Cancer care gaps and priority solutions are different in every city.
“Whilst there are commonalities across cities, it is important to recognise that the short and long term needs we’re addressing are as diverse as the cities we work with, and constantly evolving. Our metrics for success must match that diversity and fluidity, while remaining usefully comparable year to year and organisation-wide.” Rebecca Morton Doherty, Head, Policy and Global Impact, City Cancer Challenge
Tailored metrics, however, cannot come at the expense of having cumbersome, unrealistic reporting requirements. For our MELF to be practical and sustainable, we work with technical experts in each city to identify the most critical data points needed to measure results. We focus on collecting these data-points through user-friendly data collection methods such as surveys, leveraging existing data sources to reduce duplication, and streamlining reports wherever possible.
The power is in the partnerships
We know that lasting change doesn’t happen overnight, so we need to be able to monitor and evaluate results beyond the initial 2 years of C/Can’s direct support to a city. That’s why a key component of the MELF is to work hand-in-hand with our sustainability partners in each city, such as ProPacifico, in Cali, Colombia to continue measuring progress and impact, not only against defined metrics, but also through gathering case studies and stories of significant change. We will be working with our partner, ProPacifico, to gather impact stories and to collaboratively refine metrics to measure medium- and long-term changes in Cali.
Evidence-based decision making
C/Can cities, stakeholders and partners need robust data that is timely and accurate enough to make decisions on how to best design and implement solutions to benefit people affected by cancer, maximising the impact of their efforts. As a data-driven organisation, C/Can recognises the importance of having data of the highest quality on which to base decisions. This means having processes, definitions and systems of validation in place to assure data quality is integrated into our everyday work. Later this year, C/Can will be rolling out a new “Responsible Data Framework” which includes policies, processes and guidance for C/Can staff and stakeholders around data governance, privacy and quality.
In parallel, and as C/Can scale to new cities and our interventions mature, we’re expanding and systematising our data collection capabilities. This includes the development of a custom data collection platform to facilitate baseline data collection from healthcare institutions, city health authorities and patients. The platform is helping to ensure the quality of the data collected through managing user access, identifying missing data, data validation and supporting privacy.
What’s next for C/Can’s monitoring, evaluation and learning framework?
We know that it’s not enough to simply deliver results; it’s critical that we show evidence of those results and communicate them to different stakeholders in a way that is easy to use. In 2020/21 we will be focusing efforts on further equipping our champions with the data and stories they need, from videos, blogs, dashboards and infographics to case studies.
A new iteration of C/Can’s MELF will also be rolled out in October 2020 and we look forward to sharing more on C/Can’s achievements and outcomes soon. In the meantime, you can learn more about our progress and achievements in 2019 in our Annual Report.
Stay tuned for the next Inside Story in which we explore C/Can’s approach to sustainability, and how we are working with local sustainability partners to ensure lasting improvements in access to quality cancer care in cities.
