Reflections from our Monitoring & Evaluation Workshop

We recently teamed up with Code for Canada to host a workshop about monitoring and evaluating civic tech projects. Here’s what we learned!

So, what is M&E and why is it important? 

When we design a product, we make a lot of assumptions about what we need to do to reach our goal – Monitoring & Evaluation (M&E) is a tool for testing these assumptions. 

It’s all about collecting and analyzing information to learn from experiences, improve practices, and make more informed decisions in the future. M&E also enables us to better understand and communicate the impact we’re making. 

Imagine your goal is to grow a healthy houseplant. Let’s say you’ve started off with a seed, watched it grow into a bud, but soon find its leaves turn brown, and sadly, it dies. Now, let’s say you’re determined and you start again with a seed. If you track your actions (e.g. type of soil used, placement in the house, how often you water it), and make changes based on what you observe, you’ll have a much better chance at eventually getting it right. Whereas, if you just try to wing it each time, without being aware of what worked and what didn’t work the last time, you may never reach your goal. And if you do, it’d probably be pretty difficult to do it again with the same success. 

Just like with the plant scenario, it’s important that we are able to effectively monitor and evaluate our approaches and practices to civic tech work – that way we can make improvements as we go and have the documentation for easy replication in the future. 

M&E is something that a lot of civic tech organizations struggle with, and it’s something we’re trying to get better at on the Code for All team as well. For this reason, we teamed up with Code for Canada to host a workshop about monitoring and evaluating civic tech projects.

_{Attendees at our first session of the workshop in June 2022.}

About the workshop

Icebreaker: lime or lemon? Or, in our case, the green one or the yellow one?

This is how John Griffin kicked off the first session of our M&E workshop, with a light icebreaker that was easy to engage with. 

(Note: we realized soon after starting the icebreaker that different countries use different names for each fruit, so we switched to calling them “the green one” or “the yellow one”) 

John Griffin is a Project Delivery Lead at Code for Canada. He has experience implementing M&E practices at the project and organizational level in a variety of contexts from Ghana to Canada.

For this workshop, we were lucky to have John share his experiences and teach us about M&E. 

Over the course of the three-session workshop, we were joined by civic technologists from Mexico, Canada, the Czech Republic, Nigeria, Germany, Hungary, the US and the UK. 

John covered topics like:

• The process for M&E
• What we are measuring for with M&E
• Data collection methods 
• Building an M&E plan
• What to do once we have the data
• Establishing an M&E culture 

He shared real-life examples from his work at Code for Canada, and as attendees we all joined in on brainstorming how we can use M&E in our own work using a collaborative Miro board. 

_{Brainstorming as a group using Miro board during the M&E workshop.}

Key takeaways

Below are some tips that stood out over the course of the workshop:

1. M&E isn’t a one time thing, it is an active process that you do throughout the lifecycle of a project (it’s not something you just do at the end).
   
   
2. M&E is part of everyone’s job. Each person at an organization should be doing some form of M&E. Sometimes people might see it as more work – if it feels like too much and they decide not to do it, this is how you might fail. It’s important to make it part of everyone’s job without making it feel like an extra thing being added to someone’s plate.
   
   
3. Just knowing the data points isn’t enough (e.g. x increased to y in 10 months), you need to know why the change happened.
   
   
4. The success of gathering accurate and enough information sometimes depends on how comfortable participants feel – they need to feel comfortable opening up and being honest. 
   
   
5. If you want to see the change of behavior in a stakeholder group, you need to break the process down into steps that you can measure – this way, you’ll be able to measure over time.
   
   
6. A lot of opportunities to collect data come from opening up opportunities to listen (e.g. you can collect data during, not only after an event. Look at: workshop participation, technical challenges you have, observations on how people interact and how people are engaging with what’s provided).  
   
   
7. A lot of events are one-time-things, but we need to think about how we can be constantly looking at what’s happening and gathering data – any interactions with users or stakeholders is an opportunity to collect data.  
   
   
8. It’s important to be clear with staff about why we need to collect the data, that way they’ll see the value in it and will have a natural motivation to collect and review it.
   
   
9. Staff members need to understand what specific data is important – if they don’t, then you can end up with a lot of useless data. 
   
   
10. When we look for answers to our questions in the data, that’s how we figure out which questions we want to ask and what information we want to collect in the future.
    

What’s next?

We hope to host more workshops and provide resources to the global civic tech community on this subject in the future. Starting this year, we will be dedicating more time and energy to M&E, and will share back what we learn with the community. 

If you want to stay up to date with Code for All events, or join in on civic tech discussion online, you can join our Slack space or subscribe to our mailing list.