Ideally, we would like to follow every behaviour change intervention with a robust, long-term evaluation. One that spans short, medium, and long-term outcomes, capturing shifts in mindsets, behaviours, and systems with the clarity and confidence of a well-rounded evaluation framework.

But oftentimes, we are in a situation where resources simply don’t allow for it. Many of our clients in the public and third sector face tighter budgets, limited timeframes, and pressure to demonstrate value quickly. In these contexts, a well-rounded evaluation that covers all necessary and pertinent aspects may not be feasible. Through multiple such cases, Magpie’s behavioural science and insight team has learnt to develop a clear rationale and method to navigate this tension pragmatically and strategically.

Ways of evaluating behaviour change strategies 

When we set out to measure latent, complex outcomes like shifting norms, reducing stigma, or building new habits, changes aren’t always directly observable in the short term. By convention, we often rely on one or more of the following evaluation methods:

1. Engagement and impression metrics
   These act as proxy indicators, revealing how many people saw the campaign, how long they watched a video, how many clicked through etcetera. While useful and relatively inexpensive, these metrics often tell us more about attention than impact. Suitable for awareness related campaigns. 
2. RCTs and pre-post testing of concepts
   These methods involve showing campaign materials to the audience and measuring shifts in attitudes or intentions. This can provide stronger causal inference than pure engagement data, but it still doesn’t guarantee actual behaviour change in the real world. And even this level of testing may not be achievable on a shoestring budget. Good for single-strategy interventions that aim for attitudinal change.
3. Three-phase evaluation: short, medium and long-term outcomes
   This is the gold standard. We break the behaviour change journey into stages, define operationalised indicators, and track them over time. But this approach requires significant investment – financially and in terms of time, staffing, and ongoing access to participants. For many public sector campaigns, this option becomes less feasible and sometimes impractical. Ideal for a multi-strategy intervention that aims for behaviour change.

In the world of behaviour change, evaluation generally falls into two categories:

• Summative Evaluation looks at final outcomes. It tells us if the intervention worked and if the behaviours changed over time. This type of evaluation is typically carried out after implementation, often requiring longitudinal data, larger budgets, and sustained involvement. It is essential for validating large-scale impact but can be out of reach for many public or third-sector projects.
  
• Formative Evaluation, on the other hand, focuses on developing and improving an intervention before it is fully rolled out. It includes relevance testing, concept testing, piloting, and gathering feedback from the target audience. It’s about ensuring that what we build is fit for purpose and context before investing heavily in delivery.
  

When resources are limited, putting time and budget into formative evaluation is often the smarter investment. It helps sharpen the strategy, deepen audience insight, and increase the chances of genuine impact, all without the cost and complexity of a full-scale summative evaluation.

Focusing on formative approaches gives us the confidence that we’re not just delivering something but delivering the right thing.

When budgets are tight, relevance testing is a smart investment

In these resource-constrained scenarios, we believe there’s a more meaningful way to use limited budgets: invest in relevance testing of the strategy before rollout.

Relevance testing involves taking early-stage concepts, messaging, or prototypes and testing them, mostly qualitatively and sometimes even quantitatively, with a subset of the target audience. The aim isn’t just to measure interest or preference. It’s to uncover how the target audience feels and thinks about the strategy, how well it resonates, how it fits into their lives, and whether the framing or approach needs to be adapted.

This isn’t just validation or pilot testing. It’s where breakthrough insights often happen. Time and again in our work, we’ve found that relevance testing surfaces something we hadn’t anticipated – an assumption we need to challenge, a nuance we’d missed, or an opportunity to frame things more meaningfully for a specific audience. And the impact of that learning dramatically increases the chances of success.

Crucially, this testing also ensures that we’re not just building strategies for people, but with them.

Why relevance testing works

Our work across complex and layered behaviour change projects in alcohol moderation, gambling harm reduction, creating an intuitive guide to healthcare pathways, and responsible waste management has repeatedly shown that the most effective strategies are those that are deeply rooted in the realities of the people we aim to support.

When we engage communities meaningfully through co-creation workshops and concept testing focus groups, we gain more than data points. We build trust, refine relevance, and generate strategies more likely to land, resonate, and bring about lasting change.

That’s why, when budgets are tight, we recommend clients prioritise strategy refinement and hyper-local adaptation through relevance testing over small-scale or superficial evaluations. In many cases, it offers a better return on investment and lays a stronger foundation for future evaluation for when more funding becomes available.

Evaluation should never be an afterthought

At Magpie, we build evaluation into our behaviour change frameworks from the start. Even when a full three-phase evaluation isn’t immediately viable, we create adaptable frameworks that can be scaled up when circumstances change. That means the work we do now still supports deeper measurement and insight later.

If you’re navigating challenges like tight budgets, complex behaviour change goals, and the need for meaningful evaluation, we’d love to talk. Whether you already have a behaviour change aim in mind or are just exploring your options, our team can help you build effective, evidence-based strategies and shape smart, pragmatic evaluation approaches around them.

Get in touch to see how we can work together to co-create impactful, relevant interventions that make the most of your resources without compromising on insight.

Behavioural science can be a powerful tool for charities to understand and positively influence the behaviours of teams, supporters, volunteers, beneficiaries, and other stakeholders. This in turn can improve wellbeing, communications, impact and sustainable working practices. In recent years, we’ve had the absolute pleasure of workshopping with Barnardos on a framework for internal communications, co-creating mental resilience training course with Mind, re-designing fundraising packs with British Heart Foundation, connecting the public with urban plant life to boost wellbeing with Kew and reducing the demand of extreme breeding with RSPCA. 

Reporting in 2022 and 2023, the Charities Aid Foundation (CAF) revealed more than 50% of the UK’s charities are digging into reserves, half fear they won’t survive the cost-of-living crisis, and only 49% think they have enough funds to meet current demand.

According to a Pro Bono Economics survey, energy costs is charities’ third biggest worry for charities after income with 55% digging into reserves to pay fuel bills.

Amidst the rising trend of online activity, Charities Aid Foundation reports that charities are currently in the process of assessing the appropriate equilibrium between digital engagement and face-to-face interactions. There remains a lack of comprehensive understanding regarding the relationship between in-person contact and its impact on brand value, alongside potential implications of heightened digital participation on brand perception.

Responding to rising demands, here are ten ways in which behavioural science can support charities:

1. Supporter Behaviour: Behavioural science can help charities understand the psychological factors that influence supporter decision-making. By applying principles from behavioural economics, charities can design fundraising campaigns that leverage behavioural insights to increase engagement rates, support and improve retention.

2. Nudging for Good: Charities can use nudges—subtle changes in the way choices are presented—to encourage desired behaviours among supporters and frontline teams. Nudges can also be used to promote volunteering, advocacy, and other forms of support.

3. Impact Communication: Behavioural science can inform how charities communicate their impact to donors and the public. By adopting more effective lived experience storytelling strategies, social proof, and concrete examples, charities can make their impact more tangible and compelling, leading to increased support and engagement.

4. Behavioural Insights for Beneficiaries: Charities can apply behavioural science principles to design programs and interventions that effectively support their beneficiaries. This might include interventions to encourage healthy behaviours, financial literacy, educational attainment, or other positive outcomes.

5. Volunteer Engagement: Understanding the motivations and barriers to volunteering can help charities recruit and retain volunteers more effectively. Behavioural science can inform strategies to enhance volunteer engagement, satisfaction, and commitment.

6. Decision-Making Behaviours: Charities often face complex decisions about resource allocation, program design, and strategy. Behavioural science can provide insights into decision-making biases and heuristics that may affect these processes, helping charities make more informed and effective choices.

7. Collaboration and Partnerships: Behavioural science can facilitate collaboration and partnerships between charities, government agencies, academic institutions, and other stakeholders. By sharing insights and best practices, organizations can amplify their impact and address complex social challenges more effectively.

8. Improving Employee Retention and Reducing Recruitment Costs: Behavioural science can support charities to create culture shift interventions and better align teams and performance with brand values and culture. By identifying and addressing underlying factors, creative behaviour change can influence employee satisfaction, engagement, and commitment within an organisation.

9. Embracing New Technologies and Improving Tech Literacy: Some charities are experiencing inconsistencies and knowledge gaps when it comes to adopting new technologies, intranet systems, systems for reporting, training and team development. Behaviour change interventions can help leverage cognitive biases and motivational factors to enhance learning, adoption, and mastery of digital skills. 

10. Reducing Running Costs: Behavioural science and sustainability interventions can help charities reduce running costs by identifying opportunities to optimise resource usage, minimise waste, and promote eco-friendly practices within the organisation. By understanding the behavioural factors that influence energy consumption, resource utilisation, and operational efficiency, charities can implement targeted interventions such as energy-saving initiatives, waste reduction initiatives, and sustainable procurement practices. Additionally, behavioural science can inform strategies to encourage staff and volunteers to adopt cost-saving behaviours, such as turning off lights and equipment when not in use, using reusable materials, and choosing environmentally friendly alternatives. By integrating behavioural science principles into sustainability initiatives, charities can achieve significant cost savings while also advancing their environmental and social responsibility goals.

Overall, behavioural science offers charities a systematic and evidence-based approach to understanding human behaviour and designing interventions that promote positive social change and increase brand value. By leveraging insights from psychology, economics, and related fields, charities can enhance their effectiveness, efficiency, and impact in pursuing their social impact missions.

Magpie Meets is a unique exchange programme which exists to fulfil Magpie’s volunteering time and professional development needs through exchanges with a diverse range of change-making organisations.

Participants of the programme are invited to join Magpie to solve a behaviour change communications challenge using behavioural science, creativity and collaboration. The process involves an in-depth briefing, a cultural immersion activity and an exchange workshop which encourages Magpie to transfer knowledge, understand change-making leadership at multiple levels and bring creative and academic practices to problem solving.

Between January and February, Magpie worked with Women’s Health Matters, a Leeds based charity that exists to support women and girls to live safe and healthy lives. Since 1987, the organisation has provided holistic, trauma-informed services for women and girls who are socially, economically, or politically marginalised. This includes, but is not limited to, those affected by domestic abuse, those whose children live elsewhere, those accessing maternity services, women seeking asylum, women with disabilities, and women and their children experiencing trauma.

Last year, the charity worked with 2517 women & girls across 27 projects. Provided intensive support to 1214 women & girls, lighter touch support to 1303 women & girls, and supported 303 children. They delivered 869 group sessions and 1728 one-to-one sessions. 

The team are a positive force in Leeds and Yorkshire, they work tirelessly to increase confidence, wellbeing and provide transformational support. For a relatively small team, their impact is huge. Access to more funding and new partnerships are vital to their work so do get in touch if you can offer your support.

Magpie Meets brief

CEO, Rachel Kelly, set Magpie the following creative behaviour change challenge:

How, in a time-poor and resource-stretched organisation, can we grow our reputation?

Following this, a behaviour change exchange programme was designed to co-create an approach for cultural change with those responsible for delivery, resulting in a behaviour change communications blueprint and journey map to guide the organisation on this programme of work.

Meeting the team

‘Magpie Meets… Women’s Health Matters’ began with an initial immersion session to understand the challenge from different perspectives and to dig deeper. During this seesion, Magpie was able to gain a more rounded understanding of Women’s Health Matters vision for change and understand the practicalities of the challenge.

For every Magpie Meets programme, a hand-picked project team is selected with relevant skills matched to the challenge set by the beneficiary. The team for this exchange included Co-founder Ged Savva, Campaign Consultant – Louise Hallworth and Researcher – Mierla Neto.

Exchange day

In January, the Magpie Meets project team arrived at Women’s Health Matters HQ in Leeds for a visionary day of co-creation.

In attendance were five representatives from across the charity, each representing different areas of the organisation. The passion and belief alignment of the team at Women’s Health Matters was truly awe-inspiring, it was clear the vision and mission of the organisation was being lived and realised by all. 

It was delightful to connect with Women’s Health Matters at Magpie Meets, delving into behavioural change techniques and communication strategies for purposeful actions and fostering a reputation reflecting meaningful change. Huge thank you to the inspiring women at Women’s Health Matters for sharing their passion and expertise with us.

Mirela Neto, Magpie Researcher

The primary resource was the ‘co-creation canvas’ which would act as a one-page strategy for taking the desired change forward beyond the session, enabling the team to focus the outcomes of the sessions as well as actionable phases of work.

Outcome

Together, Magpie and Women’s Health Matters defined the behavioural outcomes the charity aims to achieve through their project. 

12 possible outcomes were explored and whittled down to one clear behaviour and culture shift change goal. This is:

Making space for proactive thinking to help grow reputation and increase funds.

Using a COM-B analysis for the challenge, we uncovered the capabilities, opportunities and motivations for change. The most prominent barriers for this culture shift included:

• Lack of awareness around the possibility of proactive thinking time
• Finding time in a busy schedule 
• Having inspiration to make thinking time possible and effective

The most prominent facilitators for this culture shift included:

• The importance of framing – how it is positioned and communicated
• Making the solution flexible to fit around individual roles
• Creating a framework to guide and inspire a culture shift

The sessions resulted in a blueprint strategic approach, giving the organisation focus and an outline plan for taking change forward. This included three potential frameworks for change including an open framework, a cumulative change framework and a more creative framework to align with the creative and tenacious culture at Women’s Health Matters.

I will take away how on the same page we are. That is a rewarding outcome from this session.

Hannah

Understanding that steppingstones can work. It doesn’t need to be everything right away. Small changes can take us to a better place.

Eleanor

It’s rewarding to have space and time for ideas.

Lydia

I’m really excited and inspired to action these ideas.

Molly

The time to do this together as a team is my takeaway.

Rachel

Next steps

A creative behaviour change report has been provided to Women’s Health Matters and a follow-up consultation is taking place in March to further the organisation with advice around the practicalities of adopting the recommendations. 

Interested?

Magpie is currently planning the calendar of Magpie Meets activity. If you would like to be considered as a beneficiary for our next Magpie Meets programme, please get in touch and a member of the Magpie team will be in contact.

Our Director of Behavioural Insights, Dr Grainne Dickerson, discusses the rise of Behavioural Science and how Magpie uses it to create happier and healthier communities.

What is behavioural science?

Behavioural science refers to the disciplines of psychology, behavioural economics, sociology and anthropology to understand, explain and predict behaviour. These fields of science are driven by academia and the rigorous research produced by behavioural scientists who are trained to the highest standards in research methods in their fields. Whilst the research conducted has potential to improve health outcomes for the population, there have been many missed opportunities to apply the findings of robust research in real world settings, however, this is changing.

What is the history of behavioural science?

Behavioural science started gaining popularity when the Cameron-Clegg coalition government in 2010 set up a nudge unit, inspired by the book Nudge by Thaler and Sustein (2009).

It was in the same year that I achieved my dream of becoming a behavioural scientist when I qualified as a Health Psychologist. Reading the book, Nudge and hearing about the nudge unit I realised I had been trained in all the same stuff! I then found myself determined to utilise my learning and skills within Public Health where I had been working.

“My mantra at the time was … if they can have a nudge unit in the government, they could and should have something similar in the NHS and local authorities, for they are also in the business of changing public behaviour.”

For several years after that I was actively championing behavioural science within Public Health in order to meld together the science and art of protecting the health of the public via Public Health work with the psychological processes in health, illness and healthcare through Health Psychology work. I had fantastic support at times in the various behavioural science and public health roles I worked in, and at other times I faced challenges, and this was because behavioural science wasn’t as widely valued back then. Fast forward to now, 12 years later in a world blighted by COVID-19, there’s been a proliferation in behavioural science…

Where is behavioural science now?

There has been a steady increase in the number of behavioural science teams and roles within the public sector who can apply rigorous scientific approaches to real world settings. This development reflects the recognition that behavioural sciences provide the opportunity to improve outcomes in the many areas of the public sector that are seeking to influence and change behaviour. Organisations with behavioural science teams include the Office for Health Improvement and Disparities, the UK Health Security Agency, the NHS, the Department for Transport and some local authorities such as Hertfordshire County Council.

There is a collective moving away from using ‘good ideas’ and a reliance on assumptions to utilising science based approaches to understanding target audiences, target behaviours and barriers and enablers that interventions need to address in order to enhance the health and wellbeing of population groups.

Here is a selection of highlights that have contributed to the growth of behavioural science during the last 12 years:

• The British Psychological Society’s Division of Health Psychology has worked behind the scenes to try and join up Health Psychology with policy and practice, enabling research to be translated into practice, alongside other social sciences making similar moves.
• The publication of the Behaviour Change Wheel in 2011 has made previously inaccessible theories more accessible, followed by the publication of the taxonomy of behaviour change techniques in 2013.
• More recently in 2019 an updated and practical version of the behaviour change wheel and the taxonomy of behaviour change techniques was published for local authorities in Achieving Behaviour Change: A guide for local government.
• The Behavioural Science and Public Health Network have been bringing together behavioural scientists with the Public Health workforce to develop capacity and capability for behavioural science and developing regional behavioural science hubs.
• An important strategy was published in 2018: Improving people’s health: Applying behavioural and social sciences to improve population health and wellbeing in England which marked a coming together of key national organisations to articulate what the behavioural and social sciences can offer Public Health and the theories and frameworks that can be described under the banner of behavioural science. A key message was the importance of transdisciplinary approaches, and they visualised this with a  re-invigorated version of the Dalgren and Whitehead Model (1991) of the social determinants of health.
• The biggest factor has been the pandemic which has further amplified and highlighted the importance of behavioural sciences for changing public behaviour and essentially reducing health inequalities.

Behavioural science at Magpie:

At Magpie we also adopt a transdisciplinary approach to bring our behaviour change campaigns and interventions to fruition whilst championing unheard voices.  We bring together a combination of different skills and perspectives that combine to add the value that no single approach would achieve on its own. Working in this way creates more impact, but is also more rewarding and fun!  We combine subject matter expertise from our clients and our team with behavioural sciences, community engagement and creativity.

What next for behavioural science?

You could say there is a democratisation of behavioural science; moving away from the mysterious  ‘nudge unit’ in central government and the purely academic field to a more accessible and transparent approach.

Behavioural science is now a respected part of many teams, however there is more that could be done to further enhance the contribution of behavioural science across the country, including:

• Sharing learning: Behavioural scientists working in applied settings share their learnings, and specifically their scientific approaches and impacts made. This will enable the continued learning about what does and doesn’t work for different issues within different populations and communities. Conferences like the Behavioural Science and Public Health Network annual conference and the regional behavioural science hub events are great avenues for this.
• Developing behavioural science skills: People working in practice can develop the understanding and skills they need to apply a behavioural science lens to thinking about problems and how to solve them.  There are now podcasts and several books to dive into that make the learning more accessible to people in applied settings.  For anyone wanting to make a start with a behavioural science podcast, I can recommend the real world behavioural science podcast for the perspectives of real experts in the field and the Behaviour Change Marketing Bootcamp podcast aimed at marketing and communications professionals. Happy listening! (Watch this space for learning opportunities from Magpie later this year…)
• Developing capacity for behavioural science work: For organisations who would like to recruit behavioural scientists to do in depth behavioural science work, my top tip is to use these recently published good practice guidelines for employers who wish to employ appropriately qualified and experienced people.

Written by Dr Grainne Dickerson

(Director of Behavioural Insights and Chair Elect Behavioural Science and Public Health Network)