Developing Realist Economic Evaluation Methods (REEM) and Guidance to Evaluate the Effectiveness, Costs, and Benefits of Complex Interventions
This project is funded by the NIHR Health and Social Care Delivery Research programme (Award ID: NIHR135102). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
About REEM
The ‘big problems’ in health and social care require well-designed complex solutions and robust evaluation, which itself is often complex. Realist evaluations aim to take account of that complexity, offering an explanation of ‘what works, for who, in which circumstances and why?’ Concurrently, policymakers and research funders require economic appraisals to accompany evaluations of complex interventions to inform decisions in the context of resource scarcity.
Read our protocol for more details.
You can help us
If you’ve conducted a Realist Evaluation incorporating economic factors, or an Economic Evaluation that is more theory-driven than usual, we’d love to hear from you. Please get in touch by email or by using the contact form.
However, realist evaluations do not tend to explicitly capture the economic consequences of interventions and economic evaluation methods often ignore context. This research will develop realist economic evaluation methods (REEM) and guidance to better understand and evaluate the benefits and costs of complex interventions. It will bring together realist and economic evaluation to enable evaluators to establish what works, for who, in which circumstances, why, and with what related resource impacts and opportunity costs.
Draft REE guidance
The Realist Economic Evaluation (REE) guidance and glossary were developed as a product of Phase 1 of the REEM research, drawing on literature scopes completed and methodological development workshops. We made the decision as a project team to share this draft guidance, in the hope that Open Science will allow for the best use and development of the guidance over the remaining duration of the REEM Project (end date October 2025). The REE guidance will now be tested in three pilot evaluations, in Phase 2 of the REEM project.
The public and end-users can download both our draft REE guidance and glossary for free non-commercial use.
Help us develop this guidance
If your research team is interested in using this draft guidance and becoming affiliated with our project, please contact us. We will ask that you feedback on your experience using the guidance by completing a self-reflective pilot tool. Data from this tool will be used to explore real-world uses and to further refine the guidance.
Terms and conditions for use of both documents
- You grant permission to be followed up by Northumbria University to provide information to support evidencing of impact of the REEM research, its outputs and supporting materials including this guidance.
- You will not use the REEM research outputs and supporting materials including this guidance for profit-making purposes.
- You will refer to the REEM research outputs and supporting materials including this guidance by its title as stated on the Knowledge Bank platform when using and/or sharing.
- You will appropriately reference and credit Northumbria University and the creator(s) of the REEM research and supporting materials including this guidance when using and/or sharing.
News
6th June 2023 – Professor Nick Tilley discusses EMMIE: tenses and tensions in meeting evaluation needs
Our team
Sonia Dalkin
co-Principal investigator
Professor in Applied Health and Social Care Research at Northumbria University; member of Fuse (centre for translational research in public health); co-leads Healthy Ageing Research Programme.
Sonia has significant experience in realist approaches. In 2018 she co-authored Doing Realist Research, published by Sage and, in the same year, launched the Northern Realist Research Team Hub (NoRTH). NoRTH comprises an experienced team of researchers based at Northumbria University with established regional, national and international links, who have expertise in and are regularly using and developing realist approaches to research.
Angela Bate
co-principal investigator
Professor in Health Economics with an international reputation in the development and application of health economic methods in complex (public health) evaluations.
Angela’s PhD utilised quantitative and qualitative research to interrogate approaches to primary care commissioning. Angela is actively involved in regional, national and international research organisations, including the Complex Systems research group; the National Institute for Health Research School for Public Health Research; the Social Value International Academic Research Group; and an associate member of Fuse.
Andrew Fletcher
Research fellow
Andrew’s PhD was a realist evaluation of interventions that use participatory music activities to increase wellbeing for people with mental health issues. He worked on the interdisciplinary philosophy/social science project, ‘Knowledge for Use’ at Durham University, then as an ethnographer at King’s College London. Also working on an NIHR ARC-funded whole systems evaluation of knowledge equality in the housing sector.
Andrew is particularly interested in qualitative and participatory methods, and epistemic (in)justice in health systems. Associate member of Fuse; freelance copywriter; plays the bass guitar.
Meghan Kumar
co-investigator
Meghan is a health economist and health systems researcher focusing on implementation research in low-resource settings. Methodologically, she works on economic evaluation in complex systems, policy and political economy analysis, and participatory approaches to research and systems change. Her research direction involves incorporating participatory and economic evaluation methods to improve social inclusion toward multi-level accountability in global health.
Geoff Wong
co-investigator
Associate Professor of Primary Care at the Nuffield Department of Primary Care Health Sciences, University of Oxford, UK.
Experienced realist reviewer and evaluator; led the development of the RAMESES quality and reporting standards and training materials for realist reviews and realist evaluations (www.ramesesproject.org). Also a part time General Practitioner in the NHS.
Judy Wright
co-investigator
Information Specialist team leader at Leeds Institute of Health Sciences (LIHS); chartered librarian; supports the management of the Academic Unit of Health Economics (AUHE).
Portfolio includes: custom-made literature searching, reference management and search methodology advice for systematic reviews, realist reviews, evidence syntheses, research proposal scoping, economic evaluations and cost-effectiveness models.
Cam Donaldson
co-investigator
Yunus Chair and Distinguished Professor of Health Economics at Glasgow Caledonian University.
Cam previously held the Health Foundation Chair at Newcastle University, the Svare Chair at University of Calgary and a professorship at the University of Aberdeen. The basis for Cam’s interest in REEM stems from his previous research on developing methods for priority setting and economic evaluation in health and health care.
Gill Westhorp
co-investigator
Specialist in realist research and evaluation methodologies. Professorial Research Fellow at Charles Darwin University, Australia, leading the Realist Research Evaluation and Learning Initiative (RREALI).
Gill is co-author of the international publication standards and quality standards for realist evaluation and realist literature review. She has led, undertaken or advised over 70 research and evaluation projects in diverse sectors, and provided in excess of 70 training workshops in 14 countries. She develops new methods for realist research and evaluation, and publishes in realist methodologies.
Rachel Baker
co-investigator
Director, Yunus Centre for Social Business and Health; Professor of Health Economics. Focusses on the use of mixed methods to: i) elicit societal values with respect to resource allocation for health and ii) study the benefits arising from policies, programmes or interventions that impact on health and wellbeing.
Investigating how society values different uses of public resources and how community-based organisations generate impact for participants. Expertise in health economic approaches to valuation and preference elicitation, Q methodology, and Q-based survey methods.
Rob Anderson
co-investigator
Health services researcher, economic evaluator and implementation science scholar. Interested in the evaluation and synthesis of evidence about health policies and complex health interventions; systematic reviews and evidence synthesis for policymaking (especially theory-driven approaches); implementation Science and Knowledge Mobilisation/Translation.
Director, ESMI research group; Director of Research Impact for Exeter University Medical School’s Institute of Health Research; Co-leads the Exeter HS&DR Evidence Synthesis Centre.
Felicity Shenton
ppie co-lead
Public Involvement & Community Engagement Manager, NIHR Applied Research Collaboration NENC.
Qualified social worker; managed an alternative to custody project; freelance trainer/researcher/project development manager and policy adviser. Strategic Director of Investing in Children until 2017, then Programme Manager of the North East Social Work Alliance. PhD was a co-produced participatory research project with young people in secure accommodation.
Vivienne Hibberd
ppie co-lead
Vivienne has a background in Student Support and Well-being, and Widening Participation in higher education, as well as advocacy in mental health.
She has recent experience in PPIE in health and social care research projects, including study design, chairing meetings and project management.
How you can help us
If you’ve conducted a Realist Evaluation incorporating economic factors, or an Economic Evaluation that is more theory-driven than usual, we’d love to hear from you. Please get in touch by email or by using the contact form.
