There are two RAMESES projects, both funded by the United Kingdom's National Institute of Health Research's Health Services and Delivery Research (NIHR HS&DR) Programme. The projects' goals are to produce quality and publication standards and training materials for realist research approaches.
RAMESES I - (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) developed quality and publication standards and training materials for realist reviews and the related approach of meta-narrative reviews.
RAMESES II - developed quality and reporting standards and resources and training materials for realist evaluation.
There is growing interest in theory-driven, qualitative and mixed-method approaches to systematic review as an alternative to (or to extend and supplement) conventional Cochrane-style reviews. These approaches offer the potential to expand the knowledge base in policy-relevant areas - for example by explaining the success, failure or mixed fortunes of complex interventions. However, the quality of such reviews can be difficult to assess. The RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) project has produced:
- methodological guidance
- publication standards
- training resources
for those seeking to use the realist and/or meta-narrative approach to systematic review.
In this project we used a number of different approaches to develop our project outputs. We:
- collated and summarised existing literature on the principles of good practice in realist and meta-narrative systematic review;
- considered the extent to which these principles have been followed by published and in-progress reviews, thereby identifying how rigour may be lost and how existing methods could be improved;
- used an online Delphi method with an interdisciplinary panel of experts from academia and policy, produce a draft set of methodological steps and publication standards;
- produced training materials with learning outcomes linked to these steps;
- piloted these standards and training materials prospectively on real reviews-in-progress, capturing methodological and other challenges as they arise;
- synthesised expert input, evidence review and real-time problem analysis into more definitive guidance and standards.
More details on the RAMESES project's methods may be found at:
a) Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)
b) Development of methodological guidance, publication standards and training materials for realist and meta-narrative reviews: the RAMESES (Realist And Meta-narrative Evidence Syntheses - Evolving Standards) project
The realist evaluation approach is a relatively new approach to evaluation, especially in some fields (such as health services research). Its overall place in the toolkit of evaluation approaches is not yet fully established. As with all primary research approaches, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency of studies.
This project developed quality and reporting standards and resources and training materials for realist evaluation.
More details on the RAMESES II project's methods may be found at:
a) Protocol the RAMESES II study:developing guidance and reporting standards for realist evaluation