Realist evaluation offers a valuable methodology for researchers investigating interventions utilising simulation-based education. The findings provide an insight into participants’ views on alcohol, ABI, and the barriers and enablers to change. 486 Qualitative Research 20(4) Keywords realist evaluation, realist interview, theoretical awareness, retroduction, generative mechanisms Introduction Realist evaluation is a theory-driven approach to evaluation (Westhorp, 2014; Wong et al., 2016) drawn from the seminal work of Pawson and Tilley (1997). Evaluators, researchers, and policy makers can have greater confidence in the outcome assessments which are made when a high level of implementation fidelity is achieved. A realist evaluation of whole-scale transformation in London, Greenhalgh T, Wong G, Westhorp G, Pawson R, Protocol - realist and meta-narrative evidence synthesis: evolving standards (RAMESES), Why do interventions work in some places and not others: a breastfeeding support group trial, Jackson L, Langille L, Lyons R, Hughes J, Martin D, Winstanley V, Does moving from a high-poverty to lower-poverty neighborhood improve mental health? Identifies some unique strategies and agrees with the hypotheses. Realism can be divided into two groups: direct and critical. District nurses have key skills in providing physical care and in coordinating the work of others, but struggle more with psychological aspects of care. Studies highlight the value district nurses place on palliative care provision, the importance of developing a relationship with patients, and the emotional difficulties of providing such care. Although 86% of all intentional rounding interactions were observed to be documented, fidelity to the original intervention [i.e. We report on a realist case study of a well performing hospital in Ghana and show how such a realist evaluation design can help to overcome the limited external validity of a traditional case study. Realist synthesis is an increasingly popular approach to the review and synthesis of evidence, which focuses on understanding the mechanisms by which an intervention works (or not). Background An adaptation of Pawson's realist synthesis method was used, whereby a provisional explanatory model and associated list of propositions are developed from an initial review of literature. Results Part of the problem is one of complexity. Learners were more likely to accept a course if it offered a perceived advantage over available non-Internet alternatives, was easy to use technically, and compatible with their values and norms. Realist review - a qualitative systematic review method whose goal is to identify and explain the interaction between context, mechanism and outcome. The paper discusses operational issues and lessons learnt related to studying complex community interventions, cross-disciplinarily, interfaces between practice and research and research capacity strengthening; and suggests areas for future research. Six CMOC can explain these results: four having a negative influence and two having a positive influence. Forming a community of practice could provide staff with the opportunity to share and develop their expertise in falls prevention and innovate change. Methods Best Practices: Sacred Heart Hospital, Pensacola, Florida. Past positive experiences of collaboration, a positive attitude towards OH, and the development of leadership and trust among stakeholders are present in configurations that influence positively the OH level of schoolchildren. The final programme theory suggests HCPs implement BSC when they value it as part of their role in contributing to improved patient outcomes; this is due to personal and professional influences, such as knowledge or experience. However, the context exerts a strong influence on whether BSC is implemented, or not. Introduction Objective Findings will be submitted to academic journals and presented at conferences. In the second phase, case study sites representing the different service types will collect patient data and recruit carers, service managers and commissioners to gather quantitative and qualitative data about service provision and outcomes. There is also poor indexing in databases and diversity of qualitative research [2] with inconsistent terminology [1,49,50,62] affecting the quality of the evidence. Objectives This study uses data from 15 descriptive case studies to explore high school students’ perceptions of scholarships and the forces that contribute to these perceptions. It is suggested that the insights and messages from this study inform a national conversation about whether or not intentional rounding is the optimum intervention to support the delivery of fundamental nursing care to patients, or if the time is right to shape alternative solutions. Qualitative data include semistructured interviews with primary healthcare staff and community representatives involved with Getting it Right. Realism is an approach to humanity which recognizes the real existence of social objects as well as physical objects. Future work There were a number of methodological challenges and a need to limit the depth of 'realistic unravelling'. Conclusions Increasing the available information about the contribution these services make may result in services being legitimised, understood, and resourced within mental health systems thus making the services sustainable. What aspects of intentional rounding work in hospital wards, for whom and in what circumstances? ESRC Research The paper concludes that the Pawson and Tilley methodology shows great promise for future evaluations. This paper provides an overview of critical realism, a modern philosophical perspective that seeks contextualised causal understandings of social phenomena. The aim of this study was to facilitate the implementation and operation of a falls prevention Community of Practice in a residential aged-care organization and evaluate its effect on falls outcomes. Realist synthesis is an approach to reviewing research evidence on complex interventions which seeks to explore these assumptions. Background Key contested areas included: whether to measure patient and public involvement impact; seeking public contributors to maintain a balance between being research-aware and an outsider standpoint seen as ‘authentically’ lay; scaling-up patient and public involvement embedded within a research infrastructure rather than risk token presence and whether patient and public involvement can have a place within basic science. Methods Lists the evaluation criteria, cites companies with best prractices and draws common themes. A Methods Lab Publication, ODI. 'Interactivity' led to effective learning only if learners were able to enter into a dialogue - with a tutor, fellow students or virtual tutorials - and gain formative feedback. This work was produced by Harris et al. The paper discusses four of the 13 mechanisms identified in the synthesis: collaboration and coordination; pooling of resources; individual learning; and role blurring. Context: Large-scale, whole-systems interventions in health care require imaginative approaches to evaluation that go beyond assessing progress against predefined goals and milestones. Misapplications of NRs. 25, 16 April 2018 | Evaluation, Vol. This website is intended for healthcare professionals, International Journal of Nursing Studies, Vol. 27, No. Dissemination will be facilitated by project coapplicants with established connections to national policy-making forums, in addition to publications, conference presentations and reports targeted to service providers and commissioners. It is based on the epistemological foundations of critical realism.Based on specific theories, realist evaluation provides an alternative lens to empiricist evaluation techniques for the study and understanding of programmes and policies. The intervention was successful in stimulating productive joint working, through case discussions, but often failed to ensure a review of progress. These findings are a salient reminder that what might be expected by healthcare professionals to be important influences on experience may not be perceived to be so by patients and carers. 23, No. A more detailed but accessible introduction to Realist Evaluation which highlights distinctive features of the approach, key concepts and when and where the approach is most appropriate. September, 2014. training in all aspects of M&E, from core skills development . 29, No. Unfavorable physical, social and political environment, negative past health experiences, feelings of disrespect, demotivation, mistrust and insufficient leadership are elements of negative configurations. evaluation approaches may not be feasible, with the broader aim of identifying lessons with wider application potential. but instead acknowledges that complex social interventions only ever work for certain people in particular circumstances. It is based on the epistemological foundations of critical realism.Based on specific theories, realist evaluation provides an alternative lens to empiricist evaluation techniques for the study and understanding of programmes and policies. This article joins the imbroglio by examining the logic of the two main strategies of systematic review: `meta-analysis' and `narrative review'. In some case studies, lack of coherence in defining PPI persisted, with evidence of a dual role of PPI representative/study participant. Assessing the quality of evidence in evidence-based policy: why, how and when? The third section reviews concretely the realist debate over statistics and some realist contributions to qualitative methodology, qualitative The realist approach to evaluation research: an introduction By Gillian Hewitt, Sarah Sims, Ruth Harris and University of London Kingston University and St. George's Topics: alliedhealth, health, nursing These were identified based on the scientific literature published between 1997 and 2017 addressing intersectoral SHPI. The research quality of the primary studies is checked out and only those deemed to be of high standard may enter the analysis, the remainder being discarded. It is a generic approach that can be applied to many fields of research, including health and social care. Enter your email address below and we will send you your username, If the address matches an existing account you will receive an email with instructions to retrieve your username. This paper introduces the realist approach and explains why it is particularly suited to education research. These results suggest that the program provides an effective model for building the capacity of kindergarten staff to support children with, or at risk of, developmental difficulties. Together, these can facilitate accountability at the individual, community and system levels. Methods and analysis: This is a mixed methods study in three phases informed by realist evaluation methodology. Using such an approach, elements of context denote pre-existing elements of the environment in which interventions or programmes are implemented, ... reasoning and reactions to interventions or programmes, depending on context and resources available (Lacouture et al., 2015). The matching Municipality of Odsherred serves as a control site based on its similarity to Bornholm regarding several socio-demographic and health indicators. Our use of logic models to identify theories of change could present a relatively static position for what is a dynamic programme of change. The third stage consisted of case studies tracked over 18 months through interviews and document analysis. In line with realist synthesis methodology, each study’s ‘fitness for purpose’ was assessed by considering its relevance and rigour. This review provides the background for Part II of the article (to be published in the next issue, Evaluation 8[3]), which considers the merits of a new model for evidence-based policy, namely `realist synthesis'. The realist approach to evaluation research: an introduction. Despite much research, course developers remain unsure about what (if anything) to offer online and how. The review will be conducted in four steps over 18 months to (1) construct a theoretical framework, that is, the review’s programme theories; (2) retrieve, review and synthesise evidence relating to interventions designed to develop the support workforce guided by the programme theories; (3) ‘test out’ our synthesis findings and refine the programme theories, establish their practical relevance/potential for implementation and (4) formulate recommendations about improvements to current workforce development interventions to contribute to the improvement of care standards in older people's health and social care services, potentially transferrable to other services. Methods and analysis Yet there is a great deal to be learned about the efficacy of mentoring if one digs into the specifics of each study. 78, No. There were four super-ordinate themes providing details on a range of issues identified, focusing on 1) personal reflection, 2) ABI/hospital admission, 3) social and psychological cost-benefit analysis, 4) future challenges. This paper presents and discusses an exploratory evaluation study of the ‘Motor Magic’ programme run in Adelaide, South Australia. Direct realism, also known as naive realism, can be described as “what you see is what you get”[1]. 12, 1 September 2015 | Health Services and Delivery Research, Vol. An account is offered of the evolution of this persuasion, through the works of Chen and Rossi, Weiss, and Pawson and Tilley. However, the supporting evidence for individual learning was relatively weak, therefore there may be assumptions made about learning within healthcare literature and policy that are not founded upon strong empirical evidence. Methods One promising approach is realist evaluation, which seeks to establish what works, for whom, in what circumstances, in what respects, to what extent, and why. The professional group that experienced most role change were district nurses and community pharmacists. The findings describe six themes that emerged from the data analyses: (a) awareness of scholarships; (b) perceptions of institutional scholarships; (c) motivations for pursuing scholarships; (d) barriers to pursuing scholarships; Developmental evaluation (DE), essentially conceptualized by Patton over the past 30 years, is a promising evaluative approach intended to support social innovation and the deployment of complex interventions. There are a number of randomised controlled trials in this field though few ABI studies have addressed the experiences of hazardous/harmful drinkers. Furthermore, a lack of findings about the influence of intentional rounding on patient outcomes in the safety thermometer data limits inferences on how mechanisms link to clinical outcomes for patients. The development and implementation of the intervention and its theoretical foundation is described in a separate paper. The research question is: ‘What is it about IR in hospital wards that works, for whom and in what circumstances?’ We will investigate this at the three levels of the organisation and delivery of health services: national, service provider organisation and individual ward/unit. Further research is needed to examine the effects on health outcomes of mechanisms and contexts related to cardiac rehabilitation. Overall, RE was perceived as time-consuming and resource intensive.Conclusions This paper rejects this logic, arguing that the ‘study' is not the appropriate unit of analysis for quality appraisal in research synthesis. 1, 14 March 2018 | BMJ Quality & Safety, Vol. It describes five key ideas which frame realism and their implications for impact evaluation. PROSPERO registration number Secondly, the 'attribution paradox'. You can request the full-text of this article directly from the authors on ResearchGate. It provides a range of illustrative examples. The paper provides a very brief introduction to the key ideas in realist evaluation and the implications of a realist approach for impact evaluation questions, evaluation design, programme theory, data collection, analysis and Social mechanisms identified included social comparisons, camaraderie, and social capital. 9, No. This synthesis found empirical evidence to support all four mechanisms, which tentatively suggests that collaboration, pooling, learning, and role blurring are all underlying processes of interprofessional teamwork. It seeks to understand ‘what works, for who, in what circumstances and why’ rather than providing one-off verdicts on programme … We observe that all these approaches are grounded in critical realism. Finally, we offer our thoughts on what this means for evaluators on the ground seeking an appropriate framework for their practice. 2, 3 February 2020 | Translational Behavioral Medicine, Vol. The weaknesses of the research centred on attrition where data went missing or it was not possible to find participants because they had re-located their shacks to another area or had changed cell phone numbers. An appreciation of the complex social reality inherent in the different programmes under investigation is required in order to seek the theories that explain why interventions are successful in some instances but not in others, ... 16 17 Realist synthesis seeks to understand for whom the intervention works and does not work, how, why and in what circumstances. From the ten programs that we reviewed, we identified six important and promising program strategies that reduce mental health and, to a far lesser degree, substance use problems: client choice in treatment decision-making, positive interpersonal relationships between client and provider, assertive community treatment approaches, providing supportive housing, providing supports for instrumental needs, and nonrestrictive program approaches. This article, coming out of their experiences, aims to assess the appropriateness of DE in research and describes issues related to its use. The main output from this review will be a new theory driven framework for skill development for the support workforce in health and social care for older people. 4, 22 July 2014 | Journal of Interprofessional Care, Vol. The community of practice will be supported to audit falls prevention activity and identify gaps in practice for intervention. It is a generic approach that can be applied to many fields of research, including health and social care. 3, No. Thus, intentional rounding neither improves the delivery of compassionate nursing care nor addresses the policy imperative it was intended to target. It explains when a realist impact evaluation may be most appropriate and outlines how to design and conduct an impact evaluation based on a realist approach. The purpose of this study was to identify outcomes for children, parents and kindergarten staff, as well as develop hypotheses about how these outcomes occurred and in what contexts. A process evaluation based on Pawson and Tilley’s Realistic Evaluation complemented a randomized controlled trial. Researchers will be blinded from validation data outcomes for as long as is feasible. of Manchester, Manchester, Realist synthesis: an introduction. The policy cycle revolves quicker than the research cycle, with the result that `real time' evaluations often have little influence on policy making. However, this shifting of the temporal frame for evaluation is in itself no guarantee of success. The realist approach to evaluation research: an introduction By Gillian Hewitt, Sarah Sims, Ruth Harris and University of London Kingston University and St. George's Topics: alliedhealth, health, nursing For realist evaluations (RE) this project aims to: develop quality and reporting standards and training materials; build capacity for undertaking and critically evaluating them; produce resources and training materials for lay participants, and those seeking to involve them. The Multispecialty Community Provider (MCP) model was introduced to the NHS as a primary care-led, community-based integrated care model to provide better quality, experience and value for local populations. Effect of Online Communication on Well-Being among Young Women: A Qualitative Analysis, Interprofessional teamwork in stroke care, Realist Evaluation - Oral Health Promotion Interventions, Realist evaluation of intersectoral oral health promotion interventions for schoolchildren living in rural Andean communities: A research protocol, What aspects of intentional rounding work in hospital wards, for whom and in what circumstances? The governance arrangements, policies, and administrative systems at The Station are well developed but flexibly implemented so that they support the nurturing and empowerment processes. Realist synthesis: an introduction. The first two stages comprised a scoping exercise and online survey to chief investigators to assess current PPI activity. Realist Impact evaluation: an introduction (12 pages) by Dr Gill Westhorpe. Interventions are implemented in childcare centres, schools and supermarkets in three local communities as well as in local mass media and social media during a 19 months period in the Municipality of Bornholm. Iterative data collection and analysis will inform a coding framework. The median percentage of dental plaque among schoolchildren in the targeted rural communities was 78.7% (interquartile range: 71.5-82.8) with an estimated dental caries prevalence of 98.0% (95% confidence interval: 88.3-100). The analysis unearthed perceived organisational support and reciprocity as underlying mechanisms that link the management practices with commitment. Their conclusions, which incidentally run counter to each other, are highly questionable. Data will be triangulated against administrative data and patient feedback. They have developed mainly in response to the need for evaluation of complex interventions. In this paper, we assess realistic evaluation’s articulation with evidence-based practice (EBP) from the perspective of critical realism. The second review revealed 8438 titles Participants identified several mechanisms of teamwork, but it was not a subject most talked about readily. ... 18,19 It does not seek to answer the question 'does this intervention work?' Electronic databases (Ovid Medline, Cinnahl, British Nursing Index, Embase, PsycINFO and EBM reviews) were searched, supplemented by citation tracking and grey literature searches. Mixed methods were used in data collection, including individual and group interviews, observations and document reviews. PORTER S and O’HALLORAN P. Nursing Inquiry 2012; 19: 18–28 The use and limitation of realistic evaluation as a tool for evidence-based practice: a critical realist perspective The Station's goals, policies and procedures, and the role of stakeholders were analysed in order to identify any links among these contextual factors, program mechanisms, and program outcomes. While the benefits of PPI have been widely discussed, there is a lack of evidence on the impact and outcomes of PPI in research. In the few studies offering insight regarding challenges associated with the use of RE, difficulties were expressed regarding the definition of both mechanisms and contextual factors. Realist approaches to scientific evaluation tend to be strong on theory and explanation, but lack adequate tests or means of validating theory. The term ‘realist evaluation’ is drawn from Pawson and Tilley’s seminal work, Realistic Evaluation (1997). Accordingly, there is debate about the best strategy of marshalling bygone research results into the policy process. Two evaluations of youth mentoring programmes are appraised at length. A realist evaluation, Exploring the use of Soft Systems Methodology with realist approaches: A novel way to map programme complexity and develop and refine programme theory, Factors associated with alcohol reduction in harmful and hazardous drinkers following alcohol brief intervention in Scotland: a qualitative enquiry, Improving skills and care standards in the support workforce for older people: a realist synthesis of workforce development interventions, Investigating the impact of a falls prevention community of practice in a residential aged-care setting: a mixed methods study protocol, ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study. The evidence base relating to these theory components was, for the most part, limited by initiatives that are relatively new or not formally evaluated. Data were collected through a national scoping and survey, and qualitative methods to track patient and public involvement processes and impact over time within 22 nationally funded research projects. In: Knuttson J, If an intervention to increase compassionate nursing care is required, it may be better to start afresh, rather than attempting to adapt the system currently implemented. Phase 1: theory development using realist synthesis to generate hypotheses about what the mechanisms of IR may be, what particular groups may benefit most or least and what contextual factors might be important to its success or failure which will be tested in subsequent phases of the study. Such practices can be implemented even with narrow decision spaces. A critical interpretative synthesis was performed of elements of context and underlying situational and transformational mechanisms. Funding Intentional rounding was often difficult for researchers to observe, as it was rarely delivered as a discrete activity but instead undertaken alongside other nursing activities. Results The collaborative approach nurtured by the realistic evaluation framework was found particularly helpful and there was consensus that the evaluation had become integral to the intervention itself. A realist evaluation, Improving skills and care standards in the support workforce for older people: A realist review, Theory-Based Evaluation and Types of Complexity. If an intervention is implemented with complete adherence to content, frequency, duration and coverage, then the implementation fidelity is said to be high. Phase 3: case studies to explore how IR is implemented ‘on the ground’, including individual interviews with patients, family members and staff, non-participant observation, retrieval of routinely collected patient outcomes and cost analysis. Ethics and dissemination Individual, semi-structured interviews were undertaken with 33 nursing staff, 17 senior nurse managers, 34 patients and 28 family carers from three geographically spread case study hospital sites in England. Seventeen senior managers, 33 front-line nurses, 28 non-nursing professionals, 34 patients and 28 carers participated in individual interviews. The 12 case studies represented what had happened. Collectively, these mechanisms had a positive effect on confidence that was perceived as being imperative to maintain health behaviour change. Abstract Interprofessional teamwork has become an integral feature of healthcare delivery in a wide range of conditions and services in many countries. In this article, we discuss how the epistemology of Soft Systems Methodology is compatible with realist approaches. The realist approach is a form of theory-driven evaluation developed to strengthen the explanatory power of evaluation studies and contribute to evidence-based policy and practice. Tilley N (eds.) In this article we defend the idea that theory-based evaluations—contribution analysis, logic analysis, and realist evaluation—are complementary components of a new theory in evaluation. The systematised approach of intentional rounding emphasises transactional care delivery in the utilisation of prescribed methods of recording or tick boxes rather than relational, individualised patient care. Main outcome measures The CoP was viewed as a complex intervention at the organisational level that could have differing impact across RAC facilities and the individual staff participating as members, dependent upon leadership, culture and staff behaviours [25][26][27]. Varying applications of RE across studies were examined in terms of a) reporting of core elements of RE, and b) potential feasibility of this evaluation method.ResultsA total of 14 studies (6 study protocols), published between 2007 and 2013, were identified for inclusion. Evaluation has become progressively popular within public health and healthcare programme research, with an emphasis on outcomes and a desire to improve practice and service delivery. Using such an approach, elements of context denote pre-existing elements of the environment in which interventions or programmes are implemented (De Souza, 2013; ... Mechanisms, on the contrary, are the stakeholders' reasoning and reactions to interventions or programmes, depending on context and resources available (Lacouture et al., 2015).