Searching for the Impact of Participation in Health and Health Research: Challenges and Methods
These descriptions can be organised by how people participate at each stage of designing, delivering, and evaluating interventions [ 29 ], or the processes that affect participation [ 27 , 30 ]. The description needs to acknowledge that different types of participation may occur at different stages of a project [ 31 ]. Community engagement, for example, can range from outreach, through consultation, to collaboration and shared leadership [ 3 ]. Participation may be initiated from the bottom up in communities where there is a large stock of social capital, or it may be induced by policymakers and implemented by bureaucracies [ 32 ].
These heterogeneous different approaches need to be identified in order to make decisions on whether to include different approaches. Most studies have defined impact as improvement in individual health outcomes [ 7 ]. Further, impact can be experienced at any stage of the project Box 2.
On Measuring Community Participation in Research
The period of time for impact to occur needs to be considered, as changes in partnership processes can over time lead to longer term transformation of systems [ 10 , 35 , 36 ]. Impact therefore needs to be considered as a continuum where different effects are achieved at various levels over different lengths of time.
This is quite different from an effectiveness review, which usually defines impact as the achievement of health outcomes at the end of an intervention. Questions can focus on impact within projects or beyond projects. Examples of possible questions are presented in Table 1. The examples are informed by existing theory and evidence of implementing systematic reviews [ 37 — 39 ].
Table 2 presents possible review questions to assess whether impact has been reported on social, economic, environmental, and health benefits for individuals, groups, communities, and systems. During the project, it is likely that researchers and local participants will be the main beneficiaries, while policymakers and nonacademics are the main beneficiaries after the study is completed [ 40 ].
INTRODUCTION
For reviews of participation, we would suggest that the focus of the review needs to be clarified using a scoping review. Scoping reviews are a way of mapping the territory of participation for a particular health topic. They not only serve develop definitions for participation and impact, but also help in making decisions about the boundaries of the review [ 41 ]. Four important questions to ask when scoping the literature are as follows: Boundaries for what to include in the review should be set by assessing whether the aims for participation are similar across studies and whether diversity of context is an issue.
Context is important because participation can be very different across different localities and countries. For example, the US population is largely immigrant whether recent or generations before and includes the African-American legacy of the slave trade and American Indian tribes. These identities have promulgated specific forms of community organising and activism, strongly influenced by the early labour and later civil rights movements [ 42 ]. In the United States, a particular form of partnership is the academic-community research partnership often labelled community-based participatory research or community-engaged research that forms around the development of health interventions and policy initiatives, and research on their effectiveness.
These collaborative partnerships typically involve academic researchers working with a diversity of community-based organisations or NGOs, community members and grassroots associations, policy makers, service providers, and other public and private agencies through all stages of the research process [ 14 , 35 ]. In Australia, participation of stakeholders in health research is promoted through specific grants administered by the Australian government major research funding bodies to ensure the relevance of the research and translation into policy and practice.
The model is most consistent with community-based participatory research with academics typically forming partnerships with government, industry, community, and health organisations. In other countries, social participation is embedded within structures, for example, in Germany where municipal health promotion is being integrated via government sectors, nongovernmental organisations NGOs , and citizen action groups working together to set priorities and define strategies for addressing health inequities.
These efforts are supported by coordinating centers for health promotion in each state which obtain funding and guidance through structures created by the new Law on Prevention. In Brazil, though participatory precepts were well articulated in the s with writings and activism inspired by Freire [ 43 ], social participation was codified in the constitution and in further redemocratization policies after dictatorship, including community councils for clinics and social determinant initiatives [ 44 ].
In contrast, while action research approaches are found within health research in the UK, there is little tradition of CBPR. If an international review is proposed, then these very different histories need to be taken into account as they may reflect different forms and understandings of participation. In each of these contexts, the aims of participation may be similar or different. Tables 1 and 2 can be used to categorise aims for the various studies and make decisions about which aims to include in the review.
The period of time covered by the project is important because it is related to different types of impact. During the project, impact directly related to the research may be created by those who are on the research team. Other activities may also be triggered, causing an indirect ripple effect. It is rare to find short-term and longer term impact in one publication, unless a journal is devoted to reporting a single project [ 46 , 47 ]. Journals often require authors to publish methodology and results for intervention studies separately. Articles reporting on longer term impact, as well as those describing process, may not be indexed to the original study because they are seen as separate.
This has implications for searching, as a straightforward search on outcomes will rarely produce citations for process or longer term participation impact. A method called cluster searching can be used to identify all documents related to a particular project in order to trace pathways to impact [ 48 ]. Cluster searching is an iterative process. As shown in Figure 1 , forwards and backwards chaining is done using the relevant article, the index paper, to identify all related materials.
If it is possible to cluster papers on process, outcomes, and impact, a rich picture can be produced tracing the pathway from participation to impact [ 22 ]. Mapping what exists in terms of participation aims and contexts and periods of time to achieve various impacts will lead to being able to answer the question what is the range of impacts which is possible based on these three questions i.
The scoping review will produce information on the types of studies that have been published on participation, which can be used to decide upon the type of systematic review that can be conducted. As of , 14 different systematic review types had been identified [ 49 ] and the number continues to rise. As studies exploring participation in health research are relatively new, it is likely that the most appropriate review types will involve a mapping, where an overview is presented and research gaps are identified; b qualitative reviewing where constructs and themes are identified illustrating the contribution of participation to health research; or c mixed methods reviews that combine learning from both process and outcomes studies to relate participation and intervention.
As yet, there are no guiding conceptual frameworks for the relationship between participation and impact in health research. In both primary studies and systematic reviews, a preliminary conceptual map of how participation works can be developed using existing research and stakeholder experiences.
A theoretical or conceptual framework can be developed that proposes general relationships between participation and impact see, e. Alternatively, a logic model can be developed which illustrates the relationships between participation, research design, implementation, and outcomes for specific populations in a given context [ 51 — 53 ]. Reviewers can develop their own model or use or adapt a preexisting framework such as the CBPR conceptual model, which suggests that context influences participatory processes, which then influence the interventions and research undertaken, to ultimately contribute to a range of outcomes [ 27 , 35 ].
For example, in our review of patient and wider community involvement in diabetes [ 54 ] we proposed that participation at different stages of the project could enhance the processes of clarifying problems related to diabetes, setting priorities for the research, designing the intervention, recruiting participants, collecting and analysing data, and disseminating learning Table 3. Frameworks and models can be used a priori to ensure that the search strategy explicitly looks for key concepts.
They can also be used during the review to iteratively develop explanations for how participation works [ 55 , 56 ]. When deciding which studies ought to be included in a review, concerns related to the quality of the primary research need to be addressed. Appraisal of quality generally asks whether the research was conducted in an ethical manner, whether it is relevant to practice or policy, the clarity of reporting, the coherence of the findings, and the appropriateness and rigour of the methods [ 57 ].
Filtering removes poor quality studies that may not enable decisions about the effectiveness of participatory research. Reviews that include studies with experimental or quasi-experimental research designs assessing the effectiveness of participation may use critical appraisal tools that are appropriate for the specific study design to assess methodological rigour. Where the review question wants to know how and why something works, however, qualitative studies or studies with descriptive elements may be included on the grounds of relevance because they contribute to developing the explanation.
In this instance, the appraisal process is used to make judgments about relevance. Participation adds to the complexity of an intervention, because it can mediate or moderate the effects of an intervention. Participation at one stage can create both positive and negative feedback loops, influencing the relative success of later stages.
Where the components of an intervention are a poor fit with local contexts, participation can create a more receptive setting for the intervention.
- When Death Comes Knocking Part 2!
- Account Options.
- Now Into Court.
For all of these reasons, the approach to synthesis needs to be appropriate for complex interventions [ 59 ]. As noted in the section on review types, most of the research to date on participation is descriptive, mainly qualitative in nature with some studies on process and others focusing on outcomes. This type of research asks open-ended questions about participation. The approach is configurative aiming to generate theory and explore relationships.
The recommended approaches to synthesis are outlined by Hannes [ 58 ] and include meta ethnography, thematic synthesis, critical interpretive synthesis, framework synthesis, realist synthesis, and narrative synthesis.
Editorial Reviews
The choice of approach is usually based on the material available and the skills of the review team. While it is beyond the scope of this article to explore synthesis in detail, there are several issues that will arise regardless of approach. These include selecting a framework for organising data, dealing with thin description of participation, establishing relationships between participation and outcomes, and mapping longer term impact. At the synthesis stage, the original a priori logic model or theoretical framework can be used to organise data.
Data extractors need to be trained in using an expanded and unconventional lens, as review authors have noted that information on the characteristics of partnerships and coalitions is often missing or inadequate, making it difficult to explain underlying mechanisms that promote health [ 9 ]. People may be motivated to participate when space is created for relationship building, where deliberation and dialogue is facilitated, and knowledge cocreation is promoted.
Few primary studies, however, describe how the process fosters inclusivity and involvement. The important components of the project and descriptions of process may be scattered across documents. Data may be found in descriptions of the research problem, which are often outlined in the introduction section of a paper; accounts of getting people to participate; and reflections on the process found in Methods and Discussion. We suggest that there are several reasons why causal pathways to health outcome are difficult to establish. First, health research interventions are usually conceived as consistently delivered, distinct and bounded activities that improve health outcomes for individuals independent of context.
Recently there has been acknowledgement that the success of an intervention may depend on how well it is tailored to the individual [ 61 ] and the quality of the relationship between professional and client [ 22 ]. In community interventions, it is also difficult to draw boundaries around an intervention because they are events that both affect a wider community system and are affected by it [ 34 , 63 ]. Instead of referring to interventions as linear pathways, we should be visualizing them as streams which are fed by events and relationships, which contribute in turn to larger changes in groups and networks.
People who form partnerships and coalitions, combining their perspectives, knowledge, and skills can create a synergy where the whole becomes greater than the individual contributions [ 10 ]. This process raises the question of when to assess impact in terms of individual health outcomes, and the importance of tracking interventions over time.
Capturing the ripple effect at community and system-level is difficult and may require methods that go beyond traditional data extraction from journal articles. Explanations of impact can be obtained via participatory reviews, where stakeholders are involved throughout the review process in interpreting findings based on experience and knowledge [ 22 ]. Author interviews can be conducted, to explain relationships between participation, outcomes, and longer term impact that were not reported in primary studies [ 10 ].
These examples of the evolution of individual interventions, emergent individual outcomes, and synergy and ripple effects at community and systems level illustrate several key points about impact. First, impact is time-dependent in the sense that single studies usually report on a distinct point in the intervention. Whenever possible, this point needs to be documented to reduce the risk of synthesizing data that actually represents very different stages in the process. Second, impact occurs at all stages of a project, from its inception to completion, as well as leading to further spinoff projects [ 10 ].
When defining impact in a review, these key points need to be explicitly considered with statements about whether the review covers one or more levels, whether data will be synthesized for a specific stage of participation or as an evolutionary stream, and whether nonhealth outcomes are included. A variety of issues need to be considered when determining the impact of a participatory intervention [ 35 , 66 , 67 ].
For example, historical issues of trust among the participating parties shape the nature of the participatory dynamics [ 27 ]. A poor fit between the proposed intervention and the context may lead to detrimental or ineffective processes and outcomes [ 68 ] and influence effectiveness of participatory interventions [ 66 ]. Readiness for change in organisations can influence the process of implementation [ 67 ].
The strength of this text lies in its exploration of the complex power relations involved in conducting community research, an issue that Goodson and Phillimore claim is neglected in wider literature. The diversity of both the theoretical and practical chapters provides a rich picture of the benefits but also significantly the difficulties in managing the research process. Unlike community-based participatory research where there is an equal relationship between community members and external researchers, community research can by characterised by a disparity of power between professional and community researchers.
The pursuit of funding institutions desired research products contradicts the Freireian methodology that emphasises the subjective experiences and learning processes of the community. This tension is one reason why some communities meet attempts at engagement with scepticism and hostility; and this is not the only barrier to progress.
Proper engagement with communities not only prevents researchers from entering the field solely to answer research questions generated outside local context, but trust and cohesion can be generated through shared knowledges and goals. As the case studies in this volume show, successful community research can allow for a plurality of views from which to critically consider authoritative constructions.
Questions do remain as to whether the nature of this research can truly be incorporated into traditional funding structures, particularly in the university setting where deadlines rule. Perhaps more intriguing is whether researchers really want to relinquish control over the research process.
This volume does suffer from the absence of a concluding summation that could serve to analyse the future direction and inevitable challenges of cementing community research as an established methodology. Nonetheless, Goodson and Phillimore have collated a comprehensive and timely volume into the nature of community research that highlights the benefits of working with and in communities. As such Community Research for Participation will prove an invaluable resource to those wishing to learn more about this innovative methodology.
You can follow him on twitter mattwargent. Read more reviews by Matthew. But why, Matthew, should academics share the value set that would make them want to take part in emancipation? And what if the academic reward structure is indifferent to or dismissive of the kind of research whose methodology you discuss.
In other words, why would academics be motivated to follow this path. I see no sign that conventional epistemology is being overturned. First I would suggest your questions would be better answered by the editors of this volume rather than a mere reviewer, however I will respond to your comments as best I can.