Considerations for the Design of a Systematic Review of Care Interventions for Individuals with Dementia and Their Caregivers: Letter Report (2018)

December 3, 2018

Richard J. Hodes, M.D.

Director, National Institute on Aging

National Institutes of Health

Building 31, Room 5C35

Bethesda, MD 20892

Dear Dr. Hodes:

Recognizing the rapidly growing number of people with dementia (PWD), the National Institutes of Health—and many other organizations and individuals worldwide—are interested in the state of the science on care interventions that support individuals with Alzheimer’s disease and related dementias, as well as on interventions aimed at supporting their caregivers. While research in this domain has expanded tremendously in recent years, there remain gaps in understanding. To enhance understanding of the current state of scientific evidence, and which care interventions are ready for dissemination and implementation on a broad scale, the National Institute on Aging (NIA) turned to the Agency for Healthcare Research and Quality (AHRQ) and the National Academies of Sciences, Engineering, and Medicine (the National Academies). Specifically, NIA asked AHRQ to commission a systematic review of the evidence on effective care-related interventions for PWD and their caregivers. NIA also asked the National Academies to convene a committee of experts, whose first task is to provide input into the design of the AHRQ systematic review. The names of the committee members and their affiliations are listed in the front matter, and the committee’s Statement of Task is in Enclosure A.

The National Academies committee held a meeting on November 12 and 13, 2018, to discuss and provide input on the preliminary key questions (KQs) and a draft study scope prepared by the Minnesota Evidence-based Practice Center (EPC), which AHRQ commissioned to conduct the systematic review.¹ During the meeting, the committee met in open session with representatives from NIA, AHRQ, and the EPC to discuss the committee’s task and the systematic review design. The committee also heard an overview of highlights from the 2017 National Research Summit on Care, Services and Supports for Persons with Dementia and Their Caregivers, as well as perspectives from a person living with dementia and advocacy organizations with interest in dementia and caregiving. The committee also deliberated further in closed session. This Letter Report summarizes the committee’s suggestions for refining the KQs and study design scope. The Letter Report was independently reviewed by two external individuals to ensure it meets institutional standards for objectivity, evidence, and responsiveness to the study charge. These individuals are named in Enclosure B.

In a “topic refinement” document, the EPC proposed a set of eight KQs; one guiding question for dissemination and implementation research to address in the systematic review; and a population, intervention, comparators, outcomes, timing, settings, and study designs (PICOTS) table. The EPC also posed nine questions for the National Academies’ input. Provided below are

___________________

¹ Minnesota Evidence-based Practice Center. 2018. Topic refinement: Care interventions for people with dementia (PWD) and their caregivers. Prepared for the Agency for Healthcare Research and Quality. https://effectivehealthcare.ahrq.gov/topics/care-interventions-pwd/key-questions (accessed November 16, 2018).

major comments that inform the overall design of the systematic review and KQs, followed by specific edits to the questions, specific edits to the PICOTS table, and responses to the nine EPC questions.

COMMENTS ON THE KEY QUESTIONS AND OVERALL DESIGN

The committee found that strong progress has already been made in developing preliminary KQs and a systematic review scope that will be informative for the committee’s later work and for the field in general. The comments below are provided in the spirit of further enhancing the review design.

A Broad View of “Care Interventions”

The committee recommends a broad view of “care interventions,” to encompass all those that support, enhance, or otherwise help PWD and their caregivers. These interventions contribute to a person’s well-being, happiness, identity, privacy, capacity, autonomy, or authority. They can be supports, services, programs, accommodations, or practices that include behavioral, environmental, technological, and psychological methods or approaches. They may be delivered by health care, social services, and other community organizations or caregivers with the intention of having a direct impact on either a person with dementia or their caregiver, or both.

Organizational Framework for the Interventions

To organize the available literature and understand the evidence and gaps in knowledge regarding care interventions, the committee recommends that the EPC adopt and adapt a framework that builds on a prior National Academies consensus report, Families Caring for an Aging America.² This framework focuses broadly on older adults who need help because of a limitation in their physical, mental, or cognitive functioning, but the framework concepts also can be applied specifically for dementia. This framework recognizes that interventions, supports, services, accommodations, and practices for persons with dementia and their families and support persons occur within multi-faceted contexts that encompass different settings (e.g., the home or residential setting), social/family networks, community, organizations such as the workplace and health care organizations, and societal/policy environments (see Figure 1).

Using this framework, interventions may be categorized as targeting various “levels”: the individual (older adult or caregiver), social structures (social network, neighbors, friends, community), organizations (workplace, health systems, community-based agencies), or policy initiatives, either alone or in combination. Each of these levels has a unique set of characteristics that influence individuals and other stakeholders, and there is a dynamic interplay among the various levels.

The PICOTS table already includes many outcomes that are relevant to these different levels. Adopting the framework would organize the outcomes conceptually and potentially help

___________________

² NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. Families caring for an aging America. Washington, DC: The National Academies Press.

identify gaps where additional outcomes would be valuable. This will help ensure that the systematic review includes information that is important to audiences that are in a position to support the dissemination and implementation of evidence-based interventions, including public and private payers, as well as health care, social services, and other community organizations.

Categorization of Interventions

The committee recommends eliminating the division of interventions into the categories, “care interventions” and “care delivery interventions.” This is an artificial distinction that would not add significant value in interpreting the systematic review results, but would create significant complications in attempting to categorize interventions according to this distinction.

The current proposed KQs also are divided according to interventions that address behavioral and psychological symptoms of dementia (BPSD) and interventions that address quality of life, function, or non-BPSD symptoms. Even though BPSD and non-BPSD interventions are highly linked, the literature is divided and historically has focused more on interventions for BPSD. To ensure that the non-BPSD interventions are properly covered and receive appropriate attention, the committee supports the proposed division of KQs along this dimension.

Categorization of Caregivers

The committee recommends that informal PWD caregivers and paid PWD caregivers be treated as separate groups for purposes of the search activities, review, and discussion. While both provide services and supports to PWD, each have different roles, relationships with the PWD, and goals. Therefore, there should be a separate set of KQs and outcomes for each group. Some of the interventions and outcomes are appropriate for both groups, but many are not.

With respect to the paid PWD caregivers, in addition to certified nursing assistants (CNAs), home health and home care aides, personal care aides, and hospice aides, it will be valuable to add promotoras (promotores) and community health workers. Interventions specifically targeted to paid PWD caregivers include training, peer mentoring, dementia specialist career ladders, support groups, use of technology to augment/complement service/support delivery, and bereavement support. Training describes activities such as basic dementia knowledge and understanding, as well as competency-based training in supporting and delivering services to PWD (e.g., comfort care, music therapy, behavioral change strategies to increase pleasant events for PWD and to identify and reduce environmental triggers that might cause behavioral issues, recognizing other reasons behind behavioral issues like pain and how to manage it, and cultural competence in understanding PWD and informal caregivers from different cultures and backgrounds). Outcomes include observed changes in practice; subjective well-being, feelings of empowerment, self-efficacy; emotional regulation; time spent with resident/client in relational activities (versus activities of daily living tasks); job satisfaction; turnover and retention; feeling safe; and anxiety.

Taken together, the committee recommends the elimination of one distinction (care interventions versus care delivery interventions) and the creation of a new distinction (informal versus paid caregivers). To avoid increasing the number of KQs and to group topics according to what will be most important at the interpretation stage, the committee suggests the schema shown in Table 1. In order that the review provides information that can best support the committee in developing a parsimonious set of recommendations, we suggest separating KQs according to PWD or caregiver outcomes, but do not believe that further separation of KQs by outcomes in informal and paid caregivers is necessary. However, within KQ2 and KQ4 in the schema below, separate outcomes for informal and paid caregivers should be reported when available.

TABLE 1 Proposed Schema for Key Questions

Understanding Study Participants and Cultural Modifications

The committee is interested in ensuring that the systematic review captures the demographic characteristics of the study sample, specifically in regard to representation of racial, ethnic, socioeconomic, and cultural groups. Some studies will have clear statements delineating the racial, ethnic, socioeconomic, and cultural representations of the sample, while others will

not. If studies report data that break down within groups often labeled as “Hispanic,” “Asian,” and “Native American” (which comprise many different groups and should not be bundled together) this should be captured in the systematic review. The committee realizes that this level of detail is unlikely to be broadly available, because it is only recently that organizations have advocated for this level of detail. It is suggested that the review carefully document the extraction process in order to present this information in the final report. In other words, we want to clearly delineate the specificity with regard to racial, ethnic, socioeconomic, and cultural sample characteristics.

The committee is also interested in identifying those studies that included cultural modifications or adaptions in their protocols. This includes studies developed from the outset with a target group(s) in mind or studies that integrated cultural modifications in subsequent translational efforts. These modifications can occur along different phases of a study, for example, in regard to conceptualization, design, implementation, interpretation, dissemination, and stakeholder involvement. A suggestion would be to consult the cultural adaptation and modification literature for exemplars of domains of adaptation relevant to our overall aim.³ It would even be helpful to indicate whether there was any cultural modification in a study (e.g., use of idioms, bicultural interventionists).

Dissemination and Implementation Literature

The charge to this committee is “to develop a report that informs decision making about which care interventions for individuals with dementia and their caregivers are ready for dissemination and implementation on a broad scale.” To meet this charge we will first focus on evidence on the benefits of interventions. For those interventions found to be beneficial, we will consider evidence on their successful implementation relative to a comparator group. We anticipate that there will also be many interventions that have been implemented in practice for which little evidence on benefits has been published in the peer-reviewed literature. We would like the EPC to review the grey literature, to the extent practicable, to identify evidence on the implementation and dissemination of relevant interventions. We acknowledge that this literature is vast and will provide exemplar studies to help delineate the type of projects that should or should not be included in this review.

The EPC may find it helpful to look for existing efforts to catalogue interventions in the grey literature; examining prior publications by the following organizations would be a good place to start: the Administration for Community Living, Centers for Medicare & Medicaid Services, Office of the Assistant Secretary for Planning and Evaluation, Patient-Centered Outcomes Research Institute, Salzburg Global Health Seminar Dementia Initiative, and Substance Abuse and Mental Health Services Administration.

___________________

³ For examples of previous reviews that used an ecological framework to identify how studies address cultural adaptations and modifications, see (1) Fuentes, D., and M. P. Aranda. 2012. Interventions for depression among racial and ethnic minority older adults: A systematic review of the literature across 20 years. American Journal of Geriatric Psychiatry 20(11):915–931. (2) Hays, K., and M. P. Aranda. 2015. Outcomes of faith-based mental health interventions with African Americans: A systematic review of the literature. Research on Social Work Practice 26(7). (3) Helu Fernandez, P., and M. P. Aranda 2016. Psychosocial approaches addressing sexual health and intimate relationships among patients with serious mental illness. Sexual Medicine Review 4:26–35.

This will be a critical aspect of the EPC’s work and the committee would be interested in further discussion at the next open session meeting in January 2019, after the draft protocol has been developed and literature scope has been additionally assessed.

Specific Wording of the Preliminary Key Questions

The committee has a number of suggestions for the specific wording of the KQs. The suggested edits are illustrated below using KQ1 and are intended to apply similarly to the other KQs.

For ease of comprehension, the committee suggests bringing the target of the intervention to the beginning of the sentence. The committee recognizes that there are different approaches with regard to assessing “efficacy, effectiveness, and comparative effectiveness” separately in a systematic review, versus considering them in a single category of “benefits.” For this review, the committee recommends looking at a single category of “benefits,” which refers to outcomes that are generally regarded as good, helpful, or otherwise salutary for well-being and health (as described in the section above on a broad view of “care interventions”). Terms such as “efficacy, effectiveness, and comparative effectiveness” are nuanced second-order categories of the evidence that support that an intervention is a benefit. They are well operationalized in the study of pharmacologic interventions, but less so for non-pharmacologic interventions. Efficacy and effectiveness lie on a continuum that varies with the degree of adherence and other factors—there is no bright line between the two—and different researchers may use different definitions. Therefore, the committee believes that attempting to make this distinction for every paper reviewed would add significant work without adding significant value.

The committee also suggests focusing on “harms” instead of “harms and unintended consequences,” because it will be hard to pre-determine how to look for results that are unintended, and any unintended consequences of note can be classified under benefits or harms. Harms are negative effects on the outcomes in the PICOTS table or other results identified as harms by specific studies. Together, these changes result in a congruent pair of terms “benefits and harms” (versus the typical pair of “efficacy and safety”).

The committee suggests similar edits to guiding question 1:

strength evidence for “real-world” benefits and harms effectiveness or comparative effectiveness (i.e., NIA Behavioral Research Stage Model Stages 3-4)?

ADDITIONAL COMMENTS ON THE PICOTS TABLE

In the topic refinement document, the EPC created a table outlining the preliminary PICOTS that further specify the KQs. The committee recommends some changes to the PICOTS table that follow from the more general comments above:

• The committee recommends revising the preliminary PICOTS table according to the organizational framework noted above, and potentially adding some systems-level outcomes to fill any gaps. Time spent on the intervention (e.g., participation or facilitation) and expenditures at each level of the system should also be listed as outcomes, whenever these are reported. This will help to ensure that the results are maximally informative to the various potential audiences, including those that will be using the review to inform decisions about implementing care interventions.
• Given the recommendation above to treat informal and formal PWD caregivers as separate populations, the committee recommends that the EPC include additional outcomes specifically related to the paid workforce (e.g., job satisfaction and retention), as described above.
• The committee recommends that the EPC not subjectively bundle quality-of-life measures for PWD and PWD caregivers, but rather include the actual measures used in the studies (e.g., satisfaction), unless “quality of life” is explicitly noted as a study outcome measure. Similarly, burden of care should not be labeled as a quality-of-life outcome and should be listed according to the actual measures used in the studies (e.g., financial cost), unless “burden” is explicitly noted as a study outcome measure.
• The committee notes that physical harm, abuse, neglect, exploitation, and family violence should be considered in two ways in the systematic review: interventions to reduce these harms should be examined and these should also be tracked as potential harms resulting from other interventions, because some interventions could unintentionally place great strain on the PWD or caregiver.
• The committee suggests that information on “loss to follow-up” and mortality should be captured to better interpret research findings.

RESPONSES TO QUESTIONS

The EPC also posed nine questions on which it hoped to receive advice from the National Academies committee. These questions, and the committee’s responses, are listed below.

1. Are the key questions the right questions to adequately capture the literature?

See general and specific comments above.

2. Within the key questions, further categorization will be needed to sort and present results. We listed some possibilities such as TIDieR, SQUIRE, or QI-MQCS. Do you have other suggestions for establishing a taxonomy for care interventions?

The committee defers to the EPC’s judgment on a categorization taxonomy. To evaluate the validity of the research methodology, it will be important to characterize, for each studied intervention, its duration, components, and individualized actions.

3. Which subgroups are of greatest concern for informing dissemination and implementation?

Where feasible, the committee recommends considering some metrics related to generalizability and applicability to different populations. The committee notes that the EPC may find that the concepts in the PRECIS-2 tool⁴ are useful to help characterize generalizability, but does not specifically recommend this tool over other similar approaches.

4. What outcomes are of greatest priority for decision-making, that is, which factors weigh heaviest when deciding care treatment approaches?

Above we note two areas of particular importance. First, adding any reported information regarding time spent on an intervention or expenditures will be important for any organization considering implementing or covering an intervention. Second, we note the importance of gleaning any reported information regarding study populations, including racial, ethnic, socioeconomic, and cultural aspects.

5. What is an appropriate minimum time for intervention duration/follow-up? Should the minimum intervention duration/follow-up differ by outcome category?

The committee recommends not establishing a specific minimum time for intervention duration or follow-up; all results will be informative. If this results in an overwhelming literature size, this would warrant further discussion during the next open session in January 2019.

6. What would we gain by including evidence from non-U.S. studies? If we include non-U.S. studies, what are your concerns about generalizability for home-based, community, institutional, or acute care settings?

It would be extremely valuable to include evidence from non-U.S. studies. Many of the most innovative approaches are being developed and evaluated outside of the United States, and we would benefit from understanding and considering them. The committee recognizes the practical need to restrict the search to English-language publications.

7. We plan to use a “best evidence” approach for the evidence review. Given the ultimate project goal of understanding what is ready for broad implementation, what study design/comparators are most important to include?

The committee recommends that the EPC take a broad perspective in including study designs for inclusion in the review, restricting to study designs that are informative for drawing causal conclusions (i.e., those that include comparator groups or interrupted time-series analyses). Any study without a comparator group or cross-sectional study would not be included.

8. We are looking at intervention literature and the dissemination and implementation literature using separate key questions. Regarding Guiding Question 1:

___________________

⁴ See https://www.precis-2.org (accessed November 20, 2018).

1. Do the key questions capture what needs to be asked to determine which interventions have sufficient evidence to support broad dissemination and implementation?
2. What are the important considerations for including studies that would inform dissemination and implementation (NIA Stage 5) research?
3. We are considering a staged approach to the literature search for implementation and dissemination, searching first for empirical work that informs implementation for interventions of at least NIA Stage 3 and perhaps at least low-strength evidence. Does this sound reasonable? What are your concerns?
4. What else can we provide/should we prioritize that will help you develop your recommendations?

We are primarily interested in studies that inform the evidence base on the benefits of interventions. As a secondary objective, we would like evidence on which studies are most ready to be implemented. We recognize that much of the evidence on implementation may be found in grey (i.e., non-peer-reviewed or non-indexed) literature sources such as demonstration projects and white papers. However, evidence on successful dissemination and implementation should be held to similar standards of rigor in terms of the quality of the evidence. We support the staged approach to the literature search that would prioritize searching the implementation literature for evidence about dissemination and implementation of interventions that have at least low-strength of evidence.

On behalf of the committee, thank you for the opportunity to contribute to this important endeavor. We hope that you find our comments to be constructive and we look forward to continued dialogue.

Sincerely,

Eric B. Larson, M.D., M.P.H.

Chair, Committee on Care Interventions for Individuals with Dementia and Their Caregivers