National Academies Press: OpenBook

Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs (2014)

Chapter: 5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH

« Previous: 4 EVIDENCE FOR DEPARTMENT OF DEFENSE INTERVENTIONS FOR PREVENTING PSYCHOLOGICAL DISORDERS
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

5


MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH

Building on the measurement frameworks discussed in Chapter 2, this chapter focuses on the committee’s task of identifying the best metrics for evaluating Department of Defense (DOD) resilience and prevention interventions (programs and policies) that address psychological health. It is important to note that the assessment of prevention programming and policy overlaps in part but not entirely with performance measurement of psychological healthcare treatment programs. Prevention interventions are on a continuum and can include population health risk reduction carried out through policy changes as well as indicated riskreduction screening combined with early intervention. Although “performance measurement” is typically used to refer to the assessment of health delivery services, the committee adapted this term in this chapter to refer to the metrics appropriate to all prevention initiatives.

The chapter begins with discussions about the purposes of and standards for good performance measures and the need for using measures at multiple levels (e.g., population health measures, individual- and system-level measures, and measures of process and the structure of services) when monitoring resilience and prevention programs and policies. Next is a brief overview of current efforts at DOD that can inform performance measurement of prevention interventions. Then, this chapter conceptualizes those domains that are appropriate to measure in a manner consistent with the committee’s Model for Prevention Program Development and Measurement presented in Chapter 2. Reflecting the committee’s finding that the current state of measurement for psychological health programming is inadequate, this section also illustrates how these measurement domains could be applied to prevention programs along the Institute of Medicine (IOM) continuum model. Finally, the committee presents findings from its review of performance measures promulgated by current national efforts. In this review the committee discusses the limitations and relevance of existing metrics to the development of metrics for quality improvement for the full continuum of prevention initiatives, again referring to the committee’s model for prevention program development and measurement.

STANDARDS FOR PERFORMANCE MEASUREMENT

Although they are not specific to prevention contexts, recommendations on how best to measure and improve the quality of psychological health interventions already exist and provide useful guidance. The 2006 IOM report Improving the Quality of Health Care (IOM, 2006)

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

emphasized that to measure quality effectively requires structures, resources, and expertise as well as strategic efforts among key stakeholders to

•   Conceptualize the aspects of care to be measured.

•   Translate the quality-of-care measurement concepts into performance-measure specifications.

•   Pilot-test the performance-measure specifications to determine their validity, reliability, feasibility, and cost.

•   Ensure calculation of the performance measures and their submission to a performance-measure repository.

•   Audit to ensure that the performance measures have been calculated accurately and in accordance with specifications.

•   Analyze and display the performance measures in a format or formats suitable for understanding by multiple intended audiences, such as consumers, health care–delivery entities, purchasers, and quality-oversight organizations.

•   Maintain the effectiveness of individual performance measures and performancemeasure sets and policies.

Related to these directives concerning appropriate measurement are the National Quality Forum (NQF) measure-selection criteria discussed in Chapter 2; briefly, the criteria include the importance of the measure, the scientific acceptability of measure properties (reliability and validity), feasibility, usability and use, and comparison to related or competing measures. The purpose of all quality-measurement initiatives is to improve the performance of the service delivery system (e.g., the health system). The process of conceptualizing performance and identifying quality-improvement processes has not been formally developed and applied to many population-level prevention initiatives or to prevention strategies and programs outside the formal healthcare system. Although the committee’s charge was to examine existing performance measures, current quality measurement initiatives cover only a narrow slice of possible measures for prevention activities. Given this situation, the committee provides a discussion of how to broaden measurement to include prevention programs and proposes that DOD stakeholders engage in this process as a necessary starting point.

In order to effect change, quality measurement should be linked with activities at the locus of program delivery, and quality improvement techniques should be woven into the day-today operations of all organizations that are delivering programs (e.g., the local installation, the local family advocacy program, or the Air Force suicide prevention initiative) (IOM, 2006). For performance measurement to be effectively integrated with improvement and quality initiatives, it is also necessary for organizations such as DOD, the military service branches, and program offices to invest in the infrastructure and staffing necessary to ensure the selection of relevant measurements and also in program assessment and feedback based on routine data management and analysis. The initiation and continuation of novel programs without measurement interferes with creating an evidence base and with improving the intervention’s implementation based on data concerning such things as feasibility, acceptability, and outcomes. At worst, a lack of measurement risks not a only failure to meet pressing needs or the continuation of programs that are ineffective and costly, but also a failure to detect potential harm caused by program components (e.g., increased drug addiction and suicidality associated with polypharmacy). Such quality measurement has become standard for evidence-based care, and such measurement and

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

regular review by appropriately trained individuals charged with program oversight should be included in all supported DOD programs.

SCOPE OF PREVENTION MEASUREMENT

To translate the principles of quality-improvement initiatives and performance measurement into resilience and prevention interventions requires broadening the scope of measurement application. The committee found that current quality performance measurement initiatives place significant attention on health care quality as it affects individuals in clinical settings, focusing on narrow, technical, and clinical aspects of health care. There is growing recognition that the next stage of improving health and preventing disease will emphasize measurement at the population health level (and not solely the measurement of individuals entering clinical programs), population-based strategies including environmental changes (e.g., regulating alcohol availability by controlling the number of alcohol outlets in communities), policy changes (e.g., random breathalyzer testing of service members at work), and training and support for prevention programs (e.g., training of gatekeepers for suicide prevention) (IOM, 2012).

For resilience and prevention, the focus on health care quality inappropriately encourages improvement in a narrow set of outcomes and in those population groups using health care. Yet, the goals of resilience and prevention programs are broader and include proximal and distal health outcomes at the individual and population level. Thus, the quality of interventions that are delivered outside health care, the cost and resource use by the prevention system, and the exposure and engagement of all service and family members in these interventions are also important targets to assess (Berwick et al., 2008).

MEASUREMENT IN THE MILITARY

As is the case with all large enterprises, DOD maintains many kinds of operational measurement programs, but these activities are decentralized, uncoordinated, and organized at inconsistent levels. Indeed, military populations are subjected to more data collection than the general population, including mandatory data collection (e.g., random drug testing and selfreport post-deployment health assessments) and voluntary data collection (e.g., periodic anonymous surveys of health behaviors and annual surveys of spouses). There are a variety of challenges facing these data collection programs, including that they support specific operational functions; that they are owned by disparate, uncoordinated, organizational entities; and that the data review or monitoring that occurs may benefit only one operational function, one DOD office, or one DOD level (e.g., service branch) and be uncoordinated with the informational needs of other DOD levels (e.g., operational managers and local prevention staff). In short, there is no systematic use of extant data as part of strategic DOD prevention initiatives. It was beyond the committee’s charge to review these extant data collection programs at all levels of DOD, although such a review would be an important starting point. If a broad representation of DOD prevention stakeholders (e.g., public health and commander’s programs) and content experts (e.g., headquarter sexual assault program experts and external experts, as needed) could be engaged in such a review of data collection, reporting, and interpretation, it could become a

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

forum for an associated discussion of ongoing improvement of prevention programs and policies to enhance psychological health.

Broadly speaking, measurement and reporting occur within individual services (e.g., the Navy or the Army) and at DOD-level offices of Force Health Surveillance, Personnel, and Readiness (epidemiology); Defense Health Cost Analysis and Evaluation (health behavior surveys, special analyses); and Defense Manpower and Data Center (deployment records, surveys of spouses). Another layer of data collection for operational purposes is maintained by local commanders on high-risk behaviors of unit members, often linked to drug testing programs (e.g., the Army Risk Reduction Program; see Army Substance Abuse Program, 2013) and on unit climate assessments, including reports of sexual harassment and sexual assaults (e.g., the Air Force Unit Climate Assessments; see Schogol, 2013). These unit-level data appear to be reviewed by local commanders quarterly and may result in discussions with local prevention coordinators (e.g., sexual assault prevention coordinators), but the committee found no systematic use of these reports to assess or improve the quality of the prevention strategies chosen. The reliability and validity of data elements are unclear, and the training provided to local staff collecting assessments, interpreting the data, and suggesting next prevention steps is unknown. In sum, the uncoordinated, non-standardized collection of relevant prevention data within numerous program silos and at different organizational levels contributes to the complexity of improving assessment initiatives going forward. Although the committee’s review discovered DOD offices responsible for the coordinated management of clinical quality, it did not discover any comparable entities responsible for quality review of prevention initiatives. The activities of those offices with clinical management functions are briefly reviewed here for context. The DOD office responsible for the oversight and management of clinical quality and population health is the Office of the Chief Medical Officer (OCMO), one of six directorates in the Defense Health Agency (DHA).1 Responsibilities of the OCMO include a range of programs for quality assessment and improvement, patient safety, and population-based health management which are scattered across the TRICARE Military Health System and affect both the direct care and the purchased care components of TRICARE2 (Defense Health Agency, 2013).

DOD has a number of quality-measurement activities for the clinical management of a range of health conditions. However, among the measures DOD identified in a report to Congress (DOD, 2012), the committee did not find any measures specifically addressing psychological health. For example, in its centralized Military Health System–wide database for health management—called the Military Health System Population Health Portal3—measures relate to outpatient and inpatient care for physical health conditions. Outpatient measures of care address issues from disease prevention (i.e., screenings for various cancers) to chronic disease management (i.e., diabetes). Inpatient care measures exist for heart conditions, pneumonia, asthma, and surgical care.

____________________

1 In October 2013 DOD established the DHA to manage the activities of the Military Health System, including those previously managed by TRICARE Management Activity, which was disestablished.

2 There are seven divisions in OCMO: Behavioral Health, Clinical Quality, Health Care Benefits and Risk Management, Information Management, Patient Centered Medical Home, Patient Safety, and Population Health and Medical Management.

3 The Military Health System Population Health Portal contains administrative health care data on TRICARE Prime/Plus enrollees who receive care through military treatment facilities and contracted providers.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

It is worth noting that DOD has implemented population health initiatives for tobacco cessation and education, obesity prevention and management, sexual assault prevention, suicide prevention, and alcohol abuse prevention. However, the committee is not aware of any measures being implemented by DOD to monitor the effectiveness of these population health programs. Although some data on these topics are routinely collected (e.g., epidemiological sexual assault incidence and suicide incidence), those data are limited to distal outcomes and do not appear to be used in efforts to evaluate specific prevention efforts. Quality-improvement approaches would prescribe that any system-level, program-level, or operational-level prevention initiative should always be accompanied by the careful, strategic planning of monitoring and surveillance of changes in the desired structures, processes, and outcomes with the appropriate oversight to enable utilization of such findings for ongoing quality improvement of prevention efforts.

WHAT SHOULD BE MEASURED?

Experts in the field of performance measurement agree that it is challenging to identify key indicators across the prevention continuum for psychological health for adults and children. This is because there are gaps in the empirical evidence in support of prevention programs and limitations in the breadth of existing indicators of quality (SAMHSA, 2013b). Useful and appropriate performance measurement is predicated on the existence of empirical evidence for specific interventions. As described in Chapter 2, the Substance Abuse and Mental Health Services Administration (SAMHSA) quality framework captures prevention and wellness in some population-level measures, but most measures are screening measures of health programs. Measuring the quality of population-based preventive interventions, such as policy change and stigma reduction programs, presents more challenges than measuring the quality of individualbased or family-based interventions. Generally, with selective or indicated interventions, appropriate measurement can demonstrate improved outcomes in the relative short term, while population-level public health or prevention efforts (e.g., hotlines for suicide prevention or bystander training) may require a focus on proximal outcomes and take time to diffuse to a sufficient proportion of the population before their overall impact can be detected through measurement (IOM, 2013c).

Evidence-Based Program Development

Recognizing these limitations and the current status of DOD prevention program assessment, the committee developed the Model for Prevention Program Development and Measurement (see Figure 5-1) to show the basic process for prevention program development in association with the measurement of implementation processes and program quality. This model, described in Chapter 2, identifies the development of evidence-based programming as the essential first step in performance measurement.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

image

FIGURE 5-1 Model for prevention program development and measurement.

In deliberations about the current state of resilience and prevention program measurement in DOD, the committee focused on the lack of evaluative evidence and the need for standardized validated measurement of the effects of prevention programs on targeted outcomes that are mapped to clear and documented theoretical models for improved psychological health (e.g., symptom reduction and increased positive functioning). In addition, committee discussions emphasized the need for standardized measurements of implementation processes, including training, fidelity, dissemination, and resource use. The model used by the committee (see Figure 5-1) is based on the Donabedian paradigm (discussed in Chapter 2) and includes measurements of structure, process, and both proximal and distal outcomes of DOD prevention programs. These types of measurements or assessments are needed across the full continuum of programs and policies (including local commanders’ initiatives) targeted at reducing risk, population screening, and selective and indicated prevention programming.

Structure Assessment

Application of this model to prevention includes measurement of the structural attributes of the provider and of the operational settings (e.g., the commander’s unit, community, school, or healthcare provider) in which services are delivered, a review of available and appropriate evidenced-based programs, information on program adaptation to the target populations, a documented logic model that details the link between program components and outcomes, and standardized training curriculum, supervision procedures, and delivery materials. In addition, structural assessment includes measurement of organizational readiness (e.g., of a unit or an installation) for program implementation, the availability of adequate space and material resources (e.g., the data monitoring infrastructure), and human resources (e.g., staff expertise and what training is available).

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Process Assessment

Process assessment refers to assessing the delivery of prevention services and the implementation of policy changes. This can include anything regarding participant encounters with program elements, access to prevention resources (e.g., website hits), dissemination of program messages (social marketing), and enforcement of policy changes. It can also include interpersonal processes associated with participant, leadership, and community engagement, such as collaborative partnerships, and the provision of information, consultation, and emotional support. Process measurement domains include the monitoring of participant preferences, such as tracking participation, drop-outs, and satisfaction with services. One can also measure the monitoring of the reach and consistency of implementation of policy change as well as enforcement approaches to non-compliance. In support of rigorous program implementation, process measures assess the actual services to individuals and actual changes to system operations and the fidelity with which they were delivered. Such process assessment is critical both to assuring high quality and consistency (minimizing unwanted variability among program sites) and to assuring that the intended interventions have been implemented so that conclusions about their effect on outcomes are valid.

In order to measure prevention outcomes, it is important to align standardized, measurable outcomes with the targets of the program or policy goals. When focusing on enhancing resilience and risk factor reduction (targets), prevention components may include installation-level policy changes, new protocols for enhanced screening and risk-reduction counseling, and also preventive approaches aimed at community climate and family support. Given that many prevention programs have targets in multiple domains, it is often important to assess and monitor a range of relevant outcomes, and the assessment must be done relative to the preventive intervention, the population, and the context. For example, a family-based prevention program designed to reduce social and emotional risks in children may have a variety of targets, including reduction of parental depression, increased positive parenting practices, and increased positive coping in the child. Furthermore, the domains that are targeted may be interrelated, with mutual and reciprocal impacts across outcomes over time.

Proximal and Distal Outcomes

Outcome measures should include standardized, evidence-based measurement of both proximal and distal outcomes longitudinally. Proximal outcomes refer to short-term consequences, and distal outcomes refer to long-term consequences. Examples of proximal outcomes include reduced days of binge alcohol use, decreased days absent from work, improved parenting skills, and reduced family conflict. Examples of downstream consequences—distal outcomes—include a lower prevalence of alcohol use disorder, lower mortality resulting from alcohol-drug overdoses, a reduction in incidents of spouse and child abuse, and reductions in divorce rates. For example, comprehensive measures of a suicide prevention program should use validated assessments to measure short-term consequences on suicidal attempts and calls for help to suicide hotlines as well as the distal assessment of trends in suicide rates in the target population. Furthermore, measurement of a suicide reduction intervention should include observations on reductions in known risk factors (depression symptoms, hopelessness, and identifying a supportive relationship). Thus, outcome assessments should include the multiple domains targeted simultaneously and should be followed consistently over time. Furthermore, when multiple related programs are being simultaneously implemented

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

across components of DOD and at multiple organizational levels, it is imperative that there be consistency and standardization in the measurement approaches of shared proximal and distal targets so as to enable the comparison and dissection of impact.

Validated, Reliable Measures Exist

The committee found that standardized measurement for prevention outcomes is not sufficiently covered by the national datasets or available national quality-assessment initiatives (see the section Measures Identified in National Quality Initiatives below). However, the committee notes that validated and reliable measures are available in the scientific and evaluation literature for all of the common prevention targets (proximal and distal outcomes) for children, adults, and families. Hence, DOD should not develop new measures of constructs until it has carried out a careful review of widely used valid measures, and in existing measurement programs (e.g., commanders’ risk reduction assessments and periodic health assessments) DOD should consider replacing non-validated measures with validated measures. For example, well-established evidence-based measures of adult posttraumatic stress disorder (PTSD) symptoms (e.g., PTSD Checklist, both civilian and military versions; see National Center for PTSD, 2013), anxiety (e.g., the Generalized Anxiety Disorder seven-item scale; see Spitzer et al., 2006), depression (e.g. the Patient Health Questionnaire nine-item scale; see Pfizer, 2013), and alcohol consumption (e.g., Alcohol Use Disorders Identification Test-Consumption; see SAMHSA, 2013a) are available in the scientific literature and are currently being used in both civilian and military clinical settings, yet they are underused by many prevention programs, even when such outcomes are targets of prevention and resiliency. The committee felt it was most important that the selection of such standardized measures follow an appropriate review process informed by evidence reviews and led by subject-matter experts advising a knowledgeable, empowered measurement team. It may be necessary to develop a coordinated review process in order to ensure the adoption of these measures consistently among all programs with similar targets, among services, at all levels of the organization, and for a sufficient length of time to observe trends associated with the introduction of novel programs and policy changes. Such assessment should be linked to a mechanism for prevention program oversight, with a structure for routine reporting and review to enable ongoing quality improvement, increased adoption and dissemination of the most effective prevention programs and strategies, and discontinuance of ineffective or harmful programs and strategies.

Resilience

In addition to direct health outcomes, DOD prevention and resilience programs are frequently focused on domains thought to be protective, such as positive psychological attitudes, or domains related to theoretical models of individual and family resilience (Masten, 2001; Walsh, 2006). These domains include the availability and perception of social support, parent–child relationships, positive coping, and family communication. The selection of outcome measures for a specific program should be guided by the program logic model and by evidence within the scientific literature. The committee notes that the selection of outcome measures to assess an individual’s or family’s capacity for resilience is challenging, given the complexity of defining the construct of resilience (see discussion of resilience-related programs in Chapter 4). While scales for measuring resilience have been developed, Windle and colleagues (2011) conducted a methodological review and found no “gold standard” among 15 identified measures

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

of resilience, and they noted only one scale (the Connor-Davidson Resilience Scale; see Connor and Davidson, 2003) had been used to assess intervention impacts in adults. Furthermore, current measures of resilience, including the Connor-Davidson scale, the Resilience Scale for Adults (Friborg et al., 2003), and the Brief Resilience Scale (Smith et al., 2008), were found to be of only moderate quality at best, to focus primarily on individual capacities, and to not measure systemic domains noted to be relevant to resilient processes in the context of stress or adversity.

Child Outcomes

Selective and indicated prevention programs designed to mitigate risk and enhance resilience in children often include standardized psychological health outcome measures, such as screening measures of emotional and behavioral symptoms (e.g., Strengths and Difficulties Questionnaire or the Pediatric Symptom Checklist; see, respectively, Goodman, 1997, and Jellinek et al., 1988) designed to assess a range of symptoms and behavioral problems. Additionally, validated psychological heath measures used to screen for child and adolescent mental health risks are available across developmental periods. Similar to adult outcome measures, such assessments should be selected according to the specific program context. Considerations in the selection of child outcome measures include the developmental period and the available reporters (parent, child, or teacher). Other functional domains important in the children and adolescent populations that are relevant to prevention strategies include academic outcomes and peer relationships, both of which have emerged as important consequences in military children (Chandra et al., 2010; De Pedro et al., 2011; Hazelden Foundation, 2013).

Family Outcomes

Paying attention to the ecological framework and to a prevention program’s logic model is central to the selection of measures that will increase the evidence base for prevention in military populations. For both adult and youth populations, the measurement of family-level domains is often relevant to the theory of action and to the logic model of prevention programs. For example, marital conflict, divorce, and domestic violence have been identified as risk factors for poor psychological states in military members and their partners, including increased PTSD symptoms and suicidal behaviors, while reductions in marital conflict and domestic violence are appropriate targets indicative of good psychological health in and of themselves. In children, negative parenting practices and parental depression or other psychological symptoms are known to increase the risk for childhood behavioral problems. Evidence-based prevention interventions that target improved parenting skills have demonstrated a positive impact on child adjustment (NRC and IOM, 2009; Spoth et al., 2002).

Measurement Gaps

The Example of Community Reintegration

A review of post-deployment reintegration programs conducted by the Defense Centers of Excellence for Psychological Health & Traumatic Brain Injury (DCOE, 2012) emphasized the need for standardized measures for outcome assessment that could be used across programs and the individual service branches. The dearth of instruments for measuring reintegration has stimulated the creation of at least two new self-report measures. The first is the Military to Civilian Questionnaire (M2C-Q), a 16-item self-report questionnaire that assessed difficulty with community reintegration (Sayer et al., 2011). The second new measure is Community

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Reintegration for Injured Service Members, or CRIS, (Resnik et al., 2009, 2011) and a shorter, computer adaptive test version, CRIS-CAT (Resnik et al., 2012). The committee noted that the development and inclusion of new “custom” measures inconsistently across programs can diminish generalizable knowledge and slow the growth of an evidence base for prevention programs. New measures are often developed even when other well-defined measures could provide assessment on common outcomes shared among many programs, such as relationship counseling, rates of separation or divorce, and standard validated measures of relationship and family functioning. Similarly, transition-to-work programs could measure rates of hiring and length of employment. The establishment of common program aims and the objective measurement of those aims are the key components; idiosyncratic definitions of program aims and measures could lead to so much variability in what is measured that DOD will not be able to assimilate important lessons across interventions and studies and ultimately not be able to define successful reintegration and targets of reintegration programs.

Measurement of DOD Prevention Implementation

As noted above, the systematic identification of structure, process, and outcome domains is important to the successful implementation of an intervention and relevant to the successful scaling and sustainment of effective prevention programs. Studies of DOD prevention programs have demonstrated a need for improving their implementation processes, including those associated with provider certification, training, and supervision of program personnel; ongoing fidelity to established program approaches; linkages among commander programs and care systems within the enterprise; dissemination of program information and best practices; and cost and efficient use of resources (IOM, 2013a; Weinick et al., 2011). Program implementation is a specific target of quality improvement that requires greater attention through the use of standardized measures of structure and process for all prevention programs. Furthermore, DOD needs to have in place a data collection platform and tracking system that can be used for program-level implementation of a variety of activities and also be used across service branches and organizational levels. Transparent summaries of program data and information should be made accessible across DOD to ensure that any part of the organization can review that information before embarking on development of new programs and could potentially use the information to adapt an existing program. Transparent information on programs that experienced theory ineffectiveness or implementation failure is also valuable for organizational learning. Sharing information across DOD will make it possible for proven programs to be disseminated or transferred to new settings while remaining faithful to the original approach, will reduce duplication of effort, and will help ensure that approaches that were found to be ineffective in previous efforts are not tried again (Weinick et al., 2011).

Cost-Effectiveness

The measurement of program effectiveness is necessary but not sufficient to guarantee the optimal use of scarce DOD resources, especially if there is pressure to reduce funds devoted to non-mission-driven programs. The amount of resources (costs) needed to effect a change in outcomes must be known and compared among programs to determine the cost-effectiveness of DOD prevention interventions. Paying attention to effectiveness only can result in ranking one intervention (e.g., clinic-based treatment) as more effective than another intervention (e.g., family support) on the basis of the average participant outcomes (e.g., change in depression scores) even if the other intervention may be a wiser investment. Such a situation can occur if,

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

for a given budget, the alternative intervention reaches and serves many more members of the target population and leads to significant improvement in outcome at a lower per-participant cost even though the average improvement per participant is smaller. An intervention is cost-effective if it achieves a unit change in outcome at significantly lower cost than an alternative, or if, for a given budget, it is the one with greatest change in total participant outcomes. A corollary is that the feasibility of program implementation (e.g., the ease of attracting participants) may be critical in selecting programs that will be cost-effective.

Ideally, DOD would be able to assess the value of all of its resilience and prevention programs for psychological health using a core set of metrics that reflect its highest-priority target outcomes. That would permit the department, using cost-effectiveness analysis based on current program operations, to determine how to maximize the outcomes for the resources it expends. This would result in maximal efficiency in the use of taxpayer-provided resources toward the goal of improving the psychological health and hence the readiness of service members. However, as we discuss elsewhere in this report, there is no single outcome metric that adequately measures the contributions of the very diverse set of programs that the department runs in the broad area of psychological health. This means that, practically speaking, it is impossible to directly compare all such interventions on an equal footing. Rather, evidenceguided judgment must be used to determine the optimal mix of programs and associated outcomes sought by the department. However, measuring outcomes and then estimating the cost of achieving each program’s outcomes gives the department substantial additional insight into the value of the various interventions, thereby aiding it in making decisions about resource allocation. Thus, the utilization of the same validated assessments for high-priority target outcomes that are in common (e.g., PTSD prevention goals and measurement with the PTSD Checklist) for two or more interventions supports cost-effectiveness calculations and improved information for decisions about which program is better in terms of the next dollar invested. When possible, DOD should evaluate program costs and hence cost-effectiveness using standardized metrics as part of its standard evaluation of interventions’ performance measures.

Measurement Framework with Examples

The development of performance measures for evidence-based prevention efforts will relate to the particular intervention being implemented and also be context specific. In this section, the committee offers measure examples to illustrate the measure concepts that are broadly applicable and essential to the systematic assessment of prevention programs. The measure examples reflect the goals of SAMHSA’s quality framework, the National Behavioral Health Quality Framework (NBHQF), as applied to three illustrative programs that provide universal, selective, and indicated prevention. The NBHQF defines six program goals that are useful for considering the quality of a prevention program as follows (SAMHSA, 2013b). A prevention program should be:

1. Effective—Promote the most effective prevention, treatment, and recovery practices for behavioral health disorders

2. Person centered—Assure that behavioral health care is person, family, and community centered

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

3. Coordinated—Encourage effective coordination within behavioral health care and between behavioral health care and community-based primary care providers, and other health care, recovery, and social support services

4. Healthy living—Assist communities to utilize best practices to enable healthy living

5. Safe—Make behavioral health care safer by reducing harm caused in the delivery of care

6. Affordable/accessible—Foster affordable high-quality behavioral health care for individuals, families, employers, and governments by developing and advancing new and recovery-oriented delivery models

These aspects of quality can be assessed within the structure, process, and outcomes domains, as described in the section below. These examples are not meant to include an exhaustive list of possible measures, but rather to illustrate some of the possibilities. With each example, the aspect of quality that the measure taps is noted in parentheses. Note, however, that there is overlap and that a measure may tap multiple domains.

Illustrative Example of Measures for Universal Prevention

The first example is of a media campaign designed to be delivered Army-wide to reduce suicides.

Structural Measures

•   Use of a campaign with evidence of positive impact on suicide targets, or development of one based on related campaigns by a qualified campaign developer (effective)

•   Consumer input from the target population (patient centered)

•   Consistent messaging across other Army suicide prevention efforts (coordinated)

•   Planning includes community suicide prevention partners (coordinated)

•   Consideration of unintended effects (safe)

Process Measures

•   Timing of advertisements (effective/accessible)

•   Target market(s) (patient centered/accessible)

•   Number of advertisements (accessible)

•   Cost of program (affordable)

Proximal Outcomes

•   Change in knowledge of and attitudes about suicide signs, help-seeking behaviors, help giving behaviors (effective)

•   Number who recognize the campaign message (accessible)

•   Number who saw advertisements (accessible)

Distal Outcomes (long-term population measurement to include follow-up)

•   Change in rate and type of help seeking for suicidal ideation (multiple aspects)

•   Change in rate of suicide attempts (multiple aspects)

•   Change in rate of deaths by suicide (multiple aspects)

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Illustrative Example of Measures for Selective or Indicated Prevention

The second example is of a youth mentoring program designed to be delivered to youth experiencing parental deployment.

Structural Measures

•   Use of a program with evidence of positive impact on youth targets, or development of one based on related program by a qualified program developer (effective)

•   Consumer input from youth and mentors (patient centered)

•   Attention to reduction of logistical barriers to participation such as timing and transportation during deployment (patient centered/accessible)

•   Planning includes community partners such as schools (coordinated)

Process Measures

•   Knowledge and attitudes of mentors following training (effective/accessible)

•   Type and degree of supervision of mentors (effective, safe)

•   Number of youth involved in mentoring program (accessible/affordable)

•   Number and type of mentoring contacts (patient centered, accessible)

•   Number of contacts between mentor and school (coordinated)

•   Youth and mentor satisfaction with program (patient centered)

•   Number and reason for drop-outs from the program (patient centered)

•   Cost of program (affordable)

Proximal Outcomes

•   Coping behaviors in youth during deployment cycle (effective, healthy living)

•   Social support in youth during deployment cycle (effective, healthy living)

•   Risky behaviors in youth during deployment cycle (effective, healthy living)

Distal Outcomes

•   Youth adjustment (effective)

•   Youth anxiety (effective)

As demonstrated by these examples, there may not be measures that address every framework category or domain. In addition, judgment needs to be applied to the mapping of measures to framework categories, and measures may address more than one category. Nonetheless, the framework provides a useful and systematic approach to the development and selection of measures to assure the quality and effectiveness of DOD prevention programs.

MEASURES IDENTIFIED IN REVIEW OF NATIONAL QUALITY INITIATIVES

This final section presents the findings from the committee’s review of existing performance measures from three national efforts to organize, manage, and promulgate the use of performance measures: the NBHQF, developed by SAMHSA (2013b); the NQF Quality Positioning System (NQF, 2013); and the National Quality Measures Clearinghouse, maintained by the Agency for Healthcare Research and Quality (AHRQ) (AHRQ, 2013). These sources consolidate measures that have been developed by various entities in the health field. In general,

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

the committee found that the relevant measures from these sources are primarily clinically focused screening measures that do not sufficiently address all of the domains relevant to prevention as defined by the NBHQF. As discussed earlier in this chapter, the committee believes there are many opportunities for expanding measurement to better assess the domains relevant to resilience and prevention for good psychological health.

The measures identified in this review are organized by topic area, with a brief summary of the rationale for the measures of each topic. Measure tables provide the following details: the measure title; whether it is endorsed by NQF; the NBHQF category addressed, if applicable; measure type (i.e., structure, process, or outcome); measure description; and the name of the organization that owns or developed the measure. These tables provide only general descriptions. The measure developers and owners or the NQF maintain the complete measure specifications (e.g., definitions of numerators, denominators, exclusion criteria, data sources, etc.), which are necessary for standardized implementation and reporting.

It is worth noting that in 2012, a previous IOM committee (IOM, 2013a) searched performance measures in the AHRQ measures clearinghouse and identified several screening measures associated with the performance management program in the Department of Veterans Affairs (VA). However, when this committee searched the AHRQ measures clearinghouse for this report, it did not find any VA measures; therefore, it appears that VA is no longer reporting its measures to AHRQ. The VA screening measures found in the previous committee’s search are included here because of their relevance to this committee’s review, but the committee is unsure of their current implementation status within VA.

Depression (Adult Population)

In the measure sets it reviewed (see Table 5-1), the committee found several measures specific to different aspects of early intervention (indicated prevention) for depression in adults. Although each measure has a unique algorithm (e.g., a definition of what to count, a specification of how to count, and a definition of the appropriate population denominator), the committee notes that the selection of a standardized, validated instrument to assess depression (e.g., PHQ-9 or a variant) is common to all the measures. After examining measure sets from multiple sources, the committee found two categories of measures: process and outcome. The process measures address depression screening. Evidence supports the practice of screening for depression in primary care as a population prevention initiative (USPSTF, 2009; Yano et al., 2012). To screen for depression, DOD uses the PHQ-2 in its deployment health assessments, and the Army recently introduced its use in the RESPECT–Mil demonstration. In deployment health assessments, clinicians are supposed to administer the PHQ-84 to further assess service members who have positive PHQ-2 screens. In the RESPECT–Mil program, service members who have positive PHQ-2 screens complete the PHQ-9 (IOM, 2013a).

____________________

4 DOD uses a variation of the PHQ-9, the PHQ-8, which omits the suicide-screening item because suicide screening and assessment do not occur until the diagnostic interview.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

TABLE 5-1 Depression Measures in Adults


Title NQF Endorsed (measure #) NBHQF Recommended /Category Type Description Owner/Developer

Screening for Clinical Depressiona Yes (#0418) Yes/Effective Process Percentage of patients aged 18 years and older screened for clinical depression using a standardized tool and follow-up plan documented. Centers for Medicare & Medicaid Services
Depression Utilization of the PHQ-9 Toola Yes (#0712) Yes/Effective Process Adult patients age 18 and older with the diagnosis of major depression or dysthymia (ICD-9 296.2x, 296.3x, or 300.4) who have a PHQ-9 tool administered at least once during the 4-month measurement period. MN Community Measurement
Depression Remission at 6 Monthsa Yes (#0711) Yes/Effective Outcome Adult patients age 18 and older with major depression or dysthymia and an initial PHQ-9 score > 9 who demonstrate remission at six months defined as a PHQ-9 score less than 5. This measure applies to patients with either newly diagnosed or existing depression whose current PHQ-9 score indicates a need for treatment. MN Community Measurement
Depression Remission at 12 Monthsa Yes (#0710) Yes/Effective Outcome Adult patients age 18 and older with major depression or dysthymia and an initial PHQ-9 score > 9 who demonstrate remission at 12 months defined as a PHQ-9 score less than 5. This measure applies to patients with either newly diagnosed or existing depression whose current PHQ-9 score indicates a need for treatment. MN Community Measurement
Depression: Screeningb No No Process Percentage of eligible patients who are screened annually for depression with the PHQ-2 or PHQ-9. VA
Depression: Screening Documentationb No No Process Percentage of veterans who have a positive screen during their annual depression screening and have a disposition documented in the record. A disposition is defined as a timeline for care; an arrangement for treatment, such as a mental health appointment; or the giving of instructions to the patient. VA
Depression: Timely Screening Documentation No No Process Percentage of veterans who have a positive screen during their annual depression screening and have a disposition documented in the record and have timely documentation of the disposition, which is defined as completion of the disposition by the next calendar day after a positive screen. VA

a NQF, 2013.

b IOM, 2013a.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

The Minnesota Community Measurement has developed outcome measures for depression that address relapse prevention. However, as an indication of the challenge of implementing outcome measures, the measure steward reports that only about 20 percent of the patients eligible for inclusion in the denominator of remission at 6 or 12 months currently have record of a follow-up PHQ-9 score. The definition of remission is a PHQ-9 score <5. The relapse measures promote ongoing contact between the patient and provider as, in this specification, patients who do not have a follow-up PHQ-9 score at 6 months and 12 months are also included in the denominator (and hence lower the observed rate of remission).

PTSD (Adult Population)

There are no measures addressing PTSD in the NBHQF, NQF, and AHRQ measure sets reviewed for this report (PTSD is viewed as infrequent in the general population and thus not appropriate for population-wide screening), but a previous IOM committee (2013a) identified three PTSD screening-related process measures that VA uses (see Table 5-2). Evidence suggests that identifying PTSD in this high-risk population and providing people who score positive with additional treatment can decrease symptoms as well as lessen the severity of functional impairment (VA and DOD, 2010). Two PTSD instruments are used for different purposes within DOD and VA. To screen for PTSD, DOD uses the Primary Care PTSD Screen (PC–PTSD), a four-item screen adapted from the PTSD Checklist, for both its deployment health assessments and the RESPECT–Mil program (IOM, 2013a). The PTSD Checklist is used selectively in DOD to confirm positive PC–PTSD scores and used by the VA to screen and selectively monitor the outcome of treatment.

TABLE 5-2 PTSD Measures


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

PTSD: Screeninga No No Process Percentage of eligible patients who are screened for PTSD at required intervals. VA
PTSD: Screening Documentationa No No Process Percentage of veterans who have positive PC–PTSD screens whose disposition is documented in the record. A disposition is defined as a timeline for care; an arrangement for treatment, such as a mental health appointment; or giving of instructions to a patient. VA
PTSD: Timely Screening Documentationa No No Process Percentage of veterans who have positive PC–PTSD screens whose disposition is both documented in the record and timely. Timely is defined as completion of the disposition by the next calendar day after a positive screen. VA

a IOM, 2013a.

Suicide Risk (Adult Population)

In the measure sets reviewed (see Table 5-3), the committee identified one NQFendorsed process measure for conducting suicide risk assessment in persons who have tested

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

positive for major depression as well as several VA process-of-care measures for suicide assessment (selective prevention). It should be noted, however, there is no widely accepted, scientifically validated tool to assess suicide risk directly. Thus, the assessment process that qualifies for the measure is not standardized. In the DOD suicide risk assessments are done routinely as part of the department’s periodic health assessments, and in post-deployment health assessments interviewers ask about how often a person thinks about harming himself or herself and about harming others; if the service member reports having such thoughts frequently, the interviewer escorts the service member to a health provider (IOM, 2013a). This process measure applies only to a selected population rather than to the whole population because the U.S. Preventive Services Task Force (USPSTF) found no evidence that universal screening for suicide risk among primary care patients (that is, among individuals without a current mental health disorder or history of mental illness) is an effective strategy for reducing the number of suicide attempts or the level of suicide mortality in the general population (USPSTF, 2004, 2013b).5 The measures in the table below apply to a selected population—either adults diagnosed with major depression or adults with a positive PHQ screen or adults with a positive PC-PTSD screen.

TABLE 5-3 Measures of Suicide Risk for Adults


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Major Depressive Disorder: Suicide Risk Assessmenta Yes (#0104) Yes/Effective, Safe Process Percentage of patients aged 18 years and older with a new diagnosis or recurrent episode of MDD who had a suicide risk assessment completed at each visit during the measurement period. American Medical Association—Physician Consortium for Performance Improvement
Suicide Risk Assessment: Positive PHQ Screensb No No Process Percentage of patients who screen positive on a screen for major depressive disorder (PHQ-2 or PHQ-9 or endorsement of question 9 on the PHQ-9) and have a suicide-risk evaluation completed within 24 hours. VA
Suicide Risk Assessment: Positive PC–PTSD Screenb No No Process Percentage of eligible patients who screen positive on the PTSD screen (PC–PTSD) and who have a suicide-risk evaluation completed within 24 hours. VA
Suicide Risk Assessment: Positive PHQ Screens or PC–PTSD Screenb No No Process Combines the populations of the first two measures to assess the percentage of patients who screen positive for major depressive disorder (on the PHQ-2 or PHQ-9) or positive for PTSD (on the PC–PTSD) and who have a suicide-risk evaluation completed within 24 hours. VA

a NQF, 2013.

b IOM, 2013a.

____________________

5 The draft Recommendation Statement is not USPSTF’s final recommendation; USPSTF distributed the draft solely for the purpose of pre-release review.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Alcohol Screening and Treatment Engagement (Adult Population)

For alcohol use—and, in some cases, other drug use—the measure sets reviewed by the committee (Table 5-4) contained a series of process measures, ranging from screening and brief counseling to engagement in treatment services and ease of access. In 2004 the USPSTF found good evidence that universal screening in primary care can accurately identify adult patients whose alcohol consumption elevates the risk of morbidity and mortality. The USPSTF also found good evidence that brief behavioral counseling in primary care produces small to moderate reductions in alcohol consumption (HHS, 2004). There is as yet no evidence demonstrating the effectiveness of universal screening for other drug use (e.g., marijuana) in primary care populations, primarily because the incidence level is low in most population groups. In DOD, routine screening of alcohol use with the AUDIT-C is expected at periodic health assessments, but the committee found no reports to confirm that this is happening. The AUDIT-C is contained on the deployment health assessments, but there is no requirement to offer brief counseling to service members with positive screens. DOD does not mandate alcohol screening and brief counseling in RESPECT–Mil or other primary care settings. During the past two decades DOD has operated a commander-driven mandatory drug testing program on the belief it is an effective deterrent (prevention policy) for illicit drug use (IOM, 2013b). In recent years the panel of drugs that are screened has been expanded to include prescription opioid formulations; other drugs on the panel include marijuana, cocaine, heroin, and certain amphetamines. Commanders are to refer those with positive drug tests for further assessment by the service’s specialty alcohol and drug treatment program and to begin procedures for administrative discharge.

TABLE 5-4 Measures of Alcohol Use and Abuse


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Alcohol Use: Screeninga No No Process Percentage of eligible patients who are seen in outpatient or inpatient settings and screened annually for alcohol misuse with the three-item AUDIT-C. VA
Alcohol Misuse: Brief Counselinga No No Process Percentage of veterans who are screened for alcohol misuse with AUDIT-C and meet or exceed a threshold score of 5 and who have timely brief alcohol counseling. The VA/DOD guideline indicates that although AUDIT-C scores greater than 4 points in men and greater than 3 points in women mean a positive screen, an AUDIT-C score of 5 or more is appropriate for performance measurement in a setting where brief alcohol counseling is required for everyone who screens positive for alcohol misuse. VA
Initiation and Engagement of Alcohol and Other Drug (AOD) Dependence Treatmentb Yes (#0004) No Process The percentage of adolescent and adult members with a new episode of alcohol or other drug (AOD) dependence who received
  • Initiation of AOD Treatment. The percentage of members who initiate treatment through an inpatient AOD admission, outpatient visit, intensive
National Committee for Quality Assurance

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

 

encounter or partial hospitalization within 14 days of the diagnosis.

  • Engagement of AOD Treatment. The percentage of members who initiated treatment and who had 2 or more additional services with a diagnosis of AOD within 30 days of the initiation visit.
National Committee for Quality Assurance
Average Time from First Request to First Client Treatment Sessionc No Effective—for considerationd Process Measures the time elapsed between the date a client first contacts the agency requesting service and the date the client received his or her first treatment session, which can be an individual or group session. Waiting time represents the average time for a specific client population and will be calculated as follows: Sum across all clients (date of first treatment–date of first contact)/(number of clients who receive a first treatment session) NIATx
No-Showsc No Effective—for considerationd Process Measures the number of patients who do not keep an appointment. The No-Show measure attempts to determine the number of clients who schedule a clinical assessment but fail to keep that appointment. Within NIATx, agencies typically look at client no-shows on a monthly basis, which is then calculated as follows: (number of clients with an assessment)/(number of clients who schedule an assessment appointment) NIATx
Continuationc No Effective—for considerationd Process Continuation measures the number of clients who attend 4 additional units of services (i.e., treatment sessions) within 30 days of admission to treatment. How a unit of service is defined varies by level of care. NIATx

a IOM, 2013a.

b NQF, 2013.

c SAMHSA, 2013b.

d Defined by NBHQF as measures that are either critical to broad measurement but not rising to the level of a core measure; promising but have not been tested or otherwise subjected to a consensus discussion and selection process; represent a specific level of granularity; or have emerged from the stakeholder review process.

Outside DOD other treatment process measures have been adopted by the National Committee for Quality Assurance and NIATx which focus on engagement and ease of access. NIATx is an example of a system quality-improvement approach applied to substance use disorder facilities which typically are not covered by other quality-assurance programs. NIATx

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

promotes practice and system change in order to improve access to and retention in treatment using a simplified version of the Institute for Healthcare Improvement Model for Improvement (IOM, 2013b).

Domestic Violence (Adult and Adolescent Populations)

The AHRQ database of measures includes a set of screening measures for assessing interpersonal violence (IPV). Studies show that assessing IPV in medical settings has been effective in identifying victims and that patients are not offended when asked about current or past IPV (Family Violence Prevention Fund, 2004; Moyer, 2013). USPSTF recommends that clinicians screen women of childbearing age—including those who do not have signs or symptoms of abuse—for IPV and that they refer women who screen positive to intervention services (Moyer, 2013).

Futures Without Violence (FWV) national consensus guidelines for domestic violence offers specific recommendations for assessing for and responding to IPV that may be applied to multiple health settings. There are eight process measures that may be used to assess compliance with the clinical recommendations outlined in the FWV guideline (see Table 5-5) (Family Violence Prevention Fund, 2004).

TABLE 5-5 Measures for Domestic Violence


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Domestic Violence: Disclosurea No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting within the past year and who were screened for IPV who disclosed that they were victims of abuse. Futures Without Violence
Domestic Violence: Assessmenta No No Process Percentage of adult and adolescent patients seen by a provider who received health care services in the clinical setting within the last year who were assessed* for IPV during the past year.
*See FWV guideline for IPV victimization assessment questions.
Futures Without Violence
Domestic Violence: Information and Referralsa No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting who screened negative for current or past IPV but whom the provider is still concerned may be a victim of IPV who were offered information about IPV and referrals. Futures Without Violence
Domestic Violence: FollowUp Questionsa No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting who screened negative for current or past IPV but whom the provider is still concerned may be a victim of IPV whose records include prompts for specific follow-up questions about IPV to occur at Futures Without Violence

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

  the patient's next visit.  
Domestic Violence: Suicide and Homicide Assessmenta No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting who screened positive for current or past IPV and who answered yes to initial danger assessment questions* for whom records indicate that a suicide and homicide assessment was conducted.
*See FWV guideline for IPV victimization assessment questions.
Futures Without Violence
Domestic Violence: Specified Assessmentsa No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting who assessed positive for current or past IPV for whom records indicate that the following assessments* were conducted:
Immediate safety and initial danger Abuse history (severity and extent) Impact of abuse on health issues and presence of related health care issues.
*See FWV guideline for IPV victimization assessment questions.
Futures Without Violence
Domestic Violence: Specified Intervention and Treatment Plansa No No Process Percentage of adult and adolescent patients who received health care services in the clinical setting who screened positive for current or past IPV for whom records indicate that intervention and treatment plans were offered including
  • Verbal and/or written information about safety planning (current victims only)
  • An option to talk with an advocate in person or on the phone (current victims only)
  • Verbal and/or written information about abuse and its impact on health
  • Referrals to culturally and linguistically appropriate services (when available)
  • A review of discharge instructions and a scheduled follow-up appointment or care plan with a mental health, social worker or community based service provider.
Futures Without Violence
Domestic Violence: Provider Compliancea No No Process Percentage of providers of health care services to adult and adolescent patients in the clinical setting who documented that they complied with assessment protocols.*
*See FWV guideline for IPV victimization assessment questions.
Futures Without Violence

a AHRQ, 2013.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Developmental Screening (Child and Adolescent Populations)

NQF has endorsed a number of process measures related to developmental screening (see Table 5-6). Developmental screening is defined as a standardized tool that assesses a child’s risk for developmental, behavioral, and social delays. The American Academy of Pediatrics recommends standardized screening using an approved screening tool as the best method of identifying children at risk for developmental, behavioral, or social delays (NQF, 2013).

TABLE 5-6 Measures for Developmental Screening


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Developmental screening using a parent-completed screening tool (parent report, children 0–5)a Yes (#1385) No Process The measure assesses whether the parent or caregiver completed a developmental screening tool meant to identify children at risk for developmental, behavioral, and social delays. The items are age-specific and anchored to parent-completed tools. (A majority of health care providers implementing the Bright Futures recommendations for standardized screening for all children utilize parent-completed tools due to their validity and feasibility.) The age-specific items assess whether children 10 to 71 months are screened. Maternal and Child Health Bureau, Health Resources and Services Administration
Developmental Screening in the First 3 Years of Lifea Yes (#1399) No Process The percentage of children ages 1, 2, and 3 years who had a developmental screening performed. Three rates are reported: Rate 1: developmental screening by the child’s first birthday; Rate 2: developmental screening by the child’s second birthday; Rate 3: developmental screening by the child’s third birthday. National Committee for Quality Assurance
Developmental Screening in the First 3 Years of Lifea Yes (#1448) No Process The percentage of children screened for risk of developmental, behavioral, and social delays using a standardized screening tool in the first 3 years of life. This is a measure of screening in the first 3 years of life that includes 3 age-specific indicators assessing whether children are screened by 12 months of age, by 24 months of age, and by 36 months of age. National Committee for Quality Assurance; Child and Adolescent Health Measurement Initiative
Promoting Healthy Development Survey (PHDS) a Yes (#0011) No Process PHDS assesses national recommendations for preventive and developmental services for young children. The PHDS is a survey of parents or guardians of children 3 to 48 months of age. Information is gathered on the following issues:  

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

 
  • Anticipatory guidance and parental education by a doctor or other health provider.
  • Health information.
  • Developmental surveillance: Ask about and address parents´ concerns about their child´s learning, development, and behavior.
  • Standardized screening for developmental, behavioral, and social problems.
  • Follow-up for children at risk for developmental, behavioral, or social problems.
  • Assessment of psychosocial well-being and safety in the family.
  • Assessment of smoking, drug, and alcohol use in the family.
  • Family-centered care (experience of care).
  • Helpfulness and effect of care provided.
 

a NQF, 2013.

Risky Behavior (Child and Adolescent Populations)

NQF has endorsed three process measures that assess whether children and adolescents receive preventive counseling and screening (see Table 5-7). Studies demonstrate that adolescents trust health care providers and are willing to talk with providers about recommended preventive counseling and screening topics, especially during private, confidential health care visits (National Quality Measures Clearinghouse, 2013b). Yet, few adolescents receive recommended comprehensive preventive counseling and screening services on key topics such as alcohol use, depression, sexual activity, smoking, injury prevention, physical activity, and diet (National Quality Measures Clearinghouse, 2013b).

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

TABLE 5-7 Risky Behavior Measures for Children and Adolescents


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Risky Behavior Assessment or Counseling by Age 13 Yearsa Yes (#1406) Yes/Healthy Living Process Percentage of children with documentation of a risk assessment or counseling for risky behaviors by 13 years of age. Four rates are reported: risk assessment or counseling for alcohol use, risk assessment or counseling for tobacco use, risk assessment or counseling for other substance abuse, and risk assessment or counseling for sexual activity. National Committee for Quality Assurance
Risky Behavior Assessment or Counseling by Age 18 Yearsa Yes (#1507) Yes/Healthy Living Process Percentage of children with documentation of assessment or counseling for risky behaviors by 18 years of age. Four rates are reported: risk assessment or counseling for alcohol use, risk assessment or counseling for tobacco use, risk assessment or counseling for other substance abuse, and risk assessment or counseling for sexual activity. National Committee for Quality Assurance
Young Adult Health Care Survey (YAHCS)a Yes (#0010) No Process YAHCS is a survey of adolescents 14 to 18 years of age that assesses how well the health care system provides adolescents with recommended preventive care. YAHCS assesses the provision of private and confidential care, experience of care, helpfulness of care provided, and the following aspects of preventive care:
  • Preventive screening and counseling on risky behaviors, sexual activity and sexually transmitted diseases, weight, healthy diet, and exercise, emotional health, and relationship issues.
  • Private and confidential care.
  • Helpfulness of counseling.
  • Communication and experience of care.
  • Health information.
Oregon Health & Science University

a NQF, 2013.

Suicide (Child and Adolescent Populations)

In the measure sets reviewed, the committee identified one NQF-endorsed process measure for suicide screening in child and adolescent populations that, similar to adult measures, is for indicated prevention and applies to children and adolescents diagnosed with depression

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

(see Table 5-8). As mentioned in the section on adult populations, there is no widely accepted and scientifically validated tool for screening for suicide risk.

TABLE 5-8 Measures for Suicide Risk for Children and Adolescents


Title NQF Endorsed (measure #) NBHQF Recommended/Category Type Description Owner/Developer

Child and Adolescent Major Depressive Disorder: Suicide Risk Assessmenta Yes (#1365) Yes/Safe Process Percentage of patient visits for those patients aged 6 through 17 years with a diagnosis of major depressive disorder with an assessment for suicide risk. American Medical Association

a NQF, 2013.

Depression (Child and Adolescent Populations)

NQF has endorsed three process measures relating to depression screening in child and adolescents (see Table 5-9). The USPSTF recommends screening for major depressive disorder (MDD) in adolescents (ages 12 to 18 years) when systems are in place to ensure accurate diagnosis, psychotherapy (cognitive-behavioral or interpersonal), and follow-up (USPSTF, 2013a).

Another area judged important to reduce child developmental and social emotional risk is maternal depression screening, either in prenatal or postpartum health care settings. Untreated maternal depression has been associated with negative pregnancy outcomes such as low birth weight and preterm labor as well as with negative effects on infants and toddlers such as developmental delay and cognitive impairment. Research has highlighted the negative impacts on fetal and infant development of both untreated maternal depression and antidepressant exposure. Between 14 and 23 percent of pregnant women and 10 to 15 percent of postpartum women will experience a depressive disorder (National Quality Measures Clearinghouse, 2013a).

In summary, the committee’s review of process measures related to prevention activities compiled by the NBHQF, NQF, and AHRQ found that the majority of measures relate to universal or indicated screening activities to support a limited range of prevention programming, specifically, identification of cases requiring treatment in primary care. A broader set of measurement domains are needed in order to assess the structure, process, and outcomes related to the full range of prevention programs and policies for service members and their families. The model for program development and measurement recommended by the committee and described in this chapter may be applied to develop a systematic assessment of the full range of prevention efforts.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

TABLE 5-9 Measures for Depression in Children and Adolescents and Maternal Depression


Title NQF Endorsed NBHQF Recommended/Category Type Description Owner/Developer

Depression Screening by 13 years of agea Yes (#1394) Coordinated—for considerationb Process The percentage of adolescents 13 years of age who had a screening for depression using a standardized tool. National Committee for Quality Assurance
Depression Screening by 18 years of agea Yes (#1515) Coordinated—for considerationb Process The percentage of adolescents 18 years of age who had a screening for depression using a standardized tool. National Committee for Quality Assurance
Maternal Depression Screeninga Yes (#1401) Yes/Effective Process The percentage of children 6 months of age during the measurement year who had documentation of a maternal depression screening for the mother. National Committee for Quality Assurance

a NQF, 2013.

b Includes measures that are either critical to broad measurement but not rising to the level of a core measure; promising but have not been tested or otherwise subjected to a consensus discussion and selection process; represent a specific level of granularity; or have emerged from the stakeholder review process.

CONCLUSION

As DOD advances its efforts to evaluate and improve psychological health programming for service members and their families, it faces a number of challenges, such as insufficient empirical evidence for many of the prevention programs it has implemented, no systematic use of national performance measures to assess current DOD screening programs, and the lack of a systematic process for selecting validated measures for use in judging performance of the structure, process, and outcomes of all prevention initiatives for enhancing psychological health. The measurement of performance is not as advanced in psychological health as it is in other types of care (Pincus et al., 2011; Watkins et al., 2010). Nonetheless, the DOD can focus its resources on creating a systematic approach to the measurement of structure, process and outcomes with reporting to an appropriately empowered oversight structure aimed at monitoring, selecting, and improving the quality of prevention initiatives for service members and their families.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2013. National Quality Measures Clearinghouse. http://www.qualitymeasures.ahrq.gov (accessed December 2, 2013).

Army Substance Abuse Program. 2013. Risk reduction program. http://acsap.army.mil/sso/pages/public/resources/rr-program.jsp (accessed December 13, 2013).

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Berwick, D. M., T. W. Nolan, and J. Whittington. 2008. The triple aim: Care, health, and cost. Health Affairs (Millwood) 27(3):759–769.

Chandra, A., S. Lara-Cinisomo, L. H. Jaycox, T. Tanielian, R. M. Burns, T. Ruder, and B. Han. 2010. Children on the homefront: The experience of children from military families. Pediatrics 125(1):16–25.

Connor, K. M., and J. R. Davidson. 2003. Development of a new resilience scale: The Connor-Davidson resilience scale (CD-RISC). Depression and Anxiety 18(2):76–82.

DCOE (Defense Centers of Excellence for Psychological Health & Traumatic Brain Injury). 2012. A Review of Post-Deployment Reintegration: Evidence, Challenges, and Strategies for Program Development. Arlington, VA: Defense Centers of Excellence for Psychological Health and Traumatic Injury.

De Pedro, K. T., R. A. Astor, R. Benbenishty, J. Estrada, G. R. Smith, and M. C. Esqueda. 2011. The children of military service members: Challenges, supports, and future educational research. Review of Educational Research 81(4):566–618.

Defense Health Agency. 2013. Office of the Chief Medical Officer. http://www.tricare.mil/tma/ocmo/default.aspx (accessed December 4, 2013).

DOD (Department of Defense). 2012. Department of Defense Health Care Quality Report to Congress. Washington, DC: Department of Defense.

Family Violence Prevention Fund. 2004. National Consensus Guidelines on Identifying and Responding to Domestic Violence Victimization in Health Care Settings. San Francisco, CA: Family Violence Prevention Fund.

Friborg, O., O. Hjemdal, J. H. Rosenvinge, and M. Martinussen. 2003. A new rating scale for adult resilience: What are the central protective resources behind healthy adjustment? International Journal of Methods in Psychiatry Research 12(2):65–76.

Goodman, R. 1997. The strengths and difficulties questionnaire: A research note. Journal of Child Psychology and Psychiatry 38(5):581–586.

Hazelden Foundation. 2013. Navy MORE. http://www.hazelden.org/web/public/pr091001.page (accessed November 11, 2013).

HHS (U.S. Department of Health and Human Services). 2004. United States Preventive Services Task Force: Screening and Behavioral Health Counseling Interventions in Primary Care Reduce Alcohol Misuse: Recommendation Statement. Rockville, MD: U.S. Department of Health and Human Services.

IOM (Institute of Medicine). 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series. Washington, DC: The National Academies Press.

———. 2012. An Integrated Framework for Assessing the Value of Community-Based Prevention. Washington, DC: The National Academies Press.

———. 2013a. Returning Home from Iraq and Afghanistan: Assessment of Readjustment Needs of Veterans, Service Members, and Their Families. Washington, DC: The National Academies Press.

———. 2013b. Substance Use Disorders in the U.S. Armed Forces. Washington, DC: The National Academies Press.

———. 2013c. Toward Quality Measures for Population Health and the Leading Health Indicators. Washington, DC: The National Academies Press.

Jellinek, M. S., J. M. Murphy, J. Robinson, A. Feins, S. Lamb, and T. Fenton. 1988. Pediatric symptom checklist: Screening school-age children for psychosocial dysfunction. Journal of Pediatrics 112(2):201–209.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Masten, A. S. 2001. Ordinary magic. Resilience processes in development. American Psychologist 56(3):227–238.

Moyer, V. A. 2013. Screening for intimate partner violence and abuse of elderly and vulnerable adults: U.S. Preventive Services Task Force recommendation statement. Annals of Internal Medicine 158(6):478–486.

National Center for PTSD. 2013. PTSD Checklist (PCL). http://www.ptsd.va.gov/professional/pages/assessments/ptsd-checklist.asp (accessed December 12, 2013).

National Quality Measures Clearinghouse. 2013a. Major Depression in Adults in Primary Care: Percentage of Perinatal Patients with Documentation of Screening for Major Depression. http://www.qualitymeasures.ahrq.gov/content.aspx?id=37727 (accessed December 2, 2013).

———. 2013b. Preventive Screening and Counseling on Risky Behaviors: Average Proportion Saying “Yes” to Ten Items About Whether Provider(s) Discussed/Screened on Smoking, Alcohol Use, Helmet Use, Drunk Driving, Chewing Tobacco, Street Drugs, Steroid Pills, Sexual/Physical Abuse, Violence, Guns. http://www.qualitymeasures.ahrq.gov/content.aspx?id=27446 (accessed December 2, 2013).

NQF (National Quality Forum). 2013. Measures, Reports, and Tools. https://www.qualityforum.org/Measures_Reports_Tools.aspx (accessed November 10, 2013).

NRC (National Research Council) and IOM. 2009. Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities. Washington, DC: The National Academies Press.

Pfizer. 2013. Instructions for Patient Health Questionnaire (PHQ) and GAD-7 Measures. http://www.phqscreeners.com/instructions/instructions.pdf (accessed December 13, 2013).

Pincus, H. A., B. Spaeth-Rublee, and K. E. Watkins. 2011. The case for measuring quality in mental health and substance abuse care. Health Affairs 30(4):730–736.

Resnik, L., M. Plow, and A. Jette. 2009. Development of CRIS: Measure of community reintegration of injured service members. Journal of Rehabilitation Research and Development 46(4):469–480.

Resnik, L., M. Gray, and M. Borgia. 2011. Measurement of community reintegration in sample of severely wounded servicemembers. Journal of Rehabilitation Research and Development 48(2):89–102.

Resnik, L., F. Tian, P. Ni, and A. Jette. 2012. Computer-adaptive test to measure community reintegration of veterans. Journal of Rehabilitation Research and Development 49(4):557–566.

SAMHSA (Substance Abuse and Mental Health Services Administration). 2013a. AUDIT-C Overview. http://www.integration.samhsa.gov/images/res/tool_auditc.pdf (accessed December 13, 2013).

———. 2013b. National Behavioral Health Quality Framework (Draft). Rockville, MD: SAMHSA.

Sayer, N. A., P. Frazier, R. J. Orazem, M. Murdoch, A. Gravely, K. F. Carlson, S. Hintz, and S. Noorbaloochi. 2011. Military to civilian questionnaire: A measure of postdeployment community reintegration difficulty among veterans using Department of Veterans Affairs medical care. Journal of Traumatic Stress 24(6):660–670.

Schogol, J. 2013. Critics: Climate assessments fall short. Air Force Times. http://www.airforcetimes.com/article/20130806/NEWS/307300032 (accessed November 21, 2013).

Smith, B. W., J. Dalen, K. Wiggins, E. Tooley, P. Christopher, and J. Bernard. 2008. The Brief Resilience Scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine 15(3):194–200.

Spitzer, R. L., K. Kroenke, J. B. Williams, and B. Lowe. 2006. A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine 166(10):1092–1097.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

Spoth, R. L., K. A. Kavanagh, and T. J. Dishion. 2002. Family-centered preventive intervention science: Toward benefits to larger populations of children, youth, and families. Prevention Science 3(3):145–152.

USPSTF (U.S. Preventative Services Task Force). 2004. Screening for Suicide Risk. http://www.uspreventiveservicestaskforce.org/3rduspstf/suicide/suiciderr.htm (accessed November 21, 2014).

———. 2009. Screening for depression in adults: U.S. Preventive Services Task Force recommendation statement. Annals of Internal Medicine 151(11):784–792.

———. 2013a. Major Depressive Disorder in Children and Adolescents. http://www.uspreventiveservicestaskforce.org/uspstf/uspschdepr.htm (accessed December 2, 2013).

———. 2013b. Screening for Suicide Risk in Adolescents, Adults, and Older Adults: U.S. Preventative Services Task Force Recommendation Statement. http://www.uspreventiveservicestaskforce.org/uspstf13/suicide/suicidedraftrec.htm (accessed December 2, 2013).

VA (Department of Veterans Affairs) and DOD. 2010. VA/DOD Clinical Practice Guideline for Management of Post-Traumatic Stress. Washington, DC: VA and DOD.

Walsh, F. 2006. Strengthening Family Resilience (2nd ed.). New York: Guilford Press.

Watkins, K. E., D. J. Keyser, B. Smith, T. E. Mannle, D. R. Kivlahan, S. M. Paddock, T. Mattox, M. Horvitz-Lennon, and H. A. Pincus. 2010. Transforming mental healthcare in the Veterans Health Administration: A model for measuring performance to improve access, quality, and outcomes. Journal for Healthcare Quality 32(6):33–42; quiz 42–43.

Weinick, R. M., E. B. Beckjord, C. M. Farmer, L. T. Martin, E. M. Gillen, J. D. Acosta, M. P. Fisher, J. Garnett, G. C. Gonzalez, T. C. Helmus, L. H. Jaycox, K. A. Reynolds, N. Salcedo, and D. M. Scharf. 2011. Programs Addressing Psychological Health and Traumatic Brain Injury Among U.S. Military Service Members and Their Families. Santa Monica, CA: RAND Corporation.

Windle, G., K. M. Bennett, and J. Noyes. 2011. A methodological review of resilience measurement scales. Health and Quality of Life Outcomes 9:8.

Yano, E. M., E. F. Chaney, D. G. Campbell, R. Klap, B. F. Simon, L. M. Bonner, A. B. Lanto, and L. V. Rubenstein. 2012. Yield of practice-based depression screening in VA primary care settings. Journal of General Internal Medicine 27(3):331–338.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×

This page intentionally left blank.

Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 135
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 136
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 137
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 138
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 139
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 140
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 141
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 142
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 143
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 144
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 145
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 146
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 147
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 148
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 149
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 150
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 151
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 152
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 153
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 154
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 155
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 156
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 157
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 158
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 159
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 160
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 161
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 162
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 163
Suggested Citation:"5 MEASUREMENT OF DEPARTMENT OF DEFENSE PREVENTION INTERVENTIONS RELATED TO PSYCHOLOGICAL HEALTH." Institute of Medicine. 2014. Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs. Washington, DC: The National Academies Press. doi: 10.17226/18597.
×
Page 164
Next: 6 FINDINGS AND RECOMMENDATIONS »
Preventing Psychological Disorders in Service Members and Their Families: An Assessment of Programs Get This Book
×
Buy Paperback | $58.00 Buy Ebook | $46.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Being deployed to a war zone can result in numerous adverse psychological health conditions. It is well documented in the literature that there are high rates of psychological disorders among military personnel serving in Operation Enduring Freedom in Afghanistan and Operation Iraqi Freedom in Iraq as well as among the service members' families. For service members' families, the degree of hardship and negative consequences rises with the amount of the service members' exposure to traumatic or life-altering experiences. Adult and child members of the families of service members who experience wartime deployments have been found to be at increased risk for symptoms of psychological disorders and to be more likely to use mental health services.

In an effort to provide early recognition and early intervention that meet the psychological health needs of service members and their families, DOD currently screens for many of these conditions at numerous points during the military life cycle, and it is implementing structural interventions that support the improved integration of military line personnel, non-medical caregivers, and clinicians, such as RESPECT-Mil (Re-engineering Systems of Primary Care Treatment in the Military), embedded mental health providers, and the Patient-Centered Medical Home.

Preventing Psychological Disorders in Service Members and Their Families evaluates risk and protective factors in military and family populations and suggests that prevention strategies are needed at multiple levels - individual, interpersonal, institutional, community, and societal - in order to address the influence that these factors have on psychological health. This report reviews and critiques reintegration programs and prevention strategies for PTSD, depression, recovery support, and prevention of substance abuse, suicide, and interpersonal violence.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!