Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 291
Evaluating the HRSA Traumatic Brain Injury Program APPENDIX E Stakeholders Assess the HRSA TBI Program: A Report on National Interviews and Interviews in Seven States Holly Korda, M.A., Ph.D. Health Systems Research Associates Chevy Chase, Maryland Stakeholder interviews were conducted during the summer of 2005 in a sample of seven states: Alabama, California, Colorado, Georgia, New Jersey, Ohio, and Washington State. These states were selected to provide a cross section of state and program characteristics including: length of participation in the Health Resources and Services Administration (HRSA) Grant Program; maturity of the state’s Traumatic Brain Injury (TBI) program infrastructure; state funding levels and mechanisms such as Medicaid waivers and TBI trust funds; lead agency location in state government; program accomplishments; data availability; presence or absence of other programs, e.g., TBI Model Systems, TBI surveillance, and others; and geographic and cultural diversity. The Institute of Medicine (IOM) staff consultant conducted telephone interviews in Alabama, California, New Jersey, Ohio, and Washington State. The committee chair, study director, and staff consultant participated in 2-day site visits to conduct in-person interviews with stakeholders in Georgia and Colorado. Study respondents in each state were selected based on criteria developed by the IOM Committee, and include: TBI lead agency representative, protection and advocacy (P&A) system representative, state brain injury association representative, consumer or family member of an individual with TBI, lead injury prevention representative, and other stakeholder rep- Report submitted to the IOM Committee on Traumatic Brain Injury in fulfillment of consultant contract agreement, November 13, 2005.
OCR for page 292
Evaluating the HRSA Traumatic Brain Injury Program resentatives of key state agencies, TBI trust funds, Medicaid waivers, and related interests. State TBI lead agency representatives helped to identify appropriate representatives to be contacted in their states. National program stakeholders involved with the HRSA TBI Program were also contacted for interviews, including the HRSA TBI Program Director; Executive Director of the National Association of State Head Injury Administrators/Director of the TBI Technical Assistance Center (TBI TAC); President/CEO of the Brain Injury Association of America; and Executive Director of the National Disability Rights Network (formerly the National Association of Protection and Advocacy Systems, Inc.). Interviews were conducted using a semi-structured interview guide developed by the IOM Committee to address areas of importance relating to state program implementation; program impact on persons with TBI and their caregivers; and coordination of TBI-related services, including education, vocational rehabilitation, employment, housing, transportation, and mental and behavioral health care. The interview guide includes questions about each state’s history of TBI service delivery, and their experiences with each of the HRSA TBI Program grants: Planning grants allow states to build infrastructure through the TBI Program’s four core components—(1) establishing a TBI Statewide Advisory Board, (2) identifying a Lead Agency, (3) conducting a Needs and Resources Assessment, and 4) developing a TBI State Action Plan. Implementation Grants allow states to undertake activities, e.g., implementation of the State Action Plan or activities to address identified needs, to improve access for individuals with TBI and their families. Post-Demonstration Grants authorized by the Children’s Health Act of 2000 have been available to allow states that have completed 3 years of implementation to support specific activities that will help states build TBI capacity. Protection and Advocacy Systems Grants allow 57 states, territories, and the Native American Protection and Advocacy Project to assess their state P&A Systems’ responsiveness to TBI issues and provide advocacy support to individuals with TBI and their families. The interview guide also includes questions about states’ experiences with the TBI Technical Assistance Center and with the HRSA TBI Program grant structure and processes. Each interview was approximately 40–50 minutes in length. Stakeholders of the HRSA TBI Program interviewed for this study are represented by category in Figure E-1. Interviews were conducted with a total of 66 TBI stakeholders, including: national program directors (n=4), state brain injury associations (n=6), state injury prevention epidemiolo-
OCR for page 293
Evaluating the HRSA Traumatic Brain Injury Program FIGURE E-1 TBI study respondents. gists and staff (n=6), consumers with TBI or family members (n=7), state lead agency representatives (n=9), protection and advocacy systems staff (n=16), and others, including state agency and provider representatives (n=18). STAKEHOLDER INTERVIEW SUMMARY AND KEY FINDINGS I. TBI Grant History and Program Background State TBI grantees differ widely with regard to availability of human and organizational resources, historical context, and political leadership and commitment to TBI—all factors that appear to be related to their abilities to leverage HRSA TBI Program funding for TBI services and systems coordination. States with established, well-supported services and supports have been more successful leveraging HRSA funding than states with limited or no resources. States submitted grant applications to the HRSA TBI Program with different histories and resources for TBI services coordination and systems development. The HRSA grants to states have been competitively awarded since the first grant cycle in 1997. During the first 2 years of the HRSA TBI Program, Planning and Implementation grants were available as 1-year demonstration project awards. During these start-up years, as states and the HRSA Program gained experience with these grants, it became clear that longer funding cycles were needed to accomplish grant expectations. Funding availability for Planning grants
OCR for page 294
Evaluating the HRSA Traumatic Brain Injury Program was subsequently expanded to 2 years, and Implementation grant funding was expanded to 3 years. After reauthorization of the Federal TBI Program in 2000, Post-Demonstration grants were introduced and became available, as 1-year awards. The HRSA TBI grants to P&As were added and awarded as 1-year competitive grants in 2002. The following year, the P&A grants were changed to formula-based awards. In August 2005, the HRSA TBI Program introduced new, 3-year Partnership Implementation grants that replace HRSA’s other TBI grant programs to states. Planning grants were $75,000 for up to 2 years, Implementation grants have averaged $250,000 over 3 years, and Post Demonstration grants have averaged $100,000 over 1 year. The P&A grants start at a base average of $50,000 per year. The new Partnership Implementation grants are limited to $100,000 per year for a 3-year period. Grant histories for the seven study states are summarized in Table E-1, below. California, Colorado, Georgia, and Washington State had none of HRSA’s four core TBI Program components in place when they received their first grant awards. Colorado and Georgia reported previous, established programs for individuals with TBI in their states dating to the 1980s: Colorado’s federally-funded Rocky Mountain Regional Brain Injury Center (RMRBIC) and the Georgia Department of Labor’s comprehensive programs developed at Roosevelt Warm Springs Institute for Rehabilitation TABLE E-1 HRSA TBI Grants Program History: State Award Years Grant Type Alabama California Colorado Georgia New Jersey Ohio Washington Planning — 1999, 2001 1999 1997 — — 2000, 2001 Implementation 1997, 1998, 1999, 2000 — 2001, 2002, 2003 1998, 1999, 2000 1999, 2000, 2001 1998, 1999, 2000 2003, 2004, 2005 Post-Demonstration 2001, 2002, 2004 — 2004 2004 2002, 2003, 2004 2002, 2003, 2004 — Protection and Advocacy 2002, 2003, 2004, 2005 2003, 2004, 2005 2002, 2003, 2004, 2005 2003, 2004, 2005 2002, 2003, 2004, 2005 2002, 2003, 2004, 2005 2002, 2003, 2004, 2005
OCR for page 295
Evaluating the HRSA Traumatic Brain Injury Program and its Vocational Rehabilitation programs. California and Washington State also noted early efforts to address TBI through their states. California’s Department of Mental Health established Caregiver Resource Centers and a TBI trust fund, in the 1980s. Washington State developed demonstration projects addressing the needs of individuals with TBI. Each of these states has also hosted TBI Model Systems,1 funded by the National Institute on Disability and Rehabilitation Research (NIDRR). Colorado, Georgia and Washington State hosted Centers for Disease Control and Prevention (CDC) Core Injury Programs. Still, none of the four states had developed a sustainable TBI infrastructure. Each of these states applied for Planning grants to develop the HRSA TBI Program’s four core TBI program components. California applied for, but was not awarded, an Implementation grant. Washington is in its third year of its Implementation grant, and Colorado continued on to receive a Post-Demonstration grant after completing a 3-year Implementation grant. Alabama, Ohio, and New Jersey were well under way with state efforts to coordinate services to individuals with TBI and their families when these states applied for HRSA grant funding. Alabama, which established its State Head Injury Program for adults in 1989 and designated its State Head Injury Task Force and a coordinator in 1990, applied for HRSA funding to expand its Interactive Community-Based Model for adults with TBI to create a children’s system. Ohio applied to the HRSA TBI Program with an established Advisory Council in place. The state’s collaboration with the Ohio Brain Injury Association led to development of a strategy to develop a comprehensive model service coordination continuum that would be expanded with HRSA funding. New Jersey had services with several state agencies, but did not have a coordinating group in state government. Preparing for its next grant application after two rejected attempts, the state moved to designate the Department of Human Services, Office of Disabilities as lead agency, as well as an interagency advisory board. New Jersey’s first grant involved collaboration with an established partner, the Brain Injury Association of New Jersey. Alabama and Ohio had established CDC Core Injury Programs, and all three states hosted NIDRR-funded TBI Model Systems. All three states were served by Rehabilitation Services Administration grants for TBI Regional Centers, with those centers actually awarded to programs in Alabama and Ohio. The Rocky Mountain Regional Brain Injury Center mentioned above was also a TBI Regional Center. 1 Georgia’s TBI Model System involving The Shepherd Center and Emory University is no longer operational.
OCR for page 296
Evaluating the HRSA Traumatic Brain Injury Program II. HRSA TBI Grant Experience The seven study states used diverse approaches and met with varied results implementing their HRSA TBI Program grants. Key findings are summarized below. Planning Grants HRSA’s four core TBI program components—Lead Agency, Statewide TBI Advisory Board, Needs Assessment, and State Action Plan—have been embraced as helpful elements for coordination and collaboration around TBI at the state level. Although some states have shifted agency placement and contacts since their start-up Planning grants, there is general agreement that these four components are helpful for moving TBI services and systems change forward. States have conducted Needs and Resources Assessments and developed State Action Plans, and have used this information to guide TBI efforts for the HRSA grant, as required by the HRSA TBI Program. 1. Statewide TBI Advisory Board: Requirement to develop or demonstrate the existence of a Statewide TBI Advisory Board within the appropriate health department of the state or within another department as designated by the state’s chief executive officer. The Board’s composition must include representatives of the involved state agencies; public and private nonprofit health-related organizations; disability advisory or planning groups; members of an organization or foundation representing individuals with TBI; state and local injury control programs if they exist; and a substantial number of individuals with TBI or their family members. The requirements for, and states’ experiences developing, the four core components of the HRSA TBI Program are described in the following section. The four states that established TBI statewide Advisory Boards after receiving HRSA grants—California, Colorado, Georgia, and Washington State—identified two critical elements to this activity: having membership of a manageable size and including the right people as representatives. Stakeholders in each state struggled with these elements in building and, in some cases, rebuilding a board. One respondent who served as a member of Georgia’s early Advisory
OCR for page 297
Evaluating the HRSA Traumatic Brain Injury Program Board described it as “unwieldy, with 50–60 members.” Another added, “It’s hard to work with a group that large … They tried to do too much.” Georgia’s Advisory Board held together through the state’s Planning and Implementation grants, but disbanded after its Post-Demonstration grant request was denied. The state established a Brain and Spinal Cord Injury Trust Fund and Commission in 1998. The Commission has 14–15 members, including some members of the re-established TBI Advisory Board. The new board is much smaller and is reported to be more focused. Washington State also started with a large Advisory Board that has since been pared down. One respondent describes the challenges as “participation and coordination. People come and go. Forming a sustainable board was a challenge.” Defining its mission in the state was challenging, but important to the Board’s coming together as a group. “We were also asking ourselves, were we here for the Planning grant or another, broader mission? The planning group, our planning activities, gave the Advisory Board a focus.” In Colorado, where many key participants had established working relationships in the brain injury community predating the HRSA TBI Program, there were so many people who wanted to serve on the Advisory Board that people were turned away. This Board limited the number of participants from the start, facing the challenge of “making sure we had the right people at the table.” Establishing an Advisory Board in California was complicated by the diversity of members as well as the state’s vast geography. One respondent described the Board as “initially very divisive, there was no common goal. People came to the table with different agendas, the community was divided.” With help from a skilled consultant the group did come to consensus on a State Action Plan, working through its differences. However, the Advisory Board disbanded after the state’s application for an Implementation grant was denied. 2. Lead Agency: Designation of a state agency and a staff position responsible for coordination of state TBI activities. Finding the most appropriate fit for a state lead agency is a key issue for states in the early stages of developing their state TBI programs. The TBI Technical Assistance Center (TAC) reports that throughout the history of the HRSA TBI Program, lead agencies have been located in at least nine different state agencies, including Public Health, Human Services, Social Services, Medicaid, Rehabilitation Services, Mental Health, Developmental Disabilities, and Education.
OCR for page 298
Evaluating the HRSA Traumatic Brain Injury Program States often change lead agencies in search of an appropriate “home” for TBI in state government. Placement of the lead agency is closely tied to the political and programmatic leadership, commitment, and focus of the state’s TBI activities. Three of the four states that received Planning grants—California, Georgia, and Washington—changed lead agencies as they moved forward with their HRSA grants. California shifted its lead agency from Vocational Rehabilitation to the Department of Mental Health, although respondents in this state remain unclear why the agency remains in this location given the service needs of the state’s TBI clients and families. In Washington State the Vocational Rehabilitation agency served as the lead agency for the first two HRSA grant years, then determined that TBI was not its mission. Aging and Adult Services “agreed to take us on,” recalled one respondent. The state TBI program is now with Aging and Disability Services, which state respondents observe is a better fit. Georgia designated the State Health Planning Agency (SHPA) as the lead agency for its Planning and Implementation grants, but the agency did not have the capacity or commitment to move the program forward. Georgia’s brain injury association urged the state to apply for a Post-Demonstration grant in 2002, and approached the Department of Community Health (DCH) to serve as lead agency. The application was denied. The DCH, which addressed numerous programs not related to TBI, approached the Brain and Spinal Injury Trust Fund Commission, a relatively young agency with a focus on TBI and asked the Commission to take over its role as lead agency in applying for a new Post-Demonstration grant. The Commission currently serves as Georgia’s lead agency and has made considerable progress building a Central Registry with its HRSA grant. 3. Needs and Resources Assessment: Statewide needs and resources assessment, with an emphasis on resources, completed or updated within the last five years, of the full spectrum of care and services from initial acute treatment through community reintegration for individuals of all ages having TBI. The four states that conducted Needs and Resources Assessments with their HRSA Planning grants did not have state-specific data about their states’ TBI population and needs, and used different approaches to identify needs and resources to inform their programs. The HRSA TBI Program, through TBI TAC, provides a forum for states to share information about the methods they use, and allows states to select and develop their own approaches. Information from the assessments provides the basis for states to develop the State Action Plan required as part of the TBI Program core.
OCR for page 299
Evaluating the HRSA Traumatic Brain Injury Program Approaches used by the four sample states that developed Needs and Resources Assessments for their Planning grants are included below. California used multiple methods to obtain information for their Needs and Resources Assessment, including provider and public assessments, public meetings, and extensive networking throughout the state. Washington State conducted two assessments: one for providers, and an Internet-based assessment for families. One stakeholder recalls, “When DVR had this, there weren’t clear parameters about how to do a Needs Assessment,” suggesting that approved parameters for collecting information might help future grantees. Colorado’s Needs and Resources Assessment included interviews with state agencies to determine their awareness of and involvement in TBI issues, and their perceptions of TBI needs and resources; in-depth interviews with providers; community forums; and printed questionnaires distributed to members of brain injury support groups. Georgia developed and distributed surveys, conducted regional town hall meetings, surveyed case managers, and hosted a statewide Stakeholders’ Conference to determine concerns and service needs of TBI survivors and families. 4. TBI Statewide Action Plan: Development of a Statewide Action Plan to develop a culturally competent, comprehensive, community-based system of care that encompasses physical, psychological, educational, vocational, and social aspects of TBI services and addresses the needs of individuals with TBI as well as family members. TBI State Action Plans required by the HRSA TBI Program reportedly vary in both format and length. A TBI TAC respondent noted that the plans are difficult to compare state by state: some plans are brief, one-page lists of key issues and activities; others are lengthy, detailed documents. Most State Action Plans draw on information obtained in the Needs and Resources Assessment, but states also use other sources of information and Advisory Board deliberations to target priority areas for TBI services and systems development. The TBI TAC has worked with several states to help them achieve more focus in their State Action Plans. As a technical assistance activity, the most difficult aspect of the State Action Plan is “getting grantees to understand the Action Plan is a living, breathing document,” as a TBI TAC respondent observed.
OCR for page 300
Evaluating the HRSA Traumatic Brain Injury Program Often, states must refine or revisit their planning as resources and conditions in the state change. California reportedly made great strides pulling together different constituents in developing its State Action Plan, resulting in “a very democratic process … a very sensitive, comprehensive plan,” but “the bottom has dropped out without resources.” Washington State worked with TBI TAC after the state changed its lead agency, in an effort to refine the plan to a more manageable effort that was subsequently developed through the state’s Implementation grant. Implementation Grants Implementation grants are sometimes seen as a vehicle to maintain a state’s TBI infrastructure, and have been used to expand existing programs and initiate new projects, often by leveraging the state’s resources with partner organizations. State respondents credit the HRSA TBI Program with providing the funding and political motivation to continue the focus on TBI through program and project development in areas not likely to be initially funded by the state. Partnerships with Brain Injury Associations, universities, and other organizations helped states leverage grant resources and professional expertise. Six of the seven sample states received Implementation grants. These grants allow states to implement activities identified in their State Action Plans or other activities that address identified needs, to improve access for individuals with TBI and their families. Implementation grants were awarded for 1 year during the early years of the HRSA TBI Program. Following reauthorization of the TBI Act in 2000, these grants were extended to 3-year awards, a time frame seen as more appropriate to the scope of activities states addressed than the 1-year grants. One national respondent observed that states often “struggled deciding what to choose” as a focus of their Implementation grants, facing the need to maintain infrastructure as well as select from among many identified needs. The sample states reported different experiences with their Implementation grants. California applied but was denied an award. A state respondent explained, “HRSA felt we were biting off too much. One of the problems we had was, we had priorities identified and picked out … [Federal requirements] didn’t fit well with the state’s priorities. [The application] wasn’t well put together when we tried to add on these requirements.” Georgia applied for and received an Implementation grant after completing the four core components under its Planning grant. The state report-
OCR for page 301
Evaluating the HRSA Traumatic Brain Injury Program edly achieved only modest success completing tasks proposed under the grant, with an ambitious agenda, changes in states government, and difficulties pulling together the Lead Agency and Advisory Board. Georgia’s TBI Advisory Board disbanded after the Implementation grant was completed. State respondents noted, “ there’s not much to show” for the grant. New Jersey and Washington State involved strong partner organizations that enabled these states to leverage partners’ resources to address grant objectives. New Jersey entered the HRSA TBI Program after two applications had been denied, establishing its core TBI Program components during this downtime. The state received HRSA funding for its third application, an Implementation grant to develop the Supporting Families in Crisis (SFC) program in close partnership with the state brain injury association. The SFC focused on educating families and providers about TBI-related resources, increasing the numbers of minorities and non-English speakers who access services, and increasing the identification of children with TBI in schools. Washington State developed a strong collaboration with its TBI Model System, which worked with the P&A and other state agencies to develop TBI Tool Kits, videoconferences, and other materials. The TBI Model System shared board membership with the state’s TBI Advisory Board and was seen as “a natural partner … The Model System was doing good work with telehealth [videoconferences and distance learning], so there were good opportunities to get involved.” Alabama, Ohio, and Colorado used their grants to build on and expand existing plans for TBI services development. Alabama received Implementation grant funding to develop a statewide pediatric service delivery model, PASSAGES, as an expansion of the state’s Interactive Community-Based Model for adults with TBI. Ohio did not receive funding for its first application, but was awarded funding for an Implementation grant the following year. Ohio’s grant focused on developing four Community Support Networks (CSNs) to add to two CSNs already established as part of the state’s comprehensive “Ohio Plan: Building Ramps to the Human Service System for People with Brain Injury.” Colorado worked closely with its state brain injury association and other stakeholders to continue Colorado Information, Resource, Coordination, Linkage, and Education (CIRCLE) programs that convened providers and stakeholders for information sharing and referral. Other grant activities included increasing availability of information statewide, addressing the needs of children with TBI through development of a training manual (BrainSTARS) and training materials for parents and school personnel, and increasing awareness of state agency personnel about brain injury and to identify and change barriers to effective service coordination.
OCR for page 311
Evaluating the HRSA Traumatic Brain Injury Program disabled … his dad and I both work … He’s never alone, I can’t leave him by himself.” Another consumer respondent emphasized the importance of insurance coverage in providing service access for persons with TBI and their family members, noting, “Medicaid provides options, private pay does … but others have a hard time.” IV. TBI Data, Monitoring, and Evaluation Data about TBI and TBI-related services in the study states are limited. Some state agencies collect data and track service utilization among individuals with TBI, but none of the study states reported comprehensive, cross-agency data monitoring. Several study states have established registries, which vary in scope and application, and some have conducted TBI surveillance with grant support from CDC. Respondents named lack of funding and expertise for data activities, as well as the challenges of how to monitor services and outcomes as obstacles to data-related activities. TBI Data Collection in Study States Registries and surveillance systems are the primary sources of TBI data in most states. Typically, injury prevention professionals in state health departments maintain and report these data, and report summary findings to state TBI Advisory Boards. States that do not conduct TBI surveillance rely on national estimates prepared by CDC. Registries and surveillance systems vary widely across states. According to an injury prevention specialist in one sample state, there is much confusion in the field about “what constitutes a ‘Registry’ and a ‘Surveillance System,’” and the terms are often used interchangeably. This respondent explained, “Registries usually have contact information … Surveillance is usually a data system without contact information, and is used to identify risk factors.” In practice, not all registries are used to contact individuals, and the quality and comprehensiveness of state survey data are not uniform across states. Registries. Three of the seven study states—Alabama, Georgia, and Washington State—have established TBI registries, and a fourth state, New Jersey, is involved in efforts to create a combined TBI/Spinal Cord Injury
OCR for page 312
Evaluating the HRSA Traumatic Brain Injury Program (SCI) registry. These registries were developed using different approaches and are used in various ways. Alabama’s registry was established with the passage of the Alabama Head Injury and Spinal Cord Injury Registry Act in 1998, which requires hospitals to submit their data to the state. While respondents provided differing assessments about the comprehensiveness of the registry data, state TBI program staff report having successfully integrated the registry as part of Alabama’s TBI services system, enabling followup of newly identified TBI cases. Georgia established its Central Registry for spinal cord injury in 1981, and expanded the Registry to include TBI in 1985. Expansion of the Central Registry has been the focus of the state’s 2004 Post-Demonstration grant under the leadership of Georgia’s Brain and Spinal Cord Injury Trust Fund Commission. Washington State maintains a TBI registry but does not conduct follow up of individuals. New Jersey stakeholders have been working for the past two years with the state’s Center for Health Statistics to develop registries for TBI and spinal cord injury. The registries were brought to life by the Christopher Reeve Foundation and the father of a TBI survivor. Surveillance. Four of the study states—Alabama, California, Colorado, and New Jersey—have conducted TBI surveillance with funding from the CDC; of these, only Colorado has received CDC funding for the current grant cycle. States use TBI surveillance data, which typically combines death certificate and hospital discharge data, to monitor trends and target prevention efforts. While this information can provide a basis for prevention-related public health monitoring, its utility is limited for services planning. One respondent explained, “ [The TBI surveillance system] captures incidence, or new injuries, which makes sense when the purpose is primary prevention. What service providers really want is point prevalence, that is, how many people live in the state who have had a TBI ever, and currently have a need for services?” There are a couple of ways to estimate prevalence, this respondent noted: “Some use incidence data as the starting point … However, to use incidence data one needs to know the life expectancy of people with TBI at various ages and gender and the probability of ongoing disability related to the TBI—or some other measure or definition of ‘need for services’ for various age groups and gender …” Other data collection. Several issues complicate states’ TBI data collection, monitoring, and evaluation efforts. Lack of knowledge about TBI out-
OCR for page 313
Evaluating the HRSA Traumatic Brain Injury Program comes and service use and difficulties obtaining funding to sustaining data collection efforts are among respondents’ top concerns. Respondents commented: “We don’t know a lot about outcomes regarding TBI. Do outcomes vary by ICD-9 code? Who are we designing services for? Different outcomes occur across the severity of TBI.” “We’re not good at measuring outcomes, mild TBIs, or projecting who will need what types of services. We don’t have good ways to measure information regarding TBI.” “Funding [for data collection and analysis] from several [federal] sources is drying up … we are constantly looking for grants.” “We scramble to get money together [for data collection and analysis] from wherever we can. They [federal and state government] keep cutting us.” “I’m not certain of the role of epidemiology in TBI systems change. Data alone do not create change … individuals make change. Data don’t take into account of values or politics.” TBI Data Activities in Protection and Advocacy Organizations The P&As reported variable capabilities for data collection to document and track their TBI activities. Some organizations have licensed a data system developed by their national association; others use in-house tracking systems. The P&As in the study sample recognized the challenges associated with identifying and tracking their TBI cases, as individuals with TBI approaching the P&As may be advised by phone, or seen in different established programs of their organizations. Anecdotally, all of the seven state P&As stated that individuals with TBI were now more visible as a result of the HRSA P&A grants. Colorado’s P&A, visited during a study site visit, was able to produce data demonstrating an increase in identified TBI cases addressed at the organization. The Colorado P&A uses an in-house tracking system, but plans to join the more than 40 state P&As that use the Disability Advocacy Database (DAD) operated and maintained by the National Disability Rights Network (NDRN, formerly the National Association of Protection and Advocacy Systems). Alabama’s P&A currently uses DAD. A P&A respondent explained, “I can ask for information on projects and cases easily. DAD is our whole record-keeping system. It includes information obtained at the phone in-
OCR for page 314
Evaluating the HRSA Traumatic Brain Injury Program take, demographics, and the type of inquiry. It also includes the attorney notes.” National leaders including the Executive Director of the P&A association are working to identify measures for TBI and disability advocacy that can be used by P&As nationwide through DAD. TBI Program and Systems Evaluation Few states are conducting evaluation of their TBI programs or efforts to achieve systems change. Several stakeholders called for information on “what works and what doesn’t” and suggested that HRSA establish measures against which they could assess progress achieving systems change. Stakeholders requested technical support for evaluation. Also, interest was expressed in obtaining clinical information about service needs and outcomes throughout the lifecycle of individuals with TBI. The HRSA TBI Program guidance includes a requirement that grantees evaluate their efforts. All grantees conduct some form of evaluation, but states’ capabilities and resources for evaluation differ widely. States’ evaluation activities range from basic monitoring of project task completion; to conducting process evaluations of conferences, training sessions, and products; to program impact and effectiveness studies. Both state and P&A respondents expressed interest in understanding “what works and what doesn’t” as they develop their programs, but most states have not conducted this type of evaluation. Only one of the sample states, Colorado, included a developed evaluation component that examines the systemic effects of the grant-funded projects. State respondents expressed frustration at their inability to evaluate and assess their TBI programs, citing lack of staff with skills in data analysis and evaluation, and problems understanding what and how to measure changes. A respondent in a state with mature infrastructure in place stated that, even with available data, their evaluation efforts were lacking. This respondent commented, “Evaluation is the area I’m least satisfied with and feel we need improvement. We need personnel. We have a great data system, but we don’t have anyone to pull it together. I’d love to have a data person, even if just 6 months to a year, to do this.” New Jersey and Colorado have placed a greater focus on research and evaluation than have other study states. New Jersey established a TBI Research Fund in 2004, spearheaded by the father of a son with a head injury. The Fund is supported by a surcharge on motor vehicle registrations. Colorado directs a portion of its TBI trust fund, initiated by a university-based researcher with interest in TBI, to research, including evaluation. Colorado
OCR for page 315
Evaluating the HRSA Traumatic Brain Injury Program has also engaged two evaluators to assess the state’s Implementation grant activities. The evaluators also serve as program consultants, sharing results with the state and its TBI Advisory Board. A national respondent reported that a subgroup of grantees and the federal program have started meeting to discuss evaluation issues and possible measures. The group is exploring the question, “What do you need to evaluate to tell you what you need to know?” The group expects to identify key system outcomes for the TBI Program. V. TBI Technical Assistance Center The TBI Technical Assistance Center (TBI TAC) plays a major role facilitating information sharing among state grantees and, to a lesser extent, P&As. State grantees and P&As provided strong praise for the assistance, support, and information across states available through TBI TAC. All states provided examples of how they were able to avoid “reinventing the wheel” by contacting the TBI TAC or other grantees of the program for brochures, training materials, and general advice. The annual grantee meeting and the list serv received high marks. The P&As also rated TBI TAC highly, but noted they receive information primarily through the national P&A association. The HRSA TBI Program supports TBI TAC through a contract with the National Association of State Head Injury Administrators (NASHIA). TBI TAC was established to help states in the planning and development of effective programs that improve access to health and other services for individuals with TBI and their families. TBI TAC staff specialists provide states with a range of technical assistance offerings, including: annual grantee meetings, a web-based Collaborative Space to share documents and other information, a grantee list serv, individualized site visits, and others. TBI TAC also develops and disseminates a variety of specialized documents and initiatives for HRSA’s TBI Program. TBI TAC serves all HRSA TBI Program grantees. Brain injury organizations and individuals with interests in TBI services can request access to the TBI TAC web site. Twenty-five respondents provided information about their use of TBI TAC. Respondents’ use of specific services by type is shown in Figure E-2, below. The most frequently used services include the TBI TAC grantee list serv and the Annual Grantee meeting, named by 21 and 20 respondents, respectively. TBI TAC services received above average ratings from these users. Twelve of 24 respondents rated the TBI TAC offerings as “excellent,” five
OCR for page 316
Evaluating the HRSA Traumatic Brain Injury Program FIGURE E-2 Respondents’ use of specific services, by type. respondents rated TBI TAC as “very good,” and seven rated it as “good.” No respondents rated the services as average, fair, or poor. (See Figure E-3.) Two stakeholders representing national organizations questioned the appropriateness of the amount of funding directed to the TBI TAC in the context of overall HRSA TBI Program funding, and whether or to what extent the TBI TAC should provide non-technical assistance program administrative support to the HRSA TBI Program. State respondents were not always clear about roles and relationships between HRSA and TBI TAC for program administration. FIGURE E-3 TBI TAC user ratings.
OCR for page 317
Evaluating the HRSA Traumatic Brain Injury Program While states praised the services provided by the TBI TAC, one national stakeholder raised concerns about the level of resources provided to the TBI TAC. This respondent noted that the TBI TAC receives a generous portion of HRSA’s TBI Program appropriation2 compared with the limited funding provided to states and the P&As, stating …“It’s the elephant in the living room. Do we really need to be spending this much on the TBI TAC?” Another respondent raised questions about role clarity and potential conflicts of interest involving the TBI TAC and the HRSA TBI Program, commenting: “The TBI TAC is in a difficult position. They are the TA Center, but also grant staff for HRSA. It is a conflicting role. It creates a situation where people in states can feel they’re being spied on … The same people there to help them are the ones asking for their grantee reports. It’s too much to expect, it leads to a conflict. Some people resent it.” The HRSA TBI Program Director as well as the TBI TAC Director both recognized and appreciated the dilemma and the necessity of this close relationship—and the importance of drawing a clear line between official government responsibilities that remain with HRSA, and administrative responsibilities that can be delegated to the TBI TAC. The relationship is especially significant given program staffing and resource limitations at HRSA. One respondent suggested that dedicated staff could be housed at NASHIA or another organization outside government, maintaining a separation of the technical assistance and program administration functions of the TBI TAC. VI. HRSA TBI Program Grantee Experience The HRSA TBI Program is administered by HRSA’s Bureau of Maternal and Child Health, with program oversight provided by a Public Health Services Commission Corps Program Director. The TBI Program has had four Program Directors since it was established in the Bureau in 1997. The current TBI Program Director does not have staff, and noted the challenges of overseeing several types of state grants and P&A grants, as well as providing contract oversight of the TBI TAC and other administrative activities relating to the TBI Program. The TBI Program Director currently delegates some program support activities to the TBI TAC, but retains program authority and responsibility on behalf of the government. State and P&A stakeholders were asked to comment on their experiences with the HRSA TBI Program. Respondents were asked to name ben- 2 HRSA’s TBI Program funding (FY2005) was distributed in the following approximate amounts: State Infrastructure grants, $5 million; Protection and Advocacy grants, $3 million; TBI TAC, $1 million.
OCR for page 318
Evaluating the HRSA Traumatic Brain Injury Program efits and drawbacks of their states’ participation. Respondents were also asked to provide comments and suggestions for program improvement. Stakeholders’ responses are summarized below. Benefits of the HRSA Grant Program Stakeholders credited the HRSA TBI Program with several types of benefits in their states. State grantees and P&As noted that HRSA funding has increased the visibility of TBI and related issues among state agencies, providers, and the public as a valuable benefit of the HRSA grant funding. These respondents also named TBI-specific funding for inter-organizational activities and funding for TBI-specific projects and materials as benefits. Both state grantees and P&As reported that HRSA funding, while limited, was valuable to “jump start” TBI-related activities in their states. The ability to focus on TBI and increased visibility of TBI as an issue were mentioned most frequently as benefits, followed by specific activities and materials developed with HRSA funding. Stakeholders in states with TBI trust funds noted that the HRSA Program helped the state prioritize fund expenditures. Respondents commented “The HRSA grant stimulated the state to look at the whole system. The benefit was having money for an unrecognized population.” “The biggest thing the grant has done is to bring attention to TBI. The grant helped to identify issues, the Trust Fund paid for services.” “TBI-specific funding, products. In state government we have a real lot on our plates. This allows us to focus on TBI.” Drawbacks of the HRSA Grant Program State grantees were appreciative of the funding obtained through the HRSA Federal TBI Program, and noted that even small amounts of funding could be effectively leveraged to raise visibility and awareness of TBI. However, many reported that 1-year grants, e.g., Post-Demonstration grants at the $100,000 level, provided insufficient time and resources to impact systems change goals. Stakeholders frequently shared their frustrations at the time consuming preparation required for limited funding, and problems sustaining funding. Stakeholders also commented that building relationships and trust at the community level is required for real systems change, and often requires more than a 1-year time-
OCR for page 319
Evaluating the HRSA Traumatic Brain Injury Program commitment. Many called out for formula funding, on a regular schedule to allow program continuity. Study respondents named issues related to the structure of the HRSA grant program—lack of sustainable funding, competitive grant applications, limited and restricted funding—as the top drawbacks to participation in the HRSA TBI Program. Respondents commented, “What has turned states off are limited competitive one-year grants. They’re not worth the trouble.” “Sustainability and having to write a new grant every year is difficult and time consuming. Also, it was difficult to come up with new projects for one year at $100,000.” “Limited funding. The amount of money is small for P&As, not enough for a dedicated staff member. I’d love a TBI specialist who also provides advocacy. That’s one thing that has held us back. But it also means we all have to learn about TBI in the P&A—we can’t say it’s someone else’s responsibility.” “One of the things that kept limiting us was we couldn’t do anything about services. If there could be a pilot for service delivery it would be great!” Other HRSA TBI Program Considerations The HRSA TBI Program has served as a catalyst for a host of TBI-related activities in the study states, including programs and projects funded by HRSA and others funded independently. However, sustaining state infrastructure and project activities in the absence of HRSA or other funding continues to challenge grantees. Many states reported direct and spillover impacts—and many question how they can continue to support these efforts. Spillover effects. The HRSA grants have demonstrated both direct and spillover impacts, especially when funding is skillfully leveraged. State and P&A respondents were able to point to direct impacts of the HRSA grant funding, including increased visibility and awareness of TBI, as well as spillover impacts that occurred “as a consequence but not as a direct result” of grant funding. Respondents provided many examples of TBI-related activities and initiatives undertaken in their states that occurred because awareness of TBI was heightened in multiple spheres of state government, the non-profit and private sectors, and the advocacy community. These include program ex-
OCR for page 320
Evaluating the HRSA Traumatic Brain Injury Program pansions, trust fund development, and other activities. Some examples include: New Jersey developed the core components of its state TBI program in anticipation of the state’s receiving their first HRSA grant award. New Jersey’s TBI Trust Fund and TBI Research Fund were developed through efforts initiated without HRSA funding, but were reportedly developed as TBI gained new visibility as a result of HRSA-funded grant activities. Colorado’s CIRCLE programs, established to facilitate information sharing about TBI service needs and availability at the community level, were continued through volunteer efforts when HRSA funding was no longer available. State respondents noted that these successful meetings have been embraced by local stakeholders, and have developed a momentum of their own. Sustaining program components and infrastructure. Respondents named sustainability of program components as a significant challenge to maintaining program infrastructure established under the HRSA TBI Program. Stakeholders in nearly every state named sustainability of TBI programs developed under the HRSA grant program as a great challenge. States with access to trust fund dollars were sometimes able to continue successful programs and trainings. However, states with limited resources may be unable to continue the momentum of program activity beyond the grant period. Additionally, irregular funding cycles and the uncertainty of competitive grant awards create difficulties for staff retention as well as program sustainability in general. Looking to the Future Stakeholders commented on program improvements and reforms in several areas, revealing consistent themes across respondents in the seven study states. Respondents gave positive marks to the flexibility of the HRSA TBI Program in allowing grantees to address issues relevant to their state TBI programs. At the same time, respondents called for more structure and information about state program and service effectiveness in improving outcomes for individuals with TBI and their families. One stakeholder stated, “We’d love to see the program go to a more mature level … How to promote the best outcomes …” Respondents also directed comments to changes in the grant program application process, funding levels, and time frames. The P&As currently receive formula funding for their TBI applications, and states are calling for
OCR for page 321
Evaluating the HRSA Traumatic Brain Injury Program the same, including discretionary support and mechanisms to help sustain their TBI programs. Some example comments include: “We are glad the TBI program is there, grateful for funding and support. Would like more discussion on focused issues. And more sophisticated, targeted advice.” “There should be less emphasis on processes, work more on something universally acceptable. Take the best that works and package it. Share these with other states, hospitals. There MUST be common practices that work! Evaluate them and package them.” “Continue the system as it is, with more centralized control in the Lead Agency. We need more support for statistics, research, to more accurately target dollars … number crunching capabilities … and more money!” “Being able to integrate state efforts to a basic level would help …” “Somehow, local work has to be integrated with state policy change. They need to be pulled together. There needs to be articulation of how the state system overlays [and how the trust fund fits in].” “We need to educate people … Could HRSA develop things about services, needs across the lifespan of people with TBI? Different strategies as the person ages? So people know what comes with TBI …” “The main thing is time for the grants and the irregularity of funding. It impacts our ability to get and keep staff.” “When it’s time to do the grant proposals, the HRSA proposals require more narrative than others we typically do. I’d suggest simplified paperwork for the formula funds.” States’ suggestions to move to a new structure for state grants have already been addressed. In August 2005 HRSA released its new program guidance, which provides support for 3-year “Partnership Implementation” grants. Seen as a step closer to the formula grant approach, which must be legislatively authorized, the Partnership Implementation grants “meet states where they are.” The new grants replace the Planning, Implementation, and Post-Demonstration grants previously available and, while the new grants will be competitively awarded, they are available to all states and territories.