|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. Appendix K
|
||||||||||||||||
 |
 |
Page 407 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 407
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Appendix K
Measurement of Quality in the Title X Family Planning Program
Kimberly D. Gregory, M.D., M.P.H.
SUMMARY
There exist a solid evidence base for quality domains and an extensive list of potential indicators that can be used to measure quality performance in family planning programs. The Family Planning Annual Report (FPAR), the Family Planning Council of America Performance Monitoring System (FPCA), and Healthy People 2010 reproductive health goals are explicitly specified indicators representative of the more than 200 indicators that have been suggested in this arena. There is some consistency (or overlap) in indicators among these documents, and several of the indicators reflect goals adopted by external agencies, such as the Healthcare Employer Data and Information Set (HEDIS) measures espoused by the National Center for Quality Assurance (NCQA) (specifically breast and cervical cancer screening and screening for chlamydia). Two obvious deficiencies in the currently reported measures are:
The lack of outcome data that are patient-specific about reproductive desires (specifically Helping patients Achieve their Reproductive Intentions [HARI]; patients should plan for pregnancy as well as plan to prevent pregnancy).
The lack of data on provider competency and interpersonal skills or client comprehension/literacy.
The Title X program mandate specifies three long-term measures that are to be reported annually: (1) increasing the number of unintended preg-
OCR for page 408
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
nancies averted by providing Title X family planning services; (2) reducing infertility among women attending family planning clinics by identifying chlamydia infection; and (3) reducing invasive cervical cancer among women attending family planning clinics. Additional pertinent indicators include screening for other sexually transmitted diseases (STDs) that are treatable and preventable and have significant maternal and perinatal long-term implications (syphilis, gonorrhea, HIV). Available monitoring systems adequately address these indicators.
Future primary data collection efforts should include the following patient-centered priorities:
Patient-specific reproductive desires/outcomes. Instead of counting visits and number of new visits, the focus should be changed to:
HARI: What are pregnancy plans for the year? Among those making a repeat visit, have these goals been met?
Percent clients not pregnant at next visit (denominator: those planning contraception)
Percent clients still using any method
Percent referrals for pregnancy termination or percent referrals for prenatal care for unintended pregnancy
Percent pregnant who desired pregnancy
Patient-specific evaluation of the quality of information provided
Technical competence and interpersonal skills of provider
Client comprehension (health literacy)
This appendix addresses the measurement of the quality of reproductive health services provided under the Title X program. The discussion includes an assessment of how well the FPAR measures quality, a description of quality initiatives undertaken by family planning programs, and consideration of how the quality of services should be assessed in various settings. An assessment of the costs and benefits associated with introducing quality measures into family planning clinics is beyond the scope of this discussion because of the limited data available to inform such an assessment.
OVERVIEW OF FINDINGS AND RECOMMENDATIONS
To assess quality, there must be consensus on what quality is (e.g., how it is defined), as well as agreement on what measures are to be used to monitor and report quality. Several definitions of quality are pertinent, including those of the Institute of Medicine (IOM) and the World Health Organization (WHO) (WHO, 1998; IOM, 2001). Both emphasize proper performance of care based on current standards and knowledge, recognizing the potential for individual and societal benefit. Judith Bruce offers a
OCR for page 409
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
family planning–specific definition of quality: “providing a range of services that are safe, effective, and that satisfy clients’ needs and wants” (Bruce, 1990). For this study, the author performed a focused review of the literature and evaluated the FPARs for 2001–2006 (Frost, 2001, 2002, 2003; Frost and Frohwirth, 2005; Fowler et al., 2006; RTI International, 2006), the Title X Program Assessment Rating Tool (PART) evaluation (OMB, 2005), and an advance copy of the FPCA proposed Performance Measurement System (FPCA, 1999).
This review led to the conclusion that a full assessment of the quality of the Title X program cannot be performed at this time. Based on the limited information available, primarily the 2005–2006 FPARs, the program does appear to be doing what it set out to do; however, the extent to which its services are underused, overused, or used inappropriately (measures of poor quality) cannot be determined from these reports. Whereas the readily apparent structure and process variables appear to have face and construct validity, the outputs and outcomes need further clarification. Regional and/or population data are needed to support any claim for program effects. Think tanks and advocacy groups, such as the Alan Guttmacher Institute and the Center for Reproductive Rights, believe there is a logic model to support a causal link between family planning services and pregnancies averted and dollars saved (Center for Reproductive Rights, 2004; Dreweke, 2006). Evidence of such program impact would clearly help advance the policy mandate for more funding, more marketing, and the development of more meaningful indicators to advance the reproductive health agenda.
While there are substantial data to support a framework for both quality assessment and program evaluation within the family planning field, there are limited data on the quality of national family planning services, and there does not appear to be a national consensus about the quality domains or quality indicators that should be routinely (or periodically) monitored and reported. There is a tendency to count resources, visits, and tests, with less energy directed toward capturing data on intermediate effects or long-term impact, such as pregnancies prevented (or planned) or overall reduction in population fertility rates or STD rates.
Surprisingly, the lack of data on the quality of family planning services in the United States in general and under Title X in particular is not due to the lack of an evidence base for indicators, but to an apparent failure to capitalize on the extensive work that has been done and applied internationally in this arena. Similarly, and not surprisingly, the dearth of quality-of-care research in the area of family planning in the United States contrasts with the quality-of-care work in the medical/surgical arena, likely because family planning has historically been focused primarily on women. Despite widespread acceptance internationally, only recently have the benefits of family planning been claimed to extend to improved child, family,
OCR for page 410
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
and world health in the United States (Cleland et al., 2006). The inclusion of reproductive health services that encompass infertility and STD/HIV screening and treatment has broadened family planning services to extend to outreach programs for men, but this, too, has been a relatively new phenomenon. The absence or relative paucity of indicators for women’s health, maternity services, and child health has not gone unnoticed by health service researchers, but progress on the development of these indicators has been slow (Schuster et al., 1997; Kerr et al., 2000; Gregory et al., 2005; Korst et al., 2005). In fact, the Agency for Healthcare Research and Quality (AHRQ)—the national leader in advancing the quality agenda—specifically excludes pregnancy and children from its current inpatient and patient safety indicators (AHRQ, 2004, 2006a,b). A set of pediatric inpatient indicators was recently developed (AHRQ, 2006c).
Attempts to achieve federal accountability across all federally funded programs have spurred the development of indicators for Title X. Current efforts by the FPCA to develop consensus-based performance indicators are a step in the right direction. Efforts to capture additional measures, already defined by the Department of Health and Human Services (HHS) via Healthy People 2010, would further the cause (HHS, 2000a). Examples of representative Healthy People 2010 reproductive and STD/HIV goals that would be consistent with Title X program goals can be found in Annex K-1. Similarly, focused incorporation of selected indicators from the Handbook of Indicators for Family Planning Program Evaluation, which contains more than 200 indicators, would be beneficial and could elevate family planning and preventive reproductive health services in general, and the Title X program in particular, from a relatively obscure program for the poor to a more prominent national program dedicated to improving the health and well-being of women, children, and families (Bertrand et al., 1994). Glasier et al., in an editorial about family planning services and women’s health, state that “unsafe sex is the 2nd most important risk factor for disability and death in the world’s poorest communities, and the 9th most important in developed countries” (Glasier et al., 2006). These authors contend that reproductive health services are of poor quality and underused because discussions about sexual intercourse and sexuality make people uncomfortable. Further, they suggest that the increasing influence of conservative, political, religious, and cultural forces threatens to undermine what has been achieved to date.
To make this admonishment meaningful in a different social context, the average youth watches 3 hours of television daily, whereas 59 percent of adults watch television 2 or more hours (Roberts et al., 1999; Bowman, 2006). It is inevitable that most Americans will therefore encounter sexual messages given how commonly they appear on television; approximately 64 percent of all programs have sexual content based on analysis of the
OCR for page 411
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
2001–2002 TV season. Among programs with sexual content, there was an average of 4.4 scenes per hour. Talk about sex was more common (61 percent) than overt portrayals of sexual behaviors (32 percent). However, approximately 14 percent (or one of every seven programs) included a portrayal of sexual intercourse, depicted or strongly implied (Kunkel et al., 2003; Collins et al., 2004). Since children tend to model what they see, it is perhaps not surprising that the average age of sexual debut overall is 14.2 years (13.1 for boys, 15.0 for girls) (Sandfort et al., 2008). Moreover, the rate of premarital sex continues to be high. Fully 90 percent of women aged 15–44 responding to the National Survey of Family Growth (NSFG) in 2002 had had premarital intercourse (Mosher et al., 2004). The social marketing of sexuality is rampant, while the marketing of abstinence, contraception, and preventive reproductive behaviors is glaringly absent, despite evidence that it could be beneficial (Piotrow et al., 1997).
Based on a review of the literature and a synthesis of both national and international data regarding quality measurement in family planning, there is a solid evidence base for the quality domains that should be included, and there is no dearth of potential indicators that could be used. The FPCA Performance Measurement System, coupled with the Healthy People 2010 goals, would be an excellent start. These should be fortified with a paradigm policy shift that emphasizes planning for pregnancy as much as planning to prevent pregnancy (HARI) (Jain et al., 1992). This is entirely consistent with the IOM recommendation to adopt a social norm whereby all pregnancies are intended—clearly and consciously desired at the time of conception (IOM, 1995). Further analysis of the merits of the Title X program is needed using various methodologies, such as provider observation or simulated patients, to document technical competence and communication skills. Additionally, patient exit interviews should not be limited to satisfaction surveys or closed questions about what was discussed as is currently the trend, but should include measures verifying literacy and comprehension. Community focus groups should ascertain additional perceptions of clinic quality with less possibility of courtesy bias (Sullivan and Bertrand, 2000).
The infrastructure to implement many of these suggestions exists, as evidenced by the FPARs, the Program Guidelines, application criteria, and the Office of Population Affairs’ (OPA) 2006 Family Planning Program Priorities, Legislative Mandates, Key Issues (HHS, 2006). However, data collection efforts by Title X clinic sites may already be burdensome, and future funding should support either quality-monitoring full-time equivalents (FTEs) or an electronic medical record (EMR) system without sacrificing support for existing services. The EMRs should be designed to capture indicator data. There should be a vision or capacity for shared information (regional health information network), given the transient nature of
OCR for page 412
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
the poor population. The opportunity for shared patient-level data (with appropriate consent and Health Insurance Portability and Accountability Act [HIPAA] considerations) could optimize STD/HIV prevention and treatment. Additionally, a widely acknowledged strength of the Title X program is its information–education–counseling (IEC) emphasis and the associated training and audiovisual tools directed at both clients and providers. These tools, coupled with standardized protocols, could be shared across state and federal agencies, including STD clinics and public schools, as well as incorporated into medical, nursing, and residency training curricula. There is clearly stakeholder support for enhancing the quality of family planning services at the user/provider level, but broader support at the administrative and policy levels is needed to facilitate the development of a national agenda emphasizing the maternal, child, and family benefits of family planning and preventive health services in general and the Title X program in particular.
The remainder of this appendix provides an overview of the theoretical frameworks that inform this review. It also provides an assessment of how well the FPAR measures quality based on these frameworks, a description of quality initiatives undertaken by family planning programs, and an overview of how the quality of services should be assessed. The latter overview uses representative examples of available indicators, highlighting where existing indicators may need to be modified or expanded to address clients’ needs and wants and the HARI principle.
THEORETICAL FRAMEWORKS
Several quality and reproductive health frameworks inform this discussion:
Donabedian quality model: Structure–Process–Outcome Model (Donabedian, 1968)
Bertrand et al. program evaluation model (Bertrand et al., 1994; Sullivan and Bertrand, 2000)
International frameworks for quality family planning services
Bruce and Jain et al. model of quality family planning services (Bruce, 1990; Jain et al., 1992) International Planned Parenthood Federation (IPPF) framework (IPPF, 1998)
1994 International Conference on Population and Development, Cairo (United Nations, 1994)
Lu and Halfon Reproductive Health Continuum (Lu and Halfon, 2003)
AHRQ criteria for an acceptable indicator (AHRQ, 2001)
OCR for page 413
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
These frameworks need to be integrated and ultimately accepted by various stakeholders.
Donabedian’s Structure–Process–Outcome Model
Donabedian’s model has been widely endorsed as the theoretical framework for quality measurement (Donabedian, 1968). Key representative variables critical for assessment of the quality of Title X programs using the various domains outlined by Donabedian are shown in Figures K-1 through K-3.
Structure
Where is the facility located? What are the physical and administrative barriers to accessing services? Is the site geographically convenient and accessible by public transportation? Are the hours varied enough to meet the needs of clients—for example, evening or weekend hours for people who work or teens in school? Are the building and waiting area physically appealing? Once inside, is the waiting area comfortable, are there enough chairs, and are there diversions for accompanying children? Is there an opportunity for private discussion between clients and various clinical and nonclinical staff? Both national and international studies evaluating client satisfaction have demonstrated that all of these factors may influence whether a client comes for an initial visit or returns for subsequent visits—which of necessity impacts the short- and long-term goals of initiation and continuation of contraceptive services (Alden, 2004; Zaky et al., 2007). These may be significant issues at the individual clinical sites, determining
FIGURE K-1 Examples of variables representing Donabedian’s structure quality domain.
OCR for page 414
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
FIGURE K-2 Services that must be provided by Title X programs.
FIGURE K-3 Similarities between quality monitoring and program evaluation.
whether or not a client seeks initial or follow-up services. However, at the program level, only privacy issues are relevant from a regulatory standpoint. Staffing mix and training opportunities determine technical competence. The sociopolitical and cultural milieu can impact what methods are available (e.g., termination or emergency contraception). Ready access to information technology (IT) support can facilitate notification after positive test results and timely data acquisition for audits. The strength of the advisory board can determine additional funding options for outreach, social marketing, and facility improvement.
OCR for page 415
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Process
The services that must be provided by Title X programs have been mandated by legislation (Figure K-2) (HHS, 2001, 2006). The federal Family Planning Program is authorized under Title X of the Public Health Service Act, which was created in 1970 to provide family planning and related preventive health care. The mission of Title X is to provide the information and means necessary for individuals to exercise choice in determining the number and spacing of their children (AHRQ, 2006c). In addition to providing a broad range of contraceptive supplies, counseling, and information on a confidential basis, clinics with Title X funding must provide reproductive health and preventive health services that include breast and pelvic exams to screen for breast cancer, cervical cancer, and STDs (including HIV); pregnancy diagnosis; patient education; reproductive health counseling; and appropriate social and referral services. A mechanism to determine that the information provided has been understood should be established and documented. Further, the Title X legislation authorizes funding for family planning services, training, research, information, and education. The program is administered by OPA through the Office of Family Planning (OFP). Services are intended for all who want and need them, with priority for low-income individuals. The Program Guidelines stipulate the involvement of an advisory board and community participation in the development of educational materials and project promotion.
While these are nationally legislated mandates, how these mandates get carried out is influenced by regional or local implementation strategies, largely resulting in site-specific policies and procedures (see Figure K-3). This variation is due to previously mentioned structural variables such as sociocultural or political milieus, regional demographics and/or case mix, and availability of staff and legal or licensing mandates regarding what types of providers can perform which types of services. For example, Gillian et al. found improved compliance among African American teens due to the involvement of all clinical staff in the process. All clinic employees, including clerical and professional staff, forged relationships through shared backgrounds and experiences, honesty, and additional time spent with the teens (Gilliam et al., 2007).
Outcome
See Figure K-3 and the discussion below of the Bertrand et al. (1994) program evaluation model.
OCR for page 416
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Relationship Between Quality Assessment and Program Evaluation
It is difficult if not impossible to distinguish quality assessments from program evaluations when reviewing the family planning literature. Hence, Figure K-3 shows an integrated model of quality and program evaluation. Inputs and processes are conceptually equivalent to Donabedian’s structure and process variables. Input examples pertinent to family planning include such items as personnel, financial resources, facilities, and equipment. Both output and outcomes are comparable to Donabedian’s outcome domain. However, output specifically refers to outcomes at the program level and is usually defined by service utilization (number of visits, number of new or continuing contraceptive users). On the other hand, program outcomes defined by program effect and program impact are measured at the population level. An example of program effect is the prevalence of contraceptive use as measured in a population survey, and an example of program impact is the regional or national fertility rate or desired pregnancy rate. The maturity of the program determines the type of evaluation strategy to use—the more mature the program, the more impact one would expect to be able to demonstrate. By most standards, Title X is a mature program (more than 30 years old), but it is immature in quality assessment; hence the majority of indicators will initially be focused on processes and outputs. However, the program has been in existence long enough to be capable of demonstrating long-term impact if the correct data are made available for collection and interpretation.
International Frameworks for Quality Family Planning Services
Several authors have published extensively on quality assessment as it relates to the international family planning and reproductive health care arena. Most authors build on or adapt the Bruce (1990) and Jain et al. (1992) framework, which identifies six elements of quality:
Choice of method,
Information provided to the client,
Technical competence of providers,
Interpersonal relations between clients and providers,
Mechanisms to encourage continuity of care, and
Appropriate constellation of services.
Outcome measures include program readiness, the provider perspective, the services delivered, and the client perspective and the services received, with particular emphasis on client knowledge, client satisfaction, client health, and contraceptive use—both acceptance and continuation.
OCR for page 417
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
IPPF adopted the above framework as its model for quality after adding client acceptability to the above list and emphasizing that these criteria should be defined as clients’ and providers’ rights and expectations (IPPF, 1998). Table K-1 outlines the IPPF client and provider “bill of rights.”
The most extensive and comprehensive framework for family planning quality indicators and program evaluation is that proposed by Bertrand et al. (1994). While intended for developing countries, it is easily applicable to the United States and includes both program-and population-level indicators. Program-based or performance indicators include those factors related to inputs, processes, and outputs, whereas population-based or outcome indicators usually reflect intermediate effect or long-term impact. Bertrand et al. define eight broad categories for program evaluation, which outline the pathways by which programs achieve impact in a given country:
Indicators to measure the policy environment,
Indicators to measure service delivery operations,
Indicators to measure family planning outputs,
TABLE K-1 International Planned Parenthood Federation Framework: Clients’ Rights and Providers’ Needs
Client Rights
Provider Needs
Information about family planning
Training—technical and communication skills
Access to all service delivery systems and health care providers
Information on technical issues updated regularly
Choice of adopting, switch, or discontinuing methods
Infrastructure (appropriate facility and efficient organization)
Safety in the practice of family planning
Supplies of contraceptives, equipment, and educational materials
Privacy during discussions and physical examinations
Guidance from service guidelines, checklists, and supervision
Confidentiality of all personal information
Back up from other providers
Treated with dignity, courtesy, and attentiveness
Respect and recognition from coworkers, managers, clients, community
Comfort while receiving services
Encouragement to provide good quality care
Continuity of care for as long as client desires
Feedback from managers, supervisors, and clients
Opportunity to express opinions about he quality of care received
Opportunity to express their concerns relative to clinic decision making
SOURCE: Huezo, C.M., and S. Diaz. 1993. Quality of care in family planning: Clients’ rights and providers’ needs. In P. Senanyake and R. L. Kleinman, eds. Family planning: Meeting challenges, promoting choices. The proceedings of the IPPF Family Planning Congress, New Delhi, Oct. 1992. Pearl River, New York: Parthenon Publishing Group. Pp. 235-244. Reprinted with permission.
OCR for page 448
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
ANNEX TABLE K-2 Healthy People STD Objectives with Targets and Baselines
Topic
Objective
Target
Baseline
Chlamydia
Reduce infection in those aged 15–24 Family planning clinics
3.0
5.0
STD clinics
3.0
12.2
Males
3.0
15.7
(GC)
Reduce infection
19/100 thousand
123/100 thousand
Syphilis
Eliminate primary and secondary syphilis from United States
0.2/100 thousand
3.2/100 thousand
Herpes
Decrease percent with genital infection
14 percent
17 percent
Human papillomavirus (HPV) (developmental)
Decrease percent with HPV (can help minimize the number of high-risk subtypes associated with cervical cancer)
Pelvic inflammatory disease
Reduce proportion of female who have ever acquired PID
5 percent
8 percent
Fertility problems
Decrease percent of women with fertility problems associated with chlamydia and PID
15 percent
27 percent of women with fertility problems reported history of PID
Heterosexual HIV (developmental)
Reduce HIV infections in females aged 13–24 associated with heterosexual contact
OCR for page 449
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Topic
Objective
Target
Baseline
Congenital syphilis
Reduce congenital syphilis
1/100 thousand
27/100 thousand
Neonatal STD (developmental)
Reduce neonatal consequences from maternal STD
Responsible teen sex
Increase percent of adolescents who abstain from sex or use condoms if active
95 percent
85 percent
Responsible sex on television (developmental)
Increase number of positive messages related to responsible sexual behavior on television
Hepatitis B vaccine in STD clinics
Increase number of STD programs that offer hepatitis B vaccine
90 percent
5 percent
Screening in detention and jails (developmental)
Screen within 24 hours of admission and provide treatment before release
Contracts to treat nonplan partners (developmental)
Increase percent of local health departments that have contracts with managed care providers for treatment of nonplan partners
Annual screening for chlamydia (developmental)
Increase percent of women under age 25 screened annually
OCR for page 450
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Topic
Objective
Target
Baseline
Screening of pregnant women (developmental)
Increase percent of pregnant women screened for STD, HIV, and (BV)
Compliance with recognized STD treatment
Increase percent of primary care providers who treat patients with STDs who manage according to standards
90 percent
70 percent
Provider referral for sex partners (developmental)
Annual screening for chlamydia (developmental)—Increase percent of women under age 25 screened annually.
Screening of pregnant women (developmental)—Increase percent of pregnant women screened for STD, HIV, and (BV).
Compliance with recognized STD treatment—Increase percent of primary care providers who treat patients with STDs who manage according to standards.
Provider referral for sex partners (developmental).
OCR for page 451
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
ANNEX K-2
REPRESENTATIVE INDICATORS FOR EACH OF THE EIGHT BROAD CATEGORIES IN BERTRAND ET AL. (1994)
Policy Environment
Existence of a policy development plan
Number of appropriately disseminated policy analyses
Number of awareness-raising events targeted to leaders
Existence of a strategic plan for expanding the national family planning program
Integration of demographic data into development planning
Number of statements of leaders in support of family planning
Formal population policy addressing fertility and family planning
National family planning coordination
Level of the family planning program within the government administration
Levels of import duties and other taxes
Restrictions on advertising of contraceptives in the mass media
Absence of unwarranted restrictions on providers and users
Quality of program leadership
Extent of commercial-sector participation
Service Delivery Operations
Management
Training
Commodities and logistics
Information–education–communication (IEC)
Research and evaluation
Management (illustrative indicators)
Existence of a clear mission that contributes to the achievement of program goals
Realization of operational targets
Clearly defined organizational structure
Adequacy of staffing
Awareness of current financial position
Access to current information on key areas of program functioning
Access to current information on program progress
Capacity to track commodities
OCR for page 452
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Training
Number/percentage of courses that achieve learning objectives
Number/percentage of courses that contribute to the achievement of program training objectives
Number/percentage of courses in which the training methodology is appropriate for the transfer of skills and knowledge
Number of trainees by type
Number/percentage of trainees who have mastered relevant knowledge
Number/percentage of trainees competent to provide a specific family planning service
Number/percentage of trained providers assessed to be competent at a specified period (e.g., 6 months) post-training
Number/percentage of trainees who apply the skills to their subsequent work
Commodities and Logistics
Pipeline wastage
Percentage of storage capacity meeting acceptable standards
Frequency of stock-outs
Percentage of service delivery points (SDPs) stocked according to plan
Percentage of key personnel trained in contraceptive logistics
Composite indicator for commodities and logistics
Information–Education–Communication
Number of communications produced, by type, during a reference period
Number of communications disseminated, by type, during a reference period
Percentage of target audience exposed to program messages, based on respondent recall
Percentage of target audience who correctly comprehend a given message
Number of contraceptive methods known
Percent of audience who acquire the skill to complete a certain task as a result of exposure to a specific communication
Percentage of target audience exposed to a specific message who report liking it
Number/percentage of target audience who discuss message(s) with others, by type of person
Percentage of target audience who advocate family planning practice
OCR for page 453
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Research and Evaluation
Presence of an active research and evaluation unit
Extent of use of a service system
Conduct of periodic household and/or special-purpose surveys and studies
Conduct of operations research
Regular conduct of process evaluations
Conduct of effectiveness, efficiency, and impact evaluations
Use of research and evaluation results for program modification
Dissemination of research and evaluation results
Family Planning Service Outputs
Accessibility (illustrative indicators)
Number of SDPs located within a fixed distance or travel time of a given community (i.e., service density)
Cost of 1 month’s supply of contraceptives as a percentage of monthly wages
Restrictive program policies on contraceptive choice
Percentage of the population who know of at least one source of contraceptive services and/or supplies
Percentage of nonuse related to psychosocial barriers
Quality of care (illustrative indicators)
Number of contraceptive methods available at a specific SDP
Percentage of counseling sessions with new acceptors in which provider discusses all methods
Percentage of client visits during which provider demonstrates skill in clinical procedures, including asepsis
Percentage of clients reporting sufficient time with provider
Percentage of clients informed of timing and sources for resupply/revisit
Percentage of clients who perceive that hours/days are convenient
Program Image
Number and type of activities to improve the public image of family planning during a reference period (e.g., 1 year)
Percentage of target population favorable to the (national) family planning program
Service Utilization
Number of visits to SDP(s)
Number of acceptors new to modern contraception
OCR for page 454
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Number of acceptors new to the institution
Number of new acceptors
Couple-years of protection (CYP)
Method mix
User characteristics
Continuation rates
Contraceptive Practice
Contraceptive prevalence rate (CPR)
Number of current users
Level of ever (past) use
Source of supply (by method)
Method mix
User characteristics
Continuation rates
Use failure rates
Fertility Impact
OCR for page 455
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
REFERENCES
AGI (The Alan Guttmacher Institute). 2000. Fulfilling the promise: Public policy and U.S. family planning clinics. New York: AGI.
AHRQ (Agency for Healthcare Research and Quality). 2001. Refinement of the HCUP quality indicators. www.qualityindicators.ahrq.gov/data/hcup/qirefine.htm (accessed January 15, 2008).
AHRQ.2004. Prevention quality indicators overview. Rockville, MD: AHRQ. http://www.qualityindicators.ahrq.gov/pqi_overview.htm (accessed January 15, 2008).
AHRQ. 2006a. Inpatient quality indicators overview. Rockville, MD: AHRQ. http://www.qualityindicators.ahrq.gov/iqi_overview.htm (accessed January 15, 2008).
AHRQ.2006b. Patient safety indicators overview. Rockville, MD: AHRQ. http://www.qualityindicators.ahrq.gov/psi_overview.htm (accessed January 15, 2008).
AHRQ.2006c. Pediatric quality indicators overview. Rockville, MD: AHRQ. http://www.qualityindicators.ahrq.gov/pdi_overview.htm (accessed January 15, 2008).
AHRQ.2007. Screening for chlamydial infection. Rockville, MD: AHRQ. http://www.ahrq.gov/clinic/uspstf/uspschlm.htm (accessed May 7, 2008).
Alden, D. L. 2004. Client satisfaction with reproductive health care quality. Social Science & Medicine 59:2219–2232.
Amaral, G., D. G. Foster, M. A. Biggs, C. B. Jasik, S. Judd, and C. D. Brindis. 2007. Public savings from the prevention of unintended pregnancy: A cost analysis of family planning services in California. Health Services Research 42(5):1960–1980.
Becker, D., M. A. Koenig, Y. M. Kim, K. Cardona, and F. L. Sonenstein. 2007. The quality of family planning services in the United States: Findings from a literature review. Perspectives on Sexual and Reproductive Health 39:206–215.
Bertrand, J. T., R. J. Magnani, and J. Knowles. 1994. Handbook of indicators for family planning. Chapel Hill, NC: Carolina Population Center. http://www.cpc.unc.edu/measure/publications/pdf/ms-94-01.pdf (accessed April 18, 2008).
Bowman, S. A. 2006. Television-viewing characteristics of adults: Correlations to eating practices and overweight and health status. http://www.cdc.gov/pcd/issues/2006/apr/050139.htm (accessed May 10, 2008).
Bronstein, J. M., A. Vosel, S. K. George, C. Freeman, and L. A. Payne. 2007. Extending Medicaid coverage for family planning: Alabama’s first 4 years. Public Health Reports 122:190–197.
Bruce, J. 1990. Fundamental elements of the quality of care: A simple framework. Studies in Family Planning 21(2):61–91.
Center for Reproductive Rights. 2004. America must continue its commitment to reproductive health. http://www.reproductiverights.org (accessed May 10, 2008).
Cleland, J., S. Bernstein, A. Ezeh, A. Faundes, A. Glasier, and J. Innis. 2006. Family planning: The unfinished agenda. Lancet 368:1810–1827.
Collins, R. L., M. N. Elliott, S. H. Berry, D. E. Kanouse, D. Kunkel, S. B. Hunter, and A. Miu. 2004. Watching sex on television predicts adolescent initiation of sexual behavior. Pediatrics 114(3):e280–e289.
Deming, W. 1986. Out of the crisis. Cambridge, MA: MIT Center for Advanced Engineering Study.
Donabedian, A. 1968. Promoting quality through evaluating the process of patient care. Medical Care 6:181–202.
Dreweke, J. 2006. Each $1 invested in Title X Family Planning program saves $3.80. Doubling funding could prevent 244,000 unplanned pregnancies, result in net savings of almost $800 million. New York: Guttmacher Institute. http://www.guttmacher.org/media/nr/2006/11/16/index.html (accessed May 7, 2008).
OCR for page 456
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Finer, L. B., J. E. Darroch, and J. J. Frost. 2002. US agencies providing publicly funded contraceptive services in 1999. Perspectives on Sexual and Reproductive Health 34:15–24.
Fisher, E. S., and J. E. Wennberg. 2003. Health care quality, geographic variations, and the challenge of supply-sensitive care. Perspectives in Biology and Medicine 46:69–79.
Foster, D. G., C. M. Klaisle, M. Blum, M. E. Bradsberry, C. D. Brindis, and F. H. Steward. 2004. Expanded state-funded planning services: Estimating pregnancies averted by the Family PACT Program in California, 1997–1998. American Journal of Public Health 94(8):1341–1346.
Fowler, C. I., J. Gable, and J. Wang. 2008. Family planning annual report: 2006 national summary. Research Triangle Park, NC: RTI International.
FPCA (Family Planning Council of America, Inc.). 1999. Performance monitoring system (draft provided by A. Stith, IOM).
Frost, J. J. 2002. Family planning annual report: 2001 summary, part 1. New York: AGI.
Frost, J. J. 2003. Family planning annual report: 2002 summary, part 1. New York: AGI.
Frost, J. J., and L. Frohwirth. 2005. Family planning annual report: 2004 summary, part 1. New York: AGI.
Gilliam, M. L., and K. Hernandez. 2007. Providing contraceptive care to low income African American teens: The experience of urban community health centers. Journal of Community Health 32:231–244.
Glasier, A., A. M. Gülmezoglu, G. P. Schmid, C. G. Moreno, and P. F. Van Look. 2006. Sexual and reproductive health: A matter of life and death. Lancet 368(9547):1595–2607.
Gregory, K. D., C. J. Hobel, L. M. Korst, M. Lu, and C. Reyes. 2005. A framework for developing Maternal Quality Care Indicators. Prepared for the California Department of Health and Human Services, Maternal and Child Health Branch.
Guttmacher Institute. 2008. Facts in brief. Facts on contraceptive use. http://www.guttmacher.org/pubs/fb_contr_use.html (accessed May 8, 2008).
HHS (U.S. Department of Health and Human Services). 2000a. Healthy people 2010 (2nd edition). Washington, DC: U.S. Government Printing Office.
HHS. 2000b. With understanding and improving health and objectives for improving health (2 volumes). In Healthy people 2010 (2nd edition). Washington, DC: HHS.
HHS. 2000c. Healthy people 2010—Reproductive health. www.hhs.gov/opa/pubs/hp201_rh.html (accessed April 28, 2008).
HHS. 2001. Program guidelines for project grants for family planning services. http://www.hhs.gov/opa/familyplanning/toolsdocs/2001_ofp_guidelines_complete.pdf (accessed April 24, 2008).
HHS. 2006. Family planning program priorities, legislative mandates, key issues. http://www.hhs.gov/opa (accessed April 24, 2008).
IHI (Institute for Healthcare Improvement). 2003. The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. IHI Innovation Series white paper. Boston: IHI. www.IHI.org (accessed April 28, 2008).
IOM (Institute of Medicine). 1995. The best intentions: Unintended pregnancy and the well-being of children and families. Washington, DC: National Academy Press.
IOM. 2001. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
IPPF (International Planned Parenthood Federation). 1998. Family planning programs: Improving quality (sidebars). Population Reports, Series J, No. 47.
Jain, A., J. Bruce, and S. Kumar. 1992. Quality of services, programme efforts and fertility reduction. In Family planning programmes and fertility. Edited by J. F. Phillips and J. A. Ross. Oxford: Clarendon Press.
OCR for page 457
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Kerr, E. A., C. L. Damberg, and S. M. Asch (editors). 2000. Quality of care for women: A review of selected clinical conditions and quality indicators. Santa Monica, CA: RAND.
Korst, L. M., K. D. Gregory, M. C. Lu, C. Reyes, C. J. Hobel, and G. F. Chavez. 2005. A framework for the development of maternal quality of care indicators. Maternal and Child Health Journal 9(3):317–341.
Kunkel, D., K. Eyal, E. Biely, et al. 2003. Sex on TV3: A biennial report to the Kaiser Family Foundation. Menlo Park, CA: The Henry J. Kaiser Foundation. www.kff.org/entmedia/loader.cfm?url=/commonspot/security/getfile.cfm&PageID=14209 (accessed April 28, 2008).
Lindberg, L. D., J. J. Frost, C. Sten, and C. Dailard. 2006. The provision of funding and contraceptive services at publicly funded family planning agencies: 1995–2003. Perspectives on Sexual and Reproductive Health 38:37–45.
Lu, M. C., and N. Halfon. 2003. Racial and ethnic disparities in birth outcomes: A life course perspective. Maternal and Child Health Journal 7:3–18.
Meyers, D., T. Wolff, K. D. Gregory, L. Marion, V. Moyer, H. Nelson, D. Petitti, and G. Sawaya. 2008. A summary of United States Preventive Service Task Force (USPSTF) recommendations for STI screening. American Family Physician 77(6):819–824.
Mosher, W. D., G. M. Martinez, A. Chandra, J. C. Abma, and S. J. Wilson. 2004. Use of contraception and use of family planning services in the United States, 1982–2002. Hyattsville, MD: National Center for Health Statistics.
NCQA (National Center Quality Assurance). 2007. Improving chlamydia screening. Strategies from top performing health plans. http://www.ncqa.org/Portals/0/Publications/Resource%20Library/Improving_Chlamydia_Screening_08.pdf (accessed May 7, 2008).
NCVHS (National Committee on Vital and Health Statistics). 2000. Toward a national health information infrastructure interim report. http://ncvhs.hhs.gov/NHII2kReport.htm (accessed April 8, 2008).
OMB (Office of Management and Budget). 2005. The Program Assessment Rating Tool (PART). Family planning assessment. http://www.whitehouse.gov/omb/expectmore/detail/10003513.2005.html (accessed April 8, 2008).
Paine, K., M. Thorogood, and K. Wellings. 2000. The impact of the quality of family planning services on safe and effective contraceptive use: Literature review. Human Fertility 3(3):186–193.
Piotrow, P. T., D. L. Kincaid, J. G. Rimon II, and W. Rinehart (editors). 1997. Health communication: Lessons from family planning and reproductive health. Westport, CT: Praeger.
RamaoRao, S., and R. Mohanam. 2003. The quality of family planning programs: Concepts, measurements, interventions and effections. Studies in Family Planning 34:227–248.
Roberts, D. F., U. G. Foehr, V. J. Rideout, and M. Brodie. 1999. Kids and media at the new millennium. In A Kaiser Family Foundation report: A comprehensive national analysis of children’s media use: Executive summary. Menlo Park, CA: The Henry J. Kaiser Family Foundation.
RTI International (Research Triangle Institute International). 2006. Family planning annual report: 2005 national summary. Research Triangle Park, NC: RTI International.
Sandfort, T. G., M. Orr, J. S. Hirsch, and J. Santelli. 2008. Long-term health correlates of timing of sexual debut: Results from a national US study. American Journal of Public Health 98(1):155–161.
Schunmann, C., and A. Glasier. 2006. Measuring pregnancy intention and its relationship with contraceptive use among women undergoing therapeutic abortion. Contraception 73:520–524.
OCR for page 458
A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results
Schuster, M. A., S. M. Asch, E. A. McGlynn, E. A. Kerr, A. M. Hardy, and D. S. Gifford. 1997. Development of a quality of care measurement system for children and adolescents. Archives of Pediatrics and Adolescent Medicine 151:1085–1092.
Sofaer, S., and K. Firminger. 2005. Patient perceptions of the quality of health services. Annual Review of Public Health 26:513–559.
Speroff, T., and G. T. O’Connor. 2004. Study designs for PDSA quality improvement research. Quality Management in Health Care 13:17–32.
Sullivan, T. M., and J. T. Bertrand. 2000. Monitoring quality of care in family planning by the quick investigation of quality (QIQ): County reports. MEASURE Evaluation Technical Report Series No. 5. Chapel Hill, NC: Carolina Population Center.
Thornburn, S., and L. M. Bogart. 2005a. African American women and family planning services: Perceptions of discrimination. Women’s Health 42:23–39.
Thornburn, S., and L. M. Bogart. 2005b. Conspiracy beliefs about birth control: Barriers to pregnancy prevention among African Americans of reproductive age. Health Education and Behavior 32(4):474–487.
United Nations. 1994. International Conference on Population and Development (ICPD), September 1994, Cairo, Egypt. http://www.un.org/popin/icpd2.htm (accessed April 8, 2008).
Wennberg, J. E. 1999. Understanding geographic variations in health care delivery. New England Journal of Medicine 340:52–53.
WHO (World Health Organization). 1998. WHO definition of quality. Population Reports Series J, No. 47.
WHO. 2004. Medical eligibility criteria for contraceptive use (3rd Edition, Reproductive Health and Research). Geneva, Switzerland: WHO.
Winter, L., and A. S. Goldy. 1987. Staffing patterns in family planning clinics: Which model is best? Family Planning Perspectives 19(3):102–106.
Zaky, H. H., H. A. Khattab, and D. Galal.2007. Assessing the quality of reproductive health services in Egypt via exit interviews. Maternal and Child Health Journal 11:301–306.