Appendix K
Measurement of Quality in the Title X Family Planning Program

Kimberly D. Gregory, M.D., M.P.H.

SUMMARY

There exist a solid evidence base for quality domains and an extensive list of potential indicators that can be used to measure quality performance in family planning programs. The Family Planning Annual Report (FPAR), the Family Planning Council of America Performance Monitoring System (FPCA), and Healthy People 2010 reproductive health goals are explicitly specified indicators representative of the more than 200 indicators that have been suggested in this arena. There is some consistency (or overlap) in indicators among these documents, and several of the indicators reflect goals adopted by external agencies, such as the Healthcare Employer Data and Information Set (HEDIS) measures espoused by the National Center for Quality Assurance (NCQA) (specifically breast and cervical cancer screening and screening for chlamydia). Two obvious deficiencies in the currently reported measures are:

  • The lack of outcome data that are patient-specific about reproductive desires (specifically Helping patients Achieve their Reproductive Intentions [HARI]; patients should plan for pregnancy as well as plan to prevent pregnancy).

  • The lack of data on provider competency and interpersonal skills or client comprehension/literacy.

The Title X program mandate specifies three long-term measures that are to be reported annually: (1) increasing the number of unintended preg-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 407
Appendix K Measurement of Quality in the Title X Family Planning Program Kimberly D. Gregory, M.D., M.P.H. SUMMARy There exist a solid evidence base for quality domains and an extensive list of potential indicators that can be used to measure quality performance in family planning programs. The Family Planning Annual Report (FPAR), the Family Planning Council of America Performance Monitoring System (FPCA), and Healthy People 2010 reproductive health goals are explicitly specified indicators representative of the more than 200 indicators that have been suggested in this arena. There is some consistency (or overlap) in indicators among these documents, and several of the indicators reflect goals adopted by external agencies, such as the Healthcare Employer Data and Information Set (HEDIS) measures espoused by the National Center for Quality Assurance (NCQA) (specifically breast and cervical cancer screen- ing and screening for chlamydia). Two obvious deficiencies in the currently reported measures are: • The lack of outcome data that are patient-specific about reproduc- tive desires (specifically Helping patients Achieve their Reproduc- tive Intentions [HARI]; patients should plan for pregnancy as well as plan to prevent pregnancy). • The lack of data on provider competency and interpersonal skills or client comprehension/literacy. The Title X program mandate specifies three long-term measures that are to be reported annually: (1) increasing the number of unintended preg- 407

OCR for page 407
408 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM nancies averted by providing Title X family planning services; (2) reducing infertility among women attending family planning clinics by identify- ing chlamydia infection; and (3) reducing invasive cervical cancer among women attending family planning clinics. Additional pertinent indicators include screening for other sexually transmitted diseases (STDs) that are treatable and preventable and have significant maternal and perinatal long- term implications (syphilis, gonorrhea, HIV). Available monitoring systems adequately address these indicators. Future primary data collection efforts should include the following patient-centered priorities: • Patient-specific reproductive desires/outcomes. Instead of counting visits and number of new visits, the focus should be changed to: — HARI: What are pregnancy plans for the year? Among those making a repeat visit, have these goals been met? — Percent clients not pregnant at next visit (denominator: those planning contraception) — Percent clients still using any method — Percent referrals for pregnancy termination or percent referrals for prenatal care for unintended pregnancy — Percent pregnant who desired pregnancy • Patient-specific evaluation of the quality of information provided — Technical competence and interpersonal skills of provider — Client comprehension (health literacy) This appendix addresses the measurement of the quality of reproductive health services provided under the Title X program. The discussion includes an assessment of how well the FPAR measures quality, a description of quality initiatives undertaken by family planning programs, and consider- ation of how the quality of services should be assessed in various settings. An assessment of the costs and benefits associated with introducing quality measures into family planning clinics is beyond the scope of this discussion because of the limited data available to inform such an assessment. OvERvIEW OF FINDINGS AND RECOMMENDATIONS To assess quality, there must be consensus on what quality is (e.g., how it is defined), as well as agreement on what measures are to be used to monitor and report quality. Several definitions of quality are pertinent, including those of the Institute of Medicine (IOM) and the World Health Organization (WHO) (WHO, 1998; IOM, 2001). Both emphasize proper performance of care based on current standards and knowledge, recogniz- ing the potential for individual and societal benefit. Judith Bruce offers a

OCR for page 407
409 APPENDIX K family planning–specific definition of quality: “providing a range of services that are safe, effective, and that satisfy clients’ needs and wants” (Bruce, 1990). For this study, the author performed a focused review of the litera- ture and evaluated the FPARs for 2001–2006 (Frost, 2001, 2002, 2003; Frost and Frohwirth, 2005; Fowler et al., 2006; RTI International, 2006), the Title X Program Assessment Rating Tool (PART) evaluation (OMB, 2005), and an advance copy of the FPCA proposed Performance Measure- ment System (FPCA, 1999). This review led to the conclusion that a full assessment of the quality of the Title X program cannot be performed at this time. Based on the lim- ited information available, primarily the 2005–2006 FPARs, the program does appear to be doing what it set out to do; however, the extent to which its services are underused, overused, or used inappropriately (measures of poor quality) cannot be determined from these reports. Whereas the readily apparent structure and process variables appear to have face and construct validity, the outputs and outcomes need further clarification. Regional and/ or population data are needed to support any claim for program effects. Think tanks and advocacy groups, such as the Alan Guttmacher Institute and the Center for Reproductive Rights, believe there is a logic model to support a causal link between family planning services and pregnancies averted and dollars saved (Center for Reproductive Rights, 2004; Dreweke, 2006). Evidence of such program impact would clearly help advance the policy mandate for more funding, more marketing, and the development of more meaningful indicators to advance the reproductive health agenda. While there are substantial data to support a framework for both qual- ity assessment and program evaluation within the family planning field, there are limited data on the quality of national family planning services, and there does not appear to be a national consensus about the quality domains or quality indicators that should be routinely (or periodically) monitored and reported. There is a tendency to count resources, visits, and tests, with less energy directed toward capturing data on intermediate effects or long-term impact, such as pregnancies prevented (or planned) or overall reduction in population fertility rates or STD rates. Surprisingly, the lack of data on the quality of family planning ser- vices in the United States in general and under Title X in particular is not due to the lack of an evidence base for indicators, but to an apparent failure to capitalize on the extensive work that has been done and applied internationally in this arena. Similarly, and not surprisingly, the dearth of quality-of-care research in the area of family planning in the United States contrasts with the quality-of-care work in the medical/surgical arena, likely because family planning has historically been focused primarily on women. Despite widespread acceptance internationally, only recently have the ben- efits of family planning been claimed to extend to improved child, family,

OCR for page 407
410 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM and world health in the United States (Cleland et al., 2006). The inclusion of reproductive health services that encompass infertility and STD/HIV screening and treatment has broadened family planning services to extend to outreach programs for men, but this, too, has been a relatively new phe- nomenon. The absence or relative paucity of indicators for women’s health, maternity services, and child health has not gone unnoticed by health ser- vice researchers, but progress on the development of these indicators has been slow (Schuster et al., 1997; Kerr et al., 2000; Gregory et al., 2005; Korst et al., 2005). In fact, the Agency for Healthcare Research and Quality (AHRQ)—the national leader in advancing the quality agenda—specifically excludes pregnancy and children from its current inpatient and patient safety indicators (AHRQ, 2004, 2006a,b). A set of pediatric inpatient indi- cators was recently developed (AHRQ, 2006c). Attempts to achieve federal accountability across all federally funded programs have spurred the development of indicators for Title X. Current efforts by the FPCA to develop consensus-based performance indicators are a step in the right direction. Efforts to capture additional measures, already defined by the Department of Health and Human Services (HHS) via Healthy People 2010, would further the cause (HHS, 2000a). Examples of representative Healthy People 2010 reproductive and STD/HIV goals that would be consistent with Title X program goals can be found in Annex K-1. Similarly, focused incorporation of selected indicators from the Handbook of Indicators for Family Planning Program Evaluation, which contains more than 200 indicators, would be beneficial and could elevate family planning and preventive reproductive health services in general, and the Title X program in particular, from a relatively obscure program for the poor to a more prominent national program dedicated to improv- ing the health and well-being of women, children, and families (Bertrand et al., 1994). Glasier et al., in an editorial about family planning services and women’s health, state that “unsafe sex is the 2nd most important risk factor for disability and death in the world’s poorest communities, and the 9th most important in developed countries” (Glasier et al., 2006). These authors contend that reproductive health services are of poor quality and underused because discussions about sexual intercourse and sexuality make people uncomfortable. Further, they suggest that the increasing influence of conservative, political, religious, and cultural forces threatens to undermine what has been achieved to date. To make this admonishment meaningful in a different social context, the average youth watches 3 hours of television daily, whereas 59 percent of adults watch television 2 or more hours (Roberts et al., 1999; Bowman, 2006). It is inevitable that most Americans will therefore encounter sexual messages given how commonly they appear on television; approximately 64 percent of all programs have sexual content based on analysis of the

OCR for page 407
411 APPENDIX K 2001–2002 TV season. Among programs with sexual content, there was an average of 4.4 scenes per hour. Talk about sex was more common (61 percent) than overt portrayals of sexual behaviors (32 percent). However, approximately 14 percent (or one of every seven programs) included a portrayal of sexual intercourse, depicted or strongly implied (Kunkel et al., 2003; Collins et al., 2004). Since children tend to model what they see, it is perhaps not surprising that the average age of sexual debut overall is 14.2 years (13.1 for boys, 15.0 for girls) (Sandfort et al., 2008). Moreover, the rate of premarital sex continues to be high. Fully 90 percent of women aged 15–44 responding to the National Survey of Family Growth (NSFG) in 2002 had had premarital intercourse (Mosher et al., 2004). The social marketing of sexuality is rampant, while the marketing of abstinence, contraception, and preventive reproductive behaviors is glaringly absent, despite evidence that it could be beneficial (Piotrow et al., 1997). Based on a review of the literature and a synthesis of both national and international data regarding quality measurement in family planning, there is a solid evidence base for the quality domains that should be included, and there is no dearth of potential indicators that could be used. The FPCA Performance Measurement System, coupled with the Healthy People 2010 goals, would be an excellent start. These should be fortified with a paradigm policy shift that emphasizes planning for pregnancy as much as planning to prevent pregnancy (HARI) (Jain et al., 1992). This is entirely consistent with the IOM recommendation to adopt a social norm whereby all pregnancies are intended—clearly and consciously desired at the time of conception (IOM, 1995). Further analysis of the merits of the Title X program is needed using various methodologies, such as provider observation or simulated patients, to document technical competence and communication skills. Additionally, patient exit interviews should not be limited to satisfaction surveys or closed questions about what was discussed as is currently the trend, but should include measures verifying literacy and comprehension. Community focus groups should ascertain additional perceptions of clinic quality with less possibility of courtesy bias (Sullivan and Bertrand, 2000). The infrastructure to implement many of these suggestions exists, as evidenced by the FPARs, the Program Guidelines, application criteria, and the Office of Population Affairs’ (OPA) 2006 Family Planning Program Priorities, Legislative Mandates, Key Issues (HHS, 2006). However, data collection efforts by Title X clinic sites may already be burdensome, and future funding should support either quality-monitoring full-time equiva- lents (FTEs) or an electronic medical record (EMR) system without sacrific- ing support for existing services. The EMRs should be designed to capture indicator data. There should be a vision or capacity for shared informa- tion (regional health information network), given the transient nature of

OCR for page 407
412 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM the poor population. The opportunity for shared patient-level data (with appropriate consent and Health Insurance Portability and Accountabil- ity Act [HIPAA] considerations) could optimize STD/HIV prevention and treatment. Additionally, a widely acknowledged strength of the Title X program is its information–education–counseling (IEC) emphasis and the associated training and audiovisual tools directed at both clients and pro- viders. These tools, coupled with standardized protocols, could be shared across state and federal agencies, including STD clinics and public schools, as well as incorporated into medical, nursing, and residency training cur- ricula. There is clearly stakeholder support for enhancing the quality of family planning services at the user/provider level, but broader support at the administrative and policy levels is needed to facilitate the development of a national agenda emphasizing the maternal, child, and family benefits of family planning and preventive health services in general and the Title X program in particular. The remainder of this appendix provides an overview of the theoretical frameworks that inform this review. It also provides an assessment of how well the FPAR measures quality based on these frameworks, a description of quality initiatives undertaken by family planning programs, and an over- view of how the quality of services should be assessed. The latter overview uses representative examples of available indicators, highlighting where existing indicators may need to be modified or expanded to address clients’ needs and wants and the HARI principle. ThEORETICAL FRAMEWORKS Several quality and reproductive health frameworks inform this discussion: • Donabedian quality model: Structure–Process–Outcome Model (Donabedian, 1968) • Bertrand et al. program evaluation model (Bertrand et al., 1994; Sullivan and Bertrand, 2000) • International frameworks for quality family planning services — Bruce and Jain et al. model of quality family planning services (Bruce, 1990; Jain et al., 1992) International Planned Parent- hood Federation (IPPF) framework (IPPF, 1998) — 1994 International Conference on Population and Develop- ment, Cairo (United Nations, 1994) • Lu and Halfon Reproductive Health Continuum (Lu and Halfon, 2003) • AHRQ criteria for an acceptable indicator (AHRQ, 2001)

OCR for page 407
413 APPENDIX K These frameworks need to be integrated and ultimately accepted by various stakeholders. Donabedian’s Structure–Process–Outcome Model Donabedian’s model has been widely endorsed as the theoretical frame- work for quality measurement (Donabedian, 1968). Key representative variables critical for assessment of the quality of Title X programs using the various domains outlined by Donabedian are shown in Figures K-1 through K-3. Structure Where is the facility located? What are the physical and administra- tive barriers to accessing services? Is the site geographically convenient and accessible by public transportation? Are the hours varied enough to meet the needs of clients—for example, evening or weekend hours for people who work or teens in school? Are the building and waiting area physically appealing? Once inside, is the waiting area comfortable, are there enough chairs, and are there diversions for accompanying children? Is there an opportunity for private discussion between clients and various clinical and nonclinical staff? Both national and international studies evaluating client satisfaction have demonstrated that all of these factors may influence whether a client comes for an initial visit or returns for subsequent visits— which of necessity impacts the short- and long-term goals of initiation and continuation of contraceptive services (Alden, 2004; Zaky et al., 2007). These may be significant issues at the individual clinical sites, determining Structure Faculty Staff Infrastructure Location? Number of FTEs Sociopolitics Accessible? FTE/MD ratio Cultural milieu Public transportation Exper tise of non-MDs Technology Training opportunities Convenient hours Information technology (IT) Physically appealing? Training requirements EMRs Well lit Communication skills Advisory board Comfor table waiting area Funding/revenue Private Competing services Demographics/case mix Marketing FIGURE K-1 Examples of variables representing Donabedian’s structure quality domain.

OCR for page 407
414 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM • Contraceptive services and counseling • Pregnancy diagnosis and counseling (infertility) — Level I: initial interview, education, physical exam, counseling and appropriate referral (mandatory) — Level II: semen analysis, assessment of ovulatory function, and postcoital testing (offer if a clinician with this training is available) — Level III: More sophisticated services than Levels I and II (beyond scope of Title X) • Related preventive health services — Breast and cervical cancer screening — STD/HIV screening — Human papillomavirus (HPV) screening (not explicitly stated, but indirectly through cervical cancer screening and specified referral criteria) FIGURE K-2 Services that must be provided by Title X programs. Process Outcome Structure Donabedian Effect Input Processes Output Outcome Impact Program PROGRAM Evaluation POPULATION FIGURE K-3 Similarities between quality monitoring and program evaluation. whether or not a client seeks initial or follow-up services. However, at the Figure K-3 program level, only privacy issues are relevant from a regulatory stand- point. Staffing mix and trainingvopportunities determine technical compe- ector, editable tence. The sociopolitical and cultural milieu can impact what methods are available (e.g., termination or emergency contraception). Ready access to information technology (IT) support can facilitate notification after positive test results and timely data acquisition for audits. The strength of the advi- sory board can determine additional funding options for outreach, social marketing, and facility improvement.

OCR for page 407
415 APPENDIX K Process The services that must be provided by Title X programs have been man- dated by legislation (Figure K-2) (HHS, 2001, 2006). The federal Family Planning Program is authorized under Title X of the Public Health Service Act, which was created in 1970 to provide family planning and related pre- ventive health care. The mission of Title X is to provide the information and means necessary for individuals to exercise choice in determining the num- ber and spacing of their children (AHRQ, 2006c). In addition to providing a broad range of contraceptive supplies, counseling, and information on a confidential basis, clinics with Title X funding must provide reproductive health and preventive health services that include breast and pelvic exams to screen for breast cancer, cervical cancer, and STDs (including HIV); pregnancy diagnosis; patient education; reproductive health counseling; and appropriate social and referral services. A mechanism to determine that the information provided has been understood should be established and documented. Further, the Title X legislation authorizes funding for family planning services, training, research, information, and education. The pro- gram is administered by OPA through the Office of Family Planning (OFP). Services are intended for all who want and need them, with priority for low-income individuals. The Program Guidelines stipulate the involvement of an advisory board and community participation in the development of educational materials and project promotion. While these are nationally legislated mandates, how these mandates get carried out is influenced by regional or local implementation strategies, largely resulting in site-specific policies and procedures (see Figure K-3). This variation is due to previously mentioned structural variables such as sociocultural or political milieus, regional demographics and/or case mix, and availability of staff and legal or licensing mandates regarding what types of providers can perform which types of services. For example, Gillian et al. found improved compliance among African American teens due to the involvement of all clinical staff in the process. All clinic employees, includ- ing clerical and professional staff, forged relationships through shared backgrounds and experiences, honesty, and additional time spent with the teens (Gilliam et al., 2007). Outcome See Figure K-3 and the discussion below of the Bertrand et al. (1994) program evaluation model.

OCR for page 407
416 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Relationship Between Quality Assessment and Program Evaluation It is difficult if not impossible to distinguish quality assessments from program evaluations when reviewing the family planning literature. Hence, Figure K-3 shows an integrated model of quality and program evaluation. Inputs and processes are conceptually equivalent to Donabedian’s structure and process variables. Input examples pertinent to family planning include such items as personnel, financial resources, facilities, and equipment. Both output and outcomes are comparable to Donabedian’s outcome domain. However, output specifically refers to outcomes at the program level and is usually defined by service utilization (number of visits, number of new or continuing contraceptive users). On the other hand, program outcomes defined by program effect and program impact are measured at the popu- lation level. An example of program effect is the prevalence of contracep- tive use as measured in a population survey, and an example of program impact is the regional or national fertility rate or desired pregnancy rate. The maturity of the program determines the type of evaluation strategy to use—the more mature the program, the more impact one would expect to be able to demonstrate. By most standards, Title X is a mature program (more than 30 years old), but it is immature in quality assessment; hence the majority of indicators will initially be focused on processes and outputs. However, the program has been in existence long enough to be capable of demonstrating long-term impact if the correct data are made available for collection and interpretation. International Frameworks for quality Family Planning Services Several authors have published extensively on quality assessment as it relates to the international family planning and reproductive health care arena. Most authors build on or adapt the Bruce (1990) and Jain et al. (1992) framework, which identifies six elements of quality: • Choice of method, • Information provided to the client, • Technical competence of providers, • Interpersonal relations between clients and providers, • Mechanisms to encourage continuity of care, and • Appropriate constellation of services. Outcome measures include program readiness, the provider perspective, the services delivered, and the client perspective and the services received, with particular emphasis on client knowledge, client satisfaction, client health, and contraceptive use—both acceptance and continuation.

OCR for page 407
417 APPENDIX K IPPF adopted the above framework as its model for quality after add- ing client acceptability to the above list and emphasizing that these criteria should be defined as clients’ and providers’ rights and expectations (IPPF, 1998). Table K-1 outlines the IPPF client and provider “bill of rights.” The most extensive and comprehensive framework for family planning quality indicators and program evaluation is that proposed by Bertrand et al. (1994). While intended for developing countries, it is easily applicable to the United States and includes both program- and population-level indica- tors. Program-based or performance indicators include those factors related to inputs, processes, and outputs, whereas population-based or outcome indicators usually reflect intermediate effect or long-term impact. Bertrand et al. define eight broad categories for program evaluation, which outline the pathways by which programs achieve impact in a given country: • Indicators to measure the policy environment, • Indicators to measure service delivery operations, • Indicators to measure family planning outputs, TABLE K-1 International Planned Parenthood Federation Framework: Clients’ Rights and Providers’ Needs Client Rights Provider Needs Information about family planning Training—technical and communication skills Access to all service delivery systems and Information on technical issues updated regularly health care providers Choice of adopting, switch, or discontinuing Infrastructure (appropriate facility and efficient methods organization) Safety in the practice of family planning Supplies of contraceptives, equipment, and educational materials Privacy during discussions and physical Guidance from service guidelines, checklists, and examinations supervision Confidentiality of all personal information Back up from other providers Treated with dignity, courtesy, and Respect and recognition from coworkers, attentiveness managers, clients, community Comfort while receiving services Encouragement to provide good quality care Continuity of care for as long as client Feedback from managers, supervisors, and clients desires Opportunity to express opinions about he Opportunity to express their concerns relative to quality of care received clinic decision making SOURCE: Huezo, C. M., and S. Diaz. 1993. Quality of care in family planning: Clients’ rights and providers’ needs. In P. Senanyake and R. L. Kleinman, eds. Family planning: Meet- ing challenges, promoting choices. The proceedings of the IPPF Family Planning Congress, New Delhi, Oct. 1992. Pearl River, New York: Parthenon Publishing Group. Pp. 235-244. Reprinted with permission.

OCR for page 407
448 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM ANNEX TABLE K-2 Healthy People STD Objectives with Targets and Baselines Topic Objective Target Baseline Chlamydia Reduce infection in 3.0 5.0 those aged 15–24 Family planning clinics STD clinics 3.0 12.2 Males 3.0 15.7 (GC) Reduce infection 19/100 thousand 123/100 thousand Syphilis Eliminate primary 0.2/100 thousand 3.2/100 thousand and secondary syphilis from United States Herpes Decrease percent 14 percent 17 percent with genital infection Human Decrease percent papillomavirus with HPV (can help (HPV) minimize the (developmental) number of high-risk subtypes associated with cervical cancer) Pelvic inflammatory Reduce proportion 5 percent 8 percent disease of female who have ever acquired PID Fertility problems Decrease percent of 15 percent 27 percent of women women with with fertility problems fertility problems reported history of associated with PID chlamydia and PID Heterosexual HIV Reduce HIV (developmental) infections in females aged 13–24 associated with heterosexual contact

OCR for page 407
449 APPENDIX K ANNEX TABLE K-2 Continued Topic Objective Target Baseline Congenital syphilis Reduce congenital 1/100 thousand 27/100 thousand syphilis Neonatal STD Reduce neonatal (developmental) consequences from maternal STD Responsible teen Increase percent of 95 percent 85 percent sex adolescents who abstain from sex or use condoms if active Responsible sex on Increase number of television positive messages (developmental) related to responsible sexual behavior on television Hepatitis B vaccine Increase number of 90 percent 5 percent in STD clinics STD programs that offer hepatitis B vaccine Screening in Screen within detention and jails 24 hours of (developmental) admission and provide treatment before release Contracts to treat Increase percent of nonplan partners local health (developmental) departments that have contracts with managed care providers for treatment of nonplan partners Annual screening Increase percent of for chlamydia women under age (developmental) 25 screened annually continued

OCR for page 407
450 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM ANNEX TABLE K-2 Continued Topic Objective Target Baseline Screening of Increase percent of pregnant women pregnant women (developmental) screened for STD, HIV, and (BV) Compliance with Increase percent of 90 percent 70 percent recognized STD primary care treatment providers who treat patients with STDs who manage according to standards Provider referral for sex partners (developmental) • Annual screening for chlamydia (developmental)—Increase percent of women under age 25 screened annually. • Screening of pregnant women (developmental)—Increase percent of pregnant women screened for STD, HIV, and (BV). • Compliance with recognized STD treatment—Increase percent of primary care providers who treat patients with STDs who manage according to standards. • Provider referral for sex partners (developmental).

OCR for page 407
451 APPENDIX K ANNEX K-2 REPRESENTATIvE INDICATORS FOR EACh OF ThE EIGhT BROAD CATEGORIES IN BERTRAND ET AL. (1994) I. Policy Environment • Existence of a policy development plan • Number of appropriately disseminated policy analyses • Number of awareness-raising events targeted to leaders • Existence of a strategic plan for expanding the national family planning program • Integration of demographic data into development planning • Number of statements of leaders in support of family planning • Formal population policy addressing fertility and family planning • National family planning coordination • Level of the family planning program within the government administration • Levels of import duties and other taxes • Restrictions on advertising of contraceptives in the mass media • Absence of unwarranted restrictions on providers and users • Quality of program leadership • Extent of commercial-sector participation II. Service Delivery Operations • Management • Training • Commodities and logistics • Information–education–communication (IEC) • Research and evaluation III. Management (illustrative indicators) • Existence of a clear mission that contributes to the achievement of program goals • Realization of operational targets • Clearly defined organizational structure • Adequacy of staffing • Awareness of current financial position • Access to current information on key areas of program functioning • Access to current information on program progress • Capacity to track commodities

OCR for page 407
452 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM IV. Training • Number/percentage of courses that achieve learning objectives • Number/percentage of courses that contribute to the achievement of program training objectives • Number/percentage of courses in which the training methodology is appropriate for the transfer of skills and knowledge • Number of trainees by type • Number/percentage of trainees who have mastered relevant knowledge • Number/percentage of trainees competent to provide a specific family planning service • Number/percentage of trained providers assessed to be competent at a specified period (e.g., 6 months) post-training • Number/percentage of trainees who apply the skills to their subse- quent work V. Commodities and Logistics • Pipeline wastage • Percentage of storage capacity meeting acceptable standards • Frequency of stock-outs • Percentage of service delivery points (SDPs) stocked according to plan • Percentage of key personnel trained in contraceptive logistics • Composite indicator for commodities and logistics VI. Information–Education–Communication • Number of communications produced, by type, during a reference period • Number of communications disseminated, by type, during a refer- ence period • Percentage of target audience exposed to program messages, based on respondent recall • Percentage of target audience who correctly comprehend a given message • Number of contraceptive methods known • Percent of audience who acquire the skill to complete a certain task as a result of exposure to a specific communication • Percentage of target audience exposed to a specific message who report liking it • Number/percentage of target audience who discuss message(s) with others, by type of person • Percentage of target audience who advocate family planning practice

OCR for page 407
453 APPENDIX K VII. Research and Evaluation • Presence of an active research and evaluation unit • Extent of use of a service system • Conduct of periodic household and/or special-purpose surveys and studies • Conduct of operations research • Regular conduct of process evaluations • Conduct of effectiveness, efficiency, and impact evaluations • Use of research and evaluation results for program modification • Dissemination of research and evaluation results VIII. Family Planning Service Outputs • Accessibility (illustrative indicators) — Number of SDPs located within a fixed distance or travel time of a given community (i.e., service density) — Cost of 1 month’s supply of contraceptives as a percentage of monthly wages — Restrictive program policies on contraceptive choice — Percentage of the population who know of at least one source of contraceptive services and/or supplies — Percentage of nonuse related to psychosocial barriers • Quality of care (illustrative indicators) — Number of contraceptive methods available at a specific SDP — Percentage of counseling sessions with new acceptors in which provider discusses all methods — Percentage of client visits during which provider demonstrates skill in clinical procedures, including asepsis — Percentage of clients reporting sufficient time with provider — Percentage of clients informed of timing and sources for resupply/revisit — Percentage of clients who perceive that hours/days are convenient • Program Image — Number and type of activities to improve the public image of family planning during a reference period (e.g., 1 year) — Percentage of target population favorable to the (national) family planning program • Service Utilization — Number of visits to SDP(s) — Number of acceptors new to modern contraception

OCR for page 407
454 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM — Number of acceptors new to the institution — Number of new acceptors — Couple-years of protection (CYP) — Method mix — User characteristics — Continuation rates • Contraceptive Practice — Contraceptive prevalence rate (CPR) — Number of current users — Level of ever (past) use — Source of supply (by method) — Method mix — User characteristics — Continuation rates — Use failure rates • Fertility Impact

OCR for page 407
455 APPENDIX K REFERENCES AGI (The Alan Guttmacher Institute). 2000. Fulfilling the promise: Public policy and U.S. family planning clinics. New York: AGI. AHRQ (Agency for Healthcare Research and Quality). 2001. Refinement of the HCUP quality indicators. www.qualityindicators.ahrq.gov/data/hcup/qirefine.htm (accessed January 15, 2008). AHRQ. 2004. Prevention quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/pqi_overview.htm (accessed January 15, 2008). AHRQ. 2006a. Inpatient quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/iqi_overview.htm (accessed January 15, 2008). AHRQ. 2006b. Patient safety indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/psi_overview.htm (accessed January 15, 2008). AHRQ. 2006c. Pediatric quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/pdi_overview.htm (accessed January 15, 2008). AHRQ. 2007. Screening for chlamydial infection. Rockville, MD: AHRQ. http://www.ahrq. gov/clinic/uspstf/uspschlm.htm (accessed May 7, 2008). Alden, D. L. 2004. Client satisfaction with reproductive health care quality. Social Science & Medicine 59:2219–2232. Amaral, G., D. G. Foster, M. A. Biggs, C. B. Jasik, S. Judd, and C. D. Brindis. 2007. Public savings from the prevention of unintended pregnancy: A cost analysis of family planning services in California. Health Services Research 42(5):1960–1980. Becker, D., M. A. Koenig, Y. M. Kim, K. Cardona, and F. L. Sonenstein. 2007. The quality of family planning services in the United States: Findings from a literature review. Perspec- tives on Sexual and Reproductive Health 39:206–215. Bertrand, J. T., R. J. Magnani, and J. Knowles. 1994. Handbook of indicators for family plan- ning. Chapel Hill, NC: Carolina Population Center. http://www.cpc.unc.edu/measure/ publications/pdf/ms-94-01.pdf (accessed April 18, 2008). Bowman, S. A. 2006. Television-viewing characteristics of adults: Correlations to eating prac- tices and overweight and health status. http://www.cdc.gov/pcd/issues/2006/apr/050139. htm (accessed May 10, 2008). Bronstein, J. M., A. Vosel, S. K. George, C. Freeman, and L. A. Payne. 2007. Extending Medicaid coverage for family planning: Alabama’s first 4 years. Public Health Reports 122:190–197. Bruce, J. 1990. Fundamental elements of the quality of care: A simple framework. Studies in Family Planning 21(2):61–91. Center for Reproductive Rights. 2004. America must continue its commitment to reproductive health. http://www.reproductiverights.org (accessed May 10, 2008). Cleland, J., S. Bernstein, A. Ezeh, A. Faundes, A. Glasier, and J. Innis. 2006. Family planning: The unfinished agenda. Lancet 368:1810–1827. Collins, R. L., M. N. Elliott, S. H. Berry, D. E. Kanouse, D. Kunkel, S. B. Hunter, and A. Miu. 2004. Watching sex on television predicts adolescent initiation of sexual behavior. Pediatrics 114(3):e280–e289. Deming, W. 1986. Out of the crisis. Cambridge, MA: MIT Center for Advanced Engineering Study. Donabedian, A. 1968. Promoting quality through evaluating the process of patient care. Medical Care 6:181–202. Dreweke, J. 2006. Each $1 invested in Title X Family Planning program saves $3.80. Doubling funding could prevent 244,000 unplanned pregnancies, result in net savings of almost $800 million. New York: Guttmacher Institute. http://www.guttmacher.org/ media/nr/2006/11/16/index.html (accessed May 7, 2008).

OCR for page 407
456 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Finer, L. B., J. E. Darroch, and J. J. Frost. 2002. US agencies providing publicly funded contra- ceptive services in 1999. Perspectives on Sexual and Reproductive Health 34:15–24. Fisher, E. S., and J. E. Wennberg. 2003. Health care quality, geographic variations, and the challenge of supply-sensitive care. Perspectives in Biology and Medicine 46:69–79. Foster, D. G., C. M. Klaisle, M. Blum, M. E. Bradsberry, C. D. Brindis, and F. H. Steward. 2004. Expanded state-funded planning services: Estimating pregnancies averted by the Family PACT Program in California, 1997–1998. American Journal of Public Health 94(8):1341–1346. Fowler, C. I., J. Gable, and J. Wang. 2008. Family planning annual report: 2006 national summary. Research Triangle Park, NC: RTI International. FPCA (Family Planning Council of America, Inc.). 1999. Performance monitoring system (draft provided by A. Stith, IOM). Frost, J. J. 2002. Family planning annual report: 2001 summary, part 1. New York: AGI. Frost, J. J. 2003. Family planning annual report: 2002 summary, part 1. New York: AGI. Frost, J. J., and L. Frohwirth. 2005. Family planning annual report: 2004 summary, part 1. New York: AGI. Gilliam, M. L., and K. Hernandez. 2007. Providing contraceptive care to low income African American teens: The experience of urban community health centers. Journal of Com- munity Health 32:231–244. Glasier, A., A. M. Gülmezoglu, G. P. Schmid, C. G. Moreno, and P. F. Van Look. 2006. Sexual and reproductive health: A matter of life and death. Lancet 368(9547):1595–2607. Gregory, K. D., C. J. Hobel, L. M. Korst, M. Lu, and C. Reyes. 2005. A framework for devel- oping Maternal Quality Care Indicators. Prepared for the California Department of Health and Human Services, Maternal and Child Health Branch. Guttmacher Institute. 2008. Facts in brief. Facts on contraceptive use. http://www.guttmacher. org/pubs/fb_contr_use.html (accessed May 8, 2008). HHS (U.S. Department of Health and Human Services). 2000a. Healthy people 2010 (2nd edition). Washington, DC: U.S. Government Printing Office. HHS. 2000b. With understanding and improving health and objectives for improving health (2 volumes). In Healthy people 2010 (2nd edition). Washington, DC: HHS. HHS. 2000c. Healthy people 2010—Reproductive health. www.hhs.gov/opa/pubs/hp201_rh.html (accessed April 28, 2008). HHS. 2001. Program guidelines for project grants for family planning services. http://www.hhs. gov/opa/familyplanning/toolsdocs/2001_ofp_guidelines_complete.pdf (accessed April 24, 2008). HHS. 2006. Family planning program priorities, legislative mandates, key issues. http://www. hhs.gov/opa (accessed April 24, 2008). IHI (Institute for Healthcare Improvement). 2003. The breakthrough series: IHI’s collabora- tive model for achieving breakthrough improvement. IHI Innovation Series white paper. Boston: IHI. www.IHI.org (accessed April 28, 2008). IOM (Institute of Medicine). 1995. The best intentions: Unintended pregnancy and the well- being of children and families. Washington, DC: National Academy Press. IOM. 2001. Crossing the quality chasm: A new health system for the 21st century. Washing- ton, DC: National Academy Press. IPPF (International Planned Parenthood Federation). 1998. Family planning programs: Improv- ing quality (sidebars). Population Reports, Series J, No. 47. Jain, A., J. Bruce, and S. Kumar. 1992. Quality of services, programme efforts and fertility reduction. In Family planning programmes and fertility. Edited by J. F. Phillips and J. A. Ross. Oxford: Clarendon Press.

OCR for page 407
457 APPENDIX K Kerr, E. A., C. L. Damberg, and S. M. Asch (editors). 2000. Quality of care for women: A review of selected clinical conditions and quality indicators. Santa Monica, CA: RAND. Korst, L. M., K. D. Gregory, M. C. Lu, C. Reyes, C. J. Hobel, and G. F. Chavez. 2005. A framework for the development of maternal quality of care indicators. Maternal and Child Health Journal 9(3):317–341. Kunkel, D., K. Eyal, E. Biely, et al. 2003. Sex on TV3: A biennial report to the Kaiser Family Foundation. Menlo Park, CA: The Henry J. Kaiser Foundation. www.kff.org/entme- dia/loader.cfm?url=/commonspot/security/getfile.cfm&PageID=14209 (accessed April 28, 2008). Lindberg, L. D., J. J. Frost, C. Sten, and C. Dailard. 2006. The provision of funding and con- traceptive services at publicly funded family planning agencies: 1995–2003. Perspectives on Sexual and Reproductive Health 38:37–45. Lu, M. C., and N. Halfon. 2003. Racial and ethnic disparities in birth outcomes: A life course perspective. Maternal and Child Health Journal 7:3–18. Meyers, D., T. Wolff, K. D. Gregory, L. Marion, V. Moyer, H. Nelson, D. Petitti, and G. Sawaya. 2008. A summary of United States Preventive Service Task Force (USPSTF) recommendations for STI screening. American Family Physician 77(6):819–824. Mosher, W. D., G. M. Martinez, A. Chandra, J. C. Abma, and S. J. Wilson. 2004. Use of contraception and use of family planning services in the United States, 1982–2002. Hyattsville, MD: National Center for Health Statistics. NCQA (National Center Quality Assurance). 2007. Improving chlamydia screening. Strat- egies from top performing health plans. http://www.ncqa.org/Portals/0/Publications/ Resource%20Library/Improving_Chlamydia_Screening_08.pdf (accessed May 7, 2008). NCVHS (National Committee on Vital and Health Statistics). 2000. Toward a national health information infrastructure interim report. http://ncvhs.hhs.gov/NHII2kReport.htm (ac- cessed April 8, 2008). OMB (Office of Management and Budget). 2005. The Program Assessment Rating Tool (PART). Family planning assessment. http://www.whitehouse.gov/omb/expectmore/ detail/10003513.2005.html (accessed April 8, 2008). Paine, K., M. Thorogood, and K. Wellings. 2000. The impact of the quality of family plan- ning services on safe and effective contraceptive use: Literature review. Human Fertility 3(3):186–193. Piotrow, P. T., D. L. Kincaid, J. G. Rimon II, and W. Rinehart (editors). 1997. Health commu- nication: Lessons from family planning and reproductive health. Westport, CT: Praeger. RamaoRao, S., and R. Mohanam. 2003. The quality of family planning programs: Concepts, measurements, interventions and effections. Studies in Family Planning 34:227–248. Roberts, D. F., U. G. Foehr, V. J. Rideout, and M. Brodie. 1999. Kids and media at the new millennium. In A Kaiser Family Foundation report: A comprehensive national analysis of children’s media use: Executive summary. Menlo Park, CA: The Henry J. Kaiser Family Foundation. RTI International (Research Triangle Institute International). 2006. Family planning annual report: 2005 national summary. Research Triangle Park, NC: RTI International. Sandfort, T. G., M. Orr, J. S. Hirsch, and J. Santelli. 2008. Long-term health correlates of timing of sexual debut: Results from a national US study. American Journal of Public Health 98(1):155–161. Schunmann, C., and A. Glasier. 2006. Measuring pregnancy intention and its relationship with contraceptive use among women undergoing therapeutic abortion. Contraception 73:520–524.

OCR for page 407
458 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Schuster, M. A., S. M. Asch, E. A. McGlynn, E. A. Kerr, A. M. Hardy, and D. S. Gifford. 1997. Development of a quality of care measurement system for children and adolescents. Archives of Pediatrics and Adolescent Medicine 151:1085–1092. Sofaer, S., and K. Firminger. 2005. Patient perceptions of the quality of health services. Annual Review of Public Health 26:513–559. Speroff, T., and G. T. O’Connor. 2004. Study designs for PDSA quality improvement research. Quality Management in Health Care 13:17–32. Sullivan, T. M., and J. T. Bertrand. 2000. Monitoring quality of care in family planning by the quick investigation of quality (QIQ): County reports. MEASURE Evaluation Technical Report Series No. 5. Chapel Hill, NC: Carolina Population Center. Thornburn, S., and L. M. Bogart. 2005a. African American women and family planning services: Perceptions of discrimination. Women’s Health 42:23–39. Thornburn, S., and L. M. Bogart. 2005b. Conspiracy beliefs about birth control: Barriers to pregnancy prevention among African Americans of reproductive age. Health Education and Behavior 32(4):474–487. United Nations. 1994. International Conference on Population and Development (ICPD), September 1994, Cairo, Egypt. http://www.un.org/popin/icpd2.htm (accessed April 8, 2008). Wennberg, J. E. 1999. Understanding geographic variations in health care delivery. New England Journal of Medicine 340:52–53. WHO (World Health Organization). 1998. WHO definition of quality. Population Reports Series J, No. 47. WHO. 2004. Medical eligibility criteria for contraceptive use (3rd Edition, Reproductive Health and Research). Geneva, Switzerland: WHO. Winter, L., and A. S. Goldy. 1987. Staffing patterns in family planning clinics: Which model is best? Family Planning Perspectives 19(3):102–106. Zaky, H. H., H. A. Khattab, and D. Galal. 2007. Assessing the quality of reproductive health services in Egypt via exit interviews. Maternal and Child Health Journal 11:301–306.