A Review of the HHS Family Planning Program: Mission, Management, and Measurement of Results (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Appendix K Measurement of Quality in the Title X Family Planning Program Kimberly D. Gregory, M.D., M.P.H. SUMMARY There exist a solid evidence base for quality domains and an extensive list of potential indicators that can be used to measure quality performance in family planning programs. The Family Planning Annual Report (FPAR), the Family Planning Council of America Performance Monitoring System (FPCA), and Healthy People 2010 reproductive health goals are explicitly specified indicators representative of the more than 200 indicators that have been suggested in this arena. There is some consistency (or overlap) in indicators among these documents, and several of the indicators reflect goals adopted by external agencies, such as the Healthcare Employer Data and Information Set (HEDIS) measures espoused by the National Center for Quality Assurance (NCQA) (specifically breast and cervical cancer screen- ing and screening for chlamydia). Two obvious deficiencies in the currently reported measures are: • The lack of outcome data that are patient-specific about reproduc- tive desires (specifically Helping patients Achieve their Reproduc- tive Intentions [HARI]; patients should plan for pregnancy as well as plan to prevent pregnancy). • The lack of data on provider competency and interpersonal skills or client comprehension/literacy. The Title X program mandate specifies three long-term measures that are to be reported annually: (1) increasing the number of unintended preg- 407 

408 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM nancies averted by providing Title X family planning services; (2) reducing infertility among women attending family planning clinics by identify- ing chlamydia infection; and (3) reducing invasive cervical cancer among women attending family planning clinics. Additional pertinent indicators include screening for other sexually transmitted diseases (STDs) that are treatable and preventable and have significant maternal and perinatal long- term implications (syphilis, gonorrhea, HIV). Available monitoring systems adequately address these indicators. Future primary data collection efforts should include the following patient-centered priorities: • Patient-specific reproductive desires/outcomes. Instead of counting visits and number of new visits, the focus should be changed to: — HARI: What are pregnancy plans for the year? Among those making a repeat visit, have these goals been met? — Percent clients not pregnant at next visit (denominator: those planning contraception) — Percent clients still using any method — Percent referrals for pregnancy termination or percent referrals for prenatal care for unintended pregnancy — Percent pregnant who desired pregnancy • Patient-specific evaluation of the quality of information provided — Technical competence and interpersonal skills of provider — Client comprehension (health literacy) This appendix addresses the measurement of the quality of reproductive health services provided under the Title X program. The discussion includes an assessment of how well the FPAR measures quality, a description of quality initiatives undertaken by family planning programs, and consider- ation of how the quality of services should be assessed in various settings. An assessment of the costs and benefits associated with introducing quality measures into family planning clinics is beyond the scope of this discussion because of the limited data available to inform such an assessment. Overview of findings and recommendations To assess quality, there must be consensus on what quality is (e.g., how it is defined), as well as agreement on what measures are to be used to monitor and report quality. Several definitions of quality are pertinent, including those of the Institute of Medicine (IOM) and the World Health Organization (WHO) (WHO, 1998; IOM, 2001). Both emphasize proper performance of care based on current standards and knowledge, recogniz- ing the potential for individual and societal benefit. Judith Bruce offers a 

APPENDIX K 409 family planning–specific definition of quality: “providing a range of services that are safe, effective, and that satisfy clients’ needs and wants” (Bruce, 1990). For this study, the author performed a focused review of the litera- ture and evaluated the FPARs for 2001–2006 (Frost, 2001, 2002, 2003; Frost and Frohwirth, 2005; Fowler et al., 2006; RTI International, 2006), the Title X Program Assessment Rating Tool (PART) evaluation (OMB, 2005), and an advance copy of the FPCA proposed Performance Measure- ment System (FPCA, 1999). This review led to the conclusion that a full assessment of the quality of the Title X program cannot be performed at this time. Based on the lim- ited information available, primarily the 2005–2006 FPARs, the program does appear to be doing what it set out to do; however, the extent to which its services are underused, overused, or used inappropriately (measures of poor quality) cannot be determined from these reports. Whereas the readily apparent structure and process variables appear to have face and construct validity, the outputs and outcomes need further clarification. Regional and/ or population data are needed to support any claim for program effects. Think tanks and advocacy groups, such as the Alan Guttmacher Institute and the Center for Reproductive Rights, believe there is a logic model to support a causal link between family planning services and pregnancies averted and dollars saved (Center for Reproductive Rights, 2004; Dreweke, 2006). Evidence of such program impact would clearly help advance the policy mandate for more funding, more marketing, and the development of more meaningful indicators to advance the reproductive health agenda. While there are substantial data to support a framework for both qual- ity assessment and program evaluation within the family planning field, there are limited data on the quality of national family planning services, and there does not appear to be a national consensus about the quality domains or quality indicators that should be routinely (or periodically) monitored and reported. There is a tendency to count resources, visits, and tests, with less energy directed toward capturing data on intermediate effects or long-term impact, such as pregnancies prevented (or planned) or overall reduction in population fertility rates or STD rates. Surprisingly, the lack of data on the quality of family planning ser- vices in the United States in general and under Title X in particular is not due to the lack of an evidence base for indicators, but to an apparent failure to capitalize on the extensive work that has been done and applied internationally in this arena. Similarly, and not surprisingly, the dearth of quality-of-care research in the area of family planning in the United States contrasts with the quality-of-care work in the medical/surgical arena, likely because family planning has historically been focused primarily on women. Despite widespread acceptance internationally, only recently have the ben- efits of family planning been claimed to extend to improved child, family, 

410 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM and world health in the United States (Cleland et al., 2006). The inclusion of reproductive health services that encompass infertility and STD/HIV screening and treatment has broadened family planning services to extend to outreach programs for men, but this, too, has been a relatively new phe- nomenon. The absence or relative paucity of indicators for women’s health, maternity services, and child health has not gone unnoticed by health ser- vice researchers, but progress on the development of these indicators has been slow (Schuster et al., 1997; Kerr et al., 2000; Gregory et al., 2005; Korst et al., 2005). In fact, the Agency for Healthcare Research and Quality (AHRQ)—the national leader in advancing the quality agenda—specifically excludes pregnancy and children from its current inpatient and patient safety indicators (AHRQ, 2004, 2006a,b). A set of pediatric inpatient indi- cators was recently developed (AHRQ, 2006c). Attempts to achieve federal accountability across all federally funded programs have spurred the development of indicators for Title X. Current efforts by the FPCA to develop consensus-based performance indicators are a step in the right direction. Efforts to capture additional measures, already defined by the Department of Health and Human Services (HHS) via Healthy People 2010, would further the cause (HHS, 2000a). Examples of representative Healthy People 2010 reproductive and STD/HIV goals that would be consistent with Title X program goals can be found in Annex K-1. Similarly, focused incorporation of selected indicators from the Handbook of Indicators for Family Planning Program Evaluation, which contains more than 200 indicators, would be beneficial and could elevate family planning and preventive reproductive health services in general, and the Title X program in particular, from a relatively obscure program for the poor to a more prominent national program dedicated to improv- ing the health and well-being of women, children, and families (Bertrand et al., 1994). Glasier et al., in an editorial about family planning services and women’s health, state that “unsafe sex is the 2nd most important risk factor for disability and death in the world’s poorest communities, and the 9th most important in developed countries” (Glasier et al., 2006). These authors contend that reproductive health services are of poor quality and underused because discussions about sexual intercourse and sexuality make people uncomfortable. Further, they suggest that the increasing influence of conservative, political, religious, and cultural forces threatens to undermine what has been achieved to date. To make this admonishment meaningful in a different social context, the average youth watches 3 hours of television daily, whereas 59 percent of adults watch television 2 or more hours (Roberts et al., 1999; Bowman, 2006). It is inevitable that most Americans will therefore encounter sexual messages given how commonly they appear on television; approximately 64 percent of all programs have sexual content based on analysis of the 

APPENDIX K 411 2001–2002 TV season. Among programs with sexual content, there was an average of 4.4 scenes per hour. Talk about sex was more common (61 percent) than overt portrayals of sexual behaviors (32 percent). However, approximately 14 percent (or one of every seven programs) included a portrayal of sexual intercourse, depicted or strongly implied (Kunkel et al., 2003; Collins et al., 2004). Since children tend to model what they see, it is perhaps not surprising that the average age of sexual debut overall is 14.2 years (13.1 for boys, 15.0 for girls) (Sandfort et al., 2008). Moreover, the rate of premarital sex continues to be high. Fully 90 percent of women aged 15–44 responding to the National Survey of Family Growth (NSFG) in 2002 had had premarital intercourse (Mosher et al., 2004). The social marketing of sexuality is rampant, while the marketing of abstinence, contraception, and preventive reproductive behaviors is glaringly absent, despite evidence that it could be beneficial (Piotrow et al., 1997). Based on a review of the literature and a synthesis of both national and international data regarding quality measurement in family planning, there is a solid evidence base for the quality domains that should be included, and there is no dearth of potential indicators that could be used. The FPCA Performance Measurement System, coupled with the Healthy People 2010 goals, would be an excellent start. These should be fortified with a paradigm policy shift that emphasizes planning for pregnancy as much as planning to prevent pregnancy (HARI) (Jain et al., 1992). This is entirely consistent with the IOM recommendation to adopt a social norm whereby all pregnancies are intended—clearly and consciously desired at the time of conception (IOM, 1995). Further analysis of the merits of the Title X program is needed using various methodologies, such as provider observation or simulated patients, to document technical competence and communication skills. Additionally, patient exit interviews should not be limited to satisfaction surveys or closed questions about what was discussed as is currently the trend, but should include measures verifying literacy and comprehension. Community focus groups should ascertain additional perceptions of clinic quality with less possibility of courtesy bias (Sullivan and Bertrand, 2000). The infrastructure to implement many of these suggestions exists, as evidenced by the FPARs, the Program Guidelines, application criteria, and the Office of Population Affairs’ (OPA) 2006 Family Planning Program Priorities, Legislative Mandates, Key Issues (HHS, 2006). However, data collection efforts by Title X clinic sites may already be burdensome, and future funding should support either quality-monitoring full-time equiva- lents (FTEs) or an electronic medical record (EMR) system without sacrific- ing support for existing services. The EMRs should be designed to capture indicator data. There should be a vision or capacity for shared informa- tion (regional health information network), given the transient nature of 

412 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM the poor population. The opportunity for shared patient-level data (with appropriate consent and Health Insurance Portability and Accountabil- ity Act [HIPAA] considerations) could optimize STD/HIV prevention and treatment. Additionally, a widely acknowledged strength of the Title X program is its information–education–counseling (IEC) emphasis and the associated training and audiovisual tools directed at both clients and pro- viders. These tools, coupled with standardized protocols, could be shared across state and federal agencies, including STD clinics and public schools, as well as incorporated into medical, nursing, and residency training cur- ricula. There is clearly stakeholder support for enhancing the quality of family planning services at the user/provider level, but broader support at the administrative and policy levels is needed to facilitate the development of a national agenda emphasizing the maternal, child, and family benefits of family planning and preventive health services in general and the Title X program in particular. The remainder of this appendix provides an overview of the theoretical frameworks that inform this review. It also provides an assessment of how well the FPAR measures quality based on these frameworks, a description of quality initiatives undertaken by family planning programs, and an over- view of how the quality of services should be assessed. The latter overview uses representative examples of available indicators, highlighting where existing indicators may need to be modified or expanded to address clients’ needs and wants and the HARI principle. Theoretical Frameworks Several quality and reproductive health frameworks inform this discussion: • Donabedian quality model: Structure–Process–Outcome Model (Donabedian, 1968) • Bertrand et al. program evaluation model (Bertrand et al., 1994; Sullivan and Bertrand, 2000) • International frameworks for quality family planning services — Bruce and Jain et al. model of quality family planning services (Bruce, 1990; Jain et al., 1992) International Planned Parent- hood Federation (IPPF) framework (IPPF, 1998) — 1994 International Conference on Population and Develop- ment, Cairo (United Nations, 1994) • Lu and Halfon Reproductive Health Continuum (Lu and Halfon, 2003) • AHRQ criteria for an acceptable indicator (AHRQ, 2001) 

APPENDIX K 413 These frameworks need to be integrated and ultimately accepted by various stakeholders. Donabedian’s Structure–Process–Outcome Model Donabedian’s model has been widely endorsed as the theoretical frame- work for quality measurement (Donabedian, 1968). Key representative variables critical for assessment of the quality of Title X programs using the various domains outlined by Donabedian are shown in Figures K-1 through K-3. Structure Where is the facility located? What are the physical and administra- tive barriers to accessing services? Is the site geographically convenient and accessible by public transportation? Are the hours varied enough to meet the needs of clients—for example, evening or weekend hours for people who work or teens in school? Are the building and waiting area physically appealing? Once inside, is the waiting area comfortable, are there enough chairs, and are there diversions for accompanying children? Is there an opportunity for private discussion between clients and various clinical and nonclinical staff? Both national and international studies evaluating c ­ lient satisfaction have demonstrated that all of these factors may influence whether a client comes for an initial visit or returns for subsequent ­visits— which of necessity impacts the short- and long-term goals of initiation and continuation of contraceptive services (Alden, 2004; Zaky et al., 2007). These may be significant issues at the individual clinical sites, determining Structure Faculty Staff Infrastructure Location? Number of FTEs Sociopolitics Accessible? FTE/MD ratio Cultural milieu Public transportation Expertise of non-MDs Technology Convenient hours Training opportunities Information technology (IT) Physically appealing? Training requirements EMRs Well lit Communication skills Advisory board Comfortable waiting area Funding/revenue Private Competing services Demographics/case mix Marketing FIGURE K-1  Examples of variables representing Donabedian’s structure quality domain. 

414 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM • Contraceptive services and counseling • Pregnancy diagnosis and counseling (infertility) — Level I: initial interview, education, physical exam, counseling and appropriate referral (mandatory) — Level II: semen analysis, assessment of ovulatory function, and postcoital testing (offer if a clinician with this training is available) — Level III: More sophisticated services than Levels I and II (beyond scope of Title X) • Related preventive health services — Breast and cervical cancer screening — STD/HIV screening — Human papillomavirus (HPV) screening (not explicitly stated, but indirectly through cervical cancer screening and specified referral criteria) FIGURE K-2  Services that must be provided by Title X programs. Structure Process Outcome Donabedian Effect Input Processes Output Outcome Impact Program Evaluation PROGRAM POPULATION FIGURE K-3  Similarities between quality monitoring and program evaluation. whether or not a client seeks initial or follow-up services. However, at the Figure K-3 program level, only privacy issues are relevant from a regulatory stand- point. Staffing mix and trainingvector, editable opportunities determine technical compe- tence. The sociopolitical and cultural milieu can impact what methods are available (e.g., termination or emergency contraception). Ready access to information technology (IT) support can facilitate notification after positive test results and timely data acquisition for audits. The strength of the advi- sory board can determine additional funding options for outreach, social marketing, and facility improvement. 

APPENDIX K 415 Process The services that must be provided by Title X programs have been man- dated by legislation (Figure K-2) (HHS, 2001, 2006). The federal Family Planning Program is authorized under Title X of the Public Health Service Act, which was created in 1970 to provide family planning and related pre- ventive health care. The mission of Title X is to provide the information and means necessary for individuals to exercise choice in determining the num- ber and spacing of their children (AHRQ, 2006c). In addition to providing a broad range of contraceptive supplies, counseling, and information on a confidential basis, clinics with Title X funding must provide reproductive health and preventive health services that include breast and pelvic exams to screen for breast cancer, cervical cancer, and STDs (including HIV); pregnancy diagnosis; patient education; reproductive health counseling; and appropriate social and referral services. A mechanism to determine that the information provided has been understood should be established and documented. Further, the Title X legislation authorizes funding for family planning services, training, research, information, and education. The pro- gram is administered by OPA through the Office of Family Planning (OFP). Services are intended for all who want and need them, with priority for low-income individuals. The Program Guidelines stipulate the involvement of an advisory board and community participation in the development of educational materials and project promotion. While these are nationally legislated mandates, how these mandates get carried out is influenced by regional or local implementation strategies, largely resulting in site-specific policies and procedures (see Figure K-3). This variation is due to previously mentioned structural variables such as sociocultural or political milieus, regional demographics and/or case mix, and availability of staff and legal or licensing mandates regarding what types of providers can perform which types of services. For example, Gillian et al. found improved compliance among African American teens due to the involvement of all clinical staff in the process. All clinic employees, includ- ing clerical and professional staff, forged relationships through shared backgrounds and experiences, honesty, and additional time spent with the teens (Gilliam et al., 2007). Outcome See Figure K-3 and the discussion below of the Bertrand et al. (1994) program evaluation model. 

416 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Relationship Between Quality Assessment and Program Evaluation It is difficult if not impossible to distinguish quality assessments from program evaluations when reviewing the family planning literature. Hence, Figure K-3 shows an integrated model of quality and program evaluation. Inputs and processes are conceptually equivalent to Donabedian’s structure and process variables. Input examples pertinent to family planning include such items as personnel, financial resources, facilities, and equipment. Both output and outcomes are comparable to Donabedian’s outcome domain. However, output specifically refers to outcomes at the program level and is usually defined by service utilization (number of visits, number of new or continuing contraceptive users). On the other hand, program outcomes defined by program effect and program impact are measured at the popu- lation level. An example of program effect is the prevalence of contracep- tive use as measured in a population survey, and an example of program impact is the regional or national fertility rate or desired pregnancy rate. The maturity of the program determines the type of evaluation strategy to use—the more mature the program, the more impact one would expect to be able to demonstrate. By most standards, Title X is a mature program (more than 30 years old), but it is immature in quality assessment; hence the majority of indicators will initially be focused on processes and outputs. However, the program has been in existence long enough to be capable of demonstrating long-term impact if the correct data are made available for collection and interpretation. International Frameworks for Quality Family Planning Services Several authors have published extensively on quality assessment as it relates to the international family planning and reproductive health care arena. Most authors build on or adapt the Bruce (1990) and Jain et al. (1992) framework, which identifies six elements of quality: • Choice of method, • Information provided to the client, • Technical competence of providers, • Interpersonal relations between clients and providers, • Mechanisms to encourage continuity of care, and • Appropriate constellation of services. Outcome measures include program readiness, the provider perspective, the services delivered, and the client perspective and the services received, with particular emphasis on client knowledge, client satisfaction, client health, and contraceptive use—both acceptance and continuation. 

APPENDIX K 417 IPPF adopted the above framework as its model for quality after add- ing client acceptability to the above list and emphasizing that these criteria should be defined as clients’ and providers’ rights and expectations (IPPF, 1998). Table K-1 outlines the IPPF client and provider “bill of rights.” The most extensive and comprehensive framework for family planning quality indicators and program evaluation is that proposed by Bertrand et al. (1994). While intended for developing countries, it is easily applicable to the United States and includes both program- and population-level indica- tors. Program-based or performance indicators include those factors related to inputs, processes, and outputs, whereas population-based or outcome indicators usually reflect intermediate effect or long-term impact. Bertrand et al. define eight broad categories for program evaluation, which outline the pathways by which programs achieve impact in a given country: • Indicators to measure the policy environment, • Indicators to measure service delivery operations, • Indicators to measure family planning outputs, TABLE K-1  International Planned Parenthood Federation Framework: Clients’ Rights and Providers’ Needs Client Rights Provider Needs Information about family planning Training—technical and communication skills Access to all service delivery systems and Information on technical issues updated regularly health care providers Choice of adopting, switch, or discontinuing Infrastructure (appropriate facility and efficient methods organization) Safety in the practice of family planning Supplies of contraceptives, equipment, and educational materials Privacy during discussions and physical Guidance from service guidelines, checklists, and examinations supervision Confidentiality of all personal information Back up from other providers Treated with dignity, courtesy, and Respect and recognition from coworkers, attentiveness managers, clients, community Comfort while receiving services Encouragement to provide good quality care Continuity of care for as long as client Feedback from managers, supervisors, and clients desires Opportunity to express opinions about he Opportunity to express their concerns relative to quality of care received clinic decision making Source: Huezo, C. M., and S. Diaz. 1993. Quality of care in family planning: Clients’ rights and providers’ needs. In P. Senanyake and R. L. Kleinman, eds. Family planning: Meet- ing challenges, promoting choices. The proceedings of the IPPF Family Planning Congress, New Delhi, Oct. 1992. Pearl River, New York: Parthenon Publishing Group. Pp. 235-244. Reprinted with permission. 

418 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM • Indicators to measure demand for children, • Indicators to measure demand for family planning, • Indicators to measure service utilization, • Indicators to measure contraceptive practice, and • Indicators to measure fertility impact. The categories outline the pathways by which programs achieve impact in a given country (see Figures K-4 and K-5). Bertrand et al. (1994) advise that a key prerequisite for the development of a successful family planning program is the presence of a strong, viable political and administrative system. Political support should be coupled with resource allocation and permissive legal codes and regulations that affect the number, type, and distribution of methods. A comprehensive family planning service deliv- ery program will have defined functional areas that include management, supervision, training, commodities and logistics, IEC, and evaluation. There are two primary expected results (or outputs) for a successful program: (1) adequate services from the client perspective, as defined by accessibil- ity, quality, and acceptability; and (2) increased utilization of services, as defined by number of new and/or continuing users. Other Intermediate Variables Societal Value and FP Demand Fertility and Demand � Spacing � Wanted Individual for � Limiting � Unwanted Factors Children Contraceptive Practice Development Service Outputs Programs � Access � Quality Service Other Health Family Planning � Image/ Utilization and Social Supply Factors Acceptability Improvements FIGURE K-4  Conceptual framework of family planning demand and program impact on fertility. Source: Bertrand, J. T., R. J. Magnani, and J. Knowles. 1994. Handbook of indicators for family planning. Chapel Hill, NC: MEASURE Evaluation, Carolina Population Center. http://www.cpc.unc.edu/measure/publications/pdf/ms-94-01.pdf (accessed April 18, 2008). Reprinted with permission. K-4 new 

APPENDIX K 419 External Development Assistance FP Organizational Operations Structure � Management � Service and Supervision Service Outputs Infrastructure � Training � Access � Sectoral � Commodity � Quality Integration Acquisition/ � Image/ � Delivery Distribution Acceptability Strategies � I–E–C � Public–Private � Research and Political and Partnership Evaluation Administrative System � Political Support � Resource Allocations � Legal Code/ Regulations Larger Societal and Political Governance Factors FIGURE K-5 Conceptual framework of family planning demand and program impact on fertility. Source: Bertrand, J. T., R. J. Magnani, and J. Knowles. 1994. Handbook of indicators for family planning. Chapel Hill, NC: MEASURE Evaluation, Carolina Population Center. http://www.cpc.unc.edu/measure/publications/pdf/ms-94-01.pdf (accessed April 18, 2008). Reprinted with permission. K-5 Each category has a series of indicators. A detailed listing can be found in Annex K‑2. It should be noted that there are approximately 200 discrete descriptive indicators that can provide trend data over time; however, they are meant to be used selectively depending on what outputs, processes, or outcomes are being assessed. In fact, field research used a modified “quick” 25 indicators with good success (Sullivan and Bertrand, 2000). The indica- tors and data collection method are listed in Table K-2. The research focused on indicators of quality that were related to client behaviors. It was aimed at determining the feasibility of data collection, testing the comparability of results obtained with two separate instruments (direct observation of pro- vider and client exit interview), and determining the cost of data collection, among other things. It is among the few studies that provide information on the cost of quality assessment, and also demonstrate how the data could be used and compared within and among health systems or countries. The actual dollar amount for the costs associated with conducting the studies 

420 TABLE K-2  Short List of Quality Indicators and Method of Data Collection Indicator Client Exit Facility Number Indicator Interview Observation Audit PROVIDER I-1 Demonstrates good counseling skills (composite) X X I-2 • Assures client of confidentiality X I-3 • Asks client about reproductive intentions (more X X children? when?) I-4 • Discusses with client which method she would prefer X X I-5 • Mentions HIV/AIDS (initiates or responds) X X I-6 • Discusses dual method use X X I-7 • Treats client with respect/courtesy X X I-8 • Tailors key information to the particular needs of the X specific client I-9 • Gives accurate information on the method accepted X X (how to use, side effects, complications) I-10 • Gives instructions on when to return X X I-11 Follows infection control procedures outlined in X guidelines I-12 Recognizes/identifies contraindication consistent with X guidelines I-13 Performs clinical procedures according to guidelines X STAFF (other than provider) I-14 Treat clients with dignity and respect X 

CLIENT I-15 Participates actively in discussion and selection of X X method (is “empowered”) I-16 Receives her method of choice X X I-17 Client believes the provider will keep her information X confidential FACILITY I-18 Has all (approved) methods available, no stockouts X I-19 Has basic items needed for delivery of methods available X through SDP (sterilizing equipment, gloves, blood pressure cuff, specula, adequate lighting, water) I-20 Offers privacy for pelvic exam/IUD insertion (no one can X X X see) I-21 Has mechanisms to make programmatic changes based X on client feedback I-22 Has received a supervisory visit in past ____ months X I-23 Adequate storage of contraceptives and medicines (away X from water, heat, direct sunlight) is on premises I-24 Has state-of-the-art clinical guidelines X I-25 Waiting time is acceptable X X Source: Sullivan, T. M., and J. T. Bertrand. 2000. Monitoring quality of care in family planning by the quick investigation of quality (QIQ): County reports. MEASURE Evaluation Technical Report Series No. 5. Chapel Hill, NC: MEASURE EVALUATION, Carolina Population Center. Reprinted with permission. 421 

422 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM is difficult to interpret given the lack of comparability among international salary ranges. It is noteworthy, however, that the researchers interpreted the costs as reasonable, but they were considered high by the service providers, who were concerned that those funds might need to be diverted from other family planning–related resources (range $217–1,000/site). All of the above frameworks essentially culminated in a paradigm shift away from fertility control in favor of individualized services responsive to client needs. This human rights approach was widely endorsed in 1994 at the International Conference on Population and Development in Cairo (United Nations, 1994). Recently, assessment of family planning program quality has begun to emphasize client satisfaction. WHO has specified that clients want respect, understanding, individualized care, complete and accurate information, technical competence, access, fairness, and results (WHO, 1998). Com- munication standards have been emphasized internationally. Clients should be given sufficient information and counseling to be capable of making contraceptive decisions (WHO, 2004). This information should include at a minimum: • A description and understanding of the relative effectiveness of the chosen method, • Correct use of the method, • How it works, • Common side effects, • Health risks and benefits of the method, • Signs and symptoms that should necessitate a return to the office, • Information on return to fertility after stopping the method, and • Information on STD protection. It should be noted that exit interviews on site and focused interviews in the community may reveal different pictures of quality, raising concern about the validity of exit interviews and the possibility of a positive or courtesy response bias on site (Sullivan and Bertrand, 2000; RamaoRao and ­Mohanam, 2003). There has been considerable research on concepts, frameworks, mea- surement, and methodology, but few experimental studies have been con- ducted on the impact of quality services on reproductive health outcomes. Few quality interventions have been designed within a research framework capable of measuring or quantifying their effects. In a review of 15 interna- tional studies, most had methodological flaws (RamaoRao and Mohanam, 2003). Proposed explanations for these limitations included a perception that research activities were outside the normal scope of work and were burdensome and consumed limited resources. The small number of well- 

APPENDIX K 423 designed intervention studies suggests that quality can be improved and that good care has beneficial effects. The authors conclude that conceptual frameworks to examine quality have been developed and refined. The lan- guage and vocabulary used to define them have been accepted by diverse constituents. Methodological advances have been developed for data col- lection. A variety of interventions have been tested. Intervention tools demonstrating the most promise are those that facilitate better interaction between clients and providers through provider training in interpersonal communication and information exchange or the use of audiovisual aids. Better care has been associated with higher levels of client satisfaction and contraceptive adoption and continuation. Despite these findings, the authors acknowledge a number of gaps in the literature. These gaps should serve as stimulus for discussion when considering the development of qual- ity indicators in the United States, where indicator development is in its infancy, but program infrastructure has been well established: • What is the impact of decentralization on the quality of care pro- vided or received? • What is the level of readiness and quality of care in the private sector? Studies suggest private-sector facilities do not necessarily provide better care. • Why do family planning clients choose to use some facilities rather than others? Is choice guided by perceptions of quality, and if so, how? • What changes would encourage clients to continue to visit facilities and stay with existing programs? Research designs and program evaluations tend to concentrate on new users rather than the needs of existing clients. What quality attributes predict return visits or behavioral changes that support birth spacing/limiting and STD prevention? • Many indicators have been proposed, but none have been tested on a wide scale. What is the most efficient mechanism to study new measures? Can existing infrastructure support indicator develop- ment and validation? • Can quality be improved without extraordinary financial out- lay? There is limited information on costs, cost-effectiveness, and financing. To summarize, the international literature has established the following tenets as the elements essential for quality family planning services: • A client-centered perspective sets the program objectives and standards. 

424 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM • Interactions are information based and participatory, and allow for collaborative decision making. • Clients have access and are treated with respect, understanding, and fairness. Clients want to be given complete and accurate informa- tion and be treated by technically competent providers ­(RamaoRao and Mohanam, 2003). The Reproductive Health Continuum Model This theoretical framework was initially proposed by Lu and Helfon and later used by the Maternal Quality Indicators Working Group to posit indi- cators for maternal health care quality, and is put forth here as a reminder that women are at various stages of their reproductive cycle (Gregory et al., 2005; Lu and Helfon, 2003; Korst et al., 2005). These stages include child- hood, puberty, preconception, pregnancy, postpregnancy/interconception, perimenopause, and postmenopause. The postpregnancy/interconception stage includes all postpregnancy options: miscarriage, termination, and postpartum; for this model, postpregnancy also refers to post–negative pregnancy test. All of these events are critical times when a woman is interacting with a health care provider and ready for IEC about pregnancy intendedness. During the postpregnancy/interconception stage, there are special considerations if the woman or couple is planning to space or limit children. Thus, the contraceptive needs and types of counseling and preven- tive health services required will change over time and across different life circumstances. Family planning services and IEC messages for pubescent teens should obviously be different from those for couples actively seeing pregnancy. Likewise, divorce, death of partner or child, and involvement with a new partner will all impact where a woman is in the cycle relative to interconception and her birth spacing versus limiting needs. While spe- cific to individuals, the model has some societal relevance as well. Social marketing of family planning and preventive reproductive health services that are consistent with national population goals should be geared toward age-appropriate cohorts (e.g., abstinence at puberty and more permanent methods at the birth-limiting phase of a women’s reproductive life). An explicit understanding of national population/fertility goals is critical to judging the success of the Title X program. As America becomes more cul- turally diverse, the commonly espoused number of two children per family may no longer be valid. AHRQ Framework for the Evaluation of Potential Indicators Not all indicators get the benefit of validation prior to implementation, but it is important to recognize that there is a process by which this can be 

APPENDIX K 425 done. Ideally, the following criteria, as recommended by AHRQ, should be used to assess each of the proposed indicators: • Importance—There is opportunity for improvement, as evidenced by substantial variation among hospitals and regions (or clinical sites). Face and construct validity must be established. • Scientifically acceptability — The variation in outcomes can be explained largely by ­provider/ system-related factors and not by patient case mix. — The indicator is well defined and precisely specified. — The indicator is valid, accurately representing the concept being evaluated. — The indicator is precise, adequately discriminating real differ- ences in provider performance, and a reasonable sample size exists to detect actual differences; the indicator captures all possible cases, and bias related to case exclusion or limited data is minimal. — Risk adjustment is adequate to address confounding bias. — Analytical methods appear robust. — The indicator is adaptable to a variety of settings. • Usability—Statistical testing can be applied to identify when dif- ferences in performance levels are greater than would be expected by chance. • Feasibility — Data collection methods yield consistent construction and assessment of the measure. — The indicator is feasible to calculate, and the benefits exceed the financial and administrative burden of implementation. — Data collection and presentation mechanisms allow confiden- tiality to be protected. — The quality of the data is known and consistent, and an audit strategy can be implemented. How well does the Title X FPAR measure quality? What is the FPAR National Summary? The FPAR is the only source of annual uniform reporting by all 87 Title X service grantees. It provides national-level validated data on program users, service providers, utilization of family planning and related preventive health services, and sources of revenue for the program. It provides an estimate of the impact of Title X–funded activities on key reproductive health outcomes. The Research Triangle Institute (RTI) tabulates the grantee FPARs to pre- 

426 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM pare the National Summary under a contract with OPA. Data are submit- ted electronically through a web-based electronic grant management system (eGrants). Data elements include facility-specific information such as number and type of providers; revenue generated, by payer source; user demographics (age, sex, race/ethnicity, insurance coverage, English proficiency); new user encounters; contraceptive use by male and female family planning users; summary abnormal Pap smears; and summary confidential HIV-positive test results and disease-specific rates for STD screening. As currently reported, the FPAR data provide some measures of qual- ity as related to structure and certain inputs, processes, and outputs. They could potentially be used to calculate or abstract outcomes related to effect (short-term and site-specific), but are not currently suited to determining long-term outcomes demonstrating program impact. What Quality Measures Can Be Determined by the FPAR? Structure (and Input) Variables Staffing. In 2006, there were 3,937 FTEs working at Title X sites, including physicians, midlevel providers, and other clinical service providers (CSPs). Most (51 percent, or 2,014) were midlevel providers (physician assistants, nurse practitioners, or certified nurse midwives); 13 percent were physicians and the remainder CSPs. There was one MD per 4.1 midlevel providers, and the staffing composition varied across regions. Research indicates that where variations in staffing exist, there likely exist an opportunity for improvement and associated poor quality due to both under- and over­utilization (Wennberg, 1999; Fisher and Wennberg, 2003). What is unknown from a quality perspective is what defines best practice as it relates to provider mix/FTE ratios. For example, some studies suggest that midlevel clinicians provide better care (Winter and Goldy, 1987). A further quality issue is whether fewer physicians imply better quality (enhanced efficiency) or worse quality (e.g., CSPs practicing beyond their scope). Total Revenue. In 2006, grantees reported $1.1 billion in total revenue, or approximately $217 per user. Medicaid was the major source of revenue, followed by Title X (30 percent and 24 percent, respectively). Other sources included state governments, client payment (approximately 9 percent), local government, and other third parties. The proportion contributed by other sources varied across regions. Assuming a single standard of care across all payers, the question arises of the extent to which alternative revenue sources improve other inputs, such as resources available, contraception options, additional funds for outreach, and facility enhancements. 

APPENDIX K 427 Process (and Output) Variables Percent Limited English Proficiency (LEP). In 2006, 13 percent of clients were LEP and required oral language assistance to optimize their use of Title X services. This includes services provided by bilingual staff, a com- petent agency or contracted interpreter, or a family member or friend (after the client refused the agency’s offer to provide a qualified interpreter at no cost). Data are needed on the quality of the bilingual translator, client com- prehension after the interaction, and percent and rationale when in-house translator services were refused or not used. Female Users by Primary Contraceptive Method at Last Encounter in Reporting Period. In 2006, 85 percent of Title X clients were using a con- traceptive method, while 15 percent were not (8 percent seeking pregnancy, 7 percent other reasons). The FPAR provides a breakdown of users by method, thereby demonstrating a key family planning quality domain— options in contraceptive methods. Method use varies by age, race, and region. There is a need to understand this variation; with regard to age, it is likely to be related in part to the reproductive health continuum, with increased utilization of reversible methods during the birth-spacing years and more permanent methods during the birth-limiting years. Racial/ethnic differences are likely to be related to cultural differences. As noted above, for example, the assumption that the average number of children desired by American women is two may not be valid across racial/ethnic groups (AGI, 2000). Likewise, long-standing fears about institutional racism and dis- crimination have been associated with distrust and low contraceptive con- tinuation rates among African American women (Thornburn and ­Bogart, 2005a,b). An additional output measure that could be calculated from the FPAR, but is not currently obvious, is the percent of women who use dual protection (another method plus condom) for STD/HIV prevention. Dual protection would impact pregnancy prevention given the percent of preg- nancies that occur as a result of contraceptive failure or inappropriate use. It would also protect against STD/HIV transmission. Male Users by Primary Contraceptive Method at Last Encounter During Reporting Period. In 2006, 92 percent of all male users of Title X services were using a contraceptive method, while 8 percent were not; 1 percent had a pregnant partner or were seeking pregnancy, and 7 percent had “other” reasons for nonuse. Fully 78 percent of users relied on male condoms, 6 percent on their partner’s method, 4 percent on abstinence, 1 percent on vasectomy, and 7 percent on an unknown method. The above comments on dual method use apply to this measure as well with respect to its limitations as a quality measure. 

428 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Cervical Cancer Screening: Number of Users Who Obtained a Pap Test, Number of Pap Tests Performed, Number of Pap Tests with (ASC) or Higher. In 2006, 2.4 million Pap tests were performed, and 49 percent of Title X female family planning users were tested. Ten percent (240,702) of tests revealed precursors or cancerous conditions requiring further evalu- ation or treatment. By region, the screening rates were at or above the national average (49 percent). A potential advantage of this indicator is that it is a HEDIS measure, and hence some efficiency might be gained due to the need to report elsewhere. This indicator has been proposed as a PART performance measure and will be collected by the FPCA Performance Mon- itoring System (see below) (OMB, 2005). It addresses the Title X mission to decrease cervical cancer, and there is an existing benchmark. Variation exists among regions, suggesting an opportunity for improvement; however, all regions exceeded the national threshold. Future quality measures might include referral and treatment outcomes (see below). Breast Cancer Screening: Number of Users Receiving a Clinical Breast Exam, Number of Users Referred for Further Evaluation. In 2006, 2.4 million users received a clinical breast exam, and 3 percent (65,157) were referred for further evaluation. Screening rates were at or above the national average (49 percent) in 7 of 10 regions. A potential advantage of this indicator is that it is a HEDIS measure, again offering the potential to gain efficiency. This indicator has been proposed as a PART performance measure and will be collected by the FPCA Performance Monitoring System (see below) (OMB, 2005). It addresses the Title X mission to decrease breast cancer, and there is an existing benchmark. Variation exists among regions, sug- gesting an opportunity for improvement. Future quality measures might include referral, final diagnosis, and treatment outcomes. Where feasible, information about missed cases after a clinical breast exam would also be pertinent and would enhance the feedback quality loop. STD Screening STD screening is a recognized goal of Title X. Only chlamydia, g ­ onorrhea, syphilis, and HIV are reported in the FPAR. There are more than 25 STDs, but limiting reporting to these four is reasonable in this early stage of quality measurement (HHS, 2000a,b). However, future mea- sures might include rates of diagnosis, treatment, or referral for hepatitis and HPV. Likewise, it might be useful to measure resources used; Level II infertility assessments; and costs per patient for screening, diagnosis, and treatment of more common infections, such as bacterial vaginosis, yeast, mycoplasma, and ureaplasma. These measures would be useful, especially 

APPENDIX K 429 if they could be tracked electronically, as the prevalence of these conditions diverts resources from other family planning priority areas. Chlamydia Testing by Age Group (under 24, and 25 and Older) and Gender In 2006, Title X clinics tested 47 percent of all female users and 52 percent of male users for chlamydia. Testing rates were highest among younger users. Overall, 51 percent of users under 24 were tested, and in five regions, testing rates were at or above the national average (which was not specified). However, testing rates in all regions were lower than that recom- mended by the Centers for Disease Control and Prevention (CDC) (Fowler et al., 2008). A potential advantage of this indicator is that it is a HEDIS measure, again offering potential efficiency benefits. This is a good-quality indicator in its current form. Guidelines specify contact time for notification of a positive test (2 weeks) and for documentation of a subsequent nega- tive test, confirming a program effect (successful treatment) and program impact due to implied behavioral change (condom use) preventing reinfec- tion (Meyers et al., 2008). Note that half of Title X clinics participate with CDC and OPA in a nationwide chlamydia prevention effort, and chlamydia screening has been endorsed by the U.S. Public Health Service Task Force and is a Healthy People 2010 objective (HHS, 2000b; AHRQ, 2007). This is an excellent example of cross-agency collaboration and national prior- ity agenda setting and social marketing, as evidenced by the adoption of this indicator by other external quality monitoring agencies (e.g., NCQA) (NCQA, 2007). Gonorrhea and Syphilis Testing. In 2006, Title X administered approxi- mately 2.1 million gonorrhea tests and more than 700,590 syphilis tests. This indicator is not useful as reported. No data are reported on prevalence rates, treatment, cures, or impact on congenital infection rates (a particular concern given rates of unplanned pregnancy among Title X clients). HIV Testing: Number of Positive Confidential Tests Performed and ­Number of Anonymous Tests Performed. In 2006, sites performed 652,426 confi- dential HIV tests, 1,337 of which were positive; 14,280 anonymous tests were performed. This is a good-quality indicator in its current form. The Program Guidelines specify the contact time for notification of a posi- tive test (2 weeks). Future measures could report referral for follow-up, percent and extent of partner notification, and percent of condom use postdiagnosis. 

430 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Family Planning Encounters: Face-to-Face Contact with Clinical or Non- clinical FTE to Provide Family Planning and Related Preventive Health Services to Clients Who Want to Avoid Unintended Pregnancies or Achieve Intended Pregnancies. In 2006, there were 9.8 million family planning encounters (documented in the medical record). There were roughly two visits per user; 74 percent were with a CSP (nonphysician). This indicator is not useful as reported. Given the quality domain of patient-centeredness, a more useful measure should be designed that would take into account the ideal number of visits for patient need, specifically considering patient age, race/ethnicity, pregnancy plans, and contraceptive method. Outcomes Variable The FPAR as it currently exists does not adequately address Title X out- comes at the program (effect) or population (impact) level, but performance measures have been suggested by OPA and are in development by FPCA (see below). If the Title X program were judged by the criteria of Bruce (1990) and Jain et al. (1992) (as discussed above), it would have a high likelihood of being judged as good quality on three of their six constructs: choice of method, information provided to the client, and appropriate constellation of services (assuming clinical sites are doing what they are mandated to do). However, additional information is needed to determine quality ratings on technical competence of providers, interpersonal relations between clients and providers, and mechanisms to encourage continuity of care—especially given the current audit emphasis on new users (Bruce, 1990). With respect to the IPPF framework, assumptions of good quality could be made on only four of the ten client rights, and none of the ten provider rights (IPPF, 1998). Finally, no assessment of the quality of the Title X program could be made using the comprehensive or abbreviated version of the Bertrand et al. framework (Bertrand et al., 1994; Sullivan and Bertrand, 2000). What quality initiatives have been undertaken by family planning programs? This section starts with a broad overview of the utilization of family planning services and quality assessment studies of family planning pro- grams in the United States. It then reviews what is known about the quality initiatives that have been undertaken by Title X programs, as evidenced by the PART and the proposed FPCA Performance Measurement System (FPCA, 1999; OMB, 2005). 

APPENDIX K 431 Who Needs Family Planning Services? There are approximately 62 million women of reproductive age in the United States, 70 percent (43 million) of whom are sexually active and not planning to become pregnant (Mosher et al., 2004; Guttmacher Institute, 2008). Of these women, 31 percent are not using contraception because they are infertile, pregnant, trying to become pregnant, post­partum, or not sexually active; 62 percent of women are using a contraceptive method (rep- resenting 89 percent of the 42 million fertile women at risk for pregnancy). Approximately 7 percent of women at risk for pregnancy are not using a contraceptive method; 64 percent of women use a reversible method; and of the remainder, they or their partner has been surgically sterilized. Poor and low-income women are more than twice as likely as higher-income women to use the 3-month injectable method. Approximately 7.3 million women use barrier contraceptives, such as the male condom. Condom use is especially common among teens (and is the primary method for 27 percent of teenage girls), those aged 20–24, childless women, and never- married women. Condom use declines as women grow older and marry. The proportion of women who used a contraceptive method the first time they had sex nearly doubled from 43 percent in the 1970s to 79 percent in 1999–2002. This change is due mainly to an increase—from 22 percent to 67 percent—in the proportion using the male condom at first intercourse, and likely reflects heightened concern about and awareness of HIV/STD prevention. This finding supports the programmatic impact of Title X and other family planning initiatives. Condom use at first intercourse varies by age (more likely among older women) and race/ethnicity (67 percent among whites, 60 percent among African Americans, and 46 percent among H ­ ispanics) (Mosher et al., 2004; Guttmacher Institute, 2008). Despite these encouraging statistics, most pregnancies are still unplanned and occur among women using contraception (IOM, 1995; Schunmann and Glasier, 2006). Approximately half of women experiencing ­unintended preg- nancies used some type of birth control during the month they ­conceived— albeit inconsistently or incorrectly (Dreweke, 2006). Hence, there is an ongoing need for IEC activities related to pregnancy planning, prevention, and reproductive health services. Based on data from the NSFG and other sources, 5 percent of U.S. women of reproductive age have an unintended pregnancy each year (for a rate of 51 unintended pregnancies per 1,000 women aged 15–44). Unintended pregnancy is substantially more common among women aged 18–24, unmarried and/or cohabiting women, low- income women, women who did not complete high school, and minority women. Poor women have a high rate of unintended pregnancy (112 per 1,000 women aged 15–44, or twice the national average). A poor woman is four times as likely to have an unintended pregnancy, five times as likely 

432 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM to have an unintended birth, and more than three times as likely to have an abortion as her higher-income counterpart (Finer et al., 2002; Mosher et al., 2004; Guttmacher Institute, 2008). Knowing where women receive family planning services and standard- izing the quality of care for these services should help improve compliance, the effectiveness of contraception, and patient satisfaction with respect to achieving needs and wants regarding reproductive intentions (HARI), and further validate Title X ‘s mission. More than 44 million women obtained reproductive health services in 2002. Of these women, 34.4 mil- lion (56 percent) were seen by a private physician, while 13.5 million (22 percent) were seen in a publicly funded clinic (5.4 million of these women were seen in Title X clinics), and the remainder were seen in other types of facilities. Of women seen in Title X clinics, 53 percent were at less than 300 percent of the federal poverty level (Mosher et al., 2004). Lessons Learned from the General Family Planning Quality Assessment Literature Improving access to family planning services, either by increasing fund- ing or by reducing barriers related to program enrollment or to mandated physical assessments such as pelvic exams, results in increased utilization of services and increased variety of contraceptive methods available (Lindberg et al., 2006). Both access and method choice are family planning quality domains. For example, Lindberg et al. report that the number of contracep- tive methods increased and agencies reduced barriers to oral and emergency contraception by liberalizing policies for provision (e.g., no pelvic exam required) between 1995 and 2003. By 2003, clinics were offering the ­newest methods (IUD, ring, patch); however, not every method was stocked at all facilities because of costs, suggesting that continued funding challenges limit the ability of publicly funded providers to offer all available methods to all women (Lindberg et al., 2006). In a longitudinal study of the impact of extending Medicaid coverage for family planning services, Bronstein et al. found that expanded access was associated with an increased number of family planning users who were demographically similar to Title X users (Bronstein et al., 2007). But the growth was greatest among clients of non–Title X providers (e.g., private physicians). Access to private physicians has been inconsistently associated with quality. Patient satisfaction surveys rate this as indicative of improved quality, but objective evaluations by Bronstein et al. found that private physicians provided less comprehensive services, such as less HIV screening, and more contraception services, such as provision of oral contraceptives or sterilization. The use of risk assessment and coordina- tion of care was, however, associated with more continuity as measured 

APPENDIX K 433 by return for care. Receiving less comprehensive care, unless specifically individualized based on risk assessment, would obviously be a measure of poor quality. Mathematical models suggest that improved access is associ- ated with an increased number of pregnancies averted and significant cost savings, based on both pregnancy-related costs and subsequent newborn and childhood social costs. Researchers using the Markov methodology estimate that for every dollar spent on family planning, $3.58–5.33 is saved in subsequent government spending (Foster et al., 2004; Dreweke, 2006; Amaral et al., 2007). Paine et al. performed a systematic literature review to examine the relationship between family planning services and safe and effective contra- ceptive use (Paine et al., 2000). Their aim was to identify features of family planning service provision that influenced use and the optimal effectiveness of user-dependent methods. The authors identified 142 articles, only 16 of which met inclusion and exclusion criteria, including a specified long- term outcome. The authors concluded that the quality of provider–client exchanges had a net incremental effect on contraceptive use, and available evidence suggests that training in communication with clients about side effects and an emphasis on client choice are key components of effec- tive interventions. However, evidence for the effectiveness of methods to improve uptake, continuation of method use, and safe and appropriate use of contraception is scant. In essence, providers must be client-centered and interactive and ­listen to clients’ needs and wants. This conclusion confirms findings originally publicized and widely accepted in the international family planning arena (Bruce, 1990; IPPF, 1998; WHO, 1998; Sullivan and Bertrand, 2000; RamaoRao and Mohanam, 2003). Paine et al. (2000) acknowledge that to date, almost all quality measures have looked at outcomes immediate to the site of care, with minimal or no attempt to look at the incidence of unwanted conception among users of the services. They advise embracing a broader focus on helping “individuals to achieve their reproductive inten- tions in a healthful manner,” endorsing the HARI concept as originally proposed by Jain et al. (1992). The HARI index is a mathematical number defined as the proportion of clients who meet their reproductive goals: HARI index = 100 – (% unplanned pregnancies + % unwanted pregnancies) during a specified period after the initiation of contraception % unplanned = women who are trying to space their pregnancies % unwanted = women who want to limit pregnancy (finished with childbearing) 

434 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM This index could be calculated annually per site, but would require explicitly asking about, documenting, and monitoring each user’s repro- ductive intentions for the year (confirmed by subsequent visits during that time period). In this same vein, family planning programs tend to measure continu- ation rates or discontinuation rates among new users over a specified time period (FPAR and FPCA). Some researchers have suggested that a better indicator would be the dropout ratio, defined as the proportion of new users who are still at risk of pregnancy, do not want to become pregnant, and have quit using any family planning method. This indicator would exclude women who are past menopause, are no longer sexually active, planned a pregnancy, or switched methods, thereby taking into account the client’s reason for stopping the method (WHO, 2004). Another landmark review that informs this discussion is a recently published study conducted by Becker et al. (2007). These authors pub- lished a comprehensive review of the quality of family planning services in the United States based on documents available between 1985 and 2005. They conceptualized a framework based on Bruce (1990) (specific to fam- ily planning) and Sofaer and Firminger (2005) (health care in general) that includes eight domains: accessibility, communication and informa- tion, client–staff interactions, efficiency and effective organization of care, technical competence, structure and facilities, contraceptive method choice, and patient-­centeredness (Bruce, 1990, Sofaer and Firminger, 2005). The authors identified 29 studies conducted in the United States: 15 studies were descriptive and documented levels of service quality, 10 investigated the correlates of quality, 12 examined the effect of quality on client attitudes and behavior, and 8 explored clients’ preferences and values regarding family planning service delivery. The studies were not limited to Title X programs. The methodology most commonly used in the studies reviewed was surveys of women receiving the services. Other approaches included focus group discussions, interviews, medical record reviews, direct obser- vation of client–provider interactions, surveys of providers or managers, and quasi-experimental and experimental studies. Although most studies conceptualized family planning quality as a multidimensional construct, a few explicitly defined service quality constructs; thus the domains of qual- ity evaluated were not consistent across studies. Client–staff interactions and accessibility of services have been assessed relatively frequently, while other domains, such as provider technical quality, have been explored infre- quently. A brief summary of the findings of Becker et al. (2007) by quality domain is as follows. • Accessibility—Problem areas from the client perspective included barriers involving administrative accessibility, such as waiting 

APPENDIX K 435 more than a month for an initial appointment, inconvenient hours, unable to access provider by phone, and language barriers. For example, more than 25 percent of 637 federally funded programs lacked tailored services for non-English-speaking clients in 1999 (Finer et al., 2002). Ideally, this would not be an issue in Title X clinics, based on statutory requirements, but objective performance data are needed to confirm this. • Communication and information—When queried, a high propor- tion of women reported talking to the provider about specific t ­ opics, such as the effectiveness of different contraceptives and how to use particular methods. However, studies asking clients to rate the quality of the information they received were less positive, with 14–25 percent of clients stating that they felt they did not receive sufficient information; they felt their concerns and questions were not adequately addressed; the advice was not tailored to their specific circumstances; or their concerns were dismissed, especially those related to possible contraceptive side effects. • Client–staff interactions—Issues raised by clients included a lack of respect from nonclinical staff members and of privacy while waiting for an appointment. African American women were more likely to report a perceived lack of respect (Thornburn and Bogart, 2005a). • Efficiency and effective organization of care—The most widely studied aspect of this domain is waiting time. A long waiting time has consistently been associated with poor quality by clients. Simi- larly, a lack of continuity as defined by the inability to see the same provider is associated with being rated as poor quality by clients. Not all programs have a mechanism for clients to see the same provider at each visit. A lack of follow-up mechanisms to track patients over time has also been associated with poor quality. Finer et al. found that only 53 percent of programs had a mechanism in place to contact clients who missed appointments (Finer et al., 2002). • Technical competence—This domain is defined by safe, effective care that complies with accepted clinical standards. There have been only two studies conducted in this domain in the United States, but their findings suggest technical competence is high. Clearly more work is needed in this area. • Structure and facilities—As previously mentioned, location, prox- imity, physical appeal, and waiting room comfort have been associ- ated with good quality as judged by clients. • Method choice—The range of contraceptive options varied across sites. Although oral contraceptives were the only method offered 

436 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM by virtually all providers, few clients reported being unable to obtain their method of choice. However, one study using a nation- ally representative sample of African American women reported that a family planning provider strongly encouraged these clients to adopt a specific method of birth control that was not consistent with their preference (Thornburn and Bogart, 2005b). While choice has been deemed an important quality indicator, associated with both initiation and continuation of contraceptive use, pressure or a perceived lack of choice has been associated with early discon- tinuation. This is especially concerning given that, as noted above, some African American women are suspicious about birth control (Thornburn and Bogart, 2005b). Addressing patient concerns and beliefs and providing nondirective counseling might be especially pertinent and beneficial in geographic regions where conspiracy concerns are high. • Patient-centeredness—The degree to which services are tailored to the needs and circumstances of individual clients has not been well operationalized. Studies that evaluated this domain were inconclusive. • Correlates of service quality—Correlates of service quality were grouped into four categories: facility factors, provider factors, c ­ lient factors, and consultation factors. These are comparable to the eight domains previously discussed. Facility factors were the most frequently studied. Quality ratings were generally lower for public as compared with private facilities. Hospitals and health departments received the poorest ratings from clients, while ­private physicians received the highest ratings. This contradicts other find- ings since hospitals and health departments are more likely to be Title X sites providing more comprehensive services, but they may also have longer waiting times, more rigid hours, and other admin- istrative barriers. Female providers received higher quality ratings than males, and nonphysicians were rated more highly than medi- cal doctors. Ratings varied by client demographics: those who were unmarried, under age 20, less educated, members of a minority, Spanish speaking, and males tended to rate services more poorly than others. • Quality and outcomes—Studies looked at the relationship between family planning service quality and client attitudes and behaviors. Most explored the link between quality and clients’ contracep- tive use after the visit, satisfaction with the method, likelihood of returning for services, and experiencing an unintended pregnancy. Observational and prospective studies tended to find positive relationships between service quality (individualized counseling) 

APPENDIX K 437 and contraceptive behavior. However, the evidence from quasi- e ­ xperimental and experimental studies is mixed. • Effect of service quality on likelihood of returning for care— O ­ bservational studies suggest there is an association between q ­ uality and returning for care. • Client preferences and values regarding family planning service quality—Receiving personalized attention, having staff spend time explaining issues, being able to see the same provider at different visits, and receiving affordable care are all associated with clients’ perceptions about quality. Becker et al. (2007) conclude that there was a lack of consistency in the domains of quality studied, making it difficult to draw causal associations. They suggest that future studies should formulate more explicit definitions of quality guided by previously developed conceptual frameworks with delineated domains. The methodology should be multileveled, incorporat- ing the perspectives of providers and managers as well as clients. Further, the authors suggest increased utilization of expert observations in the field, chart audits, simulated patient visits, and provider surveys to provide a more informative, multidimensional measure of quality at the site. While most studies of family planning services have focused on users, examining reasons for nonuse may reveal unfavorable perceptions of ser- vices within the community or provide insight into the role of ambivalence and pregnancy intendedness. In the same way that silence is considered implied consent, ambivalence or inconsistent contraceptive use could be interpreted as a planned or wanted pregnancy. In the developed world, most unintended pregnancies arise from inconsistent or incorrect use of contraceptives. Ambivalence about pregnancy may be associated with less effective contraceptive use. In a study conducted at the time of termination, Schunmann and Glasier developed a measure of intendedness and found that women not using contraception had higher intendedness scores than those using some type of method (Schunmann and Glasier, 2006). Of those women reporting use of a method, 44 percent were using the method incon- sistently or incorrectly (either condoms or oral contraceptives). Method choice was not linked to intendedness. The authors concluded that women who are ambivalent about the desire for pregnancy are less likely to use contraception and/or more likely to use it inconsistently. Hence, one of the many challenges to efforts to reduce unplanned pregnancy rates is to maximize contraceptive use among those who use contraception imper- fectly. Nationwide, further gain may come from increasing awareness about ambivalence and its potential consequences, and publicly endorsing preg- nancy planning and advocating that everyone specify her/his reproductive intentions. 

438 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Are There Quality Initiatives Specific to Title X Programs? What Can PART Tell Us About the Quality of the Title X Program? This review revealed no published peer-reviewed descriptions of qual- ity initiatives undertaken by Title X programs. A federal evaluation of the program occurred as part of the PART evaluation conducted by the Office of Management and Budget (OMB). The purpose of the PART evaluation was to assess and improve the performance of federal programs by identify- ing strengths and weaknesses to inform funding and management decisions. The initial PART evaluation of Title X was done in 2005 (OMB, 2005). The program received a Moderately Effective rating, just shy of an ideal rating of Effective. Programs rated Moderately Effective have ambitious goals and are considered to be well managed. However, there is a need to improve efficiency or address some other problem in their design or management to achieve better results. As a result of the PART evaluation, OMB determined that the program was strong in its overall purpose, design, and management, but performance goals for some key program activities had not yet been established. In addi- tion, OMB found that, although several focused evaluations of Title X had been completed, no broad-based, independent evaluation of sufficient quality and scope had been carried out in recent years. As a result of the PART evaluation and discussions with OMB, OFP committed to improving the performance of the program by developing performance goals for key program activities and conducting an independent evaluation of sufficient quality and scope at the federal level to demonstrate the program’s overall impact. In an effort to further address OMB’s findings, the IOM was tasked with conducting the comprehensive evaluation of Title X documented in this report. Since the initial PART evaluation, the program has successfully devel- oped, implemented, and established targets for several annual and long- term performance goals, including its efficiency measure (see Chapter 3). The Title X program has specified three long-term measures that are to be reported annually: (1) increasing the number of unintended pregnancies averted by providing Title X family planning services, with priority for services to low-income individuals; (2) reducing infertility among women attending family planning clinics by identifying chlamydia infection through screening of females aged 15–24; and (3) reducing invasive cervical cancer among women attending family planning clinics by providing Pap tests according to nationally recognized standards of care. The exact methods for accomplishing these goals (e.g., specified numerators and denomina- tors) were not identified. The program’s efficiency measure is to maintain the actual cost per client below the medical care inflation rate. The 2005 

APPENDIX K 439 PART evaluation found that the program has helped prevent more than 1.3 million pregnancies and has kept the cost per case 6 percent below the medical inflation rate (OMB, 2005). What Can the Family Planning Council of America Performance Monitoring System (PMS) Reveal About the Quality of the Title X Program? In 1999, in the absence of a national set of family planning indicators, FPCA decided to develop a measurement system to assess the performance of Title X grantees and delegate agencies within the FPCA network of family planning service delivery (FPCA, 1999). The proposed system was derived through a consensus process described in detail in the Phase I and Phase II final reports (not available for review as of this writing). The conceptual framework for the indicators was based on four priorities thought to be representative of a comprehensive family planning services model of care: • Ensure the provision of high-quality clinical services. • Enhance the skills and knowledge of clients and providers. • Maintain a leadership role in the community through education, advocacy, and partnerships. • Ensure the effective and efficient management and evaluation of councils and delegate agencies. The ultimate goal of the PMS is to provide a mechanism to achieve the following objectives: • Monitor and document the achievements of family planning providers. • Evaluate program effectiveness and impact. • Guide future program and policy development and implementation. • Communicate to consumers and policy makers the outcome of an investment in family planning. • Direct the program planning process toward improved perfor- mance of family planning providers and further improvement of health status. Ultimately, 24 indicators were selected, representing structure, process, and output variables. It is anticipated that the chosen measures will be identified as standards of care for the broader family planning service delivery field. The Measures. The PMS consists of a Clinical Module encompassing 17 clinical measures that evaluate performance in comparison with established 

440 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM benchmarks. Additionally, the system includes an optional Administrative Module with 7 measures that address operational and support services. (The Administrative Module was not provided with the advance PMS summary.) The clinically based performance measures evaluate standards of care in the following areas: Contraception/Pregnancy (CON), STD/HIV (STD), Adolescents (TEEN), Cancer Screening (CA), and Operations (OP). A worksheet will be provided for each of the indicators, describing how the data should be collected and tabulated and giving a description of the relevant standards of care and applicable government regulations. (The worksheets were not available for review at the time of this writing; a sample was provided for CON-1.) The PMS Pilot Demonstration. A pilot demonstration project involving 6 FPCA members and 30 delegate agencies was conducted between October 2001 and February 2002. This pilot tested the feasibility of using the perfor- mance measures and collected data to establish or substantiate benchmarks. In testing feasibility, the pilot examined the following: (1) the burden of data collection (e.g., time, resources, staff), (2) the availability of data to address the measure, and (3) whether the measure had meaning for assess- ing performance (validity). The results of the pilot are described in detail in the Phase II final report and informed the design of the current system. Key lessons from the pilot as described by FPCA include the following: • Data collection was not as onerous a process as originally anticipated. • Pilot sites perceived the information as beneficial, with the potential for use as a mechanism for ensuring quality services. • Pilot sites identified areas where they were meeting benchmarks, as well as areas for improvement. • The pilot process provided an opportunity to test ways in which existing data systems (e.g., billing systems) could be used to access performance data. • Sites recognized that many of the performance measures could be incorporated into the current quality assurance and auditing pro- cess required for the FPAR. Cursory discussions with two clinic sites corroborated these lessons with the following caveats. Data collection was perceived as burdensome by one site, taking CPSs away from other family planning services. It was not viewed as burdensome by the other site, as its delegate provided an external FTE to perform the chart audits for 7 of the 24 total indicators. The remaining data were directly abstracted from the FPAR data that were being prepared for submission. 

APPENDIX K 441 FPCA has established a users group, composed of administrative, qual- ity assurance, and clinical staff, which is working in the field at clinical sites to help implement the PMS by facilitating the exchange of informa- tion related to the performance measures. The PMS is intended to enhance overall program performance at the site of care. The collection of data and their measurement against established benchmarks provide an opportunity for feedback to reinforce or improve performance. Action plans can be developed to focus on areas needing improvement. New data are collected to measure improvement after the intervention. This approach essentially incorporates the Plan–Do–Study–Act (PDSA) tenets of quality improvement (Deming, 1986; Speroff and O’Connor, 2004). The FPCA indicators reflect many of the quality dimensions previ- ously established in the international family planning arena. Domains that appear to be missing include client-centered measures reflecting satisfaction, literacy after IEC encounters, and method choice. Likewise, there are no technical competence or provider communication measures. It is anticipated that indicators to measure these domains will be added over time. How should the quality of Title X services be assessed in various settings? A review of the literature reveals that a theoretical framework and a multitude of quality indicators exist that can be used to assess the quality of family planning and reproductive health services in the Title X pro- gram. One can think of the indicators presented as a pyramid becoming more complex and more comprehensive as one approaches the base (see Figure K-6). At the top of the pyramid are the PART performance mea- sures. The existing FPARs would be the next rung, but currently represent primarily descriptive indicators of structure and processes. The FPCA PMS, with its 24 indicators, will provide a mechanism to collect outputs that will yield some data on short-term program effect. Ultimately, adoption of the Healthy People 2010 reproductive health and STD benchmarks would demonstrate more significant program impact across large regions of the United States (see Appendix K-1) (HHS, 2009). Finally, individualization by site or region and selection of key representative indicators from the more than 200 indicators in the Handbook of Indicators (Bertrand et al., 1994) would help spur site-specific goals directed at quality improvement. Thus, deciding which indicators to use, how many to use, and across what settings will be a function of the goal of the evaluation. As suggested by Bertrand et al. (1994), if the goal is to advance political support, indicators showing the program effectiveness and selected indicators from the policy environment will be monitored and distributed among stakeholders who can influence policy. Small new sites might want to focus on needs assess- 

442 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM National Measures Consensus driven down to sites PART Performance Measures (quality assurance) Healthy People 2010 Annual FPAR Reports Pilot FPCA Performance Monitoring System Indicators Locally tested and validated and sent up Handbook of Indicators (quality improvement) FIGURE K-6  Integration of quality assurance (program) and quality improvement (site) activities. ments, increased volume, program revenues, technical competence, and Figure K-4 patient satisfaction. Large, established, older sites or networks might want vector, editable to emphasize networking and technical training to keep staff motivated and interested. Sites frequented primarily by teens or clients with LEP would obviously have different quality improvement interests than sites with more diverse populations. More rapid improvement could be achieved if national measures were established for all sites to report, with recognized benchmarks or standards. These objective measures would represent quality assurance activities com- ing from the top down and would serve to demonstrate both short- and long-term impact over time. Additionally, there should be an opportunity for piloting site-specific and region-specific measures. In fact, sites should be encouraged or required to identify their own quality improvement initia- tives consistent with the mission and goals of the program, but responsive to site-specific needs assessments (which are conducted routinely as part of the reapplication process). For example, sites with teen pregnancy problems would need to focus quality improvement activities differently from sites with birth-spacing, birth-limiting, or STD problems. It is conceivable that site-specific quality improvement indicators would be drawn from the uni- verse of indicators and piloted and validated using the AHRQ framework of indicator development (AHRQ, 2001). Validated indicators could then be shared horizontally across sites and vertically from the bottom up with program administrators for consideration as national indicators. Further, there should be opportunities to share best practices to help understand why variation exists, provide a mechanism to minimize variation, and move 

APPENDIX K 443 more sites toward benchmark goals. The overall process could work simi- larly to the rapid improvement cycles and learning collaboratives currently being advocated by the Institute for Healthcare Improvement (IHI, 2003). As these measures are introduced, further research and analysis of the merits of the Title X program should be conducted using a mix of method- ologies, such as provider observation or simulated patients, to document technical competence and communication skills. Additional scientific valida- tion of the effectiveness of these interventions is needed. Whenever possible, indicator development, data collection, and reporting should be electronic, or funding for FTE support should be included so that service resources will not be used to offset this expense. Given the transient nature of the poor population, there should be a vision or capacity for shared informa- tion (regional health information network), consistent with national goals for a health information highway (NCVHS, 2000). Additionally, a widely acknowledged strength of the Title X program is IEC training tools and clinical standards. Validation of the success of the IEC modules by both providers and clients should be confirmed; replicated across all sites; shared across state and federal agencies, including STD clinics and public schools; and incorporated into medical, nursing, and residency training curricula. Broader support at the administrative and policy levels is needed to facilitate the development of a national agenda emphasizing the maternal, child, and family health benefits of family planning and preventive health services in general and the Title X program in particular. While the presumed benefits are clear, the opportunity now exists to demonstrate the quality of ­family planning services within the Title X program definitively and establish a causal link with reproductive health outcomes. The American public needs to learn to plan for pregnancy from puberty on, and to acquire the skills necessary to achieve personal reproductive goals. 

444 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM annex k-1 healthy people 2010 objectives Currently, there are 13 Healthy People 2010 objectives related to repro- ductive health that could serve as indicators of quality for the Title X pro- gram (see Annex Table K-1): • 9.1. Intended pregnancy • 9.2. Birth spacing • 9.3. Contraceptive use — Contraceptive failure — Emergency contraception — Male involvement with pregnancy prevention — Adolescent pregnancy — Abstinence before age 15 — Abstinence ages 15–17 — Pregnancy prevention and STD protection — Pregnancy prevention education — Problems in becoming pregnant — Insurance coverage for contraceptive supplies and services There are also 19 Healthy People 2010 objectives related to STDs (see Annex Table K-2): • Chlamydia—Reduce infection in those aged 15–24. • (GC)—Reduce infection. • Syphilis—Eliminate primary and secondary syphilis from the United States. • Herpes—Decrease percent with genital infection. • Human papillomavirus (HPV) (developmental)—Decrease percent with HPV (can help minimize the number of high-risk subtypes associated with cervical cancer). • Pelvic inflammatory disease—Reduce proportion of females who have ever acquired PID. • Fertility problems—Decrease percent of women with fertility prob- lems associated with chlamydia and PID. • Heterosexual HIV (developmental)—Reduce HIV infections in females aged 13–24 associated with heterosexual contact. • Congenital syphilis—Reduce congenital syphilis. • Neonatal STD (developmental)—Reduce neonatal consequences from maternal STD. 

APPENDIX K 445 ANNEX TABLE K-1  Healthy People 2010 Reproductive Health Objectives with Targets and Baselines Topic Objective Target Baseline Intended Increase percent of 70 percent 51 percent pregnancy pregnancies that Note: 39 percent in Canada, are intended 6 percent in Netherlands Spacing Reduce percent of 6 percent 11 percent births occurring within 24 months of a previous birth Contraceptive Increase use 100 percent 93 percent used use People who do not use any method account for half of unintended pregnancies; rest occur to those who use intermittently or incorrectly Contraception Reduce proportion 7 percent 13 percent failure of females Focus on consistent and experiencing correct use of a specific pregnancy despite method use of a reversible contraceptive method Emergency Increase percent of Emergency contraception can contraception health care reduce risk of pregnancy by (developmental) providers that 75 percent; in 1995, fewer provide emergency than 1 percent of women contraception reported using emergency contraception; need direct access and insurance reimbursement continued 

446 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM ANNEX TABLE K-1  Continued Topic Objective Target Baseline Male Increase male Need to be Men make up 10 percent of involvement involvement in culturally and total clientele in only with pregnancy pregnancy linguistically 13 percent of clinics; an prevention prevention and sensitive in average of 6 percent of (developmental) family planning promoting clients are male—only efforts condom use and 2 percent of Title X clients in addressing HIV 1991 and 2 percent of and STD Medicaid clients in 1990 prevention, managed care marketing; emphasis on male responsibilities regarding welfare, need for rapid treatment of female partners who test positive for bacterial STD to decrease STDs, including HIV Adolescent Decrease 43/1000 68/1000 pregnancy Abstinence Increase percent of Females Females 62 percent; males before age 15 teens who have 75 percent; 57 percent never engaged in males 75 percent The later intercourse starts, intercourse the less overall exposure; need education regarding intimacy; setting limits; and resistance to social, media, peer, and partner pressure Dual protection Condoms: females Condoms: females at ages 15–17 75 percent, males 67 percent, males 72 percent; at first 83 percent; condoms +: females intercourse condoms +: 7 percent, males 11 percent females 9 percent, Condom use has increased, males 11 percent suggesting that teens do anticipate and plan for initiation; decrease in hormonal treatment 

APPENDIX K 447 ANNEX TABLE K-1  Continued Topic Objective Target Baseline Last intercourse Condoms: females Condoms: females 49 percent, males 39 percent, males 70 percent; 79 percent; condoms +: females condoms +: 7 percent, males 16 percent females Education about birth 11 percent, males control does not prevent 20 percent infections Pregnancy 90 percent 64 percent females (tracking prevention in men has just started) education Problems in Decrease 10 percent 13 percent becoming proportion of In 1995, small decline in pregnant married people infertility most marked in unable to conceive Hispanics or maintain pregnancy Insurance Institute of Medicine: one coverage for reason for unintended contraceptive pregnancy is lack of supplies and coverage; half of indemnity services plans and 7 percent of health (developmental) maintenance organizations covered contraception in 1993; methods inconsistent; bias toward permanent surgical methods • Responsible teen sex—Increase percent of adolescents who abstain from sex or use condoms if active. • Responsible sex on television (developmental)—Increase number of positive messages related to responsible sexual behavior on television. • Hepatitis B vaccine in STD clinics—Increase number of STD pro- grams that offer hepatitis B vaccine. • Screening in detention and jails (developmental)—Screen within 24 hours of admission and provide treatment before release. • Contracts to treat nonplan partners (developmental)—Increase per- cent of local health departments that have contracts with managed care providers for treatment of nonplan partners. 

448 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM ANNEX TABLE K-2  Healthy People STD Objectives with Targets and Baselines Topic Objective Target Baseline Chlamydia Reduce infection in 3.0 5.0 those aged 15–24 Family planning clinics STD clinics 3.0 12.2 Males 3.0 15.7 (GC) Reduce infection 19/100 thousand 123/100 thousand Syphilis Eliminate primary 0.2/100 thousand 3.2/100 thousand and secondary syphilis from United States Herpes Decrease percent 14 percent 17 percent with genital infection Human Decrease percent papillomavirus with HPV (can help (HPV) minimize the (developmental) number of high-risk subtypes associated with cervical cancer) Pelvic inflammatory Reduce proportion 5 percent 8 percent disease of female who have ever acquired PID Fertility problems Decrease percent of 15 percent 27 percent of women women with with fertility problems fertility problems reported history of associated with PID chlamydia and PID Heterosexual HIV Reduce HIV (developmental) infections in females aged 13–24 associated with heterosexual contact 

APPENDIX K 449 ANNEX TABLE K-2  Continued Topic Objective Target Baseline Congenital syphilis Reduce congenital 1/100 thousand 27/100 thousand syphilis Neonatal STD Reduce neonatal (developmental) consequences from maternal STD Responsible teen Increase percent of 95 percent 85 percent sex adolescents who abstain from sex or use condoms if active Responsible sex on Increase number of television positive messages (developmental) related to responsible sexual behavior on television Hepatitis B vaccine Increase number of 90 percent 5 percent in STD clinics STD programs that offer hepatitis B vaccine Screening in Screen within detention and jails 24 hours of (developmental) admission and provide treatment before release Contracts to treat Increase percent of nonplan partners local health (developmental) departments that have contracts with managed care providers for treatment of nonplan partners Annual screening Increase percent of for chlamydia women under age (developmental) 25 screened annually continued 

450 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM ANNEX TABLE K-2  Continued Topic Objective Target Baseline Screening of Increase percent of pregnant women pregnant women (developmental) screened for STD, HIV, and (BV) Compliance with Increase percent of 90 percent 70 percent recognized STD primary care treatment providers who treat patients with STDs who manage according to standards Provider referral for sex partners (developmental) • Annual screening for chlamydia (developmental)—Increase percent of women under age 25 screened annually. • Screening of pregnant women (developmental)—Increase percent of pregnant women screened for STD, HIV, and (BV). • Compliance with recognized STD treatment—Increase percent of primary care providers who treat patients with STDs who manage according to standards. • Provider referral for sex partners (developmental). 

APPENDIX K 451 Annex K-2 Representative indicators for each of the eight broad categories in Bertrand et al. (1994) I. Policy Environment • Existence of a policy development plan • Number of appropriately disseminated policy analyses • Number of awareness-raising events targeted to leaders • Existence of a strategic plan for expanding the national family planning program • Integration of demographic data into development planning • Number of statements of leaders in support of family planning • Formal population policy addressing fertility and family planning • National family planning coordination • Level of the family planning program within the government administration • Levels of import duties and other taxes • Restrictions on advertising of contraceptives in the mass media • Absence of unwarranted restrictions on providers and users • Quality of program leadership • Extent of commercial-sector participation II. Service Delivery Operations • Management • Training • Commodities and logistics • Information–education–communication (IEC) • Research and evaluation III. Management (illustrative indicators) • Existence of a clear mission that contributes to the achievement of program goals • Realization of operational targets • Clearly defined organizational structure • Adequacy of staffing • Awareness of current financial position • Access to current information on key areas of program functioning • Access to current information on program progress • Capacity to track commodities 

452 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM IV. Training • Number/percentage of courses that achieve learning objectives • Number/percentage of courses that contribute to the achievement of program training objectives • Number/percentage of courses in which the training methodology is appropriate for the transfer of skills and knowledge • Number of trainees by type • Number/percentage of trainees who have mastered relevant knowledge • Number/percentage of trainees competent to provide a specific family planning service • Number/percentage of trained providers assessed to be competent at a specified period (e.g., 6 months) post-training • Number/percentage of trainees who apply the skills to their subse- quent work V. Commodities and Logistics • Pipeline wastage • Percentage of storage capacity meeting acceptable standards • Frequency of stock-outs • Percentage of service delivery points (SDPs) stocked according to plan • Percentage of key personnel trained in contraceptive logistics • Composite indicator for commodities and logistics VI. Information–Education–Communication • Number of communications produced, by type, during a reference period • Number of communications disseminated, by type, during a refer- ence period • Percentage of target audience exposed to program messages, based on respondent recall • Percentage of target audience who correctly comprehend a given message • Number of contraceptive methods known • Percent of audience who acquire the skill to complete a certain task as a result of exposure to a specific communication • Percentage of target audience exposed to a specific message who report liking it • Number/percentage of target audience who discuss message(s) with others, by type of person • Percentage of target audience who advocate family planning practice 

APPENDIX K 453 VII. Research and Evaluation • Presence of an active research and evaluation unit • Extent of use of a service system • Conduct of periodic household and/or special-purpose surveys and studies • Conduct of operations research • Regular conduct of process evaluations • Conduct of effectiveness, efficiency, and impact evaluations • Use of research and evaluation results for program modification • Dissemination of research and evaluation results VIII. Family Planning Service Outputs • Accessibility (illustrative indicators) — Number of SDPs located within a fixed distance or travel time of a given community (i.e., service density) — Cost of 1 month’s supply of contraceptives as a percentage of monthly wages — Restrictive program policies on contraceptive choice — Percentage of the population who know of at least one source of contraceptive services and/or supplies — Percentage of nonuse related to psychosocial barriers • Quality of care (illustrative indicators) — Number of contraceptive methods available at a specific SDP — Percentage of counseling sessions with new acceptors in which provider discusses all methods — Percentage of client visits during which provider demonstrates skill in clinical procedures, including asepsis — Percentage of clients reporting sufficient time with provider — Percentage of clients informed of timing and sources for resupply/revisit — Percentage of clients who perceive that hours/days are convenient • Program Image — Number and type of activities to improve the public image of family planning during a reference period (e.g., 1 year) — Percentage of target population favorable to the (national) family planning program • Service Utilization — Number of visits to SDP(s) — Number of acceptors new to modern contraception 

454 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM — Number of acceptors new to the institution — Number of new acceptors — Couple-years of protection (CYP) — Method mix — User characteristics — Continuation rates • Contraceptive Practice — Contraceptive prevalence rate (CPR) — Number of current users — Level of ever (past) use — Source of supply (by method) — Method mix — User characteristics — Continuation rates — Use failure rates • Fertility Impact 

APPENDIX K 455 References AGI (The Alan Guttmacher Institute). 2000. Fulfilling the promise: Public policy and U.S. family planning clinics. New York: AGI. AHRQ (Agency for Healthcare Research and Quality). 2001. Refinement of the HCUP ­quality indicators. www.qualityindicators.ahrq.gov/data/hcup/qirefine.htm (accessed January 15, 2008). AHRQ. 2004. Prevention quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/pqi_overview.htm (accessed January 15, 2008). AHRQ. 2006a. Inpatient quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/iqi_overview.htm (accessed January 15, 2008). AHRQ. 2006b. Patient safety indicators overview. Rockville, MD: AHRQ. http://www.­ qualityindicators.ahrq.gov/psi_overview.htm (accessed January 15, 2008). AHRQ. 2006c. Pediatric quality indicators overview. Rockville, MD: AHRQ. http://www. qualityindicators.ahrq.gov/pdi_overview.htm (accessed January 15, 2008). AHRQ. 2007. Screening for chlamydial infection. Rockville, MD: AHRQ. http://www.ahrq. gov/clinic/uspstf/uspschlm.htm (accessed May 7, 2008). Alden, D. L. 2004. Client satisfaction with reproductive health care quality. Social Science & Medicine 59:2219–2232. Amaral, G., D. G. Foster, M. A. Biggs, C. B. Jasik, S. Judd, and C. D. Brindis. 2007. Public savings from the prevention of unintended pregnancy: A cost analysis of family planning services in California. Health Services Research 42(5):1960–1980. Becker, D., M. A. Koenig, Y. M. Kim, K. Cardona, and F. L. Sonenstein. 2007. The quality of family planning services in the United States: Findings from a literature review. Perspec- tives on Sexual and Reproductive Health 39:206–215. Bertrand, J. T., R. J. Magnani, and J. Knowles. 1994. Handbook of indicators for family plan- ning. Chapel Hill, NC: Carolina Population Center. http://www.cpc.unc.edu/­measure/ publications/pdf/ms-94-01.pdf (accessed April 18, 2008). Bowman, S. A. 2006. Television-viewing characteristics of adults: Correlations to eating prac- tices and overweight and health status. http://www.cdc.gov/pcd/issues/2006/apr/050139. htm (accessed May 10, 2008). Bronstein, J. M., A. Vosel, S. K. George, C. Freeman, and L. A. Payne. 2007. Extending Medicaid coverage for family planning: Alabama’s first 4 years. Public Health Reports 122:190–197. Bruce, J. 1990. Fundamental elements of the quality of care: A simple framework. Studies in Family Planning 21(2):61–91. Center for Reproductive Rights. 2004. America must continue its commitment to reproductive health. http://www.reproductiverights.org (accessed May 10, 2008). Cleland, J., S. Bernstein, A. Ezeh, A. Faundes, A. Glasier, and J. Innis. 2006. Family planning: The unfinished agenda. Lancet 368:1810–1827. Collins, R. L., M. N. Elliott, S. H. Berry, D. E. Kanouse, D. Kunkel, S. B. Hunter, and A. Miu. 2004. Watching sex on television predicts adolescent initiation of sexual behavior. Pediatrics 114(3):e280–e289. Deming, W. 1986. Out of the crisis. Cambridge, MA: MIT Center for Advanced Engineering Study. Donabedian, A. 1968. Promoting quality through evaluating the process of patient care. Medical Care 6:181–202. Dreweke, J. 2006. Each $1 invested in Title X Family Planning program saves $3.80. D ­ oubling funding could prevent 244,000 unplanned pregnancies, result in net savings of almost $800 million. New York: Guttmacher Institute. http://www.guttmacher.org/­ media/nr/2006/11/16/index.html (accessed May 7, 2008). 

456 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Finer, L. B., J. E. Darroch, and J. J. Frost. 2002. US agencies providing publicly funded contra- ceptive services in 1999. Perspectives on Sexual and Reproductive Health 34:15–24. Fisher, E. S., and J. E. Wennberg. 2003. Health care quality, geographic variations, and the challenge of supply-sensitive care. Perspectives in Biology and Medicine 46:69–79. Foster, D. G., C. M. Klaisle, M. Blum, M. E. Bradsberry, C. D. Brindis, and F. H. Steward. 2004. Expanded state-funded planning services: Estimating pregnancies averted by the Family PACT Program in California, 1997–1998. American Journal of Public Health 94(8):1341–1346. Fowler, C. I., J. Gable, and J. Wang. 2008. Family planning annual report: 2006 national summary. Research Triangle Park, NC: RTI International. FPCA (Family Planning Council of America, Inc.). 1999. Performance monitoring system (draft provided by A. Stith, IOM). Frost, J. J. 2002. Family planning annual report: 2001 summary, part 1. New York: AGI. Frost, J. J. 2003. Family planning annual report: 2002 summary, part 1. New York: AGI. Frost, J. J., and L. Frohwirth. 2005. Family planning annual report: 2004 summary, part 1. New York: AGI. Gilliam, M. L., and K. Hernandez. 2007. Providing contraceptive care to low income African American teens: The experience of urban community health centers. Journal of Com- munity Health 32:231–244. Glasier, A., A. M. Gülmezoglu, G. P. Schmid, C. G. Moreno, and P. F. Van Look. 2006. Sexual and reproductive health: A matter of life and death. Lancet 368(9547):1595–2607. Gregory, K. D., C. J. Hobel, L. M. Korst, M. Lu, and C. Reyes. 2005. A framework for devel­ oping Maternal Quality Care Indicators. Prepared for the California Department of Health and Human Services, Maternal and Child Health Branch. Guttmacher Institute. 2008. Facts in brief. Facts on contraceptive use. http://www.­guttmacher. org/pubs/fb_contr_use.html (accessed May 8, 2008). HHS (U.S. Department of Health and Human Services). 2000a. Healthy people 2010 (2nd edition). Washington, DC: U.S. Government Printing Office. HHS. 2000b. With understanding and improving health and objectives for improving health (2 volumes). In Healthy people 2010 (2nd edition). Washington, DC: HHS. HHS. 2000c. Healthy people 2010—Reproductive health. www.hhs.gov/opa/pubs/hp201_rh.html (accessed April 28, 2008). HHS. 2001. Program guidelines for project grants for family planning services. http://www.hhs. gov/opa/familyplanning/toolsdocs/2001_ofp_guidelines_complete.pdf (accessed April 24, 2008). HHS. 2006. Family planning program priorities, legislative mandates, key issues. http://www. hhs.gov/opa (accessed April 24, 2008). IHI (Institute for Healthcare Improvement). 2003. The breakthrough series: IHI’s collabora- tive model for achieving breakthrough improvement. IHI Innovation Series white paper. Boston: IHI. www.IHI.org (accessed April 28, 2008). IOM (Institute of Medicine). 1995. The best intentions: Unintended pregnancy and the well- being of children and families. Washington, DC: National Academy Press. IOM. 2001. Crossing the quality chasm: A new health system for the 21st century. Washing- ton, DC: National Academy Press. IPPF (International Planned Parenthood Federation). 1998. Family planning programs: Improv­ ing quality (sidebars). Population Reports, Series J, No. 47. Jain, A., J. Bruce, and S. Kumar. 1992. Quality of services, programme efforts and fertility reduction. In Family planning programmes and fertility. Edited by J. F. Phillips and J. A. Ross. Oxford: Clarendon Press. 

APPENDIX K 457 Kerr, E. A., C. L. Damberg, and S. M. Asch (editors). 2000. Quality of care for women: A ­ review of selected clinical conditions and quality indicators. Santa Monica, CA: RAND. Korst, L. M., K. D. Gregory, M. C. Lu, C. Reyes, C. J. Hobel, and G. F. Chavez. 2005. A framework for the development of maternal quality of care indicators. Maternal and Child Health Journal 9(3):317–341. Kunkel, D., K. Eyal, E. Biely, et al. 2003. Sex on TV3: A biennial report to the Kaiser Family Foundation. Menlo Park, CA: The Henry J. Kaiser Foundation. www.kff.org/entme- dia/loader.cfm?url=/commonspot/security/getfile.cfm&PageID=14209 (accessed April 28, 2008). Lindberg, L. D., J. J. Frost, C. Sten, and C. Dailard. 2006. The provision of funding and con- traceptive services at publicly funded family planning agencies: 1995–2003. Perspectives on Sexual and Reproductive Health 38:37–45. Lu, M. C., and N. Halfon. 2003. Racial and ethnic disparities in birth outcomes: A life course perspective. Maternal and Child Health Journal 7:3–18. Meyers, D., T. Wolff, K. D. Gregory, L. Marion, V. Moyer, H. Nelson, D. Petitti, and G. Sawaya. 2008. A summary of United States Preventive Service Task Force (USPSTF) recommendations for STI screening. American Family Physician 77(6):819–824. Mosher, W. D., G. M. Martinez, A. Chandra, J. C. Abma, and S. J. Wilson. 2004. Use of contraception and use of family planning services in the United States, 1982–2002. Hyattsville, MD: National Center for Health Statistics. NCQA (National Center Quality Assurance). 2007. Improving chlamydia screening. Strat- egies from top performing health plans. http://www.ncqa.org/Portals/0/Publications/ Resource%20Library/Improving_Chlamydia_Screening_08.pdf (accessed May 7, 2008). NCVHS (National Committee on Vital and Health Statistics). 2000. Toward a national health information infrastructure interim report. http://ncvhs.hhs.gov/NHII2kReport.htm (ac- cessed April 8, 2008). OMB (Office of Management and Budget). 2005. The Program Assessment Rating Tool (PART). Family planning assessment. http://www.whitehouse.gov/omb/expectmore/­ detail/10003513.2005.html (accessed April 8, 2008). Paine, K., M. Thorogood, and K. Wellings. 2000. The impact of the quality of family plan- ning services on safe and effective contraceptive use: Literature review. Human Fertility 3(3):186–193. Piotrow, P. T., D. L. Kincaid, J. G. Rimon II, and W. Rinehart (editors). 1997. Health commu- nication: Lessons from family planning and reproductive health. Westport, CT: Praeger. RamaoRao, S., and R. Mohanam. 2003. The quality of family planning programs: Concepts, measurements, interventions and effections. Studies in Family Planning 34:227–248. Roberts, D. F., U. G. Foehr, V. J. Rideout, and M. Brodie. 1999. Kids and media at the new millennium. In A Kaiser Family Foundation report: A comprehensive national analysis of children’s media use: Executive summary. Menlo Park, CA: The Henry J. Kaiser Family Foundation. RTI International (Research Triangle Institute International). 2006. Family planning annual report: 2005 national summary. Research Triangle Park, NC: RTI International. Sandfort, T. G., M. Orr, J. S. Hirsch, and J. Santelli. 2008. Long-term health correlates of timing of sexual debut: Results from a national US study. American Journal of Public Health 98(1):155–161. Schunmann, C., and A. Glasier. 2006. Measuring pregnancy intention and its relationship with contraceptive use among women undergoing therapeutic abortion. Contraception 73:520–524. 

458 A REVIEW OF THE HHS FAMILY PLANNING PROGRAM Schuster, M. A., S. M. Asch, E. A. McGlynn, E. A. Kerr, A. M. Hardy, and D. S. Gifford. 1997. Development of a quality of care measurement system for children and adolescents. Archives of Pediatrics and Adolescent Medicine 151:1085–1092. Sofaer, S., and K. Firminger. 2005. Patient perceptions of the quality of health services. Annual Review of Public Health 26:513–559. Speroff, T., and G. T. O’Connor. 2004. Study designs for PDSA quality improvement research. Quality Management in Health Care 13:17–32. Sullivan, T. M., and J. T. Bertrand. 2000. Monitoring quality of care in family planning by the quick investigation of quality (QIQ): County reports. MEASURE Evaluation Technical Report Series No. 5. Chapel Hill, NC: Carolina Population Center. Thornburn, S., and L. M. Bogart. 2005a. African American women and family planning s ­ ervices: Perceptions of discrimination. Women’s Health 42:23–39. Thornburn, S., and L. M. Bogart. 2005b. Conspiracy beliefs about birth control: Barriers to pregnancy prevention among African Americans of reproductive age. Health Education and Behavior 32(4):474–487. United Nations. 1994. International Conference on Population and Development (ICPD), September 1994, Cairo, Egypt. http://www.un.org/popin/icpd2.htm (accessed April 8, 2008). Wennberg, J. E. 1999. Understanding geographic variations in health care delivery. New ­ ngland Journal of Medicine 340:52–53. E WHO (World Health Organization). 1998. WHO definition of quality. Population Reports Series J, No. 47. WHO. 2004. Medical eligibility criteria for contraceptive use (3rd Edition, Reproductive Health and Research). Geneva, Switzerland: WHO. Winter, L., and A. S. Goldy. 1987. Staffing patterns in family planning clinics: Which model is best? Family Planning Perspectives 19(3):102–106. Zaky, H. H., H. A. Khattab, and D. Galal. 2007. Assessing the quality of reproductive health services in Egypt via exit interviews. Maternal and Child Health Journal 11:301–306.