Click for next page ( 128


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 127
7 Quality Control in Other Monetary Distribution Systems There has been increasing awareness of the importance of quality man agement in industry, including service industries, and in federal need-based social programs. Recognizing these developments, the Department of Edu- cation has been interested in the lessons that might be learned from other organizations involved in activities similar to those of the student financial aid system. In response to past recommendations that ideas for improve- ment be sought, the department commissioned a study that compared sev- eral family assistance programs and student aid programs (Advanced Tech- nology and Westat, 1987a,b). The department asked the panel to address . .. . slmllar issues. A complete benchmarking with other programs would be difficult be- cause service industries, social programs, and student financial aid pro- grams differ in regard to goals, concerns, and approaches. Further, the panel's resources limited the amount of information that could be collected. Yet, in its review of several family assistance and veterans benefit pro- grams, mortgage, insurance, and other financial entities, the panel found two practices that the Department of Education should incorporate as oper- ating strategies. First, the organizations reviewed do not hold themselves to an unattainable zero-defect standard. Second, monitoring functions are generally allocated based on the risk the activity presents to achieving important organizational goals. This chapter describes the activities of the organiza- tions reviewed, and it suggests a framework for making comparisons across programs and within which lessons can be drawn for the student financial aid programs. 127

OCR for page 127
28 QUALITY IN STUDENT FINANCIAL AID PROGRAMS FEDERAL NEED-BASED BENEFIT PROGRAMS The Congressional Research Service has identified more than 75 fed- eral benefit programs (including student financial aid) providing aid in the form of cash and noncash that are directed primarily to persons with limited income (Burke, 1991~. In fiscal year 1990, those income-tested benefit programs cost $210.6 billion, 72 percent in federal funds and the remainder in state or local funds. Total spending on those programs amounted to almost one-eighth of the fiscal 1990 federal budget. Of the spending for those programs in 1990, about 9 percent was in- vested in "human capital" programs, including education, jobs, and training. Forty-one percent was used for medical services. Cash, food benefits, and housing aid together accounted for almost 47 percent. More than 90 percent of the programs have an explicit test of income, of which there are five kinds: income ceiling related to one of the federal government's official poverty measures (Census Bureau poverty thresholds or federal pov- erty income guidelines); income limit related to state or area median income; income limit related to the lower living standard income levels used by the Bureau of Labor Statistics; income below absolute dollar standard; and income level deemed to indicate "need" (typically, student financial aid programs use this type of income test). All of these tests require that data on family or individual income be ob- tained, and that the income data be verified for all or a sample of applicants. The programs have different definitions of income and time periods for which income is reported. The panel could not look at quality control procedures in all of the programs identified by the Congressional Research Service. Thus, we chose to examine the quality control activities of several programs that were the subject of prior studies by the National Research Council (NRC) of the National Academy of Sciences. The very proliferation of programs, how- ever, suggests potential duplication of verification and quality control pro- cesses for those aid recipients who receive more than one type of aid (espe- cially if one type is not based on receiving another type). Although the panel did not have the resources to explore this area in depth, we believe coordination of verification activities and sharing of information about the development of these activities in the various programs would benefit all agencies in their attempts to improve quality. The following discussion describes the eligibility, verification, and quality control systems used in the family assistance programs.

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 129 Quality Control in Family Assistance Programs Quality control in family assistance programs offers a useful basis for comparison with the system used in the student financial aid programs. Both sets of programs serve low-income populations and involve very large commitments of federal dollars. The family assistance programs, however, pose quite a contrast in their verification strategy. First, they rely heavily on rigorous front-end verification, in general, before individuals receive any cash assistance. Then, they apply systematic and extensive post hoc re- views to a sample of recipients in order to assess the correctness of eligibil- ity and benefits received. Finally, they assess stringent penalties against the administering agencies (states) in cases of error above prescribed toler ances. Three federal benefit programs were studied by the NRC from 1986 to 1988-Aid to Families with Dependent Children (AFDC), Food Stamps, and Medicaid (Affholter and Kramer, 1987; Kramer, 1988~. Together, they represented nearly $50 billion in fiscal 1987 federal dollars. In the AFDC and Medicaid programs, states and localities pay a substantial addition in benefit amounts (between 50 and 83 percent depending on a formula based on state per capita income), and they pay 50 percent of administrative costs in all three programs. At the time of the NRC study, there were nearly 11 million recipients of AFDC, 22 million participants in Medicaid, and 20 million recipients of Food Stamps. Caseload monitoring and payment determination in these programs can be extremely complex. First, eligibility and benefit payments are deter- mined monthly (although comprehensive determinations are not repeated each month) and must reflect what is a frequently changing profile of in- come and household composition in recipient households. Second, the three programs are administered by separate federal agencies and frequently have little connection at the federal level although they are targeted at similar populations and the same individuals commonly participate in all three pro- grams concurrently. At the state and local levels, the programs may be administered separately or by the same public assistance agency, and their services may be delivered through a unified state system or by county and local agencies with differing rules and procedures. Thus, from a quality control perspective, there are two main difficulties. First, eligibility and payment levels are constantly changing. Second, federal policies and ben- efit payments may be delivered through a network of hundreds of different federal, state, and local service delivery designs, which involve differing capabilities and, in many cases, differing and at times incongruent policies and procedures. The quality control system in each of the programs employs a two- tiered review process. The system is used to monitor the correctness of

OCR for page 127
130 QUALITY IN STUDENT FINANCIAL AID PROGRAMS eligibility determinations, correct identified errors, calculate error rates, and assign penalties against the states. State staff sample and review cases monthly, and the federal agencies review a subsample of those cases and determine error rates and penalties. States also maintain a staff, in most cases separate from the program staff and separate from the quality control reviewers, to design and supervise corrective action. Error rates are calcu- lated using a combined value of the state and federal reviews, although the federal estimate from the subsample prevails in cases of disagreement. When error rates exceed specified thresholds, penalties are collected dollar-for-dollar against the federal share of benefit payments for AFDC and Medicaid and against the federal administrative share in the Food Stamp program. States are required to correct AFDC underpayments and recoup overpayments promptly, whatever the cause or identifying source, including the quality control system. There are limits on the amount of overpayment that may be recouped from current recipients at any one time; large over- payments are recouped over a number of months. In the Food Stamp pro- gram, a percentage of the value of recouped overpayments may be kept by the state under certain circumstances. Only the Food Stamp program offers rewards for good performance, and until 1988 and 1990 (when separate legislation for Food Stamps and AFDC, respectively, changed the situation), underpayments did not figure in the calculation of error rates for purposes of assessing financial penalties. In both the AFDC and Food Stamp pro- grams, neither improper terminations nor denials (so-called negative case actions) are counted as errors in determining sanctions. Eligibility Eligibility rules for each of the three programs are different. Each has differing income and asset tests, inclusions and exclusions, and periods of coverage. Thus, processing and verification requirements also differ and pose special challenges to the administering agencies. For purposes of illustration, the following brief description focuses on the AFDC and Med . . scald programs. Eligibility for AFDC and Medicaid, beyond broad limits established in federal law, is determined by the state. But states define their own stan- dards of need and income and resource limits (within federal limits), and they set their own benefit levels within those standards. States are not required to pay 100 percent of their determined standard of need, and about half the states do not. Eligibility for AFDC is determined prospectively, based on the best estimate of the income and circumstances for the month assistance will be paid, except that for the initial one or two months states may use a retro- soective budgeting method to compute the benefit payment. Generally,

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 131 eligibility must be redetermined at least every six months, and a face-to- face interview is required at least once a year. States may require certain families to file eligibility reports monthly. Families with earned income must report monthly, generally by mail, any changes in income or resources, unless specifically exempted from this requirement. The agency must ad- just the payment accordingly or terminate assistance, as appropriate. Since recipients typically move in and out of employment and sometimes on and off the benefit rolls, income and resource changes are a large source of error. Determining income in order to calculate AFDC benefit amounts is a two-step process: determining that gross income does not exceed 185 per- cent of the state's standard of need and determining that net income does not exceed 100 percent of the same standard. Totalfamily income includes gross income before taxes from a variety of specified sources (e.g., wages, child support, certain subsidies, and some income of others who are not necessarily in the assistance unit, such as stepparents, an alien's sponsor, or the parent of a minor parent, that is deemed available to the applicant household), some at state option. There is a variety of exclusions, such as grants, loans, scholarships, and income or subsidies from certain other as- sistance programs (e.g., Social Security Insurance, or SSI, payments, nutri- tion subsidies, low-income energy assistance, and at state option, certain job-training program income). Certain other income is disregarded, such as income that offsets a limited amount of work, training, and dependent care expenses. A limited amount of earned income from a job is excluded as an encouragement to work. Except for specified exclusions (e.g., primary residence and the value of an automobile up to $1,500), an AFDC assistance unit cannot have re- sources in excess of $1,000 or a lesser amount determined by the state. In general, the resources counted are those that can be converted into cash; essentials such as clothing and furniture, as determined by the state, may be disregarded. In addition to income and resource determinations, AFDC eligibility requires establishing the dependency of a child, which poses special chal- lenges. To qualify as a dependent child, a child must be "deprived of parental support or care" by the death, continued absence or incapacity of one or both parents, or by the unemployment of the principal wage-earning parent. Frequently, the most difficult task in establishing eligibility is determin- ing who should be included in the household unit and be considered to be contributing to household income. Agencies may deny aid because of in- sufficient verification of continued absence of a father, for example. States must provide Medicaid to all so-called categorically needy. These include all AFDC and SSI recipients (or, at state option, all aged, blind and

OCR for page 127
32 QUALITY IN STUDENT FINANCIAL AID PROGRAMS disabled, using stricter criteria, and all others who meet AFDC income and resource requirements). However, states may, at their option, extend cover- age to others such as those eligible for but not receiving cash assistance, those eligible except for their institutional status, and others whose income is up to 100 percent of the poverty level. States may also extend coverage to those deemed medically needy- those whose income is otherwise above the standards but whose income after deducting or "spending down" their medical expenses puts them below the state's income standards for assis- tance. For the categorically needy in Medicaid, AFDC and SSI rules apply. But for all others, determining income for Medicaid eligibility is further complicated. For example, parental and spousal income are considered available to Medicaid applicants. Complex rules related to family contribu- tions and transfers of assets are used to determine eligibility for institution- alized persons. For the medically needy using the spend-down provisions, income is determined prospectively for up to six months. Verification Verification is done at the point of initial application (and at required periodic reporting and redeterminations) by the eligibility worker, and through complete reviews of cases in the quality control sample and federal re- reviews of the subsample. In order to demonstrate eligibility, an individual applying for AFDC benefits must demonstrate U.S. citizenship or legal alien status and furnish a Social Security Number for each member of the assistance unit. Appli- cants are further required to demonstrate dependency of a child, and their ability and willingness to work or receive educational and training services, and are required to assist the state in securing spousal support payments. In all cases, with certain exceptions for incapacity, applicants apply for assistance in person, generally at a local office that administers applications for all three programs, and provide required documentation to corroborate their family status at that time. Applicants may also have to allow a home visit. If an applicant does not comply during the application process with agency procedures for demonstrating eligibility, the application is denied. Failure to comply with requirements to document continued eligibility re- sults in case closing. Nationally, failure to comply with a procedural re- quirement is the most common cause of case closing, even though the appli- cant or participant may be substantively eligible. Income is further verified through matching systems. State agencies administering AFDC, Medicaid, and Food Stamps must operate an Income and Eligibility Verification System (IEVS). The system is used to verify the accuracy of client income for the purposes of eligibility and benefit deter

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 133 minations by matching application files with data from the Internal Revenue Service (IRS), Social Security Administration (SSA), State Wage and Infor- mation Collection Agency (SWICA), and Unemployment Insurance Benefit (UIB) data bases. Since the programs serve individuals presumed to be in immediate need, a variety of requirements exist to speed application processing despite man- dated verification requirements. Applicants are required to be notified of eligibility for AFDC within 45 days of application. Benefits for the first month are prorated from the date of application. The eligibility determina- tion process may not be interrupted or delayed by IEVS checks, and data from all sources must be requested at the first available opportunity (the SWICA and UIB process requests twice a month and the IRS once a month). The SSA's Bendix system generates automatic notices to the state when updated information becomes available. Up to 20 percent of the cases may be carried beyond the prescribed application period if warranted by delays in third-party verification. If verification data are received after an applicant is determined to be eligible, a review must be completed within 20 days. Federal law ensures the right to a fair hearing of any adverse decision of the administering agency, and appeals processes can be fairly extensive. The Food Stamp program is administered by the state agency respon- sible for administering federally aided public assistance programs. The federal sponsors, the U.S. Department of Health and Human Services (HlIS) and the U.S. Department of Agriculture (USDA), are required to develop a system of single interviews to determine eligibility for AFDC and Food Stamps. Further, they must develop procedures that may permit, at state option, a single application form for Food Stamps, AFDC, or other public assistance programs, as well as permit, again at state option, a single verifi- cation based on the public assistance application. The secretaries of HHS and USDA are also required to consult in issuing regulations to ensure, to the extent feasible, that definition, valuation, and calculation of income and assets under both programs are comparable. For Food Stamps, the law specifies that states must verify income and resources to the extent practicable, prior to issuance of coupons. To verify eligibility for Food Stamps, states must request information available from the state employment service, the SSA, and the IRS. States must set up a system to protect against receipt of benefits in more than one jurisdiction, as well as a plan for automated data processing and retrieval. States are encouraged to have systems that include data elements for the determination of eligibility, calculation of benefits, issuance of benefits, computer matches for income and asset verifications, reconciliation procedures, and other pur- poses. States may not require additional verification where the state al- ready has current verification. Further, they may not deny an application

OCR for page 127
34 QUALITY IN STUDENT FINANCIAL AID PROGRAMS because of noncooperation from an individual outside the household. As with AFDC, timeliness and hearing requirements are intended to ensure prompt processing of applications and receipt of benefits, but they differ slightly from those for AFDC. For Medicaid, applications must be processed and notification given within 60 days for cases involving disability and 45 days for all others. Payment procedures ensure that 90 percent of claims are paid within 30 days of receipt and 99 percent within 90 days. States must provide for procedures for prepayment and for postpayment claims review. States are also required to check for third-party liabilities and recover monies due for services that have been paid by Medicaid. The Quality Control Systems Although HHS still does AFDC audits annually or less frequently, as determined to be necessary, the principal means of monitoring and recoup- ing misspent funds in the AFDC and Food Stamp programs is through the quality control systems. The Medicaid program has a variety of separate additional monitoring systems and audit procedures. Some are designed to detect fraud and abuse by service providers. The program also has a mechanized claims processing and information retrieval system and a utilization control program. Other systems monitor the use and appropriateness of Medicaid services and are intended to control cost and monitor the level and appropri- ateness of care. Finally, other mechanisms monitor quality of services as part of the certification of facilities that participate in Medicaid. The quality control systems for all three programs are based on the system developed originally for the AFDC system and thus are relatively similar. For purposes of description here, they will be treated as the same system unless major differences warrant mention. History The current system-except for changes in error rate thresh- olds legislated after completion of the mandated NRC and administration studies was developed in the early 1970s in the face of one of the largest increases in caseloads and costs for the AFDC program. The quality control system was specifically designed to address ineligibility, fraud, and abuse. Indeed, although early studies of the system were inconclusive about the level of error, the fact that true error rates were hard to establish only added to concern about ineligibility, fraud, and abuse, and the need for systematic error rate monitoring. The original quality control system design, developed in the 1960s, used caseworker actions, rather than active cases as is currently done, as the sample units. Such a design is better suited to management improvement but less useful for assessing overall error rates because the status of active

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 135 cases between formal redeterminations is not considered. Concern for true case error rates was heightened when the AFDC program went to a simpli- fied eligibility system, a so-called declaration system, whereby eligibility determination was reduced largely to a clerical process, and many means of verification, such as home visits, were eliminated. In revisions begun in 1970, national error standards were set at 3 per- cent for ineligibility and 5 percent for over- and underpayments. Corrective actions rather than financial penalties were imposed for states failing to meet these thresholds. Financial penalties were imposed through regulation in 1973, but they were never collected because, set only in regulation and not in law, they were ruled arbitrary and capricious by a federal court in 1976. They were reimposed by statute in 1979 and became effective in 1981. The new error thresholds were gradually made more stringent over several years, resulting in 3 percent for AFDC and Medicaid by 1983 and 5 percent for Food Stamps by 1985. Recent legislation, described below, has changed these thresholds. Sampling and Review The quality control systems for all three pro- grams are concerned with whether eligibility is properly determined and benefit payments are being properly paid. Monthly samples are selected on a continuing basis in each state. Using a separate quality control staff, states sample from their list of active cases and draw a separate, smaller sample from their negative case actions (households that were denied or terminated). Utilizing data from state quality control reviews of this sample and federal re-reviews of a subsample, the federal agency estimates a pay- ment error rate, which is used to determine the disallowance. The system also measures underpayments and erroneous negative case actions. Under- payments were only recently added to the formula that determines AFDC financial disallowances. Negative case actions still do not contribute to disallowances. Depending on caseload sizes, standard sample sizes, which are required if states use systematic sampling (the chief method used), range between 300 and 2,400 cases. States may use other sample designs and reduce the sample size, but never below the minimum sample sizes for AFDC active cases (between 300 and 1,200~. Some states use larger sample sizes when they desire information for subpopulations or subregions. Many states use an integrated quality control system (IQCS) for the three programs. The federal re-review sample is a subsample of the state sample and also a function of the size of the caseload; for AFDC, it ranges from 140 in states with caseloads below 7,000 to 360 for caseloads of 40,000 and over. Nega- tive case actions are sampled separately from a much smaller subset. Federal guidelines state that "permissible state practice" is to be used as the primary basis for determining eligibility and benefit amounts in AFDC.

OCR for page 127
36 QUALITY IN STUDENT FINANCIAL AID PROGRAMS A standardized worksheet provides a systematic means for the state's qual- ity control reviewer to analyze elements of eligibility and record case find- ings. The quality control reviewer conducts a desk review of each case, analyzing case record documentation and determining the specific elements that must be verified during a field interview. Except in special cases, all sampled cases in which a payment was issued in the review month must receive a full field review. The interview of the recipient is used to (a) establish identity, relationship, and living arrangements; (b) ensure that all significant elements of eligibility have been explored and ascertain any changes; and (c) obtain statements, docu- ments, and other evidence and cooperation and consent for identifying col- lateral sources of information. The reviewer must try to contact collateral sources to verify elements of eligibility and payment when documentation in the case record does not meet verification requirements. When a face-to- face interview is not possible, verification is attempted through other means, such as records on property ownership, work program participation, school attendance, and earned income. Verification must attempt to use primary evidence, such as a birth cer- tificate to verify age or a bank account to verify income. Secondary evi- dence may be used, such as a school record to verify age, when primary evidence is unobtainable. The required verification varies depending on the recipient's specific allegation. If, for instance, he or she claims no earned income, the reviewer must document the allegation. Evidence could in- clude employment records from the state, the IRS, or the SSA. Consulta- tion with a supervisor and other special determinations are required when documentation is questionable or circumstances make certain verification unobtainable, but the special determination cannot be used unless there is already at least one piece of evidence on the element in question. For cases in the negative sample, that is, those that were denied eligi- bility and benefits, the quality control review consists only of analysis of the case record and only for reasons for denial and adherence to notice and hearing requirements, not for correctness of the decision. Limited field investigation can be used if the record cannot be clearly established. In the Food Stamp program, documents from a governmental or public agency constitute primary evidence, and collateral information constitutes secondary evidence. When primary evidence is unavailable, at least two sources of secondary evidence must be used. The U.S. Department of Agriculture's Food and Nutrition Service (ENS) specifies minimum stan- dards of evidence for each element of eligibility, but states may use stricter standards, and thus, standards vary across states. The depth of the investi- gation depends on the nature of the applicant's case. For instance, affirma- tive statements of earnings may be documented with pay stubs, but for denials the quality control reviewer must establish a work history, contact

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 137 previous employers, and query the state employment service or the SSA records of reported wages. Reviewers are also instructed to look for other indications of employment, such as frequent or regular absences from home during working hours, which would indicate employment. Federal review- ers use the same standards of evidence. As with AFDC, however, the federal review is not independent of the state review since federal reviewers work only from the completed quality control files submitted by the states, and they may or may not seek any additional evidence. In each program, state and federal reviewers calculate the value of errors identified in the review process for purposes of assessing penalties. They also distinguish between "client-caused" and "agency-caused" errors, make judgments about willing misrepresentation, and characterize the major sources of error, such as income, resources, deductions, and numerical mis- calculations. States must submit their findings from active and negative AFDC cases within 75 to 120 days from the end of the sample month to ensure timely release of national error rates. The state quality control findings for 90 percent of the sample are due within 75 days of the end of the sample month (for active cases) and for 95 percent of the sample within 95 days. The Administration for Children and Families (formerly the Family Support Administration) attempts to issue error rates within one year after the qual- ity control reviews for the final month of a fiscal year are completed, al- though interim findings on individual cases are provided to the state as they are completed. The rules are generally similar for Food Stamp reviews. Federal Re-Review The federal agency, through its regional offices, reviews and approves state plans specifying acceptable procedures for eligi- bility and benefit determinations and state sampling plans and procedures for quality control. The federal agency, as noted, also re-reviews a sample of state quality control cases and conducts management assessments of state quality control systems. The federal re-review consists of selection of a subsample of cases for review each month, analysis of the state quality control review file and case record, interviews and collateral contacts as necessary, redetermination of eligibility and recalculation of the correct benefit amount, and sending a "difference" letter to the state if state and federal findings vary. For AFDC cases, federal reviewers may make extensive use of collateral contacts by mail and telephone; home visits are the exception. Except for Guam, Puerto Rico, and the Virgin Islands, negative AFDC case actions are not re-reviewed at the federal level, and negative Food Stamp cases are rarely reviewed. Error Rates and Sanctions Both case error rates and payment error rates are reported by the states. The official error rate, which is used to estimate financial penalties, is produced using a regression methodology

OCR for page 127
38 QUALITY IN STUDENT FINANCIAL AID PROGRAMS that combines federal and state findings on payment error rates. The regres- sion procedure in theory allows the smaller subsample to borrow strength from the larger sample. Where there is a difference in findings, the federal ^. .. . ~ nolng prevails. Different resolution and appeals procedures apply to each program. With recent legislated changes that somewhat relax error tolerances, appeal pro- cedures have been simplified and limited somewhat. Beginning with fiscal 1981, and through the time of the NRC study, states were mandated to reduce their payment error rates (overpayments and payments to ineligibles) over three years, reaching 4 percent by fiscal 1983 and 3 percent by fiscal 1984. A number of "good causes" such as natural disasters, significant caseload growth, and other conditions that are beyond a state's control- may be used to waive some or all sanctions. In AFDC, overpayments must be recovered promptly no later than the quarter following the quarter in which the error was found. States are urged to give priority to overpayment cases involving current rather than former recipients. Overpayments are collected from current recipients by reducing the amount of future aid to specified levels. States are expected to take appropriate action under state law against the income and resources of former recipients, unless the state can show that recovery will cost more than the overpayment itself. States must attempt to correct all cases involving re- cipient fraud. For Food Stamps, state agencies are also required to restore promptly benefits that were improperly denied, terminated, or underissued. The quality control system is also expected to refer cases in error to the appropriate state counterpart for action and follow-up, including referring cases involving fraud or legal liability to special administrative units for action. Historically, states were required to file annual corrective action plans (and six-month interim reports) in which they identified major concentra- tions of errors (by type, including distinguishing between agency and client errors) and their basic causes. Subsequent to the NRC studies and a round of legislated changes that principally changed error rate tolerances, the re- quirements for corrective action plans have become less stringent. Performance Over Time and System Reform For fiscal 1980 through 1984, the measured national payment error rate declined for all three programs from 7.8 to 6.0 for AFDC, from 5.1 to 2.7 for Medicaid, and from 9.5 to 8.6 for Food Stamps (although the decline was not consistent over the five years) (Kramer, 19881. The national trends, however, mask a variety of different rates among the states due to a variety of different influences on state performance. (The state rates sometimes mask extreme variation in performance across localities and local offices

OCR for page 127
/ QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 139 with differing administrative designs and differing capabilities). No state showed consistent improvement in all three programs, and only six states showed steady improvement in two programs. By 1985, when Congress mandated studies by the NRC and the administration, the states' total finan- cial liabilities were estimated to be at least $3 billion, every state but one had been found in error in one or more of the programs, almost all states had filed administrative appeals, and many had already joined in lawsuits against the federal government. Criticism of the program, by the states (in the lawsuits) and by the NRC panel, focused on the measurement and estimation systems, including the two-tiered review structure described above, the rationale for the specified thresholds, the imbalance between rewards and punishments, and the lack of incentives to meet all program objectives (as would be created, for in- stance, by incentives to reduce underpayments or improper denials as well as overpayments and payments to ineligibles). It also included substantial debate over the proper definition of error. As with the student financial aid programs, many so-called technical errors procedural errors, such as fail- ure to provide a Social Security Number, that may or may not represent true eligibility errors are included in the measurement of error and the assess- ment of penalties. The NRC panel recommended eliminating the two-tiered review struc- ture, changing error tolerances to reflect performance, and remedying the imbalance between incentives and disincentives to meet all program objec- tives. It also recommended giving states the tools-both resources and better technical designs (such as sample designs) to improve the overall ability of the systems to identify the sources of error and assist in corrective actions (Affholter and Kramer, 1987; Kramer, 1988~. Error thresholds and the components of payment error used to generate penalties were reviewed by Congress following the mandated NRC and administration studies, and several important changes were legislated. Be- ginning in 1991, states with AFDC error rates above 4 percent became subject to sanctions based on a sliding scale reflecting the degree to which their error rates exceed the national average. Error rates are then partially adjusted on the basis of the state's rate of underpayments, child support collections, and recoupment of overpayments. There are no financial re- wards or incentives in AFDC or Medicaid. However, all sanctions prior to 1991 were waived, and limited grace periods are to be provided after the new regulations are imposed. The new laws for the Food Stamp (discussed below) and AFDC programs mandate studies to determine the feasibility of measuring improper terminations and denials for purposes of sanction. For Food Stamps, as a result of the Hunger Prevention Act of 1988, states are liable for penalties beginning with fiscal 1986 if their total dollar error rate (using the point estimate of overpayments and underpayments

OCR for page 127
140 QUALITY IN STUDENT FINANCIAL AID PROGRAMS combined) is more than one percentage point above the weighted (by caseload size in each state) national average for the year (or the weighted national average of any previous year, if lower). Beginning with fiscal 1988, states can receive increased federal funding for administration (up to a maximum of 60 percent depending on their rates of terminations and denials) if their error rates are below 5.9 percent. As with AFDC, there are limited grace periods granted to allow states to adjust to new regulations before penalties are imposed. But, states are liable for interest on unpaid claims 30 days after an administrative appeal is completed, and rulings on good cause are not subject to legal appeal. Alternative Incentive Systems Much has been made of the use and effectiveness of stringent penalties to achieve desired performance. The Department of Education previously commissioned a study of the quality control systems in other federal benefit programs, including family assistance programs, in order to develop some alternative incentive and disincentive structures that could be considered for the student financial aid programs (Advanced Technology, 1987~. Some of the conclusions of that report were based on the experiences of a pilot project in which selected educational institutions assessed their manage- ment controls, estimated error rates in a subsample of awards, and devel- oped corrective actions in response to the level and sources of error found. For the pilot project, relief from verification requirements was used as an incentive for participation in the project. (The pilot project is discussed in detail in Chapter 8.) The Advanced Technology study suggests a set of four optimal charac- teristics for a system of incentives and disincentives: (1) reward and punish only the party responsible for the action; (2) rewards and penalties should be progressive and continuous (i.e., minor penalties for minor infractions); (3) rewards and penalties should be symmetrical (equal number of incen- tives as disincentives); and (4) incentives and disincentives should be fea- sible to implement and effective in achieving desired results. Item four addresses a central conclusion of the NRC panel study (Kramer, 1988~. The NRC study found that the error tolerances were not based on capability; they did not reflect system-imposed common causes of variation. Hence, the financial penalties were insufficient incentives for the programs to meet desired levels of performance. However, the Advanced Technology study suggests a strong bias against the use of sanctions, which is inconsistent with the NRC findings. The central point of the NRC finding is that tolerances, upon which sanctions are based, must be set in relation to capability or neces- sary resources must be provided to improve capability. When either of these conditions is met, sanctions should prove to be effective.

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 141 The Advanced Technology study also generated a set of five criteria of feasibility, against which the researchers assessed 32 alternative incentive and disincentive strategies. To be feasible, alternatives had to have low budgetary impact, be consistent with program intent, require minimal op- erational difficulties or legislative change, have a high cost-benefit ratio, and pose minimal political problems. The 32 alternative strategies extended through each step in the delivery process: institutional eligibility determi- nation, allocation of funds, disbursement, application, eligibility and benefit determination, benefit disbursement, verification, beneficiary account rec- onciliation, quality control, audit, evaluation of institution, corrective action planning, and agency review of corrective action implementation. It is difficult to comment on the study's assessment of each of the 32 alternatives because the empirical basis for the study's conclusions is not made clear. The panel's comments on the conclusions are thus also founded on little other than commonsensical reactions except where the earlier NRC study or other analyses can make specific contributions. In the following description the panel aggregates the study's findings into three classes of approaches and comments on them as appropriate. Tying institutional participation to performance. The study assessed strategies that would either wholly exclude institutions based on perfor- mance or adjust allocations based on performance. Few of the strategies stand the study's tests for political or operational feasibility because, in general, they threaten participation of individuals to whom the programs are aimed. Yet, such action would be appropriate for institutions that are clearly engaging in fraudulent behavior. One strategy reallocation of funds based on performance met the tests, according to the researchers, of program intent and political acceptability, but it is difficult to see why denying some institutions reallocated funds would threaten desired individual participa- tion less than other approaches reviewed. Tying financial and oversight controls to institutional performance. The study assessed tightening or lessening regulatory and other oversight controls on institutions according to performance. The options included were graduating disbursement authority according to performance, altering required quality control sample size, allowing alternative documentation for verification, altering the percentage of beneficiaries that must be verified, reducing regulatory and audit requirements, allowing reduced quality con- trol sample sizes along with diminished appeals rights, altering administra- tive allowances or allowing bonuses related to performance, allowing fund transfers across programs based on performance, and allowing flexibility in corrective action planning requirements, including waiver of sanctions tied to performance.

OCR for page 127
42 QUALITY IN STUDENT FINANCIAL AID PROGRAMS Most of these strategies were viewed favorably by the study. Many, however, create perverse consequences that might not enhance the ability of the institutions to improve performance. First, as the earlier NRC panel found, tampering with administrative funds as a carrot or stick to influence performance can be tricky if such strategies result in reducing funds to poorly performing institutions just when those funds are needed for desired corrective action. Also, using restrictions on or rewards for certain correc- tive action planning is useful in principle but often difficult to achieve. As the earlier NRC panel found and as the Advanced Technology study is also sensitive to, mandating details of program administration from above fre- quently creates inappropriate rote responses from the administering organi- zations. And, review of corrective plans by those unfamiliar with the de- tails of the administering organizations creates other problems. Tightening reporting requirements of applicants and strengthening the financial consequences of misreporting. Current law establishes a $10,000 fine and/or imprisonment for misreporting by applicants for student finan- cial aid. In general, the study views this disincentive as unenforceable and does not consider increasing reporting requirements universally or other- wise increasing beneficiary responsibility as a means of reducing benefi- ciary error. Imposing substantial and immediate penalties on the applicant for clearly willful misreporting would, however, be consistent with the fam- ily assistance programs and would, it would seem, be consistent with public fiduciary responsibilities. Rather, the strategies considered in the study include various means for restricting future eligibility or benefit amounts, or increasing reporting requirements after beneficiary errors occur, and con- versely, reducing future reporting requirements if the beneficiary establishes a history of correct reporting. Further, they would correct errors on future benefit amounts but not correct overpayments made to individuals who have graduated from or left the institution. Of 12 suggested approaches for increasing the obligations of the appli- cant, the study finds 4 that pose operational or budgetary problems. These include recovering institution-caused overpayments from the institutions rather than the beneficiary and assessing a premium to be paid to the beneficiary by the institution for underawards due to institutional error. The study also assesses the utility of collecting overawards against tax refunds or the gar- nisheeing of wages. The study deemed both budgetarily and operationally difficult. Both would also affect only the subset of students with tax re- funds or wages against which to collect. (Since the Advanced Technology study, legislation has been enacted that mandates the IRS to use tax refunds to offset defaulted student loans. Also, Department of Education staff re- port that the department is conducting a pilot activity on garnisheeing of wages.)

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS PRIVATE SECTOR MONETARY DISTRIBUTION SYSTEMS 143 Private systems that involve the distribution of money differ from fed- eral programs in many ways, most notably in profit motives and often by having an asset upon which the transaction is based. There are, nonethe- less, important comparisons that can be considered in assessing governmen- tal performance. For example, the assistant comptroller general of the Gen- eral Accounting Office (Thompson, 1991a) stressed that the quality of governmental services would benefit from program management that keeps in mind four principles found, in general, among quality leaders in the private sector: management continuity and consistency, long-range planning and visions for the future, orientation toward serving customers, and systematic strategies for measuring performance. In its inquiries the panel was repeatedly told that these principles were not characteristic of the Office of Student Financial Aid's activities related to award errors, nor its general activities. Without implying that any of these principles are more important than the others, it is the last principle on which we focus attention here. Many monetary distribution systems in the private sector have pro- cesses and quality problems similar to those found in the student financial aid system. For example, credit card lenders, mortgage bankers, auto loan providers, and insurance companies also face the problem of incorrect data being supplied by applicants and have difficulty maintaining and improving quality in a distributed network of service centers. As for student financial aid, these organizations require an application and have eligibility rules that the applicant must meet to qualify for the service (e.g., loan, policy, credit line). These systems also experience a nesting of errors within service centers and individual applicants. The panel called on a number of quality practitioners to ask about good practices in their service industries. Although no one organization provides a model for immediate use in Department of Education applications, the panel believes that two major concepts used in quality service organizations are instructive. First, organi- zations create a quality group that reports directly to senior management and is involved in decision making, especially in recommending and ap- proving actions needed to remove defects or procedural weaknesses. The second concept is targeting risk. The amount of money at risk of being lost and the propensity of an individual or service center toward "error" are the considerations. While instances of fraud were reportedly under 1 percent in, for example, the mortgage industry, the applicant is subject to intensive review prior to obtaining funds. Still, dollar losses can

OCR for page 127
44 QUALITY IN STUDENT FINANCIAL AID PROGRAMS be considerable. Thus, while the service organization must approve appli- cants to generate income, the applicant must meet certain guidelines for a profit to be made. The guidelines will vary by industry, amount of money provided, and the timing of the payment. For mortgages, credit history and employment are verified, and the value of the real property is subject to an independent appraisal. Risk considerations are used to structure the trans- action to increase the expected return on the loaned funds. For example, the portion of the appraised value that is approved for a mortgage varies, thus providing a default deterrent and a cushion against losses if a default occurs. Higher risk clients are often charged higher interest rates, which are set to provide the desired return on investment within a class of loans. Credit card companies rate credit histories to develop classes of indi- viduals to which they offer their services. Interest rates and credit limits can vary to provide expected profits within the groups. Insurance compa- nies have sources of information for verifying key application information, for example, motor vehicle department records of accidents and violations and consumer-reporting agencies to investigate other items. For large poli- cies, a health examination, or an inspection of the property, may be re- quired. Still, the insurance company is at risk only when paying a claim, and it can concentrate its inspection efforts effectively at that time. Many of the service industries the panel reviewed place the responsibil- ity for proper documentation on the local service provider. Often, this provider is an independent contractor, whose profits are based on volume. Thus, the industry also focuses on errors nested in the service center. Mort- gage-purchasing organizations have well-defined procedures for measuring the quality of underwriting as a whole and by individual servicers. The goals are to not only identify problem servicers, but also provide feedback and work toward problem resolution. Where training and such feedback do not achieve the mortgage purchaser's standards for quality, the servicer can be asked to repurchase problem loans and, in extreme cases, removed as a service provider. In addition to having such compliance monitoring respon- sibilities, the central organizations show concern for their own quality by using surveys to measure the satisfaction of the servicers as their own cus tomers. LESSONS FOR STUDENT FINANCIAL AID PROGRAMS In comparing any quality control strategy with that used in student financial aid programs, at least three issues should be explored. First, how well are the programs doing compared with other programs with similar missions? In fact, the comparative magnitude of error in programs differs markedly, but programs view errors differently in terms of who they serve and what they count. Thus, simple comparisons across programs may prove

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 145 problematic implicit as well as explicit tolerances of error must be under- stood. Second, what practices of quality control and quality monitoring are worth borrowing from other systems? Many systems have well-developed verification schemes, and some attributes may be applicable to the student financial aid system. But, verification and monitoring schemes are often very program specific; thus, simple transferability may also be problematic. Third, what incentives and disincentives are most effective in improving performance? Do stringent penalties work? Are there other incentives that also work to improve performance? The structure of incentives and the imposition of explicit penalties and rewards create important consequences for managing the institutions and for the ultimate consumers of their ser- vices. Those practices must be understood in order to keep them consistent with overall program objectives. To illustrate the first issue, comparing error levels, the definition of error itself creates wide variation in how the magnitude of the problem is perceived. For example, in the family assistance programs and the student financial aid programs, technical (procedural) error is intermingled with actual payment error. The earlier NRC panel that studied the family assis- tance programs argued, in effect, that the definition of error for the purpose of assessing financial accountability ought to be based on those measures that are direct surrogates for payment error. It reasoned that certain proce- dural requirements may provide a form of internal control that may be important in ensuring payment accuracy. Thus, certain procedural require- ments, such as the provision of a Social Security Number for certain mem- bers of the household, may give agencies a useful tool for checking for unreported income. But, extending that requirement to all members of the household, including infants and children, enlarges the magnitude of pay- ment error, due to a procedural requirement that only partially reflects true payment error. To further illustrate the first issue, the universe from which errors are measured may be markedly different in different programs or institutions. Some institutions limit their risk by limiting risky participants. Others may simply ignore areas of potential error if bottom-line profitability is not affected by the errors. Many commercial insurance carriers simply refuse to underwrite certain classes of applicants rather than make individual de- terminations of risk to guide insurance offerings. Banks have histories of so-called redlining of geographic areas or real estate markets in order to avoid altogether areas of potentially high risk. Mortgage-purchasing asso- ciations stop buying loans from servicers who do not meet basic standards. The family assistance programs, and to a large extent the student finan- cial aid programs, do not have such options. The family assistance pro- grams must require that administering agencies be scrupulous in attempting to discover all sources of income and assets in order to ensure that the

OCR for page 127
46 QUALITY IN STUDENT FINANCIAL AID PROGRAMS deserving poor are served and all recipients are deserving. They also sub- ject administering agencies to perhaps the most stringent error tolerance levels of any public loan or grant program. However, many errors stem from the number of households with earned income and the volatility of the caseload as recipients move in and out of work and on and off the rolls. Both outright eligibility and benefit amounts can change month to month as income and resources change. In the AFDC and Food Stamp programs, more errors are associated with households with working recipients than with other households. The single largest type of error in the Food Stamp program is that associated with income and with appropriate deductions from income. In the Medicaid program, a large proportion of resource errors are associated with elderly participants (provider error is measured elsewhere). The earlier NRC panel attempted to make adjustments for dif- ferences in capability among the states caused by the proportion of their caseload with error-prone characteristics. The panel recommended setting performance thresholds for groups that are differently prone to error (e.g., households with reported earnings, nonearning households with elderly re- cipients, and all other households) and adjusting thresholds applied to the states to reflect those differences (Kramer, 1988~. With regard to the second issue, quality control practices, there are surely some quality control and monitoring techniques that can be borrowed by student financial aid programs. Techniques such as simplifying applica- tion forms and improving the training of frontline workers are generic ap- proaches to process improvements that are useful tools to address problems in student financial aid programs. Newsletters, workshops, and other tools to disseminate best practices have been used in the family assistance programs. Different state and local welfare systems, however, have required dif- ferent remedial approaches. Few techniques in the family assistance pro- grams are used across all states or all local offices. Approaches that are appropriate to the context must be identified through careful research to determine common sources of error and effective techniques to control them. Other techniques for improving performance, such as an aggressive federal role in the transfer of technology across states, have also been used in the family assistance programs, but they were viewed with uneven enthusiasm by state quality control administrators. Perhaps most important in comparison with the student financial aid programs is the emphasis in the family assistance programs on front-end verification. More responsibility seems to be placed directly on the appli- cant for family assistance than on the applicant for student aid. Although the administering agencies do use independent verification techniques (e.g., matching data with IRS and UIB data), a variety of documentation intended as surrogates for truth of eligibility status is required of all welfare appli- cants at initial presentation. In the student financial aid programs, such

OCR for page 127
QUALITY CONTROL IN OTHER MONETARY DISTRIBUTION SYSTEMS 147 documentation may be a weaker proxy and is only required of applicants selected for verification. (In Chapters 8 and 9, the panel addresses the lack of such extensive and up-front verification data in student financial aid programs.) The agencies administering family assistance are responsible for ensur- ing compliance with application requirements. The quality control reviewer then undertakes-on only a sample of cases a full field investigation. The cost of administering rigorous front-end verification and other improved application procedures is important but difficult to assess. Some costs are clearly trivial, others are not. In the case of family assistance, the federal government has been willing to pay half of the costs of administering the quality control systems through its usual 50 percent share of all administra- tive costs, and even more for a variety of other improvements, such as automated data processing and fraud investigation and prosecution, in order to encourage the states to upgrade their monitoring capabilities. One inexpensive and useful technique that the panel believes is worth exploring is matching application data with the data collected by other agencies. While use of credit bureau and other such commercial data bases may not be appropriate in student financial aid programs, except for some loons f',rth~.r Am. Of m~tr.hP.~ with f~rl~.rn1 flats I. seems worth atten , _ an.. , ~^ .. _, lion. Student financial aid programs do some matching, but they do not match with IRS records. Such matches are done by the family assistance programs and by several Department of Veterans Affairs need-based pro- grams. The cost is minimal, about one cent per record for the data, and the data supplied are regarded as highly accurate by the users. Differences between reported data and the official record are often ranked by priority and assigned to field staff for investigation. (Note that such matching, while found to be accomplished with minimal procedural problems, would likely require a legislated mandate for the IRS to provide the data to the agency.) With respect to the third issue, use of incentives and disincentives, consequences other than payment error are born of the relationship between basic program objectives and verification requirements meant to serve qual- ity control, and they must be considered in any assessment of rewards and penalties built into a quality control system. For example, although many program requirements have been legislated over the years to ensure partici- pation by those who are eligible for family assistance, including explicit outreach requirements in the Food Stamp program, the intended combined effect of eligibility determination procedures and quality control monitoring is to limit participation. Until recent changes, described above, the quality control systems did not count underpayments in the penalty system at all. Improper terminations and denials of benefits are still not counted. As the earlier NRC panel found, when overall program objectives and quality con

OCR for page 127
48 QUALITY Ill; STUDENT FINANCIAL AID PROGRAMS trot requirements are in strong conflict, improving performance is particu- larly problematic. The penalty structure, the earlier NRC panel concluded, distorted incentives to serve other program objectives for example, to serve all who are eligible. Further, the emphasis in the quality control systems in the family assis- tance programs is on sanctions for poor performance. Until recently, the programs had minimal, or no, explicit rewards for good performance. A1- though few, including the earlier NRC panel, have argued against sanctions, and error rates have declined substantially over the history of the quality control systems, the stringent (financial) penalties have had varying effects, depending on their severity and on the resources and capabilities of the states to improve their performance. At the time of the NRC studies, it was widely held that the states had done all they could to improve performance with the resources they had available. The states clearly thought they had nothing else to do but to protest. At the point that the earlier NRC panel was asked to review the quality control systems, those systems had become the source of a fierce federal-state battle, and few penalties had ever been collected because most states had appealed either administratively or in the courts. Although the earlier NRC panel did not recommend against sanctions, it stressed that for sanctions to be effective, error tolerances must be tied to program capabilities or sufficient resources must be provided to help states improve their capabilities to desired tolerance levels. The NRC panel rec- ommended a simplified review structure and a much more extensive quality improvement program to improve capability. In its view, to improve pro- gram performance further, the federal government and the states would have to spend at least as much on quality improvement strategies as they had spent in preceding years on quality control. We do not make specific recommendations based on the comparisons in this chapter. In Chapters 8 and 9, we use the principles discussed here to guide our system improvement recommendations.