Click for next page ( 118


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 117
6 Financial Aid Data Systems The panel has discussed the extensive body of information available to student financial aid program managers from special studies and the edit and verification processes. The Department of Education also asked the panel to comment on the organization of information about award error and the usefulness of such data bases for making program decisions. In a simi- lar investigation, the General Accounting Office (1985) found the department's approach to issues regarding the quality of award determination to be un- systematic. That study found that decisions that might affect improvement were hindered by a lack of goals and analyses that were hurried because of operational demands. The study cited problems in coordinating the activi- ties of offices within the department that needed to work together and la- beled the resultant reactive decision making "remedial" and "not preven- tive." The panel, after discussions with Department of Education personnel and review of earlier studies, also found no particularly well-organized use of the data that focused on improving quality of award decisions. It is evident that operational demands continue to limit proactive thinking. While attempts are being made to bring the parts of the department together to work on cross-organizational issues, comments by departmental staff to the panel indicate that there is no strong management awareness of the data resulting from the quality control activities described in Chapter 4, nor is there direction in cross-functional efforts. Yet, there exist formally organized data sets related to some student financial aid programs. In this chapter, the panel reviews the data system 117

OCR for page 117
118 QUALITY IN STUDENT FINANCIAL AID PROGRAMS for entering applicant data-the Central Processing System and that used with the loan programs. Department of Education personnel reported to the panel that there is no applicant data base for Campus-Based programs- they know how many get aid, how much money goes out, and forgiven loan amounts, among other things, but not who receives it or items useful for quality improvement analyses. THE MANAGEMENT INFORMATION SYSTEM The Management Information System (MIS), the main tool used by the Department of Education to monitor and control the Title IV application processing system, draws data from virtually all aspects of the Central Pro cessing System (CPS). The MIS is part of a coordinated set of management controls, called Project Control Information Services; the other parts are quality control, configuration management, and process control. (The CPS and the MIS are operated for the Department of Education by a contractor.) The MIS enables the Department of Education to monitor data, (e.g., number of applicants, eligibility rates, and turnaround times) generated dur- ing production processes. Massive amounts of data are condensed into coherent tabulations and presented in four groups, which are aimed at vari- ous management levels (National Computer Systems, 1990-91a and 1990- 91b). The four groups of data are as follows: highly summarized information to give top management a snapshot of key CPS operational and program areas; operating management data (e.g., summary information for monitor- ing funding level and detailed contract, quality control, and project infor- mation) to allow monitoring of day-to-day operations and the performance of data entry contractors; detailed statistics for monitoring the effect of applicant characteris- tics and behavior on program costs, compliance with regulations, the effect of system changes on applicant population characteristics, and quality con- trol by contractors; and detailed information for monitoring the receipt, handling, and mail- ing of applicant documents. The contractor submits printouts of these routine MIS reports to the department at appropriate intervals, such as weekly or biweekly, during the award cycle. Some data from previous cycles are included to permit com- parisons. A few process-monitoring tools, such as control charts, are in- cluded in the reports on an ongoing basis, but their use seems limited, identifying occasional shocks to the system from causes such as peaks in work flow of applications. Several staff members have used the MIS to draw samples for additional inspection in areas exhibiting some unusual characteristics in the periodic MIS reports. Special reports are produced on

OCR for page 117
FINANCIAL AID DATA SYSTEMS 119 request, usually through a task order to a contractor. At the end of the award year, the contractor prepares a final report. The MIS reports are also available to Department of Education staff on- line. The information can be viewed on terminal and workstation screens, printed via menu selection, and downloaded in ASCII format and translated to other formats (e.g., a worksheet or data base file), manipulated, analyzed, or graphed. Some of the department's information needs can only be met through use of the entire applicant data base. However, many of the tabulations do not require an exact count; an approximation with a sufficiently small confi- dence interval provides adequate information. Such tables are produced from a Sample Data Base of approximately 50,000 cases drawn several times during each processing cycle. The department also has information needs that require statistical analysis and programming involving numerous data transformations and complex statistical procedures. The Sample Data Base can be used in such analyses with a high degree of precision and at a reasonable cost. Access to the Sample Data Base is provided through the on-line terminals and personal computer workstations at the department, which enables staff to run tabula- tions and perform statistical analyses. The Statistical Analysis System is used as the major data base management and reporting tool for the Sample Data Base. The panel commends the Department of Education and the CPS con- tractor for what appears to be a thorough development of an MIS data base that presents operational data targeted appropriately to the functions of vari- ous management levels. Improvements have also been made in recent years in the presentation of the data. The panel is concerned, however, that the data base is underutilized. While appropriate data seem to be captured with the system (almost all the data are captured), and the system recognizes the need to target data to the differing needs at various levels of management, the reports produced tend to be dollar and timing oriented and not focused on improvement of the system by identifying potential causes of error in application and award processes. There are seemingly unbounded opportu- nities to mine the data set to find ideas for process experimentation and the potential to improve by finding and removing true causes of error. Use of any data base requires some degree of expertise with computers and knowledge of statistical methodology. Use of a sample data base and a high-level statistical package as a tool for access and analysis raises the level of expertise needed. More fundamental, use requires hardware, the lack of which the panel heard mentioned by Department of Education staff. Further, the utility of the Sample Data Base for special projects is not fully appreciated. The panel, when discussing this data base with departmental

OCR for page 117
120 QUALITY IN STUDENT FINANCIAL AID PROGRAMS personnel often found a lack of appreciation for the usefulness of such a data base. There was a tendency for departmental personnel to be con- cerned that 50,000 records in the sample data base were too few of the over seven million applications, without appreciating the degree of precision that such a large sample can have for most purposes. It appears that much of the analysis of MIS data is done through con- tracts. While contracting for such analyses in itself may not be bad, a data- driven organization should encourage the development of expertise within its own staff. Several contractors commented to the panel that the depart- ment does not always encourage their innovative suggestions and that some lack of communication within the department hindered their efforts to pro- vide effective analysis. One contractor did comment that there has been a recent move toward encouraging its input, and that its contract now requires that it allocate time to consider improvements. When technical expertise, either statistical or computer, is obtained from outside sources, the depart- ment should consider the technical advisors as partners in the process and encourage unsolicited suggestions for creative analyses. The MIS has the potential to provide appropriate information to the appropriate levels of management. While the current implementation has noticeably improved over the past few years, users of the system seem hampered by a lack of equipment, methodological knowledge, and time. Similar problems were observed in other areas of the department's financial aid programs. (The panel makes a recommendation addressing these prob- lems in Chapter 9.) STUDENT LOAN DATA The panel reviewed several studies of student loan programs that com- mented on the data systems needed to monitor the loan programs, and it spoke with Department of Education personnel about those systems. In one study, the General Accounting Office (199Ob:3) describes the loan data sys- tem this way: As part of its monitoring of the program, the Department each year collects Stafford loan information from guaranty agencies and consolidates this information into the Stafford loan data base, commonly referred to as the tape dump. This data base is maintained by a contractor for approximately $900,000 a year and is primarily for the internal use of the Department. The contractor edits the data base's information to check whether guaranty agencies are reporting complete data. For fiscal year 1988, the Department required guaranty agencies to report 95-percent-complete data for 10 data fields the Department considered critical, such as students' social security numbers and loan amounts guaranteed. However, these edit checks can only determine completeness, not accuracy.

OCR for page 117
FINANCIAL AID DATA SYSTEMS 121 The department primarily uses information in the loan data base for routine trend analysis (e.g., the number of students obtaining loans). These analyses can be found in the Annual Guaranteed Student Loan Data [Book, which is prepared by a contractor. Departmental staff reported to the panel that at least some individuals within the department were dissatisfied with the quality of the data, lack of some data items of interest, and the lack of analysis with the existing data. PROBLEMS WITH THE DATA SYSTEM The General Accounting Office (1990bJ and various officials at the Department of Education (e.g., Thompson, l991b; Winkler, 1991) have written detailed critiques of several of the problems in the operation of guaranteed student loan (GSL) programs. Briefly, the findings indicate the following: Financial management capabilities and maintenance of accounting records are poor. Computer systems are inadequate. Data elements in the systems are either old, unreliable, or not collected at all. The systems are not easily adaptable and require major reprogramming to provide information when policy, regulatory, or financial analyses raise complicated questions. Data and analyses are inadequate to (a) monitor guaranty agencies and lenders, (b) predict more accurately default and interest costs, (c) im- prove collections, (d) forecast volume, (e) outline borrower characteristics, (f) analyze proposed legislative and regulatory changes, and (g) evaluate program effectiveness. Guaranty agencies are not allowed access to the loan data base, a procedure that would help prevent making a loan to someone who is in default or in excess of loan limits, and thus ineligible. To remedy these problems, the General Accounting Office (199Ob) rec- ommended that the Secretary of Education: direct guaranty agencies to ensure that their student loan data are as accurate and complete as practical, with special emphasis given to data on defaulted borrowers, and provide updated information to the department. report the ability of ineligible borrowers to obtain loans as a material internal control weakness under the Federal Managers' Financial Integrity Act. analyze the data the department now has and provide to guaranty agencies on a periodic basis the data needed to prevent defaulters from obtaining new loans. develop a more comprehensive annual report that gives guaranty agencies the data they need to more efficiently detect loan limit abuses, and require guaranty agencies to use the report.

OCR for page 117
22 QUALITY IN STUDENT FINANCIAL AID PROGRAMS A PROPOSED SOLUTION The 1986 amendments to the Higher Education Act authorized the De- partment of Education to develop a national student loan data system (NSLDS), a computer system that would make national student loan data accessible to guaranty agencies. The purposes of the system include improving data on The system default rates, reducing defaults, and reducing overpayments. could be used to assist lenders and guaranty agencies in guarding against borrower abuse. The General Accounting Office (199Ob:3) notes, however, that "because the 1986 amendments contained a provision that the Depart- ment could not require guaranty agencies to use the system before guaran- teeing new loans, the system was not developed. The 1989 Budget Recon- ciliation Act, however, allows the Department to require guaranty agencies to use such a system before approving new loans. The Department plans to develop the new system by 1993; the guaranty agencies will have to use this system before guaranteeing new loans. was awarded in early 1993. Winkler (1991) notes that data content, appli- cations, and linkages with other GSL-related systems must be defined for an NSLDS, and the prospective cost of the system must be scrutinized care- fully. Thompson (199lb) suggests that the Department of Education should expedite the development of the NSLDS in order to protect the integrity of the program more effectively. He notes that the system is crucial in provid- ing departmental officials with part of the information they need to prevent student-borrowers from abuses, such as exceeding statutory loan limits and receiving additional loans when they are already in default. The need for an NSLDS, as included in the 1986 amendments to the Higher Education Act and as reemphasized in the 1992 reauthorization of the act, will be even greater in the future. While the direct lending demon ~ " A contract to develop the system stration project authorized in the act would affect the loan distribution sys- tem, there will still be difficulties with students' address changes that have to be kept up-to-date as they leave school, enter the work force, reenter schools, and/or change schools. Generally, young people at this point in life are difficult to track. In addition, the vast increases in amounts autho- rized under the Parent Loans for Undergraduate Students program will re- quire very careful attention if default rates are to be kept in check. Parents who borrow $25,000 or more and are not able to pay it back may go into default as much as a year before the current tracking system could identify that default and deny another $25,000 loan. Recommendation 6-1: The panel believes that a totally integrated ap- plicant data base is essential for problem identification, the develop- ment of solutions, and monitoring progress important activities if true quality improvements are to take place. The data base would compile

OCR for page 117
FINANCIAL AID DATA SYSTEMS 123 information from all the Title IV aid programs and across years to provide, for each applicant, a complete history of participation in the financial aid system, thus facilitating a broad range of cross-sectional and longitudinal statistical analyses. The Department of Education should allocate sufficient resources and personnel to accomplish this task without further delay.

OCR for page 117