Click for next page ( 198


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 197
APPENDIX CApplying for Federal Student Financial Aid for ~ 992 93 Margaret Weidenhamer Consultant Students who wish to receive federal student financial aid must com- plete either the Application for Federal Student Aid (AFSA) produced by the U.S. Department of Education or one of the forms developed for states or geographic regions by multiple data entry (MDE) processors. For the 1992-93 school year, applications supplied by MDEs were required to in- corporate verbatim all the instructions and data items from the federal ver- sion; they were permitted to append additional items needed to administer financial aid programs, including nonfederal programs, at the schools they served. For 1992-93, the AFSA was the appropriate form for the following federal programs: Pell Grants Stafford Loans Supplemental Educational Opportunity Grants College Work-Study Perkins Loans THE 1992-93 AFSA: PAPER VERSION The paper version of the AFSA for the 1992-93 school year became available to applicants late in 1991. English and Spanish versions were printed. An electronic version, in English, was available at some school 197

OCR for page 197
98 QUALITY IN STUDENT FINANCIAL AID PROGRAMS locations. The electronic version, known as Stage Zero, is discussed later in this paper. The English AFSA for 1992-93 consists of a 4-page form and preaddressed mailing envelope stapled in the middle of a 12-page booklet. The AFSA booklet, as distinguished from the application form itself, consists mainly of instructions, definitions, and work sheets needed to fill out the applica- tion form. A limited amount of information on the five programs is pro- vided, some on the first page and a little on pages 10 and 11. The following information appears on page 11: This booklet gives you only a brief summary of the student financial aid programs offered by the U.S. Department of Education. Each student financial aid program has its own special features and procedures. You can get more information from the booklet: The Student Guide: Financial Aid from the U.S. Department of Education Grants, Loans, and Work Study 1992-93. To get a free copy, write to.... The 1992-93 Student Guide (U.S. Department of Education, 1992-93) is a 60-page booklet that describes the five programs listed above, as well as Supplemental Loans for Students and Parental Loans for Undergraduate Students. It discusses key topics, such as eligibility requirements and bor- rower responsibilities and rights, and provides Federal Student Aid Infor- mation Center telephone numbers. The telephone numbers are not given or referred to on the AFSA, however, where the need is probably much greater. On the other hand, the student guide does not provide definitions of dislo- cated worker and displaced homemaker, special circumstances that it states can trigger some flexibility in eligibility requirements. Instead, it directs readers to the financial aid application for those definitions. The student guide also gives advice on finding out about nonfederal sources of student aid. A cursory reading of the 1992-93 AFSA when it first became available revealed some quirks one would not expect in such a document. For ex- ample, the first question on the form asks for last name, first name, and M.I.; those who don't know what M.I. stands for find no explanation any- where in the booklet. And the Jacquelines, Christophers, Marguerites, and other applicants whose first name contains 10 or more letters also have a decision to make on their own, because they are not told how to squeeze their name into the 9 spaces allotted on the form. Only one copy of the form is included in the AFSA booklet. There is no working copy for the applicant to use as a draft and keep as a record for reference if verification or correction of any of the information is required. Yet, the instructions state "Use a pen with black or dark ink; don't use a pencil." A black-and-white photocopy of the form does not adequately reflect the color coding that is used. Many applicants apparently get an

OCR for page 197
APPENDIX C 199 extra copy of the complete application to get another copy of the form, wasting 12 pages to get 4. The application form itself gives the overall impression that a decision was made to stay within a four-page limit without sufficient regard for other form design considerations (e.g., accuracy of item wording and clarity of directions). Perhaps because of a determination to save space, the form also uses a potpourri of formats. For example, the first two items in Section A cover the full width of the page the answer spaces are next to the ques- tions asked. Items 3 through 10 are in triple columns and the answer spaces are below the questions. Section B starts with items [la-d, which span half the page; the answer boxes are to the right, and directions about what to do next are on the far right. Item 12 follows that style, but for item 13, the response categories are below the item, and each response is followed hori- zontally by two more questions and the answer boxes. The directions at the bottom of the first page state that everyone is to answer the items in the white areas of the form. Students whose answers to the preceding items indicate they are "dependent" also answer items in the areas shaded red; those considered "independent" also answer items in the areas shaded gray. The mixture of formats, complicated by the addition of the white, red, and gray areas, continues for the remaining three pages. Further, item numbers are duplicated when identical, or nearly identical, questions are asked of parents and students. These design variations might present challenges for data entry personnel as well as applicants. Panel members were concerned that the current design of the AFSA could cause inadvertent response errors that would complicate the processes of advising applicants, checking the accuracy of responses, and correcting erroneous data. They were also concerned because inadvertent response error, if undetected, can result in award error. The panel therefore requested help from selected federal agencies and other organizations thought to have experience in collecting and processing data similar to the information re- quired for the 1992-93 AFSA. Reasons for including specific data items on the form were not mentioned in the letter. The panel's request was mailed in mid-March of 1992. It stated in part: The panel is especially interested in expert advice concerning the complex- ity.of the form and instructions especially for typical applicants (lower income, high school graduates and their parents). The panel would also like information on response errors measured for similar data. Comments or suggestions based on your organization's experience with the collection and study of similar data would be helpful to determine whether errors made by applicants or data processors might be reduced through revisions in wording, sequence, instructions, format, or other aspects of the applica- tion and related tasks. References to or copies of reports from relevant qualitative studies or statistical research that your organization has con- ducted would be appreciated.

OCR for page 197
200 QUALITY IN STUDENT FINANCIAL AID PROGRAMS None of the replies cited statistical research conducted by the organiza- tions. One reviewer did mention that in "SIPP" (the Survey of Income and Program Participation conducted by the Census Bureau) substantial response error was measured on some similar income/asset items. However, that reviewer mentioned the possibility that, in the AFSA, the warning about penalties for supplying false or misleading information may improve the accuracy of responses. Some reviewers indicated they had not had any recent experience collecting similar data items, and a few declined to com- ment for that reason.) Panel staff and consultants also commented on the AFSA based on their experience and an examination of material provided by the Department of Education, such as Office of Management and Budget (OMB) clearance documents for the 1992-93 AFSA and comments submitted by MDEs in reports to the Department of Education for the two previous years. The opinions of reviewers from 13 organizations are reflected in this discussion. The excerpts listed below indicate the range of responses. The comments merit thoughtful consideration by those with responsibility for the AFSA. Reviewers were generally critical of the AFSA: "The aid application forms and process are much more complex and onerous on applicants than they need be, more onerous than a 1040 tax form. Ironically, most of the students seeking aid come from low-income families which either file no tax return or file a 1040EZ. The student aid application form is probably the most complicated financial form they have ever seen." "We agree that this form and accompanying instructions are exacting and complex due to the myriad of financial requirements which must be documented by law. As a result, student-level applicants and low-income parents are likely to encounter much difficulty in both understanding and completing the AFSA." "I found the form to be difficult to follow. Part of the difficulty is due to the amount and complexity of the information required, but the overly complicated appearance and structure of the form also contribute to confusion." "Looking at the form, I don't think any of the questions are very difficult. But it has the appearance of being overpowering. The first page is cluttered and the rest is red and gray. From the face of it, my guess is that it is confusing." iThe panel and staff are very grateful to those who did respond; their comments and sugges- tions were extremely helpful. Regrettably, it was not feasible to include all their comments in this paper.

OCR for page 197
APPENDIX C 201 "I think the similarity to income tax forms might underline the im- portance of completing the form correctly, but many people are intimidated by tax forms. I am concerned that this general similarity and the overall lack of "white space" make the task appear more formidable than it is." "Needed improvements would require a total redesign of all features mentioned in the letter wording, sequence, instructions, format, and so on." . "The form itself appears manageable for typical applicants.... instructions that accompany the application form are quite difficult and complex. We would expect significant confusion and incorrect responses if the instructions are not modified." The Reviewers gave a wide range of specific comments and suggestions for improving the AFSA. For example, Section B. "Student Status," was singled out by several reviewers as difficult to understand. The crowded format took a lot of the blame. (A brief explanation in the instructions of the overall logic might help somewhat.) Some reviewers thought the sole purpose of this section might be to get the applicant to determine whether he or she should answer the questions in the red-shaded areas or gray- shaded areas. If the information is not used in determining eligibility for aid, they thought the section should become a work sheet in the instruc- t~ons. _ Marital status plays an important part in some key decisions applicants must make about which questions on the application form they are supposed to answer. It figures in the determination of whether the student's status is "dependent" or "independent" in Section B. It also figures in the definition of the term parents which applies in Section C, "Household Information," and the remainder of the questionnaire. The answer categories for marital status differ in each of the items on the form in which it is used; definitions are not provided anywhere. Four versions of marital status categories are found in the 1992-93 AFSA: Marital status of the student, Section A, item 8. I am not married. (I am single, divorced, or widowed.) I am married. I am separated from my spouse. Use of student's marital status in part of the directions that follow questions lla-d in Section B. "Separated" is no longer a category. Unmarried now (single, divorced, separated, or widowed) - Married now In the instructions for "Parents' Information - red areas," the unmar- ried grouping is broken into two pairs. Married to each other - Divorced or separated - Widowed or single

OCR for page 197
202 QUALITY IN STUDENT FINANCIAL AID PROGRAMS In Section C, item 16, the categories for current marital status of "parents," as defined in the instructions for this section, are not grouped. They appear in this order: single separated married divorced widowed The marital status categories, as worded, are not mutually exclusive. "Separated" is not clear: couples may be separated legally or the arrange- ment may be informal, but in either case they are still married. "Divorced" and "widowed" are ambiguous for applicants who have remarried. The lack of precision in terminology and the variety of groupings used are a potential source of confusion that could lead to errors throughout much of the appli- cation. The errors might not be detected in routine edit checks. The instructions for Section B state, "'Parents' in questions 12, 14a- 14f, and 15 means your mother and/or father, or your adoptive parents, or legal guardian. 'Parents' does not mean foster parents, and for this section, it does not mean stepparent. Later the instructions will tell you if you should supply information about your stepparent." Information about stepparents is wanted in Section C, "Household In- formation," under certain circumstances, which are described in the instruc- tions labeled "Parents' Information - red areas." For some applicants, the term "parents," as used in questions in the remainder of the form, is thereby redefined. The use of different definitions in different parts of the form . . may cause misreporting. The labels for questions 18 and 19 in Section C do not correspond to what the instructions say is meant. A reviewer suggested that the following wording would be closer to the intent: (for 18) "How many people will your parents support in 1992-93?" and (for 19) "Of the number in 18, how many will be in college at least half-time in 1992-93?". Section E, "Stafford Loan Information," provides an example of puz- zling instructions. The first paragraph in the explanation on page 6 states: "Answering the questions in this section does not commit you to accept a Stafford loan nor does it guarantee that you are eligible for a Stafford Loan. For most schools you will have to complete additional forms. Check with your financial aid administrator." This statement has no apparent relation- ship to the questions in that section, all of which deal with loans received in the past. The instruction to enter "0" if the applicant has not received a Stafford Loan contradicts the direction given on the questionnaire. The statement about what loans should and should not be included would be easier to follow in a list format. Other aspects of the instructions were also scrutinized. Reviewers cited such concerns as excessive cross-references from one part to another, unclear definitions, duplication of information, and illogical sequencing.

OCR for page 197
APPENDIX C 203 Several reviewers thought the color-coding system currently in use was not helpful to applicants. Suggestions included clarifying the student's task by splitting the application into two forms, one for students who are depen- dents and the other for those considered independent. The determination of which form the applicant should use would be made through screening questions on a separate one-page form. (Split forms were used for two years during the 1980s The panel was told the reasons for abandoning this approach were not documented.) Another idea for simplifying the process was consolidating the items to be filled out by all applicants, by parents, and by independent students, so that all white sections are together, fol- lowed by red areas, then gray areas. Both approaches should reduce confu- sion about which instructions apply and which items should be answered. Another option that might be useful for this complex form is to convert the AFSA instructions into a working copy of the application. The ques- tions and the answer categories would be interspersed with the complete instructions, examples, definitions, and work sheets, if any, needed for each data item. The student would retain the working copy for his or her records and copy the answers to a recording form containing question numbers, brief descriptions (e.g., name, address, title) for each, and answer boxes. The recording form, which would fit easily on four pages or fewer, would be the application the student submitted. More of the reviewers' ideas are summarized in the box on the next page. Many are linked to comments reported elsewhere in the text. It is possible that informal testing or formal research would indicate that some of the "faults" discussed here do not in fact create problems for applicants. It is also possible that testing and research would reveal that aspects of the form that were not criticized by reviewers do cause problems for applicants. The following suggestions for review and testing were offered by the reviewers: A careful review should be conducted to determine minimum data requirements; then extensive questionnaire design work should be under- taken to develop a foe that collects these data in a way that is simplest for applicants to provide. The questionnaire design work should address not only wording of questions but also form layout and general instructions provided. Cognitive psychology techniques could be very useful. It would be desirable to do some small-scale testing to see how well people follow the form, and to identify questions which are not understood or for which people have difficulty providing information. . We believe it would be helpful . . . to convene a focus group or panel of actual AFSA users (students and parents). ...Ethe Department of Educa- tion] would then be better able to identify those structural changes in format and content that are needed to improve the AFSA.

OCR for page 197
204 QUALITY IN STUDENT FINANCIAL AID PROGRAMS Why don't they- list the Federal Student explain how to get IRS supply two copies of Aid Information Center forms rather than the form in each telephone numbers on supplying work sheet booklet the AFSA #1 ``to answer question 24 if you cannot get a list consistency checks combine some AFSA 1991 tax form, but will students can do sections (A and B. for be filing one" themselves example) replace "big" words use the same number every question (such as self- terminology and explanatory, optional, definitions the I RS uses define terms, such as pertaining, wherever possible spouse, rollover, determination) exemption tell if AFSA definitions explain all are not the same as refer to IRS forms abbreviations, IRS definitions in items precisely (e.g., year, acronyms (apt., M.l., such as 11d and 18 line number) whenever VEAP, FDC, and they are cited so on) designate each item in every form consistently restrict the AFSA to an put work sheets on the application form and same page as the give the "completed" instructions; place all instructions for them or tax return and program information in put all work sheets in "estimated" tax return the student guide one place in item 22 more accurate names and move the AFSA number the pages of define them reference to the the form student guide from warn students not to page 11 to page 1 delete the chart on apply too early (e.g., page 1 of the AFSA before January 1, expand the form to 1992) in the "deadline more than four pages move the supplemental box" on page 1 of the information sections AFSA and at the document the reasons ahead of certification bottom of the third and major format changes signatures (item 38) fourth pages of the were made in the form AFSA reduce the clutter on the form delete duplication and give each question a overlap in the unique number instructions

OCR for page 197
APPENDIX C 205 Pretest with respondents typical of those who would use the form. If you can observe them completing the forms, and probe where they appear to be having difficulty or are making errors, it may help to explain why errors are made and how to correct or avoid them. As Yogi Berra reportedly once said, "You can observe a lot by watching." The informal testing methods suggested by the reviewers are not neces- sarily expensive to conduct, and they have been employed by many federal agencies. The use of such techniques is discussed in Approaches to Devel- oping Questionnaires (DeMaio, 19831. Some examples of research are de- scribed, including qualitative group interviews (focus groups) conducted for the Social Security Administration among teenagers and adults to evaluate a proposed revision of the application form for a Social Security Number (SSN). Research on a larger, more scientific scale would be indicated for evaluation of any major design revisions under consideration. The AFSA has never been field-tested among students. A predecessor form, the basic grant form, was field-tested under contract in 1981. Far fewer data elements were required by the legislation in effect at that time. The prototype used in the test was an uncrowded, two-page form color coded with shading for dependent and independent applicants. Two alterna- tives were also tested, including a version with separate forms for depen- dent and independent applicants. Much helpful information was gathered during this project and used to improve the form. The following quotation is from the contractor's report on the field test (Rehab Group and Macro Systems, 1981:6-7~: There are certain ways in which OSFA tOffice of Student Financial Assis- tance] would save money in future years and other ways in which it could improve its forms research projects. OSFA should consider actions in the following areas: (1) OSFA should better utilize existing routine activities that could provide information about problems applicants are having with the form; such activities include receipt of and response to calls and letters from applicants with problems and analyses of applications received at the pro- cesslng site; (2) OSFA should consider utilizing formal testing procedures as a meth- od of establishing the best terms, definitions, and data collection formats; (3) OSFA should explore major variations in the application form struc- ture, including additional forms for special populations; (4) OSFA should provide for methodological improvements to be made in future field tests, including, for example, use of longer data collection periods, use of experimental application forms in the actual application process, use of sampling schemes that more directly identify specific sub- populations, employment of probability sampling techniques, and use of larger samples; and

OCR for page 197
206 QUALITY IN STUDENT FINANCIAL AID PROGRAMS (5) OSFA should consider making a comprehensive review of other comparable benefit application forms as a means of developing standards against which to measure performance of the Pell Grant form. Over a decade later, those recommendations are still sound. The re- viewers' suggestions outlined above for testing the 1992-93 AFSA are clearly in accord with them and should be implemented. Utilization of data from routine activities, as mentioned in research area (1) above, should include . examining AFSA answers item-by-item separately for first-time ap- plicants and renewers to identify those items that cause problems (not nec- essarily errors) for students and/or processors, and analyzing the apparent impact, if any, on answers to core data items of variations among application forms used by various states, such as font size, layout, and specific additional questions. THE 1992-93 STUDENT AID REPORT Applicants who submitted the paper version of the 1992-93 AFSA should have received a Student Aid Report (SAR) within about four weeks unless the AFSA was rejected. Rejections may occur because the form is unread- able, the application was dated or filed too early, or the signatures from all those who must sign the certification-of-accuracy statement were not pro- vided. The SAR contains the data given by the student on the AFSA, informa- tion from the Department of Education about the student's eligibility for federal student aid, and instructions on what to do next. It also includes a form (either an Information Review or an Information Request) on which to provide additional information, make necessary corrections, and verify any "assumption edits" that may have been inserted during processing for se- lected blank or inconsistent data elements. For example, if the question about whether parents' assets include a farm was not answered, the editing system assumes the answer is no; if the item about whether either parent is a displaced homemaker was not answered, the edit again assumes the an- swer Is no. Applicants who appear to qualify for a Pell grant (on the basis of the information they provided and the assumed values, if any, inserted by the editing system) also receive a payment voucher to submit to the school of choice. If the student submits corrections or additions, a revised SAR is prepared and sent to the student in about two or three weeks, according to the student guide. The Information Review Form is of concern in the context of improv- ~ng the forms used in the application process. The panel's interest was sparked by the comments of over 600 applicants who wrote to the Depart

OCR for page 197
APPENDIX C 207 ment of Education during 1991. They did so in response to the invitation in the SAR to voice their comments or ideas for improvement to either the department or OMB. OMB also received many public comments on the SAR which were forwarded to the Department of Education for review and appropriate action. The response was "ED [the Department of Education] has also reviewed each of the letters provided in OMB's package to determine if further revi- sions to the SAR . . . are needed. Based on our review of those letters, no changes are required . . . " (U.S. Department of Education, 1991:5~. Ac- cording to a statement in the clearance file, the SAR design has been essen- tially the same each year since it was completely reformatted for the 1984- 85 processing year. As noted earlier, the AFSA has become longer and more complex since then because of added data items. One of the applicant letters the OMB forwarded to the Department of Education stated that the line item identification on the SAR should con- form to the identification on the original submission to make it easier to review or correct and, therefore, more accurate. Three-fourths of the com- ments received by the department referred to confusion in reviewing or correcting data because of the lack of correspondence between the item sequence on the SAR and the item sequence on the AFSA. Table C-1 illustrates the back-and-forth movement required in the AFSA to locate items cited on the SAR and the attention students must pay to the color coding for some items; it makes the criticisms of the SAR very understand- able. It is not necessary to burden applicants with deciphering the relation- ship between the SAR items and the AFSA items. Two simple labeling changes on the SAR would make the task much easier: Delete the letters in the section headings shown in the "We asked for" column of the Information Review or, for sections E through O. substi- tute the letters that identify those sections in the AFSA. In the "You told us" column, identify the information source by AFSA section letter and item number. If, despite these changes, applicants still find the form is more difficult than it need be, several other possibilities are obvious: Renumber the SAR and the AFSA so the item numbers match. Revise the SAR to conform to the sequence and item numbering in the AFSA. Revise the sequence of the AFSA to conform to the SAR. The third solution would be in line with the suggestions made earlier in this paper, that is, that AFSA items be grouped according to who should answer them or split into two forms on the same basis. .

OCR for page 197
208 V: Cal ; - o ._ a, o .~ 4 - C~ 50 - o U) o .~ Ct C) JO a) - Ct o o U) .~ Cal Q o V V Cal o z - . _ o ._ C) a, C~ o Z C) o L~ ._ o ._ ._ C: ~ ~o P ~._ V) U~ _ oo o ~ ~ ~ ~ oo ~ oo o o - C~ C~ ~ ~ ~ ~ ~ C~) - C~ ~ ~t _ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 - O CM =\ - 00 0 ON ~ '~) C~ ~ - O - - C~ C~ ~ ~ ~ ', C~ - , - C~ ~ ~t Ir) C~ ~ ~U. ._ , ~ ~ V) ._ V ~S - 3 ~ _ Ce _ C/, 4_ ,~ .0 .) ~ V' =, ~: ._~ ~ ~ o ~o C ~ 3 ~._ (,;, ~ ~- ~ , , c ~ ~ e - ~A ~ ~ ~ ~ ~ _ ~ ~ o c) _ ~ - ~ _ _ _ e e _ O - ~ O C o 5 o G~ 5, ~ C C 0~ 7` ~ ~ V ~ ~ CC ~ ~ ~ [L V ~ c) ~ -, _ 1 1 1 ~D = _ o ._ Ct s~ ~ 00 ~ ._ C~ Cq C~ ~ , . ~ S:: V) ~ ~ O o0 o0 - o ._ Ct ._ = ~V =) ( o D O ~ C~ ~ ~ Ct o a' U, - ~ _ C ~ C~ C~ 5 3 V ~ oo [~ oo C~ 1 1 1 1 1 1 1 _ o ~ O ~ ~ ~ U~ ~ ~ ~ ~ oo C~ ._ a; D Ct C,7 - C) C~ =, {004 X ~ .= C~ C~ - Ct ~ C~ C) ~ ~s 3 ~ . ~ U) ~ ~ O o . _ ._ ] ay ~ ~ D - - O X ~ ;> ~ Ct cn c~ O _ e ~ C~ ~ ~ X ~ . _ C~ C) . ._ C~ C~ e~ C) 00 - 5_ . ~ ~ _) ~ z ~ - V) O - C>` - O _ C~ ._ D C~ C~ ._ D ~O ~ ~ '- {~ C~ O C O O X ~ ._ ._ C~ ~ U) _ Ct C~ ~ C) ~ i ~ C~ o ._ C~ - C) a' X C 1 Ct . . Z O

OCR for page 197
APPENDIX C 209 THE 1992-93 AFSA ELECTRONIC VERSION The Department of Education's Electronic Data Exchange application process was introduced during the 1990-91 award year and has been ex- panding steadily since then. At participating schools, federal student aid application data can be entered and reviewed on a personal computer or an IBM-compatible mainframe system. Many financial aid administration and security features are provided in the electronic application form (Stage Zero) for 1992-93 (version 3.0.0, released in December 1991), some of which are discussed in this section. The comments in this section are based on information in the Stage Zero user's guide (National Computer System, 1992-93), a demonstration of the system held at the Department of Education, OMB clearance materials, and experimenting with the software by entering answers for imaginary appli cants. In Stage Zero's Electronic Needs Analysis System, financial aid person- nel can perform the following functions on student data already keyed: calculate the expected family contribution (to the student's financial needs), using the Congressional Methodology, as well as alternate family contribution amounts, calculate the estimated Pell Grant Index for determining Pell eligi- bility and an estimated Pell award amount, and apply verification selection criteria. Electronic filing of renewal applications was introduced for the 1992- 93 school year. It is available to returning students at participating institu- tions. (Renewals for other students require filling out the entire paper AFSA, no matter how much of the data filed previously is still correct.) Renewal records for students who meet certain criteria, such as having a calculated Pell Grant Index and not being in default on a student loan, are loaded into the software at the institution. Renewal applications are then printed; they resemble the Information Review Form in the paper SAR, but the section sequence and item numbers on the two forms do not match. The renewal application lists the data previously reported for many of the AFSA data elements; only changes to the data that must be updated each year need be entered. The updated data are edited, and a copy of the completed form is generated by computer. Signatures for release and certification are re- quired solely on this form. An "expert" application entry mode enables financial aid personnel to enter data from initial paper applications submitted by students. Three rather crowded screens are used; answer categories and the codes for them are omitted, but help messages are available. Eight user fields defined by the institution are available in the expert entry mode. Edit and review functions can also be performed. .

OCR for page 197
210 QUALITY IN STUDENT FINANCIAL AID PROGRAMS According to the user's guide, the electronic AFSA "is intended for student use in the financial aid office environment, with monitoring and counseling done as needed by the financial aid staff. Some institutions may want to have aid administrator staff walk through the questions as the stu- dent enters his/her information. Others may have staff members use this entry mode while interviewing applicants" (p. 15-1~. Learning to use Stage Zero was easy. Experience with the keyboard was helpful, but almost any applicant should be able to adapt quickly. Even hunt-and-peck typists would not have serious problems. Answers to most of the questions are preceded, and those that are not, such as name, address. and SSN, do not generally require lengthy answers. The first screen contains some simple instructions on using the system. It is in white letters on a blue background, the basic color scheme through- out the electronic application. Additional information about the questions mentioned in the instructions is available for every data item, even those for which no instructions appear in the paper AFSA. These help messages usually contain the information from the paper AFSA and information spe- cific to Stage Zero, such as which characters are permitted (no commas in the address, for example), and what address abbreviations to use if the answer space is insufficient. However, no solution is offered for the prob- lem of a name that does not fit in the allotted space and, obviously, writing in the margin is not an option. The help messages pop up mid-screen. However, they cover less than half the screen and often the message is too long for the space available. Continuations are brought to the screen by pressing the "page down" key. That is not explained, and it isn't obvious there are continuations unless the first part ends in mid-sentence. Since it often is not possible to see the questions when the help messages are displayed, the help messages could be enlarged to the full size of the screen, as the work sheets discussed below are, to reduce the number of continuations needed. Adding "more," "continued," or an arrow when appropriate would be helpful. The screen that follows the instructions contains a warning about pur- posely giving false or misleading information, a reminder that "you" and "your" refer to the student, and a place to enter the student's SSN. The SSN becomes the identifier for the record. The wording, sequence, and numbering of the items throughout the electronic application match the contents of the paper version. A limited number of questions appear on each screen; none of the screens is crowded. Answer cells are consistently located on the right-hand side, except those for the longer items, such as name and address. The person keying in the data can proofread entries on the screen and correct errors by backing up and rekeying. Edit or query messages appear on the screen during data entry, when appropriate, to alert the typist to

OCR for page 197
APPENDIX C 211 incorrect, incomplete, inconsistent, or suspect data. The typist can stop at any point, but if the application is not completed, the entries made cannot be saved. In some data fields, "wild" characters, as defined by the program, can- not be keyed (such as the letter o instead of a zero in a numeric field). But the software is tolerant of some unlikely answers, such as one-letter first and last names and numbers only for a street address. Section A of the electronic AFSA is divided into three screens contain- ing questions 1-4, 5-7, and 8-10, respectively, and the answer categories for them. The student's SSN is inserted automatically in item 5. (In later items, such as whether the student was born before 1969, previously reported data are also inserted automatically.) Section B contains the items that determine whether, in the remainder of the application, the questions meant only for "independent" applicants or those meant only for "dependent" students should be answered. However, only the questions and the answer categories are shown; the complex skip directions on the paper version are omitted. Instead, the skips are decided by the software. Comparing the screens with the cluttered appearance of the same items on the paper AFSA makes one appreciate a key advantage of a computerized application system. If the applicant is deemed dependent in Section B. a pop-up message on a red background states: "The remaining questions will ask you for infor- mation about your, (your spouse's), and your parents' finances." The help message for the definition of parents that applies in the remainder of the application does not include all the information in the instructions for the paper AFSA: the contents of the red box labeled "Parents' information - red areas," which explain the complex rules governing parents who have divorced, separated, and perhaps remarried, are not included in the elec- tronic AFSA. (Stage Zero could be even more helpful if such crucial in- structions popped up unbidden.) In the remainder of the application, the section identification in a box at the top of screens on which information about parents or the household is requested appears in black letters on a white background, rather than the white-on-blue scheme used for student information. Some distinction is essential. In sections D, G. H. and J. the questions do not specify whether they are for the student or the parents. They are verbatim from the paper version, where the questions are printed just once, followed by two sets of answer boxes. Duplication of some item numbers, again because they are that way on the paper version, adds to the opportunities for confusion. (It took this author a while to notice the color coding; it should be made more dramatic where the items about parents start and where the subject switches to the applicant. The color coding also should be mentioned in the pop-up message that precedes these items.)

OCR for page 197
212 QUALITY IN STUDENT FINANCIAL AID PROGRAMS For the income, earnings, and benefits portion of the form, a pop-up message informs the student that the two work sheets in the paper AFSA are available. The Stage Zero work sheets do the mathematical calculations called for on the data entries and then automatically insert the totals into the proper answer fields on the application. (Even the programmers were mis- led in this section: The query about whether the work sheet for parents is needed says "your" instead of referring to "parents.") In Section F. items 34a-c and 35b on the paper version were omitted from the computerized version. The user's guide explains that: "students are not asked for the name of their school or for permission to release their data to the school. It is assumed, by the choice to apply for aid at a particular school, that the student is interested in attending that school" (p. 13-14~. The calculations in work sheets A (for dependents) and B (for indepen- dents) are used in the paper AFSA to determine whether the supplemental information in Sections G through J is required. In Stage Zero those work sheets are not needed; the software makes the calculations automatically. The on-screen notice to those who have a choice is worded more clearly than the equivalent information on the paper work sheets. Finally, the typist is asked if he or she wants to review the application, and choosing "yes" causes the display to return to the beginning of the application. Corrections can be made easily. When the data are okayed by the typist, "your application is now being validated" appears on the screen. Corrections can be made during validation if inconsistencies are flagged. If the application passes the validation stage, the program automatically saves the file and prints a summary of the data entered. The two-page printout includes a section for office use only and a certification statement t"I have reviewed the information on this document and agree with its contents"] for the student to sign. The Quality Control Guidelines in the user's guide say the student should compare the responses on the printout with the responses on the paper form, one item at a time. The specific instructions say the student should "match the item numbers and names to make sure you are comparing the same items" (p. H-4~. It is not possible to do so, however, because item numbers are not shown on the printout, the section names do not match those on the paper AFSA, and the item sequence differs. If the student finds errors on the printout, the student is to circle the incorrect response and "write the correct response in the space immediately to the right of the incorrect response" (p. H-41. That can't always be done either; for some items, very little space is available on the printout. It is surprising that the software does not provide for recording who has keyed the data for the electronic application. For quality control analysis, it

OCR for page 197
APPENDIX C 213 The following equipment is required for operation of the Electronic App software: IBM or IBM compatible PC with approximately 2.5 megabytes of available space on the hard disk to accommodate the software and data records and 512K of free RAM (Use the DOS CHKDSK command to verify that you have at least 512,000 bytes free) a disk drive that accepts 5.25 or 3.5 inch, double-sided diskettes in DOS format (note that double or high density drives are acceptable) MS-DOS operating system 3.0 or greater for non-LAN users or 3.3 or greater for LAN users . A printer that is capable of printing standard 8.5 x 11 reports. Installation of the Electronic App software requires approximately 2.0 mega- bytes of available space on the hard disk. Your additional data storage needs will depend on the size of your database, with each application record requiring about 600 bytes of storage space. A 360K floppy (or 360K on your hard disk) can hold about 600 application records. Note to network users: The Electronic App software now supports LAN capabilities. Please refer to .... SOURCE: National Computer System (1 991 :14-1 ). would be valuable to know whether the information was entered by the applicant or by a specific member of the financial aid office. Applications can be sent to the Central Processing System electroni- cally, or data files can be mailed in on a floppy diskette. According to the user's guide, applications are processed and an Electronic Student Aid Re- port (ESAR) is sent to the institution within 72 hours. Even for applica- tions submitted on floppy diskettes, the processed data are returned to the institution in the form of ESAR records in regular Electronic Data Ex- change transmissions. At present, the student completes the paper form, gets the necessary signatures, and then enters the data in Stage Zero or someone else does the keying. This sequence is backwards. It does not take advantage "up front" of the many Stage Zero features that make it easier to understand what information is required on the application and to provide it accurately. If applicants could start with Stage Zero, working from notes, and fill out the paper form concurrently or subsequently (perhaps after more than one session on the computer), errors and burden would be reduced. In addition, a slightly modified version of the software would be appropriate for use off campus. Adding the capability to save and print incomplete applications is another change that would be helpful. Personal computers in many homes and most offices would meet the

OCR for page 197
214 QUALITY IN STUDENT FINANCIAL AID PROGRAMS requirements for installation of the 1992-93 electronic AFSA. The descrip- tion in the user's guide of the equipment required to use Stage Zero appears in the box on the previous page. If they could use a tailored version of Stage Zero, high schools, churches, community organizations, civic groups, and mentors with access to a computer could aid applicants in filling out the form. REFERENCES DeMaio, Theresa, ed. 1983 Approaches to Developing Questionnaires. Statistical Policy Paper 10. Prepared by Subcommittee on Questionnaire Design, Federal Committee on Statistical Meth- odology. Available through NTIS Document Sales, PB-84-105005. Washington, D.C.: U.S. Office of Management and Budget. National Computer Systems 1992-93 U.S. Department of Education, General Electronic Support, Electronic App User's Guide, 1992-93 Cycle, Vol. II. Presented to the U.S. Department of Education. National Computer Systems, Iowa City, Iowa. Rehab Group, Inc. and Macro Systems, Inc. 1981 Field Testing of 1982-1983 BEOG Application Forms, Executive Summary. Sub- mitted to the U.S. Department of Education, Office of Student Financial Assis- tance. Contract No. 300-81-0014. Rehab Group, Inc., Falls Church, Virginia. U.S. Department of Education 1991 OMB Number 1840-0132: The 1992-93 Student Aid Report (SAR) Clearance Package. Memorandum from Director, Operations Improvement Section, Division of Program Operations and Systems, to Ira Mills, Federal Information Review Branch, IMCD, Office of Management and Budget. Washington, D.C. October 31. 1992-93 The Student Guide: Financial Aid from the U.S. Department of Education, 1992- 93. Washington, D.C.: U.S. Department of Education.