5
Recommendations

We have briefly reviewed the need for education reform and the difficulties in achieving it (Chapter 1). We have discussed the various roles of education research and some of its contributions (Chapter 2). We have examined in detail the structure, programs, operations, and funding of the Office of Educational Research and Improvement (OERI) (Chapter 3). Finally, we presented our appraisal of the agency and the environment in which it operates (Chapter 4). In this chapter we first summarize our conclusions and then present our detailed recommendations for OERI.

Education research has been used and useful in far more ways than is commonly thought. Research has contributed new perspectives and conceptualizations of learning, teaching, and schools. It has guided the development of innovative curricula, teaching strategies, special programs, and technologies. Evaluations have assessed the consequences of programs, practices, and policies. And statistics have monitored the status of the nation's schools and warned of impending problems.

Despite the contributions of education research, education reform is difficult to accomplish successfully. Authentic and sustained reform requires the confluence of many elements—appropriate school organization, improved curricula and pedagogy, more public understanding and community support, better teacher education, supportive policies, and the mobilization of adequate resources. In addition, the contributions of research to practice often have not been direct and predictable; rather they have slowly percolated through established policies and practices. Numerous supply-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement 5 Recommendations We have briefly reviewed the need for education reform and the difficulties in achieving it (Chapter 1). We have discussed the various roles of education research and some of its contributions (Chapter 2). We have examined in detail the structure, programs, operations, and funding of the Office of Educational Research and Improvement (OERI) (Chapter 3). Finally, we presented our appraisal of the agency and the environment in which it operates (Chapter 4). In this chapter we first summarize our conclusions and then present our detailed recommendations for OERI. Education research has been used and useful in far more ways than is commonly thought. Research has contributed new perspectives and conceptualizations of learning, teaching, and schools. It has guided the development of innovative curricula, teaching strategies, special programs, and technologies. Evaluations have assessed the consequences of programs, practices, and policies. And statistics have monitored the status of the nation's schools and warned of impending problems. Despite the contributions of education research, education reform is difficult to accomplish successfully. Authentic and sustained reform requires the confluence of many elements—appropriate school organization, improved curricula and pedagogy, more public understanding and community support, better teacher education, supportive policies, and the mobilization of adequate resources. In addition, the contributions of research to practice often have not been direct and predictable; rather they have slowly percolated through established policies and practices. Numerous supply-

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement side and demand-side problems impair the linkages between the research and practice. In answer to the question—How can the Office of Educational Research and Improvement better contribute to the improvement of schools?—the committee has reached a two-part answer: by strengthening the agency's ability to support and achieve benefits from traditional approaches to R&D and by expanding the agency's functions to encourage establishment of learning communities—collaborative relationships between teachers, administrators, and researchers in the pursuit of new understandings and practices that will improve education. The traditional approaches and this new one should not be pursued separately. The fostering of learning communities will need the insights and innovations of the traditional approaches. In turn, the traditional approaches will function better if informed by the needs, craft knowledge, and inquiries of teachers and administrators. OERI can foster learning communities through support for research, development, demonstrations, and evaluations aimed at encouraging the needed collaborations; through obtaining diverse input into the planning of research and development; and through wide dissemination of its work. However, OERI cannot singlehandedly encourage learning communities. Leadership and support must also come from those who educate teachers, administrators, and researchers; from those who hire them, manage their work environment, and structure the incentives and rewards for outstanding performance; and from national, state, and local policy makers who establish the broad parameters that affect both. The mission of OERI is inherently difficult, and the committee does not expect that to change. The disagreements and conflicts over education are endless. Probing the mysteries of human learning is not easy. Linking research with practice remains a challenge. And improving schools is always difficult. But OERI is also faced with many problems that are not inherent in its mission or responsibilities. If these problems are eliminated or reduced, the agency could be more effective. Frequent changes in leadership have caused organizational instability, false starts, abandoned efforts, and unfulfilled agendas. Having the head of any research agency serve at the will of a high political appointee creates the appearance, if not the reality, of politicization. So does requiring a research agency to submit its reports for clearance by a politically controlled public affairs office. Congressional actions have also weakened OERI. In addition to substantial budget cuts through most of the 1980s, set-asides in the appropriations have almost eliminated field-initiated research, and mandated studies have occasionally been politically skewed. Fragmentation within OERI, and between it and other federal agencies, has resulted in agenda setting with little benefit from what the others have learned and accomplished. The paucity of sustained research has often

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement limited the advance of understanding. The paucity of sustained development efforts has resulted in many innovations that are less effective and more expensive than necessary. Inadequate mechanisms for quality control and accumulation of results have forced practitioners and policy makers to wade through large literatures with little guidance as to what is valid, important, and widely applicable. Weak links with teachers, administrators, and policy makers have often limited researchers' knowledge about the realities of schools and public policy making and denied practitioners the benefits of R&D. Inadequate funding has contributed to most of these problems and undermined OERI's capacity to deal effectively with them. Along with these conclusions, four caveats should be kept in mind when considering our recommendations. First, much that needs to be done about U.S. education is beyond the responsibilities and authority of OERI. OERI's mission is to expand understanding and assist in the improvement of education. It has no authority over teacher education institutions, state education agencies, school boards, district administrators, principals, teachers, or parents. The agency also is tightly constrained in the extent to which it can promote or induce change. The role of OERI (and its predecessor, the National Institute of Education [NIE]) has always been limited to generating new knowledge, developing new techniques and approaches, disseminating information about both, and assisting interested parties to apply the education research and development. Second, no two or three of the committee's recommendations, by themselves, are likely to make a big difference. The committee has not found an isolated fatal flaw in OERI. Rather, it has found an agency with a very difficult role, severely constrained resources, and a number of organizational and functional weaknesses. OERI needs to be rebuilt, not merely repaired. Third, even if OERI is rebuilt, it cannot fulfill the unreasonable expectations that have often been placed on education R&D. OERI cannot provide quick answers to long-standing and complex questions. It generally cannot find simple, universal solutions for problems embedded in diverse causes. And it cannot, alone, reform education in the United States. What OERI will be able to do is respond more fully to needs for education R&D, support the sustained work that is necessary for scientific and technological advances, improve quality control, and better assist teachers and administrators in a quest for improved education. Fourth, unless there is a substantial increase in OERI's budget, no amount of leadership and organizational change can accomplish what needs to be done. Our recommendations would require a large increase in the agency's budget. If such an increase is not provided, the agency's mission and responsibilities should be dramatically narrowed. Our recommendations are organized into four groups and numbered

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement accordingly: (A) mission, governance, and agenda; (B) organization and functions; (C) operations; and (D) funding. The committee has made rough estimates of the additional resources needed for most of the recommendations. These estimates are based on the committee's judgments about the intensity of effort that is desirable and its knowledge of the staffing levels used in other federal research agencies to conduct similar activities. Unless otherwise indicated, all estimates are for permanent staff and for annual program expenditures in 1991 constant dollars. The committee has not estimated the increases that will be needed for support services personnel or for the staff and expenses budget, though increases will obviously be needed for both. MISSION, GOVERNANCE, AND AGENDA The legislative authorization for OERI directs the agency to ''provide leadership in the conduct and support of scientific inquiry into the educational process.'' That mission is accompanied by a substantial list of specific responsibilities. The authorization also outlines the composition and duties of OERI's 15-member advisory council and prescribes a mechanism for generating and publishing priorities for research. Our first seven recommendations apply to OERI's mission statement, leadership structure, and mechanisms for setting research priorities. They simplify the mission statement, replace the advisory council with a more representative policy-making board, establish a new mechanism for setting priorities, specify the board's responsibilities for monitoring and reporting on the federal system for education R&D, and provide for a balance in various forms of needed research and development. A-1 The mission of OERI should be to provide leadership in: • expanding fundamental knowledge and understanding of education; • promoting excellence and equity in education; and • monitoring the state of education. The mission should be accomplished in collaboration with researchers, teachers, school administrators, parents, students, employers, and policy makers. OERI's current mission statement includes a long list of responsibilities for the agency, which drives the agency to try to be everything to everybody. As a result, OERI has spread its resources so thin that there is little chance of fulfilling any of the responsibilities well. The recommended mission statement does not preclude any of the agency's current responsibilities, but it does direct the agency to focus more narrowly if its budget is not sufficient for credible work in all areas or if the subsequently discussed governance and agenda-setting processes suggest doing so.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement We have added to the mission statement a reference to expanding fundamental knowledge and understanding in education. Basic research is critical to progress in research (see Chapter 2), and OERI has a long history of giving it short shrift. The Office of Education, in the late 1960s, and NIE, in the early 1970s, provided substantial support for basic research, but OERI has provided very little funding for it: $1.9 million in 1989 (the last years for which data are available), or 5.5 percent of OERI's R&D budget, a far smaller percentage for basic research than prevails in other major federal research agencies. We have retained from the current mission statement a reference to promoting excellence and equity in education because both are in keeping with the goals of this country. We have also retained a reference to monitoring the state of education because it is essential for knowing what is actually achieved. OERI's current mission statement does not explicitly indicate that the agency should work closely with the major groups that are importantly involved in, or affected by, education. Although OERI (and NIE) has not been oblivious to this need, at various times it has operated with less participation from some of these groups than would be optimal. Each of the groups included in our recommended mission statement has unique perspectives and expertise that are essential for reforming U.S. education: researchers apply the theories and methods of various sciences in developing new knowledge; educators and parents are the main agents of a child's education; employers are a key "consumer" of education; policy makers determine the distribution of public resources and set the broad parameters for the conduct of education; and students are both the targets of education and key actors in them. The perspectives and expertise of all these groups needs to be infused into OERI's work. That will not be easy under any circumstances, but it is a critical challenge that must be met by the agency. Some of the committee's subsequent recommendations should help with the task. This recommendation would not require changes in OERI's staffing or funding levels. A-2 OERI should support a balanced portfolio of activities: basic research, applied research, statistics, development, evaluation, dissemination, and technical assistance; field-initiated and institutionally based R&D; and long-term sustained efforts and responses to newly identified needs and opportunities. To do so, OERI must substantially expand support for basic research, field-initiated research, and sustained R&D activities. OERI supports many kinds of education R&D activities. Each makes different contributions, and a mix of them is necessary to fulfill its mandate.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement Over the years, the distribution of these activities has skewed away from basic research, field-initiated research, and long-term efforts. This change has been the result of declining budgets, congressionally specified set-asides for specified activities, and decisions of OERI and NIE administrators. Although there is overlap among the three categories of basic research, field-initiated research, and long-term sustained efforts—much basic research is field-initiated, some basic research is part of long-term sustained efforts, and some field-initiated research is part of long-term sustained efforts—each needs to be recognized for its contribution. In 1989 OERI spent only 5.5 percent of its R&D budget on basic research; in the same year, the Agriculture Research Service spent 46.6 percent of its R&D budget on basic research, NIH spent 59.8 percent, and NSF spent 93.5 percent. Basic research explores the fundamentals of the studied phenomena, generates new views of reality, and proposes new visions of the achievable. Since 1988 only about 3 percent of OERI's R&D budget has been available for field-initiated research; in comparison, NIH uses about 56 percent and NSF about 94 percent of their budgets, respectively, for field-initiated research. This research permits scholars and others throughout the country to propose topics for research rather than only to respond to announcements prepared by the agency. It also helps train the next generation of researchers by providing assistantships for graduate students throughout the country. Field-initiated research has been a key contributor to scientific and technical advances in all fields. OERI and NIE have often been criticized for failing to provide sustained support for R&D activities. The actual record is varied, and the criticism is only partly justified, but more sustained efforts are needed. Long-term efforts are needed for advances in basic research as well as for development of new applications. Major advances in the natural sciences and technology applications often have taken a decade or longer. Without substantially enhanced programs of basic research, field-initiated research, and long-term sustained efforts, OERI will be a feeble partner in the nation's quest for substantial education reform. Research and development require long-term investments. The staff and budget implications of this recommendation are discussed under Recommendations B-2 and B-3. A-3 OERI should have a director appointed by the President, in consultation with the agency's board and with the advice and consent of the Senate, for a 6-year renewable term. Nominees for the directorship should be distinguished researchers, proven managers, and persons with substantial knowledge of U.S. education. The

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement director would be responsible for implementing the agenda developed by a policy-making board specified in Recommendation A-4. Under the delegated authority of the secretary, the appointee would also guide and manage all the agency's activities. NIE had six Senate-confirmed directors over a 13-year period, and since it was dissolved into OERI in 1985 there have been three confirmed assistant secretaries of the latter agency. During the same 19-year period, there have been six directors of NSF, five at the National Institutes of Health (NIH), and three at the Agriculture Research Service. The rapid turnover at OERI has been dysfunctional to an agency that needs sustained leadership in planning for, investing in, and supporting the long-term efforts that are required for major scientific and technological advances. There are several precedents for 4- or 6-year terms of office in federal research agencies, though none of them is perfectly analogous to OERI's situation (see Chapter 4, "Governance"). These arrangements have been used to assure sustained professional management and to minimize the opportunities for politicization. They cannot assure either—appointees are still free to quit and both the President and Congress retain discretion over agency budgets. In addition, there are some potential problems with terms of office for the top administrator. They make the administrator less accountable to the executive branch, and an incompetent could linger for years unless there is a precaution against this, such as specifying removal by the President with the consent of the Senate. Nevertheless, we have specified the longest term of office that can reasonably be supported by precedent. We think the potential benefits outweigh the potential risks. Four-year terms are more common and provide each President with an appointment opportunity. When they have been used there often is a tradition of reappointments. Given OERI's history, we fear that there would be few reappointments and 4 years is a short period in the management of R&D. It should be noted that a fixed term of office does not give the director unlimited discretionary powers. On the contrary, he or she would still be subject to many of the normal management processes and to the full budgetary process that involves the Secretary of Education, the Office of Management and Budget, the Executive Office, and Congress. This recommendation would not require additional staffing or funding for OERI. A-4 OERI's agenda setting should be guided by a 24-member policy-making board. At least one-third of the membership should be distinguished researchers who have done work on education issues, complemented by a balanced representation of practitioners, parents, employers, policy makers, and others who have made note-worthy contributions to excellence in education.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement OERI currently has a National Advisory Council on Educational Research and Improvement. It is composed of 15 members serving 3-year terms, appointed by the President with the advice and consent of the Senate. Congressional legislation requires the council to be "broadly representative" of several groups. For at least the past 3 years, however, the council has apparently had no active education researchers or other social scientists among its members. The council is also widely considered to have little influence. NIE had a policy board. Under both Republican and Democratic administrations in the 1970s, it was widely considered competent and hard-working, but was unable to help NIE gain the support of educators, the public, or Congress. In the early 1980s the board was considered less distinguished, more politicized, and less effective. Three options for OERI's future have been analyzed (see Chapter 4, "Governance"). There are potential advantages and disadvantages to policy-making boards, advisory councils, and administration of the agency without either. With OERI's history of controversy, constant charges of politicization, and fragmentation, bringing focus and stability to the agency is a bigger job than any one person is likely to manage. We have chosen a policy-making board because persons of accomplishment are more likely to agree to serve on such a board and because a policy-making board is likely to be more influential. Our analysis indicates, however, that policy boards for federal agencies have little unilateral power—they must win the respect and support of the President and Congress. The board we propose would differ significantly from the boards of NIE. It would be larger and more diverse, helping to ensure that its members understand the views of the major groups concerned with education. It would be limited to people who have already proven their ability to make important contributions to research on education or to excellence in education, thus ensuring competence and some common understandings during its deliberations. In addition, as indicated in the next recommendation, the board would not set OERI's agenda on the basis of its member's own predilections, but rather would distill priorities from the needs and capabilities of the country after wide consultation with those concerned about education. The board we propose is modeled closely on the National Science Board of the National Science Foundation. Most observers believe that this board has served its agency well. But there are no assurances that the same structure will work effectively for OERI: the National Science Board has the advantage of making decisions about astronomy, geodesy, and computer engineering, topics about which only a small number of people have any knowledge; OERI's board would be making decisions related to education,

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement a topic with which almost everyone has prolonged experience and strong opinions. Though there are reasons to be uncertain about the efficacy of the proposed board, the committee thinks it is the best option available for stabilizing and guiding OERI. Its task will not be easy, and its success is not guaranteed, but the alternatives are less promising. The board members should be appointed by the President, with the advice and counsel of the board, for 6-year staggered terms. For initial appointments to the board, the President should seek advice from professional organizations of those who are involved with education and research on educational issues. The board should be led by a chair, elected by the board members from among their own ranks, for a 2-year renewable term. The OERI board should monitor the health, needs, and accomplishments of OERI's R&D and all federally sponsored education R&D; report periodically to the President and Congress about both; and guide the agenda setting of the agency. This recommendation will not require additional staff or funding. A-5 The OERI board should establish a process to develop priorities for OERI's agenda. The process should involve active participation of the various groups concerned with education. These priorities should be set so as to maintain the continuity, stability, and flexibility needed to conduct high quality research and to effect educational change. The current authorization for OERI requires the Secretary of Education to publish proposed research priorities in the Federal Register every 2 years, allowing 60 days for public comment. This process results in the establishment of a new set of priorities every 2 years, which can interfere with the continuity and stability needed for many education R&D activities. The committee proposes long-term plans, with a limited biennial update. Development of 5-, 10-, and 15-year national research plans have been standard practice at NIH for many years. It would be desirable that the agenda-setting process begin after the director has served for about 1 year—assuring his or her familiarity with the realities of the agency. OERI's agenda for education R&D must reflect the priority needs of researchers, teachers, administrators, parents, students, employers, and policy makers. Publication in the Federal Register is far from sufficient outreach to these groups. OERI did hold some meetings before competing the laboratories and centers in 1985 and 1990, but several of its other programs have not had the benefit of such outreach. The agenda-setting process must also reflect the capabilities of the education R&D enterprise. Unrealistic objectives of quick fixes to complex problems or universal solutions to problems with multiple causes serve only to disappoint researchers and potential users of their work. Without the

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement integration of needs and capabilities, the productivity, effectiveness, and applicability of the education R&D will suffer. The board should not impose its own priorities, but rather it should design, oversee, and participate in a process that distills priorities from the needs and capabilities of the country. Preparing priorities that fairly reflect diverse views of all the groups concerned with education will not be easy. Education has been the subject of fierce loyalties and long-standing conflicts. Facts and values frequently clash; needs and capabilities seldom coincide. The priority-setting process will be messy and at times confusing, but only through such a process can there be an agenda that truly reflects the country's needs. In the context of the previous and following recommendations, this one would require OERI to have about three additional staff persons and $100,000 annually to support the board's priority-setting process. A-6 The OERI board should publish a biennial report on federally funded education R&D that describes its accomplishments, summarizes the programmatic activities and funding levels throughout the federal government, identifies unmet needs, and makes recommendations for future directions. For most of the history of OERI and NIE, the board's (or council's) report has not portrayed the extent and nature of federal involvement in education R&D nor reflected on the progress and contributions of research and development to the improvement of education. This has limited its ability to identify needs, chart a course for OERI in light of what other agencies are doing, and illustrate the utility of proposed work. This lack has also deprived the agency of institutional memory, hindered researchers' and practitioner's efforts to build on what has been learned by past efforts, and denied policy makers information on the extent of progress. OERI and other federal agencies involved in education R&D have done a poor job of describing the contributions of their work (see Chapter 4). There is a need to synthesize—succinctly and saliently—what has been learned from education research, how it has extended prior knowledge, the implications for practice and school reform, the development and assistance activities that have used the research, and the effects of those efforts. This information is important not only for OERI, but for researchers, teachers, administrators, other agencies engaged in education R&D, the President, and Congress. This recommendation directs OERI's policy-making board to prepare a more ambitious report than is currently the case. The resources needed for some of the data collection are discussed in Recommendation A-7. Additional resources will be needed to analyze the collected data, to assemble

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement information on the accomplishments of federally supported education R&D, and to prepare a written report of that information. This would require about two additional staff members and about $25,000 annually. A-7 The Office of Management and Budget (OMB), the National Science Foundation (NSF), or the Federal Coordinating Committee for Science Engineering and Technology (FCCSET) should extend data collection programs, in consultation with OERI, to provide annual data on federal agencies' program activities and expenditures for education R&D. There has rarely been comprehensive information on the programmatic activities and funding levels of education R&D throughout the federal government. For two decades NSF has been responsible for monitoring the extent and nature of R&D in the United States: it collects and reports federal R&D expenditures by agency, fields of science and engineering, and budget functions, as well as several other categories, yet none of these data provide even a rough picture of federal support for education R&D. In addition to the Department of Education, several other federal departments and agencies sponsor education R&D. NSF, the Department of Health and Human Services, and the Department of Defense have long-standing and substantial involvements in this work. Reasonably comprehensive and current information on federal agencies' education R&D activities is needed to identify existing priorities and unmet needs. The committee has not analyzed which agency would be best suited to collect the needed information nor what specific data should be collected. Determination of those matters should be made during consultations among OMB, NSF, FCCSET, and OERI. This recommendation will probably require one additional staff person at OERI to coordinate preparation for the consultations and follow-up from them, for a period of 1 or 2 years. The agencies that undertake the data collection will also need some increase in staffing and budget, but if the work can be melded into another data collection effort, only a fraction of one person's time and $25,000–$50,000 for additional expenses will probably be needed. ORGANIZATION AND FUNCTIONS OERI currently has six offices: the Office of the Assistant Secretary; the Office of Research, which administers the 25 R&D centers, the Educational Resources Information System (ERIC) and field-initiated studies; Programs for the Improvement of Practice, which includes the ten regional laboratories, the Program Effectiveness Panel and the National Diffusion Network

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement Both NIH and NSF support programs aimed at developing more minority and women researchers. As women now earn 58 percent of the doctorates in education and 46 percent of the doctorates in the social sciences and now comprise 49 percent of the membership of the American Educational Research Association, the training of racial and ethnic minority researchers should be given priority. A solid program of dissertation grants for graduate students in all the social sciences would provide about 100 2-year grants of $10,000 per year, requiring about three additional staff and $2 million annually, phased in over 2 years. These grants would be incentives for dissertation research on issues relevant to education and therefore not have to cover the full expenses of graduate students. A viable program of minority fellowships would provide about 100 3-year grants, averaging $23,000 per year, to cover living expenses at the same level provided by major NSF and Department of Education fellowship programs, plus full payment of tuition and fees. This program would require another five staff members and $6.9 million annually, phased in over 3 years. OPERATIONS OERI depends on the Department of Education for hiring and other personnel matters, for procurement, and for report clearance, and OERI has experienced various problems with all three. It has had limited success in recruiting highly qualified personnel. Its proposal review process has commendably involved both researchers and practitioners, but they have been allowed to make decisions in areas outside of their respective expertise. And despite its research and development in areas of considerable dispute, OERI has rarely used a formal process for developing consensus. The four recommendations in this section address these problems. They would give OERI its own personnel authority, its own contracts and grants office, and sole responsibility for its publications; encourage the agency to recruit top-flight personnel and create an intellectually attractive work environment for them; modify the peer review of proposals for grants and contracts; and have the agency use a consensus development process to determine the implications of important and controversial bodies of research and evaluation. Each of these should help the agency better fulfill its mission. C-1 OERI should have independent authority for staffing, contracts, grants, and reporting. Many individuals and institutions that have dealt with the Department of Education's contracts and grants office report inordinate delays, a lack of responsiveness to legitimate needs, and incompetence. Several OERI staff have echoed those complaints and voiced similar ones about the Depart-

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement ment's personnel office. Such complaints are not uncommon at other federal agencies, but the vehemence of the criticism of OERI does appear unusual. In addition, many of the critics have had the benefit of experience with other federal agencies and claim the problems are more severe at the Department of Education. Providing OERI with its own contracts and grants office would not guarantee improved service, but it would put that possibility within the control of the agency. Similar complaints of delays and nonresponsiveness have been frequently voiced about the Department of Education's personnel office. In addition, independent staffing authority would give OERI its own salary and expenses budget as well as its own personnel office. NCES has had its budget and number of projects tripled over the past 5 years with virtually no increase in its staffing slots. That represents a serious failure of the current salaries and expense budget process. Independent staffing authority will not guarantee avoidance of this in the future, but it will make disparities between staffing levels and agency responsibilities more obvious to those involved in the budget process. There is at least one disadvantage in giving OERI its own personnel and procurement offices. As a small organization, OERI will not be able to acquire the depth or breadth of expertise that is possible at the departmental level. Indeed, because of this, one knowledgeable administrator in OERI has expressed doubts that the quality of service would be markedly improved. All things considered, however, the committee believes OERI should have its own authority for staffing, contracts, and grants. All OERI reports, except those of NCES, must now be cleared by each assistant secretary of the Department of Education who has jurisdiction over matters discussed in the report. They also must be cleared by the Editorial Policy Division of the department's Office of Public Affairs. The criteria for approval include "consistency with ED's mission and goals" and "conformity with legislation, regulations, and policy" (see Chapter 3), and there have been many allegations of ideologically motivated delays and modifications, resulting from this review process. The committee has not attempted to verify these allegations, but thinks that the current clearance procedures are inappropriate for a research agency, whose work should be characterized by the highest standards of objectivity. In place of the Department of Education's clearance procedure, OERI should establish a rigorous review and revision process for all reports that are formally issued under its imprimatur. The report review procedures currently used by NCES might be a good model for the rest of OERI to follow. Independent staffing authority, contracts and grants authority, and reports authority are common among the larger research agencies, such as the Agriculture Research Service, NSF and NIH (with a few exceptions for

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement reports) and have been granted to some relatively small federal statistics units. The Bureau of Labor Statistics within the Department of Labor has all three authorities. Recent legislation offered the former two for NCES, but the then Secretary of Education decided against implementation. This recommendation will require additional staff for OERI, but there could be some offsetting reductions in departmental staff. No budget increase would be needed. The committee cannot estimate the number of staff needed for the personnel office and the contracts and grants office; the Office of Personnel Management should do that. OERI would need about four additional staff for its own rigorous report clearance process. OERI currently issues about 160 reports a year, and staff also prepare many other professional papers and speeches in their official capacities. C-2 OERI should actively recruit highly qualified personnel from various disciplines for OERI staff positions and should create an intellectually stimulating working environment. OERI can provide effective leadership in education R&D only if it is staffed with high-quality personnel. A cadre of professionals whose qualifications are known and respected is essential to develop a partnership between the agency and the field. NIE recruited many outstanding researchers during its first years, but many left as its budget dwindled, and for years OERI and NIE have held little attraction for top scholars. Although OERI has retained some excellent staff members and recruited some others, many observers within the agency and outside believe it does not have enough. If this committee's various recommendations are implemented, OERI should have a better chance of attracting the kind of personnel needed. It is particularly important to secure excellence at the top levels—the director and associate directors—both because of the responsibilities of those positions and because such persons serve as magnets for attracting outstanding mid-level professionals. Another mechanism that would help bring top talent to OERI is to implement a ''rotators" program, bringing top researchers, developers, teachers, and administrators to the agency, for significant periods of time, involving them in the planning and the execution of OERI's work. Such a program would strengthen the agency, make it less parochial, enrich the intellectual environment, and improve OERI's relationships with the fields of research and practice. There is widespread opinion in OERI, by staff and several managers, that the agency has provided very limited opportunities for staff to engage in professional activities such as attendance at professional meetings and advanced training seminars. This is a serious problem in a research agency. Staff need to be on the cutting edge of new developments when helping to develop the research agendas, soliciting research proposals, evaluating pro-

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement posals, monitoring the work, and assessing its importance after completion. A perennially tight salaries and expenses budget is blamed for the problem. This recommendation will not require additional staff positions or program budget. The rotators would fill existing staff slots. C-3 OERI's contract and grant application review process should provide an appropriate balance between expertise in research and in practice for all proposals, with technical research merit judged by research experts and programmatic relevance judged by program experts. NIE relied primarily on researchers to review proposals. OERI has considerably increased the use of teachers, administrators, policy makers, and others. The inclusion of practitioners as reviewers is commendable—and not unprecedented in federal agencies—but changes should be made in the procedures. Administrators and teachers are generally not qualified to judge the technical adequacy of proposed research or evaluations. Similarly, researchers are generally not qualified to judge the programmatic promise of proposals, except those in their areas of substantive expertise. OERI has had both types of experts judge both the technical and programmatic merit of proposals, and all their ratings have been averaged. Proposals for basic research should be reviewed only by researchers, because, by definition, basic research is directed at probing fundamental understandings and is not expected to have direct programmatic application. Proposals for applied research, development, and technical assistance should be reviewed by panels with a broad range of expertise. Even then, however, given the nature of OERI's work, we think a minimum of one-third should be researchers—and a higher proportion when the proposed activities primarily involve research. In addition, the review process should be structured to ensure that only reviewers with technical research expertise make judgments about scientific and technical merit, that only reviewers with the appropriate programmatic expertise make judgments about those matters, and that proposals that are judged to be technically or programmatically inadequate are not recommended for funding. This recommendation would not affect the staffing or budget needed by OERI. C-4 OERI should implement a consensus development process involving distinguished experts to review and report on the quality and implications of potentially important bodies of research and evaluations that appear to have unclear or conflicting results. There are disputes over many education issues, including the interpretation of research results. Teachers, administrators, parents, journalists, and

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement policy makers repeatedly state they have difficulty determining what is true in a large body of literature with apparently conflicting results. Is bilingual education or English as a second language (ESL) instruction most effective for students with limited English proficiency? For which students and under what conditions? Is the phonics or the whole-language approach superior for teaching beginning reading skills? Superior in what respects and inferior in what respects? Is homogeneous or heterogeneous grouping best—for the slower students and for the faster students? These and other contested issues have been subject to considerable research, but few practitioners know, or could themselves assess, what the weight of evidence says about each. Although OERI (and NIE) has long sponsored and conducted reviews of research and evaluation literature—some of which have been high-quality scholarly works—reviews differ from the consensus process that we are advocating. Reviews are usually the product of one or two people; a consensus process typically relies on a panel of 10–15 people with diverse expertise and perspectives, who can be expected to raise more questions, apply a wider spectrum of knowledge and skill, and arrive at a broader set of insights. After considering the available evidence, they reach agreements about what is known from the evidence and what is not yet well established and issue a public statement reporting their judgments and rationales. Sometimes they also indicate needed lines of future research. The findings of a consensus development process are likely to be more credible to teachers and administrators than a review because a substantial number of prominent people must agree on the findings. This recommendation would require some additional staffing and funding. If contracted out, each consensus development process would require about $400,000–$900,000 and perhaps 0.2 staff for liaison and monitoring over 12–24 months. If conducted in house, each would require about 1.5–2.5 full-time staff for 12–24 months and about $100,000 of program expenses. Given all the controversy in education, there could easily be two or three topics a year that would benefit from this scrutiny. FUNDING OERI's budget was $380 million dollars in fiscal 1991, of which most was spent on Library Programs and development work that is not based on research. Under the narrow definition of R&D used by NSF and OMB, the agency invested only an estimated $58 million in research and development work. By a broader definition that includes routine statistics collection and dissemination of R&D, the agency spent $126 million. Neither of these amounts is adequate for the mission of OERI and the education needs of the country.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement The committee recommends increases in OERI's budget for more basic and applied research, more research-based development, laboratory staff with state liaison responsibilities, a minority fellowship program, more extensive refinement and evaluation of promising innovations, and consensus conferences to reach findings about important and contested bodies of research and evaluation. If that increase is not forthcoming, the mission and activities of the agency should be significantly narrowed. D-1 To implement the committee's recommendations, OERI should be given substantial, phased-in, increases in its budgets and staffing levels. The funding of OERI (and NIE) declined dramatically over the past two decades while its mission remained largely unchanged. As a consequence, the agency has carried on a broad array of activities with little depth and little chance of making substantial contributions. Over the same period, the need for significant reform of the nation's schools has gained prominence. There is now major public support for the six National Education Goals, but little agreement about how to achieve them. Other needs for research and development have long gone unfilled. Basic research on many education issues is almost nonexistent. Promising innovations have gone without rigorous evaluation. And there has been little effort to help teachers be full partners in the quest for school change. A substantial increase in the budget and staffing of OERI is required if it is to play an important role in filling these needs and assisting the nation with effective education reform. In trying to determine how much money is needed for a given field of research, three lines of analysis are often brought to bear: a comparison of R&D investment in various fields; computation of likely rates of return on investments; and examination of what activities are important to undertake. Federal expenditures for education R&D are much lower than federal expenditures for R&D in agriculture, transportation, health, and other major fields, not only in terms of dollars expended but also as a percentage of all federal expenditures in each field and as a percentage of national expenditures in each field (see Chapter 3). By this line of analysis, education R&D is significantly underfunded by the federal government. In 1986 the Office of Technology Assessment completed a review of the return-on-investment approach for guiding scientific research and concluded that, ''while valid conceptually, [it] does not provide a useful practical guide to improving Federal research decision making (U.S. Office of Technology Assessment, 1986:9)." The applicable return on investment is not dollars, but rather a broad spectrum of social benefits that are difficult to measure and assign dollar values. The social benefits of improved education include both the personal

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement satisfactions of a good education and the social contribution of education to "human capital." The productivity of any economy depends on its human capital—the knowledge, skills, and habits that employees bring to their jobs. If education R&D improves education's contributions to human capital, the returns to society will be in the form of increased economic productivity, improved competitiveness in world trade, and a heightened standard of living in the United States. Although it is not possible to apply returns-on-investment analysis to research investments, it is obvious that a small percentage increase in the productivity of the nation's $350 billion education enterprise could yield substantial cost savings, or alternatively, increased outputs in thousands of schools, and the benefits are likely to be reaped for many years. The third line of analysis for deciding on the level of investment in a field of R&D is to list all the things that are considered important to do and price them. This report has noted many serious problems in the nation's schools for which the nation should be diligently seeking solutions, as well as needs and opportunities for R&D that have resulted from recent work. For example, progress in understanding reading difficulties is incomplete; there is much dissatisfaction with traditional means of measuring student learning, and several proposed alternatives need careful exploration; international comparisons have yielded attention-getting results but require several refinements to be fair gauges; intriguing new teaching approaches are in early stages of development and need much more work; several promising curriculum innovations have not yet been rigorously evaluated; and efforts to restructure schools offer the promise of large and sustained gains, but need further development and evaluation. Another need for education R&D results from the current ferment in educational policy at both the state and federal level. New ideas are being advanced and implemented with little or no research foundation, such as proposals about school choice, national testing, and student mentors. There is a need for these ideas to be rigorously studied so that their full potential can be realized and assessed. Without well-designed and careful evaluation, new ideas will be judged by press releases and public opinion, rather than on the basis of rigorous evidence. The committee's prior recommendations are based on the problems and opportunities identified in this report. For most we have estimated the cost and number of additional staff needed. Together our recommendations will require an addition of about $267 million annually for OERI's program budget. About 214 more staff will be needed for expanded program operations, several more for OERI's own personnel and procurement offices, and whatever additional number is required to provide support services for the enlarged staff (accounting and payroll, supplies, mail, computer support,

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement etc.). Some of the additional staff will have considerable liaison or monitoring responsibilities, which require more travel and communications costs than those for other employees; therefore, the OERI staff and expenses budget will need to be increased somewhat more than would be expected by the number of added staff. Several of the above recommendations could be implemented within 1 year; others will have to be phased in over 3–6 years. The new governance, agenda setting, and organizational changes should be implemented soon. Funding of the new multiyear grant programs should be phased in over the number of years for which the grants are to be awarded. A large portion of the additional staff should be brought on board as quickly as practical to meet the critical needs of NCES and the forthcoming needs of the reorganization. The committee recognizes that its funding recommendations would require a large expenditure. Some people will simply dismiss it as too expensive. The committee sees it as a critical investment in the nation's future. Without the investment, and concomitant efforts at state and local levels, the country is not likely to come close to meeting the national education goals. Over the first 6 years, our recommendations will cost the nation approximately $1.3 billion in additional expenditures. Over the same period, the nation will spend about $1,500 billion on elementary and secondary education in this country. It is clear that this added investment in R&D will be paid back many times over if it improves the effectiveness or efficiency of our education system by even 1 percent. It also should be noted that even with full implementation of all our recommendations, federal investment in education R&D will still be significantly less than federal investment in agriculture, transportation, or health. Our committee cannot address the future of OERI much beyond the next 5 or 6 years. Recommendations A-4, A-5, and A-6 establish a mechanism for monitoring the health and needs of OERI's R&D and that of the entire federal government. This monitoring should be supplemented with a strategic planning process that anticipates the needs for education R&D 5, 10, and 15 years into the future and develops a strategy for effectively meeting those needs. It should be noted that any research enterprise is much like an airplane accelerating down a runway. In the early stages, there is much noise and vibration, but the plane just rolls along the tarmac. Only when it has gained enough speed do the aerodynamic forces exceed the gravitational ones, allowing the plane to lift off. The committee believes that its recommendations, if implemented, will accelerate the education R&D enterprise in this country, but lift-off, moving the enterprise into a new mode of functioning, is several years away.

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement D-2 Unless OERI's budget is substantially increased in the near future, the mission and activities of the agency should be significantly narrowed. Some of our recommendations require little, if any, additional funding. Given the federal budget difficulties, there will be the temptation to implement those and postpone the others requiring larger expenditures. We believe that would be a serious mistake. The past and present strategy of sprinkling NIE's and OERI's limited funds across an enormous array of disparate activities has not proven to be an effective national strategy for education R&D. Basic research has almost dried up. The sustained efforts that are so important for progress in science and technology have been limited. Promising innovations are disseminated without the iterations of R&D that are needed to maximize results and minimize costs across diverse situations, and without evaluation of their long-term effects. Important successes have been achieved, but for every accomplishment of OERI and NIE, teachers, researchers, and policy makers can cite at least ten unfulfilled needs that are fully within the agency's mission. And even if all the problems in organization and functioning identified in this report are remedied, the agency could not possibly fulfill its responsibilities with its current funding levels. There are at least two alternatives for paring back the breadth of the agency's activities. First, OERI could focus exclusively on dissemination and technical assistance activities. Some people suggest that the biggest bang for the buck, at least in terms of school improvement in the short term, would come from putting the existing research knowledge base into wider use through enhanced dissemination and technical assistance. This approach, however is like eating one's seed corn, and would become self-defeating within a few years. Second, the agency could invest the entire amount of its limited funds in research and statistics and a few highly targeted development activities, eliminating all dissemination and technical assistance. Several programs of high-quality research can be pursued at much less expense than widespread programs of dissemination and improvement. As these programs bear fruit, there would be increased motivation for dissemination, and perhaps the states or for-profit companies would move to meet the need, although history does not suggest this is very likely. On several occasions OERI and NIE have tried to concentrate their limited resources, but they have always come under heavy pressure to do the opposite—from researchers of all the excluded fields of inquiry, from curriculum associations of all the excluded subject areas, and from policy makers concerned with all the excluded issues. Only an act of Congress

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement would be likely to ensure a narrowing of focus. And such legislation would alienate so many participants in education that the travails of the agency might well worsen. We emphasize that neither of these reductions to OERI's activities is consistent with the widespread, pressing, and diverse needs for education reform. They are not in the best interest of the agency, the federal government, or the nation's education system. Yet it would be more honest to drastically cut OERI's mission than to pretend that it can fulfill that mission without significant increases in funding. CONCLUSION OERI needs to be rebuilt for its complex and difficult mission. The agency has made some valuable contributions, but it has been seriously compromised by governance, organizational, operational, and funding difficulties. Our study has led us to conclude that the agency can best fulfill its mission by strengthening its traditional roles and by supporting a research enterprise that is a powerful and enduring partner for learning communities—communities comprised of researchers, teachers, administrators, policy makers and parents. School reform is rarely achieved by mechanically plugging in innovations. Rather it is a developmental and systemic process that requires new learning on the part of both researchers, practitioners, and policy makers. We recommend major changes in the governance of OERI to improve the agenda setting of the agency, balance its portfolio of work, and bring stability to the top management. Our recommendations for reorganization would expand, focus, and integrate much of the research and development work; strengthen PEP and NDN; correct a critical shortage of staff in NCES; coordinate, extend and supplement the various dissemination and technical assistance efforts; strengthen quality assurance; and initiate efforts to encourage and foster the above mentioned learning communities. Our recommendations about operations would give OERI more control over staffing, contracts, grants and reporting; strengthen its peer review process; and enhance the quality of the staff. We think these changes will make a difference, an important difference, if they are sustained for a substantial period of time. The story of OERI and its predecessor does not offer much hope for implementation and maintenance of these changes, but conditions are changing. Never before has education received so much attention simultaneously from the administration, Congress, and the governors. Despite the differences of opinion on many issues, there is widespread agreement that business cannot continue as usual in most of the nation's schools, that more than marginal improve-

OCR for page 135
Research and Education Reform: Roles for the Office of Educational Research and Improvement ments are needed, and that the nation's diverse resources must be mobilized to meet this challenge. Our recommendations for OERI are in full accord with that agreement. Our recommendations will not be easy to implement, they will not be painless, and they will not yield immediate results. But we think they are a wise long-term investment in the future of the country.