Improving Student Learning: A Strategic Plan for Education Research and Its Utilization (1999)

Designing a strategic research program for education is a difficult task. Improving the contribution of knowledge to education practices requires more than is encompassed in research-to-practice formulas as traditionally conceived. The challenge is understanding how to make improvements, understanding when knowledge can contribute to education. Moreover, the knowledge needed when it comes to improving social systems like education is not only the product of those who identify themselves as researchers; important kinds of knowledge also come from the halls of policy and from school teachers and administrators. Yet the kind of collaborative creativity that is the key to solving most complex problems—including that of improving student learning—is as difficult to effect among these separate professional cultures as it is essential (Pressman and Wildavsky, 1973; McLaughlin, 1991b; McDonnell and Elmore, 1991).

This strategic plan envisions building a collaborative effort focused on a small set of very important problems to improve the contribution of scientific knowledge to education. It will require participants with diverse expertise and viewpoints to develop a workable common language and a shared agenda. A program of research that aims to be strategic cannot apply the model of individual, field-initiated research projects that has characterized education research in recent decades: it must be much more focused and coherent. At the same time, the directed, highly specified approach to research that has been so effective in military research and development (R&D) and certain other high-technology fields is not well suited to this task—the research effort must be more flexible, more able to learn incrementally.

The National Institutes of Health (NIH) research program offers a more promising model: by drawing on the two research traditions, it combines central program direction designed to advance public health with robust field-initiated project development. In doing so, NIH has embraced an approach to research that has advanced fundamental understanding of biomedical processes (e.g., the mechanisms that transform healthy cells into malignant ones), while also being responsive to public calls for better medical treatments—the so-called war on cancer (Stokes, 1997:137–138). But there are important aspects of medical research that reduce the salience of the NIH model to education. Perhaps most striking is that there are no equivalents of pharmaceutical companies to support education research. And an educational intervention is not like a drug or a serum: in education, even when you have a promising intervention, it has to be incorporated into a highly complex social system.

Making research more useful for education requires a new approach, combining the best features of the two research traditions and crafted to suit the nature of the complex social organism called the education system. We believe that this new approach has six crucial features:

Given the decentralized character of education and the diffuseness of the federal research effort, the problem is how to get sustained attention over a decade or more by the best researchers to a program that is attuned to the needs of the policy and practice communities. The committee considered a range of possible mechanisms, but ultimately found the MacArthur networks of greatest interest. Envisioned as an "experiment in scientific organization," the MacArthur Foundation's Program on Mental Health and Human Development has been in existence for nearly two decades and has created 13 interdisciplinary networks (supported with long-term funding) devoted to addressing key challenges in the fields of mental and physical development and health (Kahn, 1993; Bevan, 1989; Prager and Kahn, 1994).¹

At the core of the MacArthur model is the insight that complex social problems can only be effectively addressed through interdisciplinary research. Yet both academic structures and incentive systems and federal funding mechanisms militate against sustained investigations that transcend disciplinary boundaries. The networks were devised as a way to promote effective collaboration across the biological, behavioral, and social sciences, to encourage scientists from many disciplines to make common cause to understand complex social processes and to translate that understanding into practical benefits (Kahn, 1993:iii).

Foundation funds are used primarily to facilitate communication and collaboration among the network members (technology, meetings, etc.), as well as some collaborative research projects. The foundation does not typically support projects being conducted by individual members of the teams, but it does fund some big network projects. In the case of the Successful Midlife Development Network, for example, the foundation provided funding for a major survey that has generated a wealth of data for all members of the network (and others) to exploit.

The 13 MacArthur networks evolved over the years as the foundation learned from experience. Some were more successful than others; some endured while others did not; some

became the springboard for rich interdisciplinary collaborations while the influence of others was transitory. A great deal was learned about intellectual collaboration and about the elements of success. And there is evidence that the networks have produced an array of innovative methods, significant data sets, and important findings. Perhaps most important, network members describe the excitement and creativity unleashed by collaborations across areas of expertise, which challenge the old assumptions and mental ploys that often hamper innovative research.

The plan that follows is obviously deeply indebted to the MacArthur networks, but there are some differences worth noting. The MacArthur networks had a strong basic research orientation, although the foundation was also deeply concerned about applications. The SERP plan would reverse the emphasis. And in an important sense the SERP idea is more ambitious—and risky—for it expands the goal of collaboration beyond scientific disciplines to include the expertise of policy and practice. If successful interdisciplinary collaboration requires researchers to adapt to one another's professional culture, intellectual traditions, and analytical methods—and the MacArthur experience shows that this is not easy, then how much more challenging it will be to create successful collaborations of scientists, practitioners, and policy makers. It is, nevertheless, essential if the potential of education research to improve practice is to be realized.

A final point for consideration has to do with the balance of research support. The MacArthur program was a high-leverage design that assumed that network members would have other sources of support for their individual research. SERP would not be able to depend on all network members, for example, practitioners, having independent support for their time spent on SERP activities. Nevertheless, SERP is designed to leverage existing investments in research. It would aim, for example, to draw in the technology centers of the National Science Foundation (NSF) and the regional laboratories and R&D centers of the Office of Education Research and Improvement (OERI) of the U.S. Department of Education as partners in advancing the work of the SERP networks.

We see the SERP mission carried out by four networks of expert and committed education researchers, working in collaboration with practitioners and policy makers. Their mandate would be both to pursue research and to develop strategies for getting the best available knowledge used in everyday education practice, all with the goal of improving student learning. Each network will address one of the four hub questions:

• The Learning and Instruction Network addresses the question: How can advances in research on human cognition, development, and learning be incorporated into educational practice?
• The Student Motivation Network addresses the question: How can student engagement in the learning process and motivation to achieve in school be increased?
• The Transforming Schools Network addresses the question: How can schools and school districts be transformed into organizations that have the capacity to continuously improve their practices?
• The Utilization Network addresses the question: How can the use of research be increased in school and school districts?

The committee accorded a special status to the utilization network. While all of the networks will be committed to promoting and studying the use of research findings relevant to the hub questions, this network will try to develop general principles for theory and practical guidance on how to remove barriers and facilitate the use of research knowledge in education practice. It will work closely with one or more of the other three networks on particular projects, using them as cases for learning and experimentation. Also important is for all four networks to confront the questions and serious problems that arise in attempting to implement research's best ideas in schools serving poor or underachieving students. If SERP can figure out how to substantially improve student learning in schools with the most problems, it will be the key to getting every teacher, school, and school district interested in using research.

Each network would have a director who devotes half time to the network. The director should be chosen on the basis of ability to address the network's hub question and skill at leading a team with members from diverse backgrounds. The commitment and leadership of the directors will probably be the single most important factor in making SERP work. The core membership in the networks would range from 7 to 15 people from different institutions and from different disciplines and professions. It is essential that the network members be creative, committed, and productive people, open to the possibilities of collaborative work and ready, as Kahn (1993:19) put it, to ''make a sustained effort to link their work and their ideas to those of others.''

At the outset of SERP, each network director would work with a governing board of the whole to develop a general 12-year plan. The keystone of each network plan would be a synthesis of the state of knowledge and the extent of knowledge utilization within its domain.

The MacArthur experience suggests that the members of each network would need to meet as a group at least four times a year. The network as a whole would work together to design and evaluate projects that advance the network objectives, solving problems and maximizing the chances for impact on the network's hub questions. Between network meetings, the members would communicate regularly by telephone and computer communications and work as needed in subgroups with task forces on specific projects.

Each network would be responsible for carrying out three key functions: (1) assessing the state of knowledge and the extent of knowledge utilization within its domain; (2) expanding that knowledge base by undertaking and commissioning research, with an emphasis on "usable" research; and (3) increasing knowledge utilization in districts and schools, in collaboration with the utilization network.

Each network would produce a strategic plan for achieving its mission and carrying out its key functions. An important starting point would be a critical synthesis of existing research,

practice, and policy relevant to the network's hub question. A basic version would be developed during the start-up period for the network. A part of each network meeting and each network activity would be spent revising that synthesis through the incorporation of new developments from the network's activities, the interlocking networks of SERP, and the field at large.

An essential design element of SERP is that it is cumulative in nature—that new work takes advantage of and builds on earlier findings. The ongoing synthesis represents a concrete expression of that cumulation of understanding. The syntheses will identify and analyze:

• Strengths of the research base: What is known? With what level of confidence?
• Gaps in knowledge: Cases of problems that have been well documented (by practitioners, policy makers, or researchers) for which the existing knowledge base provides few workable solutions.
• Unused and underused knowledge: Cases where there is authoritative professional consensus on research findings or best practices, but applications have not yet been developed or are not as widespread as they should be.
• Incipient knowledge in the field: Cases where there is progress toward consensus on research findings or best practices, but further development and research are needed before applications can be confidently promoted.
• Productive conflicts in the field: Cases where the development of data, instruments, or theory would resolve conflicting claims and permit research and development to proceed.
• Barriers to and facilitators of utilization: The social, institutional, and individual factors that can work either as barriers or facilitators of the use of research findings. Each network will track specific cases concerning its hub question but will also plan joint work with the utilization network to identify and address potential barriers or facilitators.
• Priorities for the network: On the basis of the analysis described in the first five parts of the synthesis, agreement on the areas of greatest need and greatest opportunity for making progress on the hub question. Planned objectives for the network will be stated.

Each year's synthesis is a crucial step in the strategic planning process, but is not an end in itself. Based on this synthesis, each network plan would identify the areas of greatest need and greatest scientific opportunity and develop activities (both research and implementation) that have the greatest potential for making progress on the hub question. Planned objectives could be organized in a nested set with very specific yearly objectives for the first 4-year phase and more general ones for later phases.

Each network would undertake to meet the objectives identified in its strategic plan. Working within and across networks, the members would carry out a variety of activities.

Quarterly network meetings will serve as forums, where core members would present, discuss, and assess research, policies, and programs related to the network's hub question. These meetings would also serve as planning sessions for activities aimed at expanding knowledge or increasing its utilization on the hub question.

Each network will carry out research activities in its domain, consistent with the mission of SERP. This work might be done solely by network members or, more likely, by task forces headed by at least two members of a network with others from outside of network. The quality of the design and the results would be the responsibility of the sponsoring network. Undertakings might include, for example, reanalyzing existing data sets to shape a new hypothesis; replicating or expanding an effective but limited study; or conducting a study that tests a hypothesis under different conditions. Ideally, SERP would offer the opportunity and funding for centralized large-scale data collections. All network research would be expected to look beyond the traditional disciplinary boundaries and to take advantage of the full array of research fields, findings, and methods relevant to each hub question. Network research would also be expected to be informed by the needs and perspectives of the users and to meet accepted standards of excellence.

Task forces convened by networks could carry out activities aimed at enhancing instructional prac-

tice in K-12, consistent with the SERP mission. These might include, for example, running brief pilot studies to gauge the viability of a proposed multiyear project; coordinating with developers and practitioners to develop or evaluate promising materials; or sponsoring a design competition to produce several good solutions for well-defined problems in schools. All network activities and products will be expected to meet rigorous standards of excellence within the disciplines and professions.

Some projects will be directed and funded by the network from start to finish. Other projects, however, could be carried out as a part of the ongoing activities of a core network member (funded by another private or public agency) or another research group or program. In such cases, the network might add components to planned or ongoing work, increasing the value of projects outside the network while fulfilling network objectives. For example, the network might support a set of questions piggy-backed on an existing survey; it might enhance a study by providing longitudinal evidence on a specific finding; or it might extend a study to test it under varying conditions. By enhancing existing projects, networks would cement relations with others in the field while making the most of their expertise and resources. This is one way in which SERP would operate on a wider scale or for a longer period than it could on its own.

The networks would be encouraged to use electronic and print media in an aggressive dissemination program. Publication of the annual synthesis from each network and, at the end of each 4-year period, the collected synthesis documents would be part of the dissemination effort. A SERP congress would be held periodically for network members to discuss their progress and findings with leading members of the research, practitioner, and policy communities. Congress proceedings and specialized materials could be prepared for targeted audiences, to translate network findings and recommendations into guidance for users. Network members would contribute articles based on network projects to professional journals and make presentations at conferences. Technology offers important opportunities, and the growing popularity of the Internet suggests the possibility of creating not just web pages devoted to network activities, but also databases for educators (theory-based curricular materials)

and students (learning activities designed to promote deep understanding).

Each network would undertake activities aimed at building capacity within the field to further education research and its increased utilization by districts and schools. For example, networks might establish task forces to develop human resources, data banks, or survey instruments intended to extend beyond the life span of SERP. These might include:

• workshops cosponsored by professional associations (e.g., American Federation of Teachers, the National Education Association, the International Reading Association, the National Association of Elementary School Principals, the Council of Chief State School Officers, the National Governor's Association);
• summer institutes cosponsored with universities and state departments of education;
• preprofessional courses offered through teacher education institutions;
• professional development sequences cosponsored with local school districts and state departments of education;
• mid-career fellowships to expand the capabilities of researchers, practitioners, and policy makers by immersion in the activities of the networks;
• data banks useful for researchers, practitioners, and policy makers who want to address questions about cognitive development, learning, motivation, etc.; and
• instruments to measure learning that would be sensitive to changes in student performance and causally related to changes in instructional situations.

The menu of network activities outlined above is very ambitious. It is important to reiterate that the intention is not to replace other education research and reform programs, but to strengthen the contribution of research to education by providing a powerful focus for related activities and by finding unrealized synergies. The SERP networks are not expected to be primarily a conduit for resources, but rather, a vehicle for focusing the attention and energies of the research community, sponsors, practitioners, and other stakeholders. For example, the annual syntheses—and the discussions they generate—could

help weave together the programs of many different sponsoring agencies. The interdisciplinary character of the network activities promises to produce innovative approaches, new ways of looking at education and learning that can be incorporated by others and, if the MacArthur experience holds true, these collaborations will unleash creativity and an excitement that could spread far beyond SERP.

If it is not intended to replace other education research and reform programs, a successful SERP could nonetheless have a profound influence on research and practice. Its focus on four questions studied in national networks over 15 years would affect the organization of knowledge in education. This has implications for such things as graduate training, academic publishing, the allocation of new faculty positions. Furthermore, by building better bridges between the practitioner and research communities, SERP could illuminate the most productive avenues for investments in use-inspired research, and thus affect the research priorities of funding organizations.

We make no attempt in this document to say where the governance of the Strategic Education Research Program should reside, or even to say whether it should be public or private or some combination of the two; there are too many potential participants and possible locations for that to be sensible at this stage. But it is an issue of tremendous importance to the success of SERP. Could government provide leadership for a long-term, strategic research program? Would an agency like the Department of Education's Office of Educational Research and Improvement, which would seem a logical place to lodge such a program, have the political continuity or indeed the authority to enter into the kinds of agreements needed to sustain a 15-year effort? Would some sort of interagency partnership be feasible? Would a federal-state partnership offer advantages? Do the federal procurement regulations afford the flexibility needed to empower the thoughtful research managers so vital to the success of SERP? Would a consortium of foundations be a more likely alternative? Would leading foundations be willing to give up some of their autonomy to join forces in a strategic

program. What are the serious bottlenecks likely to exist in the relevant public and private institutions, and can they be overcome? Could a public-private partnership be constructed that would enable the sponsoring institutions to make the most of the SERP idea.

All of these are questions for another day. Nevertheless, there must be a host organization, and we have given thought to the characteristics that it will need to embody.

In order to assure continuity, focus, coordination, mutual reinforcement, and quality control, SERP will need to be carried out under the auspices of a host organization. The host will need to be nationwide in scope, to have the capacity and experience to work over time with multidisciplinary groups, and to influence the dispersed world of education. Also, it will need to be prepared to make the connections between research, policy, and practice. The host organization will need to have the status, structure, and longevity needed to initiate SERP and to administer, monitor, and maintain it for a sustained period; it must also be able to provide the corporate structure needed for legal, fiscal, human resource, development, and public interface functions.

The nature of the host organization will have implications for the location of the networks and their relation to the host. It might be in a position to manage the networks in-house, or it could make sense to place their management in other institutions, for example, the director's university, or a research organization, or an entity like the OERI centers. The entire question of governance requires more thought and discussion by the interested parties, with particular attention to arrangements that will promote the coherence of the overall program and its ability to build cumulatively on the ongoing work.

To maintain the strategic nature of SERP, the host organization will need to establish a governing board to function as a policy board and also have coalition-building and development responsibilities. The governing board we envision would be capable of sophisticated and sustained interface with constitu-

encies, stakeholders, and decision makers in education. We believe the governing board would need to have at least 16 members, drawn in roughly equal proportion from four categories:

• policy makers—from local, state, and federal governments, and from nongovernment organizations (businesses, professional organizations) that support improvement in education;
• practitioners—teachers, school-level administrators, and district-level staff;
• researchers—whose work is relevant the hub questions; and
• fundraisers—leaders in foundation, business, government, and other spheres who would shape the campaign to generate financial support for SERP.

The Governing Board would set overall policy for the Strategic Education Research Program. Among its policy activities, the board would:

• develop and refine the SERP mission and mandate;
• establish criteria for selecting and evaluating network personnel, select network directors;
• develop criteria for reviewing proposals and reports, upper and lower limits for the duration of networks, the number of network members and amount of honoraria for each, the percent of salaried time for the network directors and central staff, and the proportion of expenditures for administrative and substantive aspects of the program;
• assess the progress of the overall program at specified intervals, using independent reviews as needed, and correcting its course as needed; and
• promote strong linkages among the networks to assure the overall coherence of SERP.

The governing board would work with each of the networks in three ways: appointing the network director, reviewing each network's plans and allocating financial support, and assessing progress for each network. Annually, each network would provide the board with a sub-

stantive written and financial report and a plan for the following year's activity. Each network would also contribute to the board's 4-year reviews of SERP as a whole; these reports will be the building blocks for a quadrennial SERP congress.

In defining the relationship between the networks and the host organization, a careful balance needs to be struck between flexibility and coherence. The governing board and its staff would need to promote coherence and integration of the network activities so that together they advance the SERP mission.

The governing board would convene a national congress every 4 years, inviting the various stakeholders in education to participate. Each of the four SERP networks would present an updated report on the state of the art and conditions of knowledge use related to its hub question. Formal comment and discussion of these reports would be prepared by other network directors, board members, and key scholars outside of the networks. Following critical discussion at the congress, the board would prepare a response to the proceedings, charting new plans as needed for the subsequent periods of SERP's operation.

The quadrennial congresses would be a useful device for focusing attention on the most promising advances in research and development. The board might issue a series of reports addressed to diverse audiences for SERP—practitioners, policy makers, and researchers—ensuring, in particular, that policy makers and practitioners have the concrete detail and motivation to carry out changes that can improve student learning.

A central undertaking of the governing board would be to generate the financial support and political will to carry out the Strategic Education Research Program. Both the size and the nature of SERP make it desirable to build a coalition of funders, spread among the parties who have a stake in the fulfillment of its mission. The federal government has long supported education research and would be an important member of the coalition. There are also a number of foundations, old and new, that have a strong interest in education and numerous business organizations concerned about the quality of tomorrow's workforce.

The states have, by and large, not had a strong tradition of funding education research. A high priority of the SERP governing board should be to involve each of the 50 states and the District of Columbia so that state and local policy makers and practitioners would be customers and shareholders in the program. This would, over time, cement a circle of supply and demand with respect to getting good research used in education.

The success of a Strategic Education Research Program is predicated on the willingness of political leaders and the public to make a long-term commitment to it. The committee strongly recommends that, if the program is implemented, it be done with the expectation that SERP will continue for at least 15 years—2 preparatory years, 12 years of operation for four networks, and a final year devoted to completing projects and ensuring that the impetus continues for using the best knowledge in education practice. (See timeline on page 65.) Research initiatives in education have tended to be so abbreviated that the planned time frame may seem long, but it is important to keep in mind that it is less than the time that it takes for one cohort of children to graduate from high school.

During the 12 years that the networks would operate, reviews would take place at 4-year intervals. The host organization and the SERP board would take advantage of the periodic reviews to make informed decisions about continuing or correcting the course of each network. These reviews would ensure accountability and allow plans for the next period to be refined.

In the preface to this report, National Academy of Sciences' President Bruce Alberts conveys his enthusiasm about the potential of the SERP idea and his hope that it will catalyze major new investments in educational research—both by federal and state governments and by foundations. As a first step in that direction, the National Academies propose to launch a year-

long national dialogue during which the idea for a Strategic Education Research Program is discussed with all the professional groups concerned with education. The feasibility of the plan needs to be widely discussed. The general design features suggested in this report need to be forged into workable specifications for a large-scale, long-term research and development program. Above all, a year of dialogue is needed to see if this plan can generate the kind of political will and financial commitment that will be needed for its operation.

We call upon the federal government, and in particular the Department of Education and the National Science Foundation, major foundations whose mission includes improving education, state and local education leaders, and education research organizations to join in this year of dialogue to see if, together, we can transform the SERP idea into a productive collaboration to use the power of science to improve education in the United States.