Enhancing Professional Development
The need for a diverse pool of well-trained education researchers to generate high-quality scholarship and to lend their expertise to deliberations about educational practice and policy is great. As in any other professional endeavor, attention to ways in which education researchers are trained and provided with opportunities for continued growth over the course of their careers is essential. We focus on a few such opportunities: doctoral training in schools of education, and peer review panels for judging both education research proposals (submitted to federal agencies), and products (articles submitted to journals for publication).
NATURE OF THE FIELD
Approaching the crucial topic of how to frame and develop high-quality professional development for education researchers requires an appreciation of the diversity of the field and the kinds of students who pursue advanced training. Education researchers come from virtually every discipline and from many interdisciplinary specialties, reside in a great variety of institutions—schools of education, university arts and science departments, and now even policy schools and schools of management, think tanks, and school systems—and focus on an enormous array of research topics.
Furthermore, novices in education research usually enter the field relatively late—beginning academic study of education research only when they become doctoral students. Many have previously studied to be teach-
ers and gained considerable teaching experience, but they have no prior training or experience conducting research. For some, their undergraduate and master’s programs were in content areas (e.g., mathematics, English, history, biology). For others, undergraduate and master’s work may have been in one of the social science disciplines. Still others completed master’s degrees oriented toward the skills of practice in teaching, counseling, or school administration, with limited attention to the conduct of research.
Available data suggest that there are 1,000 Ph.D.-level degrees awarded in education research each year (Levine, 2003). Compared with the production of Ph.D.s in social and behavioral sciences fields, this number is quite large. However, the real issue may not be one of quantity, but one of quality: Are current and prospective education researchers capable of tackling the complex questions that policy makers and practitioners want answers to? And more to the point: Are existing training and professional development activities producing a capable cadre of investigators?
MECHANISMS FOR ENHANCING PROFESSIONAL DEVELOPMENT
These are tough questions without simple answers. Our work only scratched the surface but nonetheless resulted in several promising suggestions for positive change. In this chapter, we focus on three leverage points: schools of education, peer review processes in federal agencies, and the policies and practices of publishers, especially peer-reviewed journals.
Schools of Education
Although education researchers pursue advanced training in a variety of institutions and university departments, we chose to focus our recommendations on the role of doctoral programs in schools of education because of their central role in preparing the next generation of leaders in education research. Our focus is on those schools of education that prepare researchers (many do not, their sole purpose being to provide advanced training for educators and administrators), recognizing that even among this smaller set of schools, there are substantial differences in the origins and nature of schools of education across the country.
As we have said, the kinds of students who choose to enroll in doctoral programs in schools of education vary dramatically. Because there are so many pathways for pursuing academic preparation in education research,
recruitment strategies at doctoral programs in schools of education must reach out to students with a range of academic and professional backgrounds to tap the available pool of talent. This necessity means that groups of future education researchers who embark on doctoral training in schools of education have a wide range of skills, experiences, and world views to bring to their work—a real asset to a field as complex as education research. Schools of education can and should recruit students from a range of social, racial, and economic backgrounds as well (Hancock, 2003). This diversity can strengthen the quality of research in the field by effectively tapping a broad pool of talent and by promoting research that is relevant to a range of education issues and student populations.
When a diverse group of students comes together in a doctoral program, however, they do not share previous experiences, a common language, or norms regarding the value and conduct of education research. Doctoral programs in schools of education play a particularly crucial role in helping to define and instill common principles and habits of mind that will grow with them as they pursue their careers in education research and contribute to the knowledge base (Labaree, 2003).
Still, doctoral training of education researchers and the challenges associated with it are at their core very similar across social and behavioral science disciplines, as well as professional fields like social work, business, and nursing. For example, scholars and educators in social, behavioral, and economic sciences are increasingly aware of the need for advanced skills and methodological tools to address the vexing problems facing society, not unlike education research. In these disciplines there is also appreciation that training requires enhanced, interdisciplinary integration across related fields, again, quite similar to issues facing education research trainers. There are also similarities between education research training and training in other professional fields. Social work, for example, is much more akin to education in aspiring to build conceptual and methodological scientific skill, and many of the current concerns in the social work field about doctoral training have parallels to education research. Further, the ongoing tension in doctoral programs in social work between the objective of preparing high-quality researchers to go to top schools and the objective of training faculty with broad knowledge and excellent teaching skills, who can work in social work programs ranging from very large to very small, is present in doctoral programs housed in schools of education. Furthermore, while some doctoral programs in social work focus on clinical practice,
more focus on research, and some are trying to make the transition to a research focus or create a balance between the two (Levine, 2003).
Mindful of these similarities and differences, we adopt the premise of the Carnegie Initiative on the Doctorate (see Box 4-1) as an organizing idea for our recommendations regarding doctoral programs for education researchers in schools of education. This initiative is supporting reforms in doctoral programs in a range of fields, including education, with the overarching goal of using doctoral training to develop a professional community with shared norms, language, and ways of knowing. Doctoral training can and should be a mechanism for instilling common habits of mind—not rote standardization, but a sense of purpose around which research and teaching can be framed.
Recommendation 9: Schools of education that train doctoral students for careers in education research should articulate the competencies their research graduates should know and be able to do and design their programs to enable students to develop them.
Schools of education did not begin life as research centers. Rather their origins lie in 19th century normal schools, which were designed to provide roughly one additional year of schooling to would-be teachers.1 Subsequently, as normal schools evolved into teacher’s colleges and as research universities added schools or departments of education, research was undertaken either as a basis for generating a curriculum or as outreach to local schools. As a result, research that is congruent with the principles of science is a very recent arrival in schools of education. Developing earlier in some fields—notably psychology—scientific research became generally important in schools of education only in the 1960s, when the marriage of education studies and social science disciplines—history, philosophy, anthropology, psychology, sociology, and economics—established the importance of rigorous methods (Lagemann, 2000).
Not surprisingly, given this situation, no two schools of education prepare researchers in exactly the same way. Some schools award Ed.D. degrees; others give Ph.D.s; others give both. (Indeed, it is not even clear what the differences between the two degrees are, as their purposes and require-
Lee Shulman emphasized the idea that while doctoral training in education research is different in important ways from other fields, lessons can be learned from exploring the experiences of other institutions and departments. Echoing other speakers’ remarks about the striking similarities between education research training issues and those in other fields, Shulman offered the example of recent discussions about doctoral training in mathematics, in which leading scholars identified as a core problem the development of an understanding of the boundaries between mathematics as a discipline and mathematics as a profession; education is similarly vexed. He also pointed to law schools—which, like education schools—draw a student body with a diverse set of educational backgrounds, yet have common courses and other similarities that cut across the first year of their doctoral programs, providing a common basis for their advanced training and professional lives—unlike education schools. He further suggested that in some ways, reform of doctoral education in education schools is far ahead of what it is in other sciences to which education research is often derisively compared.
Shulman described some of the efforts to foster the improvement of doctoral training that he has spearheaded in the Carnegie Initiative on the Doctorate (see http://www.carnegiefoundation.org/CID/). The initiative endeavors to enrich and invigorate doctoral education through a multi-year research program. Envisioned is the idea of preparing “stewards of the disciplines”—individuals who can generate new knowledge and defend against challenges and criti
ments vary substantially across universities.) Some schools encourage research on teaching practice; others focus on policy research. Some schools require training in statistics; others do not. Some require experience collecting data in the field; others do not. Some require participation in supervised research and preparation of a scholarly publication prior to graduation; others do not. As currently constituted, such doctoral programs do little to ensure that the preparation of education researchers meets at least a minimal standard of scholarly rigor.
The workshop on doctoral programs at education schools illustrated the importance of articulating a common core of education that can shape
cisms while conserving important ideas and findings from past work in their disciplines. Such individuals will be able to transform knowledge into “powerful pedagogies of engagement, understanding, and application” (Golde and Walker, 2002).
The initiative has been explicitly designed to promote cross-disciplinary and cross-departmental thinking and learning in the pursuit of improving doctoral training in the areas of English, chemistry, mathematics, history, neuroscience, and education. It has provided financial support for 50 partner departments at colleges and universities across the United States to conduct “design experiments” in which they examine the purposes and desired outcomes of their doctoral programs and consider what changes to their programs are needed to better achieve outcome goals, including fostering stewardship of the discipline. The initiative has also convened discipline-specific conferences where representatives from the partner departments can discuss and learn from one another’s efforts. Several products will result from these efforts, including models of experimental doctoral programs, research and analysis of the sponsored efforts, and institutional and policy-level recommendations.
the development of advanced research training. What are the core courses every education researcher ought to have to earn their doctorate? Without an articulation of what graduates need to know, there is no reference point from which to design course sequences. How should research apprenticeships and experiences in these core competencies be structured? It depends on what it is you want students to gain from those experiences. Lacking direction on the goals, mapping out roles and responsibilities across organizational lines is problematic.
Specifying the competencies that every graduate of doctoral programs in schools of education—that is, the future leaders in education research—
should have is critical. Not only would such an exercise provide an essential lens for developing and implementing training programs, it would result in an articulation of a minimum breadth of skills and experiences all education researchers should have. Such an articulation is important because the field is so diverse and because its participants need to recognize the role, value, and points of convergence across a range of theoretical ideas, epistemologies, and methods.
It is likely that many schools of education would need to articulate these competencies at the level of the program—for example, an educational psychology program that trains education researchers would be likely to articulate some different goals for its doctoral students than a similar program in a curriculum and instruction department. However, there also may be some goals that schools have for all of their students, as is the case in the School of Education at the University of Colorado, Boulder (see Box 4-2). In either case, schools or programs would articulate what skills and experiences they want students to have when they complete their doctoral training and then design the curriculum, apprenticeships, mentoring, and research experiences needed to develop that base of substantive and methodological knowledge.
At the workshop we learned about some promising initiatives by administrators and schools across the country working to find creative ways of approaching this issue. With careful planning, study, and leadership, the potential is great.
Recommendation 10: Schools of education that train doctoral students for careers in education research should design their programs to enable research students to develop deep substantive and methodological knowledge and skill in a specialized area.
In addition to facilitating a breadth of knowledge in education research among doctoral candidates, schools of education should also provide opportunities for students to develop a depth of expertise in selected subfields of education research. As students progress through their doctoral training, their course work and research experiences should hone their skills and understanding in the theoretical ideas, methodological tools, and existing research in the subfield they intend to pursue over the course of their careers.
It is in the development of deep expertise in subfields of education research that collaborations with disciplinary departments and other orga-
Margaret Eisenhart described a recent effort at the University of Colorado that she led to develop a core curriculum for their doctoral students studying to become education researchers. She and her colleagues embarked on this work after the faculty agreed that the existing doctoral program was not adequately preparing students to conduct high-quality education research. To inform development of a new program, Eisenhart and a faculty committee examined basic data on leading education schools, including the content and number of required courses. They found that beyond one or two required methods courses, there was very little that was common across or even within most schools. The committee found this lack of a common set of courses especially problematic given the nature of their graduate students: many are former teachers, and many lack the kind of undergraduate preparation that is typical in discipline-based research trajectories. Furthermore, students without research training also sometimes approached their graduate work with a deep skepticism of research and its utility for helping to solve problems in education.
Eisenhart and her colleagues designed a set of core courses for their doctoral students in light of these facts: they explicitly sought to develop a common language, to convey and discuss a shared set of issues and arguments in the field, and to instill common norms and standards for the conduct of research. Eisenhart described several challenges in designing their curriculum, including the difficulty of balancing breadth and depth: the faculty believed that students should have a general understanding of the field of education and education research, but also develop deep expertise in a substantive area and a set of research methodologies.
The process that Eisenhart led at Boulder was supported by students but complicated by some faculty resistance. She told the group that many faculty were wary of change for several reasons, including tradition, the burden of developing and teaching new courses, and fears that those specialty areas represented in the core curriculum would have disproportional influence and control over all students’ thinking in the early stages of their training.
The faculty involved in this effort are also focused on ensuring that all students, regardless of their backgrounds prior to beginning
doctoral work, have access to both teaching and research experiences as part of the program. It is relatively easy to assign former teachers to teaching assistantships, and it is easy for students with research training to be included in faculty research grants. It is less easy but crucial to the success of their program to cross those lines, according to Eisenhart.
The faculty committee that spearheaded the new program worked hard to overcome these obstacles.a One strategy was to assemble a broadly representative planning committee and to offer multiple opportunities for faculty updates and input on the committee’s work. The planning committee also received strong backing from the dean and the help of the Carnegie Initiative on the Doctorate, a national effort to consider reforms in graduate research training across a range of disciplines (see Box 4-1).
After a year and half of work, Eisenhart and her colleagues received approval from the full faculty for a three-year doctoral curriculum, anchored by a required core for first-year students. The new curriculum includes the following courses:
Year 1 (all first-year students will take these courses together as a cohort)
nizational entities in a university can pay off, particularly for in-depth methodological training. Indeed, education faculty expertise and school of education offerings are not likely to cover all specializations adequately. Opportunities to take courses or to work with faculty outside the school of education can facilitate the honing of specialized expertise by expanding access to a wider range of subfields and specialty areas (Raudenbush, 2003).
We readily acknowledge that collaborations across units of complex organizations like universities are difficult to establish and sustain. Difficult or not, we conclude that to promote the highest caliber of education re-
searchers, they are essential, and they have been brokered successfully in the past: for example, progress in statistical modeling and causal inference can be attributed at least in part to scholars trained in statistics with joint appointments in education and statistics departments (Raudenbush, 2003).
Why are such collaborations necessary? Training education researchers exclusively in disciplinary departments robs the profession of a common core and can detach investigators from the central issues in education they espouse to study. Training education researchers exclusively in schools of education shuts students off from the social and behavioral sciences and
related departments, stunting opportunities for in-depth explorations and depriving the future leaders in the field of relevant disciplinary models and methods. The way to address this “essential tension” (Shulman, 2003) is to make the boundaries between the two organizational units highly permeable: that is, to actively encourage research collaborations among the faculty and students in each around common areas of inquiry (Tobin, 2003). Formal arrangements like dual degrees, requirements for minors, or joint appointments are possible; informal links among individual faculty members are essential. Typical incentive structures at some universities will have to change in order to support the forging of such links.
Thus, the university itself will need to enable these connections by actively supporting and encouraging them. New syllabi for courses need to get approval from the university curriculum committee. Scheduling changes to accommodate greater interactions within cohorts of students and across university departments will be required. Beyond these more rudimentary challenges are long-standing funding discrepancies and shortfalls. Universities can and should actively support such collaborations financially.
Recommendation 11: Schools of education that train doctoral students for careers in education research should provide all students with a variety of meaningful research experiences.
It is hard to overstate the importance of providing regular and increasingly sophisticated opportunities to design and conduct education research in doctoral programs. Research experience is absolutely essential—without applying the concepts covered in course work, they are mere abstractions, and research skill is difficult to develop. Furthermore, opportunities for publication and other professional networking experiences are limited without contributing to ongoing research projects.
In addressing how such experiences might be structured, the traditional vehicle of the dissertation needs reconsideration. One lone project conducted alone at the end of a program squelches opportunities to learn from multiple experiences and flies in the face of the idea of teams of researchers working together on common problems. A better model might be for doctoral students to develop a portfolio of research products (Cohen, 2003; Shulman, 2003; Whitehurst, 2003). Staging a series of research experiences would also provide opportunities to publish research findings in peer-reviewed journals, and to present at conferences.
Ensuring meaningful research experiences for doctoral students re-
quires that they engage in research under the guidance of multiple faculty members who themselves are active in the field. Indeed, such mentorships are a key feature of the professional development of junior scholars in many scientific disciplines. Seasoned investigators active in education research can help facilitate the kinds of access, opportunities, and networking so critical to establishing junior researchers as contributors to their profession.
In doctoral programs for education researchers, traditional one-on-one mentoring might be augmented or even replaced by participation in interdisciplinary networks that connect faculty and students in schools of education with their peers in other departments. This kind of work may be especially important for doctoral students who have no prior research experience—former teachers, for example—to help orient them into the culture of research and to provide a setting in which their expertise can be tapped systematically. The importance of collaborations with disciplinary departments is relevant for promoting meaningful research experiences as well, because they provide opportunities for students to develop their specializations and to interact with faculty members conducting research outside schools of education.
Finally, an important element in the research experiences of doctoral students pursuing careers in education research is that they engage in research-based interactions with schools or other educational settings. Such interactions are likely to pay off in a number of ways. Working in schools provides the real-world conceptual context for graduate research training—the understanding of major issues facing educators and administrators today. In the context of conducting research, working together with practitioners also serves the very practical end of developing researchers who know how to set up and implement research projects conducted on site: providing these opportunities could help address concerns we heard at our workshops that many education researchers do not know how to work with urban schools.
To seasoned administrators in schools of education, these recommendations may seem wholly unattainable in the face of available resources. It is absolutely true that high-quality doctoral training for researchers requires a good deal of time for both students and the faculty members supervising them and time, of course, means money. The ways in which schools of education are financed, however, vary, and similarities and differences in the financial underpinnings across institutions are not widely understood. An analysis of these issues would greatly facilitate institution- and policy-level action to strengthen doctoral training for education researchers.
One aspect of such an analysis would need to focus on student financial support and patterns of program completion. Strong research training typically requires full-time study. And full-time study typically requires financial support. However, National Center for Education Statistics data reveal that 50 percent of doctoral students in education receive no financial support. By contrast, in the humanities and the physical, life, and social sciences, just 21 percent of students receive no support. The federal government needs to provide funds to enable in-depth research training (Whitehurst, 2003) of the kind we call for here; programs funded by other organizations like private foundations will also likely be necessary.
The improvement of education research training is a core matter of professional responsibility for the field (Cohen, 2003). Marshalling the financial and intellectual resources for it is a crucial task.
Professional development in education research does not begin or end with doctoral programs in schools of education. In this section we highlight participation in peer review panels in federal agencies that fund education research as an effective mechanism to provide ongoing professional development of education researchers at all stages of their careers.
Although not typically viewed in this way federal agencies can contribute to the professional development of the field through their peer review processes. Participating in peer review has a powerful influence on shaping education research professionals and the quality of their future work. Several ways in which these peer review systems can be designed to promote the professional development of the field are described in detail in Strengthening Peer Review in Federal Agencies That Support Education Research (National Research Council, 2004b). Here, we reprise the role of peer review as a tool for enhancing the professional development of education researchers.
Recommendation 12: Peer review panels in federal agencies that fund education research should be composed to promote the participation of people from a range of scholarly perspectives and traditionally underrepresented groups and provide opportunities for professional development.
In Chapter 2 we argued that diversity in peer review panels promoted quality. Here, we focus on another way that actively recruiting panelists
from diverse backgrounds to participate in the process can improve the process: by extending professional opportunity to a broader pool of researchers, building capacity in the field as a whole. Social characteristics affect the training researchers receive (because of the schools they attend, the topics and designs they are likely to pursue in school, and the jobs they anticipate for themselves) and in turn affect the experiences and expertise they develop (Harding, 1991; Howe, 2004). Thus, explicit attempts to engage traditionally underrepresented groups in the peer review process can improve access and opportunity, resulting in an overall stronger field and more relevant research.
Peer review can provide a rich context for further developing researchers into the culture of their profession and should be explicitly designed to promote the attainment of this objective. This function of peer review is often underutilized in the push to make funding decisions efficiently. Opportunities for engaging panel members in activities that further their professional development are facilitated when panels include broad representation of relevant experience and expertise, when panel members deliberate together, and when time permits differences of perspective and position to be aired and debated. Such opportunities for developing investigators—both experienced and inexperienced with respect to sitting on review panels—to the research ethos are facilitated when clear requests for proposals are available and when good feedback is provided to proposers. These conditions also create incentives for strong researchers to contribute their time and expertise to peer review: Why should they contribute if so little will come of their efforts and if they will gain so little from the experience?
Involving promising scholars in peer review at early stages of their careers can also target professional development opportunities for up-and-coming researchers who have solid credentials but less experience reviewing. The testaments of many workshop participants citing early experiences serving on National Institutes of Health (standing) panels as career-changing are indications of the potential of peer review to develop early career researchers. It is important, however, that promoting the participation of rising scholars in the context of peer review be balanced against the need to tap the best intellectual talent for review.
We need to be clear that by supporting peer review as a mechanism for developing researchers, the committee is not arguing for inculcating researchers to a culture based on cronyism and traditionalism. To prevent the isolation of perspectives and favoritism for well-established names and institutions from taking hold, checks on the system must be in place. That
said, the very foundation of the research process rests on the development of a commitment to scientific norms and values, which can and should be reinforced in the context of peer review (National Research Council, 2002).
The rationale to target the publication process as a means for professional development is exactly as it is for federal funding agencies: publishers of peer-reviewed education research have the opportunity and the responsibility to view and to develop their manuscript review procedures in ways that promote the professional development of those involved, particularly researchers who submit manuscripts for publication.
Recommendation 13: Publishers of peer-reviewed education research should design their editorial and manuscript review systems to promote the professional development of education researchers who participate in that process.
If done well, reviews of manuscripts are conducted by established researchers in the field with a range of perspectives and types of expertise, and their judgments can be helpful in shaping the future work of the prospective author. Thus, the ideas we put forth about diversity with respect to proposal review in federal agencies apply equally to manuscript review: encouraging highly qualified researchers from a range of backgrounds, perspectives, and career levels can promote a vibrant stock of published articles and facilitate growth and development among a wider range of scholars. The main difference is that professional development opportunities are likely to center largely on those submitting manuscripts, because reviewers and editors do not typically collaborate closely on reviews of manuscripts and therefore have less opportunity for the interaction that is the core of professional development.
All of the journals represented at the committee’s workshop provided feedback one way or another to authors of submitted manuscripts (usually in the form of blinded copies of the original reviews)—and many of them expressed their view that the review process served an important educative function in addition to its role as a screening device. Several of the editors at the workshop cited this goal as an explicit one in their work, one calling the work “developmental” editing (Emihovich, 2003). Others pointed to the inclusion of more junior scholars as reviewers as evidence of their com-
mitment to this idea (Coughlin, 2003). Yet the ability to provide this kind of feedback, as many editors and others involved in publishing journals attested, is limited by the logistical burdens that many social science and education journals place on their editors (McKeown, 2003). See Box 3-3 for a description of a tool designed to ease this burden and free up time for editors to engage in the professional development of scholars in their respected fields.
Our recommendation to publishers has a number of corollaries. One is that we think most manuscripts deserve some kind of review. While the participants at the workshop all rejected some small fraction of submissions without formal review, even in those cases, many of the editors revealed that they would write personal rationales and critiques to encourage high-quality future submissions. While enacting this recommendation will require more time on the part of editors, if the logistical burdens of publishing are eased, editors will have more time to engage in important substantive interactions that can slowly build capacity. The second corollary is that journals should consider revise and resubmit policies to enable the results of the reviews to promote improved future manuscripts.
Our recommendations for the professional development of education researchers are necessarily limited in scope. Doctoral programs in schools of education, as well as participation in peer review within federal funding agencies and journals, play important roles in reinforcing the norms and standards of the community. However, in our view the professional development of education researchers is a critical area for sustained analysis. Future work might seek to look at how to recruit and prepare education researchers earlier in their careers—in undergraduate programs, for example, and to conceptualize the professional development as a continuum of experiences over the course of a variety of career trajectories.