Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Administration of the Vocational Education R&D Program Vocational education R&D has not had as great an impact as it could have had partly because of certain characteristics of the administration of the R&D program. In this chapter, the Committee recommends several changes in the administration of the program that are intended to im- prove the resulting R&D. Before these recommendations are presented, however, the structure and management of the R&D program are de- scribed. The administration of the vocational education R&D program is com- plicated by the need to accommodate two major factors: three categories of R&D (research and development, demonstration, and curriculum de- velopment), and three levels of organization (federal, regional, and state). This chapter discusses the administration of the vocational education R&D program with respect to planning, administration, and management, outlining the roles of the major national, regional, and state organiza- tions: the U.S. Office of Education, the two national R&D centers, the National Advisory Council on Vocational Education, USOE regional offices, the National Network for Curriculum Coordination in Vocation- al and Technical Education, state education agencies, state research co- ordinating units (RCUS), and State Advisory Councils. (See Appendix B for a discussion of sources of information. ~ After the organizations involved in the administration of the R&D pro- gram are described, a separate discussion of dissemination and utiliza- tion of R&D products is presented. The problem of dissemination and 44
Administration of the Vocational Education R&D Program 45 utilization involves many different types of organizations and so is treat- ed separately. NATIONAL INSTITUTIONS U.S. OFFICE OF EDUCATION The U.S. Office of Education of the Department of Health, Education? and Welfare plays the major role in the administration of the vocational education R&D program. It is responsible for overall planning, including coordination of Part I and of the federal shares of Parts C and D, and for setting priorities. It announces the availability of money for grants and contracts, reviews proposals received in answer to its requests for propos- als, and monitors projects once they are funded. USOE iS also responsible for some dissemination of information and materials produced by those projects. It is becoming more concerned with evaluation of individual projects as well as evaluation of the entire vocational education R&D effort. Although the Office of Education in Washington, D.C., has received some guidance from USOE personnel in the ten regional offices of HEW, the functions of the regional offices have not been well defined. Under President Nixon's move to decentralize the government, the regional offices were to be given increased authority, including some decision- making power for the administration of the Commissioner's share of Part D funds; however, after court rulings nullifying certain decentralization actions, decision-making authority was withdrawn from the regional offices. At the present time, the regional offices have very little responsi- bility for Parts C and I funds or projects: they can review applications for Part C awards and review requests for proposals for Part I awards prior to publication. The regional offices have a greater role in connection with Part D, participating in planning and setting priorities. They review applications for Part D grants or contracts from states in their regions and send rec- ommendations to the federal office. Other regional office functions in connection with Part D include negotiating grants and awards, providing technical assistance and information to researchers, monitoring ongoing projects, participating in site visits for evaluation, closing out completed projects, and participating in federal office seminars and workshops. On the average, one person in each region is available for these Part D func- tions, and that person usually has additional responsibilities for other programs.
46 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Administrative Location Since 1964, the vocational education R&D program has been managed by many different divisions within the Once of Education. To administer the research program, which came into existence as a result of the 1963 Vocational Education Act, an Occupational Research and Planning Unit was established in 1964 within the USOE Division of Vocational and Technical Education. Branches within the Unit were established to man- age research in each of three substantive areas identified by USOE: em- ployment opportunities, human resources development, and education resources development and training. Table 2 lists changing adm~nistra- tive locations of the R&D program since 1964. At least partly as a result of these shifts in administrative structure, there is evidence of a rapid suc- cession of contradictory long-range plans. TABLE 2 Location of Federal Administration of Vocational Education R&D within the U.S. Office Of Education Fiscal Year 1964 1965-1 967 1968* Administrative Location Bureau of Adult, Vocational and Technical Education Division of Vocational and Technical Education Occupational Research and Planning Unit Bureau of Research Division of Adult and Vocational Research National Center for Educational Research and Development (bureau level) Division of Comprehensive and Vocational Research 1969-1971: National Center for Educational Research and Development Division of Comprehensive and Vocational Research and Bureau of Adult, Vocational and Technical Education Division of Vocational and Technical Education 1972-1973 Bureau of Adult, Vocational and Technical Education Division of Vocational and Technical Education 19.74 1975t Bureau of Occupational and Adult Education Division of Vocational Education Research Bureau of Occupational and Adult Education Division of Research and Demonstration _ *This change is mostly a change in the title of a group. Administration was split between the two divisions. tThis change reflects only the renaming of a group. In this case, the major organizational structure and personnel involved remained the same. However, in other instances, espe- cially in fiscal 1964-1965 and fiscal 1972-1974, there were major shifts in the personnel and structure involved in administration of the vocational education R&D program.
Administration of the Vocational Education R&D Program 47 Coordination Since 1975 the Division of Research and Demonstration in the Bureau of Occupational and Adult Education (BOAE) has administered the federal half of the Part C research program, the federal half of the Part D dem- onstration program, and the Part I curriculum development program. The Division attempts to coordinate these efforts by developing plans encompassing all three Parts and specifying their interrelationships. In general, Part C funds support applied research and developmental stud- ies; Part D, demonstrations; and Part I, development of nationally need- ed curricula. The USOE staff tries to move useful research products into developmental and later into demonstration stages. The stab also strives to coordinate vocational education R&D work with general educational R&D, with R&D in special education, and with research supported by the National Institute of Education (NTE). Despite those attempts, Parts C, D, and I have not been coordinated to produce a well-integrated research and development program. There is little evidence that Part D demonstrations are based upon information gained from Part C or from products developed under Part I. Instead, Part D funds have generally supported career education models. In addi- tion, each of the three programs has its own set of priorities that may or may not coincide with the priorities of the other two programs. To some extent, attempts at coordination are hampered by the legislat- ed purposes of the three parts. Part D funds have been used appropriate- ly for career education. In general, they cannot be used to support dem- onstrations of Part C projects not related to career education. Thus, lack of coordination of Parts C, D, and I is a legislative as well as an adminis- trative problem. Planning and Setting Priorities Most of the planning in the Office of Education has been on a year-to- year basis although there are some longer-range plans. Long-range plan- ning of a specific nature is exemplified by the multi-year commitment of the Part I staff to develop curricula in each of the usoE-designated occu- pational clusters in the early 1970s. The development of occupational cluster curricula was established as a Part I priority in response to pres- sures in BOAE but outside the Division. ES '70 (Educational System for the 70's) and career education are nota- ble examples of attempts to initiate long-range planning, and both are also examples of the use of vocational education research funds for pur- poses that extend well beyond the goals of the authorizing legislation.
48 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Each program was advocated strongly by one administrator and, there- fore, was dominant in vocational education R&D for a period of time. David S. Bushnell, Director of the Occupational Research and Planning Program in the Division of Vocational and Technical Education between 1965 and 1969, was the major proponent of ES '70. ES '70 has been closely associated with the idea of an "organic" curriculum that would prepare students for a variety of post-high school activities. The organic curricu- lum was designed to include both academic and occupational training, as well as components of personal development, real work experience, and post-high school placement. Career education became a high priority with the strong advocacy of Sidney P. Marland, Jr., Commissioner of Education between 1970 and 1972. The career education program, as originally supported by USOE, was transferred to NIE in 1972. Poorly defined roles created some difficul- ty for NIE and USOE in formulating an R&D plan for career education. NIE has defined its role with respect to career education as the "examination of the relationship of education and work, and the development of pro- grams and products to improve this relationship" (U.S. Department of HEW 1975, p. 8~. USOE'S role has been "assistance to states and local education agencies to use, demonstrate and improve the practice of edu- cation in relationship to the world of work" (U.S. Department of HEW 1975, p. 8~. Not only is it difficult to see the difference between these definitions of career education R&D, but also both clearly overlap with vocational education R&D. The notions of both ES '70 and career education were dominant (and almost exclusive) themes of the federal vocational education R&D pro- gram at various times. For example, several of the Section 4(c) projects funded during fiscal 1967-69 were directed toward the development of the ES '70 program in such areas as career guidance and modern manage- ment practices for education. In addition, in fiscal 1972 and 1973, Parts C, D, and I were all oriented towards research, development, and dem- onstrations that would increase the knowledge base for career education. There is no evidence that a national dialogue for planning involving a representative segment of the vocational education community or its RED sub-community occurred during the years 1964 to 1974. A five-year or even a two- or three-year vocational education research agenda has never been~published. While priorities are set yearly for Parts C and I, Part D priorities are set every three years. Procedures for setting priorities are similar for all three programs; the Part C procedure is described here. The yearly procedure has three phas- es. During the first phase, which lasts about one year, the Director of the Division of Research and Demonstration meets with branch chiefs and
Administration of the Vocational Education R&D Program 49 various groups to determine possible priorities. These groups include: The Research Committee of the National Advisory Council on Voca- tional Education; the State Directors of Vocational Education Research Liaison Committee; an ad hoc group of state RCU directors; Curriculum Coordinating Center directors; USOE regional personnel; and ad hoc in- terest groups. Recommendations from sources such as General Account- ing Office reports, USOE program evaluation reports, the Commissioner of Education, and influential national leaders are also sought. During the second phase, the Deputy Commissioner for the Bureau of Occupational and Adult Education makes the final decision regarding yearly priorities. Typically, new priorities are favored over the continu- ance of old priorities. During the third phase, grant announcements and REPS are prepared, moved through the administrative levels, and approved- for publication in the Federal Register (or in Commerce Business Daily for contract an- nouncements). The procedure for setting priorities is much more responsive to politi- cal pressures than to scientific pressures. It is clear that researchers have had little representation in Phase I. Strong and Jarosik (1975, p. 6) note that in Phase II "leadership in vocational education tended not to have control of how research funds were to be spent," and offered the example of the use of those funds predominantly for career education. In fiscal 1972 and 1973 at the direction of Commissioner Marland, the Deputy Commissioner specified that vocational monies were to be spent for ca- reer education; thus he redirected money that could have been used to support program categories that were more specifically implied by the vocational education legislation. The identification of particular priorities is partly dependent on the composition of the ad hoc groups that are convened during Phase I. The level of sophistication and particular interests of these groups can and do influence the priorities established. Hence, the people who convene them influence priorities, and there have been different conveners over the last ten years. Obviously, the use of ad hoc groups that change from year to year results in lack of stability in the research program. Concern has also been raised about the extensive participation of certain state leaders in setting federal priorities. A nationwide program of research should take state priorities into account, but it is not clear how great a role the states should have in setting federal priorities. Nor is it clear on what basis certain state leaders have been invited to participate. The lack of stability in priorities has been perceived as detrimental by the vocational education researchers interviewed. A concern with short- term, product-oriented research, a lack of concern for long-term, pro
50 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT grammatic studies, and a lack of emphasis on high-risk efforts that have promise of high payoff have characterized Part C priorities. Moreover, USOE itself reports that identification of new priorities takes precedence over continuation of last year's priorities, regardless of the probability of payoff from further effort toward a goal partly reached during the past year. Policy Development Both understaffing at USOE and frequent administrative shifts may ac- count for the lack of long-range planning. Whatever the reason, policy decisions have not been derived in a consistent and systematic manner and often have been determined externally. The need for quick answers to pressing problems, changing goals with each new Commissioner of Education, and reaction to political pressures have tended to increase the emphasis on targeted, product-oriented priorities. Policy and decision making have generally not been influenced by past R&D activities, to continue research needed in some areas and to allow for learning from past R&D. Stronger and continuing national leadership in policy develop- ment is needed. Awarding Grants and Contracts Announcements The availability of funds and priorities for the federal halves of Parts C and D are announced in the Federal Register, which is sent to all state departments of education, RCUS, and others and is avail- able nationally. Winning applicants are usually awarded grants, unless they are profit-making institutions. For Part I projects, requests for pro- posals (RFPS) for contract awards are announced in Commerce Business Daily, also available to all. Some practitioners and researchers believe that the announcement of availability of funds in these publications is, in itself, discriminatory. Although the publications are distributed widely to departments of edu- cation and school districts, they are not available without charge. Small private organizations such as consulting firms and academic researchers can be especially handicapped. Moreover, simply keeping abreast of an- nouncements and information in these publications is a time-consuming task that may be too expensive for some would-be applicants. In addi- tion, the time for writing applications or proposals can be as short as two weeks once the announcement is located and discussed with collabora- tors; however, it is generally estimated that it takes four weeks to write a proposal and secure necessary local approvals.
Administration of the Vocational Education R&D Program 51 Nearly all awards are made on a competitive basis and few proposals outside the stated priorities are funded. Because priorities change from year to year, researchers who want to follow a consistent plan of research over several years are likely to be denied an opportunity to compete on an equal footing for funds each year, regardless of the quality of their past work. Announcements requesting grant applications or RFPS are essentially the only form of dissemination of information about proposed research used by USOE. Grant announcements and RFPS specify quite clearly what research USOE wants, including goals and methods of accomplishing these goals. This procedure requires that USOE have well-defined priori- ties, that USOE know exactly how the research should be planned and conducted, and, preferably, that the priorities adequately and accurately reflect the needs of vocational education. If announcements and RFPS are to be well-written and responsive to those needs, good communication must exist between USOE and vocational educators. In addition, USOE must be adequately staffed with competent vocational researchers or it must employ such researchers as consultants. The current limitations on salary and expense budgets prevent either of these options from being effectively implemented. Review Procedures Proposals are reviewed by panels, usually composed of five people and chaired by the USOE Part C program chief. Boo more than three of the panel members can be federal employees, and they must be from outside the Division of Research and Demonstration. Hjelm and Boerrigter (1974, p. 43) specify: A typical panel will consist of two members being content specialists, one mem- ber being a design specialist, one member being an evaluation specialist, and a fifth member being an educator or user of the products of the R&D program. Attention is given to the geographical spread and the spread by type of institu- tion of the non-federal reviewers. The panels are representative in terms of m~- nor~ty groups and women. These often are not peer reviews (reviews by other vocational education researchers) like those often used in social science, physical science, and medical research. The review procedure is characterized by a tendency to fund projects at the lowest possible cost. Award Recipients Figure l shows the flow of vocational education R&D funds; the dollar figures are based on fiscal 1974 data. The agencies and institutions iden
o ~- _E- ~ _ _d ~. C o ._ Ud o _ CJ o C' ·- ._ _ . _ o E · oo cn ~ j 69 o . _ E o - 0 = (_) - U' ~ j ~ c ._ = . _ E -CD- 69 c o . _ . _ E ° E U) CO j ~ Ct Ct ,o _ .O C~ _ ~ C - C} cn o - .~ C .> ~_ CO g C~ 0 ~' ._ ~ ~ ~ ~L ~-9 .> = C~ C o =o ~ ro -J L~ . _ C C~ ~, .C) =0 C C < _ C _ C ~ .-0 C ~ Ct Z ~ ~ ~ E - 52 o - cc C1 _o o: c _. o ._ C) o :~ o 3 o Ct c) ¢
Administration of the Vocational Education R&D Program 53 tiffed in Figure 1 did not necessarily receive awards in every year of funding. Section 4(c) and Part C The types of institutions that most frequently have received grants or contracts have varied from program to program and from time to time. Throughout the Section 4(c) program (up to 1969), colleges and universities predominated as award recipients, while in the Part C program (since 1971), there has been a greater percentage of state education agencies (SEAS) as grantees. In fiscal 1972 and 1973, all federal share Part C funds went to SEAS to support career education and experimental, developmental, and demonstration projects. Since the state half of Part C funds is awarded directly to SEAS, there has been extensive state control over research funds; in fiscal 1972 and fiscal 1973, SEAS received all Part C funds. Research awards are displayed by type of recipient for the Section 4(c), federal share Part C, and state share Part C programs in Tables 3, 4, and 5. It should be noted that in a few cases award recipients may not be the actual researchers because they may subcontract the work. TABLE 3 Sample of 149 Research Awards by Recipient Institution, Section 4 (c) Percentage of Dollar Awards Private Fiscal State Education Local Education Universities or Nonprofit Year Agencies Agencies Colleges InstitutionsTotal = _ .. 1965 22.8 - 65.8 1 1.2 99.8 1966 5.7 0.8 65.4 20.1 100.0 1967 10.6 1.3 78.6 9.3 99.8 1968 2.3 6.2 57.7 33.6 99.8 1969 8.3 5.5 76.6 9.4 99.8 TABLE 4 Research Awards by Recipient Institution, Federal Part C Percentage of Dollar Awards ~- Fiscal State Education Local Education Universities or Private Year Agencies Agencies Colleges Institutions Total 1971 1.9 - 43.3 54.8 100.0 1972 100.0 - - - 100.0 1973 100.0 - - - 100.0 1974 39.7 10.9 27.7 21.4 99.7
54 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT TABLE S Research Awards by Recipient Institution, State Part C Percentage of Projects Schools and State and Local Fiscal Education Universities Vo-Tech Year Agencies or Colleges Schools Other* Total 1971 37.6 45.1 9.4 8.0 100.1 1972 38.6 41.0 10.2 10.2 100.0 1973 3 1.7 44.9 9.2 14.2 100.0 *Private organizations, individuals, state departments other than education, unknown or unclassifiable agencies. Part D Because of the nature of the Part D program, the majority of the grants are made to local education agencies. Data collected by De- velopment Associates (1975, p. 142) on the states' share of Part D indi- cate that over 80 percent of the grants made during the first three years were to local education agencies or schools, 16 percent were to universi- ties or colleges, and only three percent were to SEAS. In many cases, federal share Part D funds were awarded to SEAS (see Table 6~. Part I Throughout the Part I program, a significant number of research projects have been conducted by private agencies (see Table 7). Many of these private agencies are private-for-profit: in fiscal 1972, 23 percent of all award recipients were private-for-profit; in fiscal 1973, 20.4 percent; and in fiscal 1974, 46.6 percent. (Data on percentage of nonprofit and for-profit agencies are not available for fiscal 1970 and 1971.) Universi- ties or colleges also received a substantial portion of Part I funds. TABLE 6 Research Awards by Recipient Institution, Federal Part D Percentage of Projects State Local Education Education Universities Funding Round Agencies Agencies or Colleges Other* Total 1st (fiscal 1970-73) 23.1 63.1 4.6 9.2 100.0 2nd (fiscal 1974-76) 34.6 59.6 5.8 .0 100.0 *Private organizations, individuals, state departments other than education, unknown or unclassifiable agencies.
Administration of the Vocational Education R&D Program 55 TABLE 7 Research Awards by Recipient Institutions, Part I Percentage of Dollar Awards State Local Fiscal Education Education Universities Private Year Agencies Agencies or Colleges Agencies Total 1970 4.6 1.4 33.6 60.4 100.0 1971 41.8 2.5 21.5 34.2 100.0 1972 21.7 0.6 31.3 46.4 100.0 1973 22.7 3.9 47.9 25.4 99.9 1974 0 0 16.0 84.0 100.0 Project Directors Data on the sex of project directors have been collect- ed for Section 4(c), federal Part C, and Part I programs. Tables 8 and 9 display project awards for these programs by sex of project directors. There seems to be a random fluctuation in the rate of female participa- tion in the Section 4(c) and Part C programs, with a ten-year average of 8.3 percent. Under Part I, female project directors outnumbered males in one year (fiscal 1974), possibly due to a large curriculum project awarded to the American Home Economics Association (see Table 8~. Data on the ethnic minority representation among project directors are not avail- able. TABLE 8 Section 4 (c) and Federal Share Part C Project Directors by Sex Fiscal Year Males Females Total N Section 4 (c) 1965 79.37O 20.770 100.070 29 1966 88.9 1 1.1 100.0 45 1967 100.0 0 100.0 22 1968 95.0 5.0 100.0 20 1969 91.0 9.0 100.0 33 Part C 1970 0 0 0 0 1971 100.0 0 100.0 33 1972 89.9 10.1 100.0 92t 1973* ~ 1974 89.8 10.2 100.0 981: *No data available. t3 projects had two co-directors. As projects had two co-directors.
56 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT TABLE 9 Part I Project Directors by Sex Male Female Unknown Total 1972 25 1 2 28 1973 9 3 1 13 1974 8 10 0 18 TOTAL 42 (71.9~o) 14 (23.7~o) 3 (5.1~o) 59 (100~) NOTE: Only new starts are included. Monitoring and Evaluating Projects Monitoring ongoing grants and contracts is seen as an important func- tion by USOE in an effort to ensure adequate benefits from the expendi- ture of taxpayers' money. In practice, however, limited numbers of staff and insufficient funds for travel prevent USOE from doing as much moni- toring as it thinks desirable. In general, contracts require closer monitor- ing than do grants. The Office of Education has only infrequently required evaluation to be a component of R&D projects. (In fiscal 1972 and 1973, career educa- tion projects funded under Part C were required to have third-party evaluations.) Instead, USOE has awarded contracts to three groups- De- velopment Associates, Inc., Project Baseline, and this Committee to perform post hoc evaluations of the Parts C, D, and I programs. All three groups have faced difficulties because evaluation was not given early consideration in the design and conduct of most R&D projects. Therefore, criteria for "success" and desired outcomes were not defined at the out- set of each project, and projects did not keep complete records that would allow post hoc measurement and evaluation of outcomes. In addition, evaluation of the quality of research performance has been missing. There is essentially neither a quality control procedure nor a mechanism to ensure that research performers who have done unsatis- factory work in the past will not be awarded grants or contracts in the future. Dissemination The Office of Education relies heavily on written reports for the dissemi- nation of R&D products. Abstracts of completed projects are published in a periodic USOE report and are sent to Abstracts of Instructional and
Administration of the Vocational Education R&D Program 57 Research Materials (AIM/A~) and the Science Information Exchange of the Smithsonian Institution. Final reports are sent to Educational Re- sources Information Center (ERIC) and AIM/ARM. Beginning in 1975, in- formation about Part D projects funded by USOE and NTE will be com- piled into an annual report. USOE believes commercial publication to be the most effective means of disseminating curriculum materials and, therefore, encourages Part I project directors to seek such publication. However, the curricula developed are often highly specialized and the market is considered to be speculative because of the Part I focus on new occupations. Therefore, it is often impossible to secure commercial pub- lication for these materials. The view of dissemination currently most popular among educators and social science researchers involves two principles: 1. To be effective, dissemination must be planned at the outset of a research project. 2. Dissemination modes must be flexible and must take into account the varied needs of the audiences addressed. However, USOE does not routinely require that researchers and develop- ers carefully consider the appropriate means of disseminating the results of R&D efforts before a project is completed. (The National Network for Curriculum Coordination and the state RCUS were established partly to aid in dissemination of R&D projects. These institutions are discussed later in this chapter.) NATIONAL VOCATIONAL EDUCATION R&D CENTERS In 1965, USOE established a series of national and regional R&D institu- tions to respond to specific substantive needs in education. The Division of Vocational and Technical Education established institutions related to vocational education: an RCU in each state, two national R&D centers, and four research development units. Due to funding reductions, the four research development units have been phased out; the other institu- tions still exist but receive little funding from USOE. The two R&D centers are the Center for Vocational Education at the Ohio State University and the Center for Occupational Education at North Carolina State University. Both were funded initially under Sec- tion 4(c) and are now supported by several sources on a project-purchase basis. Each center stresses slightly different aspects of R&D. At present, the Ohio Center is much larger than the North Carolina Center.
58 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT The North Carolina Center was established in 1965 to serve 14 south- ern states. Its expressed mission is to provide a national resource for policy analysis and development: (1) to inform constituency groups of issues that may affect vocational education; (2) to assist federal agencies in working with others in policy development; and (3) to respond quickly to questions of vocational education policy. It also provides technical assistance. This Center stresses a multidisciplinary approach to R&D in vocational education. The mission of the Ohio Center is "to increase the ability of diverse agencies, institutions, and organizations to solve educational problems relating to individual career planning and preparation." In order to fulfill its mission, the Center conducts national programs and projects related to six objectives (Council for Educational Development and Research, Inc., p. 384: 1. Generating knowledge through research 2. Developing educational programs and products 3. Evaluating individual program needs and outcomes 4. Installing educational programs and products 5. Operating information systems and services 6. Conducting leadership development and training programs. Whenever possible, the Center undertakes multi-year endeavors that ad- dress national priorities. The two national Centers received general institutional support from USOE until fiscal 1972 when they were transferred to NIE, which estab- lished a project-purchase policy. They currently receive grant and con- tract awards for R&D projects from NIE, USOE, SEAS, local school districts, and business and industry. The project-by-project funding focuses the Centers' efforts on short-term, product-oriented research. Both Centers usually receive grants from the Commissioner's share of Part C. Although the Centers study problems of regional and national scope, these awards are charged against the Ohio and North Carolina state allotments of the Commissioner's share; this penalizes both the states and the Centers and can strain the working relationship between a Center and the state in which it is located. NATIONAL ADVISORY COUNCIL ON VOCATIONAL EDUCATION The National Advisory Council on Vocational Education (NAC^) was established by the 1968 Amendments, which authorize funds for its sup
Administration of the Vocational Education R&D Program 59 port. It consists of 21 members appointed by the President for terms of three years. The legislation specifies that the members be representative of labor and management as well as of the general public. They should also be familiar with manpower problems and the administration of manpower programs, the administration of state and local vocational education programs, the training of the handicapped and disadvantaged, and post-secondary and adult education. NACVE'S three tasks, set forth in the 1968 Amendments [Sec. 104(a)~2~], are to: (A) advise the Commissioner concerning the administration of, prep- aration of, general regulations for, and operations of, vocational educa- tion programs supported with assistance under this title; (B) review the administration and operation of vocational education programs under this title, including the electiveness of such programs in meeting the purposes for which they are established and operated, make recommendations with respect thereto, and make annual reports of its findings and recommendations (including recommendations for changes in the provisions of this title) to the Secretary for transmittal to the Con- gress; and (C) conduct independent evaluations of programs carried out under this title and publish and distribute the results thereof. NACVE has a research committee that commissions research and evalu- ation activities and informs the Council of state vocational education research work. The research committee is concerned primarily with the second and third tasks listed above. Indicating NACVE'S interest in R&D, in April 1975 Roman Pucinski testified for NACVE before the House Sub- committee on Elementary, Secondary, and Vocational Education. Pucin- ski suggested that a major portion of federal funds for vocational educa- tion should be used for supporting innovative programs. NATIONAL DISSEMINATION SYSTEMS There are two national dissemination systems supported by federal funds for vocational education. One is the ERIC Clearinghouse in Career Edu- cation (ERIC/CICE), supported by Central ERIC at NIE; the other is AIM/ARM, supported by USOE. ERIC/CICE was preceded by the ERIC Clearinghouse in Vocational and Technical Education (VT-ERIC), which was originally established at the Ohio Center in 1966. VT-ERIC was the first clearinghouse in the ERIC system to: (1) develop information analysis products; (2) partition the ERIC file; (3) engage in dissemination activities; (4) computerize its own
60 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT file; (5) conduct user studies; (6) develop a user training package; and (7) develop supplementary abstract publications (AIM/A~. The Clearing- house benefited from the resources and contacts of the Ohio Center: a research library; a large, interdisciplinary stab of vocational education R&D specialists; a research program on dissemination; and links to the vocational education community. For 1966-68, VT-ERIC was totally supported with vocational education funds authorized by the 1968 Amendments at about $300,000 per year. In 1969, Central ERIC funded the Clearinghouse at reduced levels and, for the next few years, funding ranged from $175,000 to $240,000 per year. Another cut in funding came when Central ERIC, which had moved to NIE in 1973, merged VT-ERIC and the Adult and Continuing Education ERIC Clearinghouse at Syracuse University into a new clearinghouse, ERIC/CICE at Northern Illinois University. The contract awarded was for about $152,000, far less than the previous year's funding for VT-ERIC and about one half of the combined funding for the two displaced clearing- houses. Lacking experience in clearinghouse operation and limited by inade- quate funding, ERIC/CICE could not continue the activities of VT-E~C. Central ERIC had drastically curtailed information analysis at clearing- houses and discontinued support for local clearinghouse collections. ERIC/CICE has not developed a comprehensive information resource sys- tem for vocational education even though it has received increased fund- ing from NIE in recent years. Its principal emphasis has been upon acqui- sitions of research reports, input to Resources in Education, and computer searches of the ERIC files. Many potential users of vocational education research products have not been totally satisfied with the ERIC system. At the summer 1975 meet- ing of the six curriculum centers, the two principal complaints against ERIC raised by participants were the time delay (about six months) before materials submitted to ERIC are accessible, and the screening performed by ERIC. In many cases, states want immediate access to all curricula however. Other criticisms of the ERIC system are that it takes too long for requests to be filled and that it is difficult to work with microfiche. Because of curtailed ERIC clearinghouse services, in 1974 BOAE decided to continue support of AIM/A~ as a project. MM/ARM had been con- ceived originally as an ERIc-compatible, supplementary publication pro- viding comprehensive coverage of research and instructional materials for the vocational education audience. With its new project status AIM/A~ had the additional responsibility of reporting projects in prog- ress, conducting literature searches for BOAE, supplying bibliographies in
Administration of the Vocational Education R&D Program 61 support of solicitations for Parts C and D project applications, develop- ing interpretative papers, and consulting with dissemination network components. AIM/A~'S bimonthly publication, Abstracts of Instructional and Re- search Materials, includes sections entitled: Instructional Materials Abstracts Instructional Materials Subject Index Research Materials Abstracts Research Materials Subject Index Curriculum Development Projects in Progress Research Projects in Progress The projects-in-progress sections announce current activities funded under the 1968 Amendments. In addition to the bimonthly publication, AIM/ARM products include (Magisos 1975~: Annual indexes to AIM/ARM Computer search tapes, including summaries of at least 16,000 docu- ments. Microfiche of research documents and products Information searches for the sponsor Interpretative papers Consultation with affiliates in the linked information dissemination network Repackaged indexes to instructional materials Pilot testing of state information dissemination services Training workshops for information specialists Development of a guide to operating information dissemination sys- tems and a guide to existent information resources User training and development of user training materials. To some extent, AIM/ overlaps with ERIC/CICE. Neither is a com- plete information and retrieval facility, and both omit certain classes of information for reasons other than quality. However, to the extent that they are utilized, both systems serve to provide the vocational education community with completed and in-progress instructional and research materials. The state RCUS and regional curriculum centers (discussed be- low) help to feed information into these systems, encourage more wide- spread use, and disseminate microfiche to requesters.
62 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT REGIONAL ORGANIZATIONS NATIONAL NETWORK FOR CURRICULUM COORDINATION The National Network for Curriculum Coordination in Vocational and Technical Education is made up of six regional centers. It was estab- lished by USOE in June 1972 with five centers, after 30 states responded to the initial REP; in June 1973 two other centers were added. One year later the seventh center was discontinued because of lack of funds, and the states were regrouped into six regions more nearly consistent with the ten USOE regions. The National Network is supported under Part I; current funding is about $40,000 per year per center. Originally, there was dual motivation for funding the centers: to im- prove the curriculum management capabilities of states and to put cur- riculum development and management in the context of career educa- tion (Simpson 1975, p. 23~. Secondary goals were to improve communi- cation among states, especially among neighboring states, to enhance coordination in reducing duplication of effort, and to promote coopera- tion in developing, validating, evaluating, disseminating, and installing curricula. In a statement prepared by USOE for Congressional oversight hearings in the spring of 1974, four primary purposes of the National Network were outlined: · Information sharing: to provide a mechanism for the sharing of information on curriculum materials available and under development, and for reporting on coordination efforts. · Standards: to develop and recommend guidelines for curricula and curriculum development with the ultimate goal of increasing the e~ec- tiveness of curriculum materials and enhancing their transportability. · Curriculum needs, as a basis for planning: to establish and main- tain a system for determining curriculum needs in vocational-technical education and reporting conclusions to the field. ~ Coordination: to coordinate activities in curriculum development dissemination and utilization with the aim of avoiding unwarranted du- plication, enhancing quality of effort, increasing the transportability of curriculum materials, and improving the acceptance and use of curricu- lum materials. Curriculum development is viewed by the National Network as dis- tinct and separate from the rest of R&D. Communication between curric
Administration of the Vocational Education R&D Program 63 ulum developers and other researchers is not presently one of the Na- tional Network goals. It is important to note that Part C functions over- lap considerably with the functions of the National Network centers. Each center is presently staffed at a marginal level and operates as best it can under the circumstances, but the centers' efforts are fragmented and isolated. The centers duplicate some of the work of ERIC and AIM/A~, in part because they are not satisfied with these services. Weir efforts at dissemination are neither as far reaching nor as concentrated as might be desired. Their efforts at evaluating their dissemination work have been severely hampered and sometimes eliminated because of lack of funds. Although the centers are organized along USOE regional lines, there is no evidence that geographic differences in curricula are larger than the similarities or that the differences that do exist are related to variability among USOE regions. Because regions lack the revenue base of either local or state government, it is not possible for the centers to ob- tain a multiplier effect (matching federal funds with state or local mon- ey). There is no apparent reason for regionalization of curriculum devel- opment efforts. STATE ORGANIZATIONS On the state level, institutions involved in the vocational education R&D program are concerned primarily with the states' share of Parts C and D funds. These institutions are the SEAS (which may be called state educa- tion agencies or state departments of education), state departments of vocational education, which may or may not be part of the SEAS, RCUS, and state advisory councils on vocational education. Vocational education R&D programs vary greatly by state, reflecting factors such as the amount of allotments from USOE, the amount of local and state matching funds, the scope and function of the RCUS, the admin- istrative structure, and the state's philosophy toward research. State ap- propriations to match Parts C and D funds vary widely. In most states, Parts C and D funds are not matched equally by state funds. Small states receive very small Parts C and D allotments and seem to be uniformly unable to provide sufficient state revenue to have enough money for productive vocational education R&D. RCUS, which play a key role in states' research programs, usually function as a branch of the state voca- tional education department, but in a few states they are located at and partly funded by a university or are part of a more general educational research operation in the SEA.
64 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT RESEARCH COORDINATING UNITS Al1 RCUS administer their states' share of Part C research funds. Part C funds pay up to 75 percent of the costs of operating the RCU and up to 90 percent of the costs of R&D grants and contracts. Some states support the operations of their RCUS with state funds and use the Part C funds to expand the research effort within the state; other states supply only the required matching. In addition to managing Part C funds, RCU functions include setting research priorities, coordinating state R&D efforts, and disseminating R&D results and products. In some states, the RCUS play a large part in con- ducting research; in other states, research funds are awarded to individu- als, local education agencies, and other organizations that conduct Part C research. In addition, RCUS have the potential to form a national net- work and link the national, state, and local levels of the R&D program. Usually, RCUS set research priorities with the approval of the state director of vocational education. About half of the ten RCUS visited by the staff (see Appendix B) claim to consider the federal priorities for the Commissioner's projects in setting their own state priorities, but some RCU directors believe that federal priorities change too frequently. The ten RCUS sampled review their priorities annually, some using a more highly refined process than others. Most gather input from teachers and state and local administrators and sometimes from lay or labor people. A few RCUS use management by objective and therefore stress measurable goals and objectives under their priorities. They believe that this is a good way of ensuring that state priorities will be addressed and of assess- ing the progress of projects done by either the RCU or other researchers. In awarding grants and contracts for Part C, there is considerable variability in the use of RFPS. (In a few RCUS, all research is in-house consequently, they do not use RFPS.) Some RCUS use some form of an- nouncement, usually not as formal or as structured as the federal REP. There is some hesitancy to use RFPS, partly because it is difficult to write them. Perhaps even more important is that RFPS require researchers to prepare lengthy proposals without any guarantee of being funded, and this is thought to waste time and energy and to alienate the research community. All but two of the ten RCUS visited fund researcher-initiated proposals, although some RCUS require that these projects fit within the state priori- ties. In one small state where communication between the RCU and re- searchers is excellent, all projects result from such proposals. Half of the states surveyed use sole-source awards to some extent. Usually they are
Administration of the Vocational Education' R&D Program 65 used for specific ideas judged by the RCU to be important and most appropriately handled by one well-known performer. Minigrants (awards of approximately $5,000 or less) are frequently awarded to teachers or schools because they seem to generate interest and enthusiasm and result in substantial informal matching of funds or services from the school or locality. However, two states visited do not use minigrants because of the high administrative costs involved. A common problem faced by the RCUS in making awards is the time lag between federal appropriations and obligations. According to some RCU directors, the uncertainty of funding is a factor that influences the use of REPS, the quality of proposals, and even the quality of projects funded. Some RCUS have additional uncertainties with regard to state support, which further handicaps planning. In some states RCUS take on additional functions such as training re- search personnel, developing a management information system, and maintaining a research information clearinghouse. For those RCUS with little in-house research, typical activities include providing technical as- sistance to both producers and consumers of research products and mon . . . . torlng ongoing projects. In about half the states, RCUS also manage projects funded by Part D, while in the other states a separate section within the state department of vocational education or the SEA administers Part D. While almost all SEAS (or RCUS within SEAS) monitor Part D projects, the extent of their involvement varies widely. Data collected by Development Associates, Inc. (1975) indicated that in 20 states, Part D project sites were visited at least monthly; in 22 states, they were visited three or more times per year; in seven states, one or two times per year; and in one state, as needed. The amount of coordination between Parts C and D is closely related to whether or not the two parts are administered by the same agency. When Part D is administered by a separate section within the SEA and not by the RCU, communication problems between the SEA and the RCU seem frequently to inhibit coordination between the two programs. These problems are more acute when the SEA staff shows a lack of recep- tivity to research. Dissemination activities, although variable in type, are carried out to some extent by all RCUS. In all but one of the ten states visited, final reports are required of every project and some of these reports are even- tually included in the ERIC dissemination system. Most of the RCUS have ERIC collections with computer retrieval systems, and microfiche is avail- able upon request.
66 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Beyond involvement with ERIC, RCU dissemination efforts vary consid- erably. Dissemination in about half of the states visited is geared primar- ily to requests for information. All but three states prepare abstracts of projects done in their states in addition to microfiche of final reports. The abstracts are usually available upon request and, in some states, are dis- tributed routinely to a selected audience. In one state, each request for information is handled individually and copies of searches are not kept for future use. Other states answer some requests with inflation gath- ered for earlier requests. The former method has a personal touch and is felt to increase the interest of users; the latter approach is less costly and probably less time-consuming. Other RCUS take the initiative in circulating information. Several send out newsletters or journals that announce the availability of research products. Less common vehicles for dissemination are personal contacts through workshops, demonstration sites, in-service training, and regional centers. It is generally felt that personal contact is the most effective dissemination technique, particularly since many potential users are un- familiar with research processes, research methodologies, and the use of microfiche. Personal contact as a means of dissemination is widely recog- nized as very expensive and time-consuming for RCUS with relatively limited budgets and few stab. STATE DEPARTMENTS OF VOCATIONAL EDUCATION Each year the state boards of vocational education, in conjunction with the state advisory councils, develop state plans for vocational education as specified in the 1968 Amendments. These plans, which identify both annual and long-range objectives, focus primarily on instructional pro- grams, but they include R&D as well. The plans are submitted to USOE and must be approved before a state can receive any money under the 1968 Amendments. In addition to their involvement with the states' share of Parts C and D, state departments of vocational education review and approve pro- posals from local education agencies for federal Part C awards. In many cases, they even assist in the writing of those proposals. In fiscal 1975, the SEAS were also asked to review all proposals submitted from their state and to advise USOE of any duplications. Since SEAS are also eligible to receive discretionary awards, there may be a conflict of interest in this SEA function. Some SEAS avoid this problem by choosing not to compete for discretionary federal money or by approving all proposals from local education agencies that are submitted to them.
Administration of the Vocational Education R&D Program 67 STATE ADVISORY COUNCILS ON VOCATIONAL EDUCATION The state advisory councils on vocational education (SACVES) are in- volved in planning with regard to the states' share of Parts C and D. Functioning autonomously, the councils are responsible for evaluating the effectiveness of state and local vocational education programs in terms of the goals and objectives outlined in the state plans. Recommen- dations for change resulting from state evaluations, contained in annual evaluation reports, are sent to the National Advisory Council on Voca- tional Education, USOE, the particular state board of vocational educa- tion, and other state agencies. The relationships of the SACVES to RCUS differ widely. For example, among the sample of ten states visited, five SACVES advise the RCUS on research priorities and read and react to re- search proposals. In the other five states, the SACVES do not do so and are not seen by the RCUS as being very helpful to them. SACVES vary considerably in the size of their budgets. Those SACVES whose budgets provide for more than a minimal staff are frequently in- volved in supporting research to aid their evaluative functions. For ex- ample, Illinois had funded 25 projects and Texas had funded 17 projects as of June 1975. Most projects were funded by the annual federal allot- ment for SACVES, but the availability of state funds for other SACVE activi- ties frees federal monies for SACVE research and evaluative activities. DISSEMINATION AND UTILIZATION Dissemination and utilization activities have been inadequate for voca- tional education R&D to have had measurable impact. After R&D projects are completed, final reports are usually prepared (as required by USOE), the required number of copies are supplied to USOE, and some distribu- tion is made. Reports are then entered in the vocational education in- formation storage and retrieval systems (ERIC and AIM/A~) and occa- sionally reviewed or mentioned in other publications. Although a wide variety of techniques for dissemination have been developed, widespread dissemination is rare, and little attention has been given to increasing the use or adoption of disseminated products. Systematic identification of target audiences and packaging of information to meet the needs of different audiences have been lacking. Joseph F. Blake (1975, p. 25) iden- tified some special needs: The practitioner groups need information for policy decisions and program de- velopment. The researcher-developer group needs information about completed
68 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT and on-going work to avoid duplication and to benefit by existent knowledge. Among these groups the need is for collection and processing of materials acces- sible and available on a continuing stable basis. They need a system searchable through different levels of sophistication and capable of providing repackaged and digestible versions for special target audiences. There is little documentation of the utilization of R&D products and frequent failure to distinguish between dissemination and utilization. Ex- isting documentation of utilization is questionable because it often does not really measure the actual adoption of products, but rather the extent to which products have been disseminated. For example, some states document "utilization" by keeping records of requests for microfiche copies of research reports. There is apparent difficulty in identifying utili- zation of the results of information-seeking research as well as utilization of the products of development-oriented research. For example, it is dif- ficult to determine if the new information gained by a certain research activity was considered in the decision-making process that resulted in a certain change. Blake (1975, p. 52) identifies some of the present problems in improv- ing utilization by practitioners: ~ Many practitioners do not understand how R&D can help them deal with their daily problems. · Many practitioners need help in identifying, locating, and acquiring potentially useful materials. · Practitioners need more information or help to determine the prob- able utility, reliability, or validity of R&D products and results. · Many practitioners need help in interpreting and applying the findings of research reports. At present, many administrative agencies have some responsibility for dissemination, yet the vast majority of R&D products and results are not widely disseminated, and there is no systematic way of determining which ones should be disseminated. It is not clear which level of project administration (federal, regional, state, or local) has or should have pri- mary responsibility for encouraging practitioner utilization of R&D prod- ucts and results. CONCLUSIONS AND RECOMMENDATIONS The following conclusions and recommendations in this section are based on the information presented above on the administration of the
Administration of the Vocational Education R&D Program 69 vocational education R&D program. The Committee recommends changes in only the functions and institutions that are most important to the vocational education R&D program. These conclusions and recom- mendations are organized differently from the descriptive material above; they are grouped into four categories: program structure, pro- gram planning and administration, institutions involved in the R&D pro- gram, and dissemination and utilization. PROGRAM STRUCTURE Consolidation and Use of Parts C, D, and I of the Vocational Education R&D Program Conclusion The results of research funded under Part C have not been fully used as a basis for developing curriculum products under Part I and designing demonstrations under Part D. The lack of coordination and the separate setting of priorities among Parts C, D, and I have reduced the potential for a cumulative impact of the R&D program. In addition, there has been some overlap in the activities funded under the separate Parts, due in part to legislative mandate. At the state level, the points at which the Committee was able to find links between Parts C and D were in states in which both programs were administered by the RCU. In some states there is virtually no communica- tion between the Part C program administered by the RCU and the Part D program administered by a division within the SEA. Recommendation Without reducing the overall appropriation, the Con- gress should change existing legislation to provide that the vocational education R&D programs be consolidated. In addition, the legislation should mandate that not less than 20 percent be allocated for research, defined as scientific inquiry designed to generate new knowledge. Research, development, demonstration, dissemination, and evaluation are related elements of the same R&D system and belong together for purposes of administration, coordination, and communication. R&D products resulting from research programs should be considered in defining some of the priorities for demonstration and curriculum devel- opment. Unexpected results from demonstration and curriculum devel- opment should be further studied by researchers. Results from research, development, and demonstration projects should be considered in setting priorities for R&D and should be used in the training of vocational educa- tion personnel.
70 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Administrative Location Conclusion Rapid shifts in policies and goals of the vocational educa- tion R&D program are due in part to frequently changing administrators. Further, the three parts of the R&D program have been shifted and reorga- nized within USOE, fragmenting the total effort and, at times, removing research from its proximity to the operating unit for vocational educa- tion, the Bureau of Occupational and Adult Education. Recommendation The Committee recommends to Congress, the Secre- tary of HEW, and the Commissioner of Education that the research, dem- onstration, and curriculum development parts of the vocational educa- tion R&D program (Parts C, D, and I) continue to be administratively located within the Bureau of Occupational and Adult Education in USOE in order to help ensure stability of planning and to facilitate coordination of research with program operations. Commissioner's Share of Funds Conclusion Because both the Commissioner's (federal) half and states' half of Parts C and D funds are distributed to states on a formula basis, SEAS have received a sizable portion of vocational education R&D funds, and problems of multi-state or national scope have not received ade- quate attention. Recommendation Congress should designate 50 percent of all vocation- al education R&D funds (Parts C, D, and I) as the Commissioner's share, reserved for attempts to solve national or multi-state problems; these funds should not be subject to state formula allocations. Career Education Conclusion Career education has been supported heavily with vocation- al education R&D funds since 1971, and activities supported with these funds have contributed materially to the development of career educa- tion. Career education is a broad concept in which vocational education plays an important role. However, there are major conceptual differences . . In t. near programs. Recommendation The Congress should authorize separate funding for career education R&D and vocational education R&D. The Commissioner
Administration of the Vocational Education R&D Program 71 of Education should articulate or integrate, as appropriate, career educa- tion and vocational education R&D. PROGRAM PLANNING AND ADMINISTRATION Procedures for Setting Priorities Conclusion There has not been a long-term, scientifically based sched- ule of national priorities for the support of research, demonstration, and curriculum development. Only one research topic, guidance and counsel- ing, has been included in the list of research (Part C) priorities each year of the past decade. Some research topics have appeared as priorities once or twice during the past ten years, and some important research topics have never appeared as priorities. During two years, fiscal 1972 and fiscal 1973, there was only one designated research priority- career education, with an emphasis on guidance and counseling. There appears to be no rational or empirical basis for the inclusion, exclusion, or repetition of particular priorities. While priorities are established through suggestions from vocational educators and administrators, rarely have researchers been adequately represented in the process of setting priorities. Those involved in vocational education R&D have not been informed about the process and have no formal means for participating in it. State priorities for research and related activities are often determined administratively without any involvement of research producers and consumers in the state. In many states there appears to be no formal procedure for setting priorities. Recommendation A systematic, open, cumulative, and data-based pro- cess should be initiated by the Commissioner of Education for identify- ing national priorities for vocational education R&D and by the state directors of vocational education for identifying state priorities. The pro- cess should involve advisory groups at both national and state levels that represent the clients of vocational education, including students, employ- ees and employers, and professionals in vocational education. More in- put from researchers is necessary to determine what R&D iS feasible from a scientific viewpoint and to identify the most appropriate mode of work- ing on each problem and the most productive sequence of working on different problems. Terms of advisory group members should be suf- ficiently long and overlapping to allow setting priorities that are long- term and programmatic as well as those that address problems requiring a more immediate response. Priorities should be reviewed and updated
72 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT annually. The process should be well-publicized so that those who want to participate can do so. The process should also use the cumulative data base developed by management information systems. Administration of Awards Conclusion The availability of the Commissioner's share of federal funds for vocational education R&D projects has been advertised as RFPS in the Commerce Business Daily and as grant competition announce- ments in the Federal Register. Awards for Parts C and D are distributed according to a state formula, and applications (or proposals) are subject to veto by state directors of vocational education if the directors judge them to be duplicative. Using only RFPS, especially when further restric- tions are imposed by state veto, does not adequately allow or encourage researchers with an exceptional idea outside the announced research priority areas to submit proposals or applications. In some states, the states' share of federal funds for R&D iS allocated primarily for the study of state administrative problems. In other states, priorities for allocating resources are determined solely by administrators (state directors of vocational education or RCU directors). Some states announce priorities and issue RFPS or announcements of award competi- tions. Only a few states follow procurement policies that permit open competition in response to state priorities determined through open pro- cedures. Recommendation A broad mix of announcements and funding proce- dures should be used by the Commissioner of Education and the state directors of vocational education. The appropriateness of contracts, grants, or sole-source funding will vary with the nature of each project and the general availability and interest of competent researchers. It should be recognized that field-initiated applications and proposals that are not in response to contract or grant announcements also have advan- tages. In the interest of encouraging innovation, a portion of R&D funds should be reserved for good proposals or applications that do not ad- dress federal or state priorities or are not in response to RFPS. In order to accommodate the variable capacity of contenders to pro- duce proposals rapidly, USOE and the states should allow potential appli- cants at least two months to respond to the announcements of all compe- titions for federal research funds. A pre-announcement of the date on which a grant competition is to appear may be useful. USOE and the states should also experiment with a two-phase announcement process: in the first phase, an initial announcement would solicit project prospect);
Administration of the Vocational Education R&D Program 73 in the second phase, those applicants whose projects are considered promising would be encouraged to develop full proposals. The Role of Women and Minorities in Research Activity Conclusion Women have been involved as researchers in vocational education far less often than men. There are virtually no data on the involvement of minorities in R&D projects. Further, vocational education researchers have not taken into account research on perceptions held by different groups within society concerning the acquisition of desirable skills, role performance, and attitudes toward time and work. Recommendation The Commissioner of Education and state directors of vocational education should ensure that researchers and administra- tors representative of population subgroups (women and minorities) are involved in the R&D program. All REPS and announcements of research opportunities should state that women and minorities as well as others are encouraged to apply. In addition, for the purpose of ascertaining trends in the participation of various population subgroups in vocational education research activities, the Commissioner and the state directors should keep annual status reports on the percentage of R&D project di- rectors who are members of various population subgroups. The Commis- sioner should also encourage the involvement of researchers from popu- lation subgroups in R&D concerning those subgroups. Funds for training personnel should be used to build the R&D capacity of these groups. It should be determined how data on cultural differences can be used to create opportunities for equitable access to vocational programs and jobs. Management Information System Conclusion USOE does not have an efficient system for collecting and recording information on many aspects of the vocational education R&D program. Little information on research performers is collected; records of project impact are not usually kept; and those project evaluations that have been done have rarely been analyzed in depth and used to improve programs. Moreover, as in any research program, not all project di- rectors submit interim and final reports. Therefore, it is extremely dif- ficult to measure the impact that R&D projects have had on vocational education. There is essentially no evidence of quality control of research performance to ensure that the quality of past work of researchers affects the probability that they will receive awards in the future. There is no
74 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT system for collecting information on the research needs of vocational educators or on unnecessary duplication of research projects. In addi- tion, there is no way to determine whether or not isolated charges of duplication of effort are accurate. i, Some states rely heavily on management information systems while others lack systems for information collection and use. The extensiveness of monitoring and evaluating activities and the extent to which systems are dynamic and interactive also vary greatly among states. Recommendation The Commiss.ion~r of F`l,~r~timn ch~l.lr1 Or; it_ h ~ BEAT Amp ~lVVl~c Io1 system for vocational education R&D. Such a system should include data for monitoring and evaluating projects, measuring dissemination and utilization of project results, keeping track of the aualitv of r~nrrh t~t~f~rm~n~P anA ~q;~+~;~;~ __ ~1 ~. . . ~! __~^ ~ ~4 ~-~01 ~11 c, a~u ma~n~a~n~ng records of the characteristics of the re- search program (including data on research performers, institutional affiliation of award recipients, types of projects funded, and amount awarded to each state). Analysis of these data would help USOE identify factors that are critical in determining the success or usefulness of R&D oroiects. The devel`)nm~nt roof state If ;_` _ `:_ . 1 J ~ Rae ~ ~ =~- 1tl~l1CL~lll~llt 1lllullllatlon systems ~ 1 1 1 ~ _ _ snoula also be encouraged by state directors of vocational education. An important function of a management information system for R&D would be to provide a systematic means for collecting information on the needs of vocational educators. At the same time, topics that have re- ceived repeated and duplicative attention could be identified and unnec- essary duplication eliminated. INSTITUTIONS National Vocational Education R&D Centers Conclusion The national vocational education R&D centers serve useful and essential functions. The centers have increased the research capabili- ty of vocational education and have studied topics of national impor- tance. They have considerable potential for filling the present need for national leadership in policy making for vocational education R&D. However, the centers must seek support by pursuing contracts and grants on a project-purchase basis from diverse agencies. Thus, only in scattered instances have they been able to devote themselves to long- range vocational education projects of national importance. In addition, the centers are forced to compete with agencies within their states for Part C funds. This has tended to impair the relationship between the centers and the vocational education communities within their states.
Administration of the Vocational Education R&D Program 75 Recommendation Congress, the Secretary of HEW, and the Commission- er of Education should ensure that there is at least one adequately fund- ed national vocational education R&D center. The national R&D effort in vocational education needs continuity, from setting priorities to the use of validated results. The efforts would be enhanced by comprehensive- ness that goes beyond that normally achieved in single, unrelated proj- ects. Variety and quality of work would be increased through the opera- tion of two or three national R&D centers. The cemetery) should receive support from federal vocational education funds, including adequate re- sources for activities initiated by the centers. The centers) should address priorities that meet primarily national or multi-state (as opposed to state or local) needs and should build R&D capacity. However, they should also be free to contract with any state or locality to provide needed services so long as these do not interfere with the primary task of meeting national and multi-state needs. The center~s) should aid in planning and policy development for the national voca- tional education R&D system, including providing USOE and states with data needed for planning. USOE should view the center~s) as the appropri- ate place for conducting high-risk research where the payoff may be high. The cemetery) should communicate with the research operations within each state; they should assist in dissemination of research prod- ucts and in the training necessary for carrying out R&D activities. To increase the relevance of research, practitioners (teachers, counselors, etc.) should be involved in center operations, including planning, policy decisions, and improving dissemination capacity. There should be fund- ing to allow practitioners to work with centered), through means such as grants and summer institutes. Curriculum Centers Conclusion The National Network for Curriculum Coordination in Vo- cational and Technical Education funded by Part I has received less than adequate federal financial support for its intended activities. It is a re- gional effort, but the only regional financial support that is available comes through voluntary cooperation of groups of states; in only rare instances have groups of states contributed the necessary finances. Con- sequently, the efforts of the network have been fragmented and ineffec- tive. Further, the centers are duplicating some of the functions of a com- prehensive information system as well as some Part C functions usually performed by RCUS. There is no apparent reason for separating the coor- dination and dissemination of curriculum development products from that of other R&D products.
76 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Recommendation The Commissioner of Education should require that the following curriculum coordination functions be performed: (1) iden- tifyina common curriculum ne,~`l~ Mad ctot-~. I..; 1~_~1 ~,,^~-~e, ~ All ~ll~wula 111~ 1~1 . . . ~ practitioners to become involved in curriculum development; ( ) feeding curriculum information for the states into ERIC and AIM/A~; (4) per- forming curriculum searches of these systems for vocational educators; (5) improving techniques for curriculum development, and (6) discourag- ing unnecessary duplication in curriculum development. Since curricu- lum development is very expensive. elimin.~tins, ,~nn`~r'`~r`, Al'^~;^q+;~ is economically desirable. ~O _ ^ ~vow ~Fll~cLbl~ll If adequate funding for the Curriculum Center Network cannot be provided, the Network should be disbanded and the functions should be assumed by the national vocational education R&D centerts). The R&D center~s) should receive additional funding, which would probably be less than that needed to support separate institutions partly because of economies of scale. Research Coordinating Units Conclusion RCUS vary widely in organization, function, and effective- ness. All RCUS have attempted to stimulate state and local interest in the R&D process and disseminate information on R&D products, thereby legi- timizing R&D within the states. However, many states have had difficulty in disseminating research results and products and in promoting their utilization. In addition, some states deliberately separate the administra- tion of research from that of development and demonstration. In small states, RCUS have not had enough money to support research, a full-time director, and clerical services. In some states, Part C funds are being used for puIposes that should be supported by program operating funds (Part B), for example, operation of state management information sys- tems and routine program evaluation (although development of manage- ment information systems and plans for evaluation are legitimate R&D functions). Recommendation Congress and the Commissioner of Education should ensure that the Office of Education allocate funds specifically for the RCUS based on a periodic evaluation of each RCU'S activities. States that have effective RCUS should receive a minimum allocation (approximately $25,000) plus a population-determined amount, even if this necessitates a cutback in the present funding levels of the larger states. The states should be encouraged to provide additional funding from program mon- ies (Part B) and from other state sources. One way of doing this would
Administration of the Vocational Education R&D Program 77 be to require that federal research funds for RCUS be matched (perhaps at a 25-percent rate) from federal funds for program operation or state sources. The Office of Education should provide guidelines and organizational support for the management of R&D in RCUS and for the establishment of cooperative activities among RCUS. RCUS should be responsible for state-level management of demonstra- tions (Part D), research (Part C), and curriculum development projects (Part I). They should be required to develop a process for defining re- search priorities for their states and providing input for setting priorities at the federal level. RCUS should also be concerned with the validation of R&D products before these products are widely disseminated or imple- mented in the state. RCUS should require that intensive evaluation be built into a sample of their research projects. RCUS should be required to continue dissemination and utilization efforts in order to develop their capacities to serve as brokers of R&D. RCUS should cooperate with the national R&D centers and with other national dissemination efforts. RCUS should both provide intellectual leadership for research and involve local practitioners in research in order to facilitate the utilization of R&D re- sults in vocational education. DISSEMINATION AND UTILIZATION The Committee views dissemination and utilization as extremely impor- tant aspects of the vocational education R&D program. Both are essential in moving R&D products into operating vocational education programs. Three components of the dissemination and utilization process are dis- cussed below in separate conclusions and recommendations: informa- tion collection and retrieval, information analysis, and utilization. Dis- semination is discussed in conjunction with each of these components. 1 Information Collection and Retrieval Conclusion Research reports and other products of vocational educa- tion R&D have been made accessible by the -sponsored ERIC system, supplemented by the usoE-sponsored AIM/ project. However, the development of a comprehensive information resource system linked to a dissemination network has not received adequate support from Central ERIC. The work of ERIC/CICE and MM/A~ have not been sufficiently integrated and, taken together, have only partly met the needs of voca- tional education personnel. Audiovisual materials are one of the major omissions from existing retrieval systems.
78 ASSESSING VOCATIONAL EDUCATION RESEARCH AND DEVELOPMENT Recommendation The Secretary of HEW and the Commissioner of Edu- cation should ensure that vocational education has a comprehensive in- formation resource system linked to a dissemination network serving practitioners. The vocational and technical education portion of ERIC/CICE should be administered separately by USOE'S Bureau of Occu- pational and Adult Education until Central ERIC iS able to support a comprehensive system for vocational education. The system should be operated by an organization that has demonstrated ability to coordinate divergent activities and develop strong links with practitioners. An ade- quately funded clearinghouse for vocational and technical education should include AIM/ARM and articulate with other vocational education . . . R&D activities. Every vocational education R&D project should be required to submit its reports and products to ERIC'S Resources in Education and Current Index to Journals in Education, to AIM/ARM, and to vocational education research libraries, where they will be available for selection, adoption, adaptation, and installation by users through the linking dissemination network. In addition, selected projects should be required and funded to widely disseminate their own R&D products. There should be an intensive program of activities to help RCUS in their dissemination role in a com- prehensive dissemination network. The ERIC and MM/A~ systems should cooperate with other agencies to maintain system compatibility and avoid duplication of effort. The vocational education information resource system should cooperate with Central ERIC, ERIC contractors, other information systems, and the vocational education community to overcome the persistent technical problems related to copyrighted, non- print, and poor print materials. Some provision should also be made for establishing a system for maintaining and disseminating audiovisual ma- terials. Information Analysis Conclusion Many vocational educators are either unaware of R&D re- sults and products or are unable to understand (and subsequently use) them. Simply mailing out copies of reports is not always adequate. In- formation synthesis and analysis have not been given adequate or long- lasting support by USOE. Various kinds of information analysis are often desirable: summary and synthesis of research on the same or related topics, and research analyses perfo~ed for different purposes or differ- ent audiences. Nonprint media are often effective but not often used. Personal contacts between the research and practice communities at workshops and demonstration sites have been more successful than writ
Administration of the Vocational Education R&D Program 79 ten forms of communication in disseminating R&D but have been used much less frequently. Recommendation The Commissioner of Education should establish an information analysis program to transform information on critical prob- lems into appealing, new forms targeted to diverse user groups. These new products should include interpretations of research and commis- sioned analyses of research. Special collections of information on topics such as vocational education consultants, nonprint media, legislation, and exemplary programs should be developed. The Commissioner of Education should fund studies of the needs of users of R&D to determine the most effective methods of dissemination and the most appropriate forms of information analysis for different situations and different users. New products and dissemination strategies based on the results of user studies should be developed and implement- ed. Utilization Conclusion In general, neither the federal sponsors (in requests for pro- posals and grant announcements) nor researchers have planned for ade- quate dissemination and utilization activities. There has been little effort to increase user receptivity to R&D outcomes, to provide technical assis- tance in utilization, or to document utilization. Dissemination and utiliza- tion have not been included among federal priorities for vocational edu- cation R&D. Recommendation USOE must assume responsibility for ensuring that R&D results and products are disseminated and utilized and should desig- nate a significant proportion of federal R&D funds for these activities. RCUS should be funded and encouraged to serve as state-level links in the dissemination network and to provide practitioners with opportunities for involvement in the R&D process. The national center~s) should be made responsible for assessing R&D outcomes, for creating user aware- ness of promising innovations, and for assisting users with application of suitable R&D outcomes. Periodically, selected SEAS, local education agen- cies, universities, or professional organizations should be funded to dem- onstrate dissemination strategies, to assist with dissemination and utili- zation of specific products, and to conduct research on dissemination and utilization. User training programs involving R&D personnel, admin- istrators, and educators designed to improve the flow of information from the resource system to the practice community should be conducted.