6
Fostering High-Quality Training

The findings in the preceding five chapters have led the committee to conclude that the primary aim of federal postsecondary training policy must be the development of state and local systems of training that provide high-quality and coherent training options to people throughout their working lives.

The committee came to realize that we, like many others, were grappling with a more general challenge of improving the performance of the agencies that administer public programs.1 This is directly analogous to the situation described in Chapter 1, where private firms are faced with the challenge of restructuring themselves because the old ways of doing business are no longer adequate. We found ourselves, like other critics of "business as usual" in the public sector, searching for management approaches that are compatible with the need to "grope along" (Behn, 1988): we were looking for approaches based on clear objectives, with the knowledge that achieving the objectives will involve continuous adaptation and improvement, not the mechanistic implementation of some predetermined grand strategy.

What might improved public management mean for federal postsecondary training policy? The committee investigation into how to improve the quality of postsecondary training yielded one answer. We gave a great deal of attention to whether performance management and standard-setting, which are increasingly being adopted in federal training programs, are likely to be effective tools to improve the quality of postsecondary training. We concluded that performance-management and standard-setting approaches that produce information useful for continuous improvement and that in



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy 6 Fostering High-Quality Training The findings in the preceding five chapters have led the committee to conclude that the primary aim of federal postsecondary training policy must be the development of state and local systems of training that provide high-quality and coherent training options to people throughout their working lives. The committee came to realize that we, like many others, were grappling with a more general challenge of improving the performance of the agencies that administer public programs.1 This is directly analogous to the situation described in Chapter 1, where private firms are faced with the challenge of restructuring themselves because the old ways of doing business are no longer adequate. We found ourselves, like other critics of "business as usual" in the public sector, searching for management approaches that are compatible with the need to "grope along" (Behn, 1988): we were looking for approaches based on clear objectives, with the knowledge that achieving the objectives will involve continuous adaptation and improvement, not the mechanistic implementation of some predetermined grand strategy. What might improved public management mean for federal postsecondary training policy? The committee investigation into how to improve the quality of postsecondary training yielded one answer. We gave a great deal of attention to whether performance management and standard-setting, which are increasingly being adopted in federal training programs, are likely to be effective tools to improve the quality of postsecondary training. We concluded that performance-management and standard-setting approaches that produce information useful for continuous improvement and that in

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy crease the capacity of training organizations to provide coherent and high-quality training are the most promising tools for ensuring quality in the postsecondary training world. Moving postsecondary training policy in the direction of high quality and coherence is not an easy task. It demands careful thinking about federalism and about how to coordinate three levels of government, training providers, and employers; it requires marshaling resources from multiple financing sources. To improve quality and coherence, policy makers must focus on adding value that benefits those who are trained and that justifies the costs to those who pay; and changes must include mechanisms that are flexible enough to adjust as the needs of workers and employers change. Chapter 3 showed that performance management is currently in vogue as a way of using the federal government's financial and political leverage to encourage institutions to move toward high-quality, effective programs. The chapter also demonstrated that this approach does not always have desirable consequences: performance management normally focuses on outcomes, but the performance that should be measured relates to the value added by a program. Outcomes and value added may not be highly correlated. This chapter also shows that the institutional structure of education and training complicates the standard-setting approach in ways that have important implications for how the federal government can help improve the quality of postsecondary training. THE DEBATE OVER STANDARDS2 Using the language of performance management, Chapter 3 defined performance measures as indicators of how well an entity is performing. Performance standards were defined as the specification of acceptable levels of performance. Hill et al. (1993:12) point out that measures and standards can be defined for a host of entities: students, courses, programs, institutions, school districts, regions, and states. The Job Training Partnership Act (JTPA) performance system, for example, sets highly quantifiable measures and standards for service delivery areas. A different kind of standard, this one for curriculum, is growing out of the current policy debate over what should be taught in elementary and secondary schools. Standards in this view are statements embodying a coherent vision of what an individual should know and be able to do at a given stage in his or her educational development. 3 Such standards can also be used to establish the level of accomplishment that must be achieved to receive an award, such as a high school diploma or a certificate of mastery. The growing popularity of performance management systems coincides with increasing debate over how standards of performance might help to

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy achieve national education and work-readiness goals. Advocates assert that standards of various kinds can produce a variety of benefits. Standards can provide clearer guidance and direction for education and training institutions. A common critique of American education and training institutions is that they lack a common set of goals or expectations for what students should learn and why. Advocates argue that if standards are based on clearly defined goals, and if these goals are based on realistic appraisals of what students or trainees will need to know in order to perform effectively in society and the economy, then education and training institutions will link what they teach and how they teach to the goals. Standards can focus education and training institutions primarily on outcomes, rather than inputs. Education and training policies have, until recently, emphasized regulating spending to ensure that money reaches its intended target. This approach meant that institutions had to focus on compliance with regulations rather than on performance. A major theme in the current standards debate is that policy should instead primarily promote and reward higher level performance, allowing institutions maximum flexibility in how they use money, technology, and teaching to meet performance standards. Standards can encourage greater coherence and coordination among education and training services targeted on similar populations. The variety and complexity of postsecondary institutions encourages each one to think of itself and its clients as unique, rather than as a part of a larger system of institutions organized around a common set of goals. Advocates of national standards, especially standards of student performance, argue that expressing national goals and translating them into standards will emphasize the common purposes of training institutions and produce greater coherence among them. Standards can improve accountability. Fiscal pressure at the federal and state level, coupled with concern about international economic competition, has caused policy makers to demand greater accountability from education and training institutions. Advocates of standards argue that it will be extremely difficult to make a case for public expenditures on education and training in the absence of solid evidence on performance. They add that evidence on performance has little meaning in the absence of agreed-upon standards against which the performance of individuals and institutions can be assessed. Standards can improve evaluation and diagnosis. Most policy makers and opinion leaders are convinced that U.S. education and training institutions have serious problems meeting the emerging demands of society and the economy. Yet few agree on what to do about it. Advocates of standards argue that it is impossible to evaluate the effectiveness of particular education and training programs or institutions, much less to assess the overall

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy effectiveness of these institutions, without significant agreement on what they are supposed to do. Standards can provide a set of benchmarks for evaluating the effectiveness of programs and institutions and for diagnosing where the key weaknesses are. Standards can provide a basis for certifying institutions and establishing credentials for individuals. It is frequently alleged that a major weakness of American education and training is that institutions and individuals often do not know whether they are performing acceptably, because they seldom receive clear feedback on their performance. Employers seldom ask to see school transcripts or the comments of teachers from training programs when they hire new employees (Bishop, 1990). Schools and other training organizations often know little about the immediate effect they have on their students or the long-run effects of their education and training programs. Advocates of standards argue that institutions that succeed in preparing individuals for society and work should be recognized and rewarded for their success, and those that do not succeed should be encouraged to change or close down. Likewise, students or trainees who succeed in learning should be recognized as having achieved important levels of knowledge and skill; those who do not should not be recognized. In this view, agreed-upon standards of knowledge and performance are a basic condition for being able to certify institutions and give credentials to individuals honestly and fairly. These six purposes—guidance, focus on performance, coherence and coordination, accountability, diagnosis and evaluation, and certification and credentialing—figure prominently in the current debate over national education and training standards. Advocates often disagree on the relative importance of these purposes, however, and even on whether certain purposes should be embraced. Some argue, for example, that standards should be used primarily for guidance and diagnosis, but not for awarding credentials to individuals, because the costs of error in assessing individuals are too great. Some critics of national education and training standards argue that they will lead inevitably to the institutionalization of existing inequalities, because the standards will be manipulated to protect the privileged and exclude the underprivileged. Other critics argue that, while standards are generally a good idea, national standards are suspect, because the responsibility for most governance decisions related to education and training is located at the state and local level. No one claims that standards alone will improve education and training institutions. Standards are only as good as the measurement and assessment techniques that accompany them. In the absence of incentives and sanctions that reinforce them, standards may be only symbolic. Standards that attempt to influence complex processes, such as teaching and learning, re

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy quire the organizations and individuals that implement them to have expert knowledge and skills. Finally, standards imply that a standard-setting body commands sufficient authority to influence the institutions that are expected to implement the standards. In sum, standards must be viewed as part of a complex web of factors that must be made to work in parallel in order to influence the delivery of education and training. Definitions Performance measures and standards fall into two broad categories: design, related to inputs and processes; and outcome, related to results. A design measure for inputs might, for example, be the percentage of a particular client group (low-income, the unemployed) enrolled in a program. The parallel standard, established by law or regulation, would identify the minimum percentage of the client group that the program must enroll in order to perform satisfactorily. Design standards relating to processes can be used as instruments to improve the quality of services, in contrast to input standards, which are used to target services. Using process-oriented design standards requires the introduction of benchmarking. A process-oriented design standard can be based on empirically verified practices that seem to be effective in achieving desired outcomes. This conception of design standards is more similar to the Underwriters' Laboratory's industry standards for safety in electrical appliances, the Society of Automotive Engineers' standards for automobile parts and fluids, or the advanced standards set by certain firms for the processing time of computer hardware. The basic premise is that one can establish a tighter link between quality and performance by making information available about practices that seem to be effective in achieving certain kinds of performance. Leading-edge standards, which will be discussed below, are an application of this idea. Outcome measures and standards, by contrast, focus on results accomplished. If an outcome measure is how many people are employed 3 months after completion of a training program, for example, the standard for satisfactory performance might be set at 60 percent or better (McDonnell and Elmore, 1987; Bardach and Kagan, 1982). Recent conventional wisdom has been that outcome standards are preferable to design standards. There are a number of complex problems, however in designing policies that focus on standards with an outcome emphasis. The Mix of Design and Outcome Standards The first problem is determining the best mix of design and outcome standards. While conventional wisdom prefers outcome standards to design

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy standards, performance-management systems typically include both. The performance-management system for JTPA illustrates this point. The federal government sets standards for, among other things, the proportion of trainees expected to be employed at the end of a follow-up period and the average wage at follow-up, both of which relate to outcomes. There are also, however, requirements for such design standards as the proportion of welfare recipients to be served and the proportion of funds to be spent on youth. So when speaking of policies aimed at improving public services, it is important to note that no policy is likely to consist entirely of outcome standards to the exclusion of design standards. It is more accurate to say that such policies shift the balance between design standards and outcome standards, giving more weight to the latter. Identifying Appropriate Measures The second design problem in constructing results-oriented policies is determining what measures to use as the basis for judging performance and providing incentives. A fairly robust principle is, ''You get what you measure and reward.'' We have already discussed one example, the "creaming" problem in JTPA that resulted, at least in part, from cost standards. Welfare policy offers another example, this one stemming from the distinction between outcomes and impacts of programs. An outcome-based assessment of a welfare-to-work program, for example, might reveal that participants with some work experience and income prior to entering the program have an average employment rate of 61 percent after their participation, while those with no prior work experience or income have a 30 percent employment rate. Other things being equal, a service provider rewarded on the basis of outcomes would prefer clients with some minimum amount of work experience. An impact-based assessment, on the other hand, might give a completely different picture. A comparison of participants who had prior income and work experience with others from similar backgrounds might well reveal that the comparison group had an employment rate of 59 percent without the program, leaving a net impact of 2 percent. At the same time, a comparison of those without prior income and work experience to their peers might reveal an employment rate of 22 percent for nonparticipants, meaning the program had a net impact of 8 percent. Judged in terms of raw outcomes, then, the program would be more effective with participants who have prior income and work experience. Judged in terms of impacts, however, the program would be more effective with participants who have had no prior income or work experience (Friedlander, 1988). The message of these examples is two-fold: first, outcome standards,

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy while they are relatively easy to administer, may not give a very accurate picture of the actual performance of programs. When outcomes and impacts are not highly correlated, mandating outcomes may focus program administrators in precisely the wrong direction. Second, linking incentives to performance involves complex judgments about what constitutes adding value as opposed to simply learning how to do well within the parameters of the performance system. Setting Design and Outcome Criteria The third problem involved in developing standards policies is determining where to set design and outcome criteria. In the literature on the political economy of regulation (McKean, 1980; Stigler, 1971; Zeckhauser and Viscusi, 1979), the problem of standard-setting is portrayed in the following way: administrative agencies have limited resources for monitoring and enforcing standards and, in the case of outcome standards, limited resources for rewards and sanctions. The agencies that are the objects of these standards, rewards, and sanctions—the implementing agencies—have interests of their own. Implementing agencies calculate the costs and benefits of compliance with the administrative standards for them and their clients as well as the likelihood that noncompliance or nonperformance will be detected. From the perspective of the administrative agency charged with enforcing standards, the central problem is to set standards high enough, taking account of oversight and enforcement costs, to induce implementing agencies to produce desired results. At the same time, the administrative agency must not set standards so high that large numbers of implementing agencies cannot comply. There are at least three broad standard-setting strategies that administrative agencies can follow. The first might be called the minimum standards approach. The agency sets standards to reflect the minimum acceptable level of performance given the political preferences of policy makers and the oversight and enforcement capacities of the agency. The second might be called the leading-edge standards approach. This involves deliberately setting standards at a level that only a few implementing agencies can achieve, and investing resources in pulling the rest of the implementing agencies along. Variants of this approach in the private sector are sometimes called benchmarking, because they involve setting design and outcome standards for a firm at the leading edge of industry practice rather than at the average level. A third approach might be called problem-driven. This approach involves identifying an existing condition that needs fixing and setting standards consistent with a solution to that condition. The minimum standards approach is the usual solution to most prob

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy lems, because it is well-adapted to political and administrative constraints. An example of the minimum standards approach is the JTPA performance standards. The performance targets that the federal government and the states set for local service delivery areas and service providers in such areas as placement rates and post-program wages are, of necessity, based on expectations that it is reasonable to expect the average provider to meet. Hence, these performance targets provide a reasonable standard against which to judge the performance of all providers, with the expectation that a large proportion of providers will exceed the expectations. In the area of education and training, one can see the clearest evidence of the emergence of a combination of leading-edge and problem-driven standards in a report by the Secretary's (of Labor) Commission on Achieving Necessary Skills (SCANS) (1991). The SCANS standards are leading-edge because they are based on what workplace skills should look like in the future, not on what employers currently demand. They are problem-driven because they are directed at solving the problem of workplace readiness, as defined by evidence on emerging skill needs, rather than being based on a judgment about what regulatory agencies can currently do. How would one develop standards compatible with the SCANS report? It might seem obvious that education and training programs should be judged on the basis of their success in placing students. But the market for students with the kind of competencies the SCANS report envisions might be limited, as the report admits, because of the time lag in restructuring firms to accommodate people with high-performance skills.4 Over the long run, economists might tell us, this is a trivial problem; in the short run, however, it presents an important problem of evaluation and standards-setting. Should one continue to push education and training institutions to use leading-edge design standards and benchmarking based on the SCANS model, even though the capacity of firms to absorb the better-trained workers is limited? Determining where to set criteria in the design of standards systems, then, is a major strategic problem not amenable to simple solutions. A few things seem clear, however. In the absence of strong countervailing political pressures, standards systems tend to gravitate toward the minimum standards approach. Moving standards systems toward the leading-edge or problem-driven approaches requires some clever institutional design. One solution can be found in California, which is implementing leading-edge curriculum frameworks in elementary and secondary education. The state has tried to avoid the minimum standards problem by using experts and leading-edge practitioners to set design standards and by not insisting on short-term compliance, but rather focusing on long-term changes in performance.

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy Creating an Institutional Structure The final problem is determining the institutional structure within which standards will be set and implemented. Most analytic discussions of standards-setting ignore the issue of institutional structure, yet, as discussed below, institutional structures have an enormous influence on the way standards work. For example, ignoring for a moment the problem of whether the change resulted in a better program, it is generally acknowledged that JTPA performance standards have shifted the service delivery system's focus on inputs to a preoccupation with mandated outcomes. The institutional conditions of JTPA, however, are almost unique, and the same conditions would not necessarily apply more broadly in the education and training sector. Specifically, JTPA is what might be called a high-leverage program: it is wholly funded with federal money; the system that administers it at the state and local level is devoted primarily to carrying out this responsibility; and the service providers in the system either draw their support mainly from JTPA, or administer JTPA programs separately from other activities in ways that maximize external accountability. One would expect that standards would be quite effective in directing and focusing resources under these institutional conditions. Imagine what would happen to the development and implementation of standards if these institutional conditions were relaxed. While moving from a single source of funding to multiple sources, for example, different standards are likely to apply to the same or similar functions within a given organization. Moving from a single administrative structure to multiple structures, each with its own governance system, raises other issues: where does the authority come from to set standards and who adjudicates differences among standards? When providers of training no longer draw their support primarily from one source or provide certain services that are funded completely from one source and instead become relatively autonomous actors drawing support from multiple sources, how much leverage can any single funding agency exercise over the performance of any single provider? Standards imply authority. Where authority is clearly defined and focused, standards seem to be an efficient mechanism of influence and control. Where authority is diffuse, the role of standards in influence and control seems more problematic. Much of postsecondary training clearly falls into diffuse authority category. It is possible to define a role for standards where authority is diffuse. But, as discussed below, doing so requires a broader view of standards than exists in the current approach to performance management in federal programs.

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy CONCLUSIONS The impulse behind performance management and the spread of performance standards in federal programs is praiseworthy, but the complexities involved suggest that caution is warranted. Even in programs like JTPA, where federal authority is clear, the problem of distinguishing impacts from outcomes means that standards-setting is not a simple process. We believe it is important to recognize the dangers involved in tying performance standards to outcomes; to be careful about the use of incentives and sanctions where they may lead administrators to emphasize inappropriate objectives; and to work continuously to develop better ways of measuring impacts and reflecting them in the standards-setting process. Outcome standards are not always preferable to design standards, especially when design standards are process-oriented and focus on best practice. Therefore, the federal government, along with improving outcome-oriented standards, should ensure that research and evaluation efforts in federal programs give attention to determining what works. The federal government should also attempt to encourage the adoption of best practices through its standards-setting efforts. In the many parts of postsecondary training where federal authority is diffuse, the federal government must recognize that much of the delivery system lies outside its direct influence. The standards-setting approach the federal government chooses should reflect the strengths and limits of federal authority as well as the inherent technical complexities of standards design. This suggests that the use of performance management to improve quality needs to go beyond the traditional focus on establishing measures and standards. Performance management should include building capacity into the training system to meet performance standards. The federal interest in ensuring quality must extend beyond federal programs if the entire postsecondary training system is to improve; this suggests standard-setting should emphasize changing institutions rather than regulating federally funded activities. Current federal performance management systems assume that any federal effort to set standards should focus on the recipients of federal funds and that money is the federal government's main source of leverage. We prefer an approach that assumes that the federal government's interest in the quality of training goes further. The federal government has two main sources of leverage: its funding and its capacity to set a national agenda and mobilize key institutional interests behind it. The federal government might pursue at least three strategies in support of a quality assurance system that reaches beyond the providers and recipi

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy ents of federally funded training: fostering the development of voluntary, national, occupation-based skills standards for job entry and advancement; improving information systems; and helping the providers of postsecondary training increase their capacity for improved performance. Skills Standards National skills standards already exist in a few sub-baccalaureate occupations, and the federal government is currently encouraging the development of others through some 20 demonstration projects sponsored jointly by the U.S. Departments of Education and Labor. Skills standards potentially offer a number of solutions to problems we have previously identified in this report: Even in the developmental stage, skills standards can turn the attention of providers toward performance, as demonstrated by curriculum standards in elementary and secondary education. The effort to develop skills standards often gets key players talking and working together in new ways. In particular, skill standards can become a vehicle for closer communication and joint planning between employers and educators. Skills standards can lead to the development of portable credentials for workers that can enhance mobility and improve the job-matching process. Skills standards can improve the content of training and provide a measure of accountability for training providers. Skills standards can make civil rights enforcement easier by making it more difficult to discriminate against qualified workers. If they involve a framework that measures skills at various levels of an occupation, skills standards can create ladders that define career pathways and help document and monitor the progress of people as they move through various kinds of postsecondary training. The existing federal demonstration-project approach, in our view, fails to provide enough of a framework for the development of a national skills standards system. A number of common questions of definition, structure, and governance should be addressed in more than an ad hoc way. For example, commonly used terms, such as skills, competencies, tasks and duties, certification, occupation, and occupational cluster, need to be defined. Currently they mean different things to different standard-setting groups. Berryman and Rosenbaum (1992) point out that a system for creating national standards for work preparation must include national skill-defining

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy and skill-certifying entities that they call boards. These boards face common design issues: What functions should national boards have? Options include core functions like defining and updating generic or industry and occupational skills, establishing needed levels of performance, certifying individuals, etc. Other possible functions include fostering the adoption of skills standards by employers and educators, accrediting training programs, and designing training curricula. When should individuals be allowed to apply for certification? This relates to the kind of foundation skills people will be required to have before being allowed to pursue narrower skills certification. Can they begin after 10th grade? Only after high school graduation? During postsecondary education? What framework should boards use to organize skill standards? How broadly or narrowly will occupations or occupational clusters be defined? Should skill standards be restricted to entry level requirements or to the sub-baccalaureate level, or extended to include skilled and professional workers as well? In part, the issue is how useful skills standards will be for experienced workers as well as for new labor force entrants. What are the implications of the restructuring economy for conceptualizing skills? Will skills standards reflect the requirements of high-performance workplaces, even when these do not reflect current industry practices? How should national boards accommodate existing standards, licensing, certifying, and accrediting processes? Should certifying processes be based in any way on completing recognized formal training? Or should individuals be allowed to try for certificates simply on the basis of demonstrated knowledge and skill? How should national skills standards boards be governed? Should they be public agencies, some blend of public and not-for-profit organizations, or not-for profits, perhaps linked to professional associations? What should be the structure and membership of national boards? How can the boards be organized to ensure that employers have a sense of ownership? How should national boards be financed? Is federal subsidy desirable or essential? The federal government needs a mechanism to address these issues and within which individual skills standards boards can operate. We will propose such a mechanism in the final chapter.

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy Better Information Systems A second strategy through which the federal government can help ensure quality is to build better information systems to enhance the influence of consumers (trainees and employers, alike) who are in the market for training services. We described in Chapter 3 the problems facing would-be trainees and employers in learning enough about the training system to use it well. A great deal of information exists about labor markets and training activities, but it is not always the right information, nor readily accessible. Improving information has several dimensions: improving the federal and state statistical system that provides labor-market data and broadening it to include more useful information about local labor markets; coordinating federal requirements, so that integrated management information systems can be developed at the state level; and improving the availability and accessibility of information to users at the local level. Information can also become an indirect means for ensuring accountability and oversight by providing details of institutional performance that can be used as a basis for client choice, professional sanctions, or political pressure. If understandable information can be made available to those exploring training options, the clients themselves, by the choices they make, could exert a great influence on the improvement of quality in the training system. Those exploring options would benefit greatly from information about the rewards of entering certain occupations, the costs of obtaining the necessary skills from various providers, the relative success of providers in imparting the skills, and the ability of providers to place those whom they serve in jobs and at what wages. This indirect approach contrasts with a more direct-effect model that uses standards as the basis for the administration of formal rewards and sanctions. The indirect-effect model works best when the agency administering the standards has limited direct control over the implementing agency but relatively open access to professional and public channels in order to publicize the performance of the implementing agency. Capacity Building Finally, a federal strategy for quality improvement in postsecondary training should focus on capacity building. Standards can promote continuous improvement when people and institutions know how to produce results consistent with the standards. This truism is routinely violated in the design of regulatory systems and performance standards. The JTPA uses performance standards, for example, but it does not attempt to identify what sort of practices distinguish more- and less-effective employment-training pro

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy grams, except for such general practices as classroom instruction, work experience, on-the-job training, and so on. Systems of performance standards, especially those heavily oriented toward outcomes, tend to rely on a definition of program success that is based on a kind of "natural selection." As long as some organizations succeed by the standards that are set, according to the natural selection view, someone must know what to do. Efforts to improve performance then become a matter of ratcheting up the standards, thereby selecting the fittest and driving out the unfit, rather than attempting to identify the practices that work best. Whatever the utility of this approach, it is not consistent with introducing the philosophy of continuous improvement into education and training organizations. To engage in continuous improvement, an organization must be able to diagnose its shortcomings; it must have access to the knowledge and skills required to remedy whatever shortcomings it finds; and it must have the capacity to do these things distributed throughout the organization in a way that allows for broadly based participation. For the most part, the postsecondary training system is relatively poorly equipped to engage in this sort of continuous improvement activity. People who work in training institutions, with some notable exceptions, occupy relatively low-status roles characterized by working conditions that do not favor continuous learning. The internal structure and management of postsecondary education and training institutions, again with notable exceptions are not conducive to the diagnosis of shortcomings or to creative solutions. Teaching loads are high, pedagogy is relatively primitive, materials and technology are often not advanced. Most institutions that operate training programs are toward the bottom of the educational hierarchy in the state and local structures. If policy makers want to introduce continuous improvement as part of a system for ensuring quality in postsecondary training, the constraints on continuous improvement have to be removed or relaxed. A number of possible measures might signal a serious commitment to change. One measure, as we have already seen, is to encourage the development of voluntary, jointly developed skills standards in which employers, trainers, and public officials all have an important stake. Another is serious investment in developing the competence and enhancing the status of practitioners who train postsecondary students and in identifying and disseminating information on proven effective training practices. Still another measure would be support for the development of the institutional structures that are needed to undergird systemic approaches to training. For several reasons, we favor an emphasis on continuous improvement, rather than top-down regulatory compliance, in the federal government's approach to quality assurance. We noted the compatibility of the indirect

OCR for page 129
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy model of accountability with the diffuse authority exercised by the federal government in many parts of the postsecondary training world. An emphasis on improvement rather than compliance will also help to avoid the risk that standard-setting for postsecondary training will degrade into a minimum standards approach. Leading-edge (or world-class) standards, which we believe should be the benchmarks, run the risk of devolving toward minimum standards if compliance is too heavily stressed. We reiterate that it is important to avoid an emphasis on short-term compliance and prefer a quality assurance strategy that focuses on long-term changes in performance. NOTES 1.   See, for example, Barzelay (1992); Behn (1988); Golden (1990); Nathan (1993); National Commission on the State and Local Public Service (1993); Osborne and Gaebler (1992); and Wilson (1989). In September 1993, President Clinton announced the findings of a 6-month "national performance review," designed to reform federal agencies and programs. 2.   Parts of the remainder of this chapter are from a paper prepared for the committee by Richard F. Elmore. 3.   For example, the National Council of Teachers of Mathematics has developed curriculum and evaluation standards for school mathematics. The effort involved creating "a coherent vision of what it means to be mathematically literate both in a world that relies on calculators and computers to carry out mathematical procedures and in a world where mathematics is rapidly growing and is extensively being applied in diverse fields" and creating "a set of standards to guide the revision of the school mathematics curriculum and its associated evaluation toward this vision" (National Council of Teachers of Mathematics, 1989:1). The standards are meant to be used as criteria against which local and state curriculum and evaluation ideas can be judged. 4.   The report also acknowledges that the skills, knowledge, and personal attributes that schools should be developing in young people are not even broadly demanded by existing firms; it says only that they should be demanded if firms are to be successful in the future. Thus, the report puts part of the onus on private firms for developing high-performance workplaces that will be compatible with the competencies of students who emerge from new educational programs.