Developing and sustaining a skilled technical workforce is a high-priority goal for employers and for policy makers at the local, state, and federal levels. The nation is making substantial public and private investments in achieving a broad range of education and training goals, spending an estimated $618 billion on public K-12 education and an additional $701.5 billion on postsecondary education and training each year.1 This substantial and broad-based commitment has made it possible for U.S. workforce development practices to evolve so they now include a wide range of options that potentially accommodate the diverse needs of American students, workers, and employers. A key issue for the nation, then, is how effectively these substantial resources are used to develop and sustain a skilled technical workforce.
To address this issue, it is necessary to look more closely at the institutions—the rules, norms, and incentive structures—that shape the ability of students, workers, employers, educational organizations, and governments to work together to develop and sustain a skilled technical workforce.2 The economic data reviewed in Chapter 2 capture the visible patterns created by these actors within prevailing institutions. At the microeconomic level, the data
1 The estimate of annual spending on public K-12 education and training is based on NCES (2016c). The estimate includes federal education and training services ($17 billion), employer spending ($167.5 billion), and the total expenditures of postsecondary Title IV institutions ($517 billion). The source for federal spending on education and training services is The White House (2014). The source for employer spending is the Association for Talent Development (2015). The source for postsecondary Title IV institution expenditures is NCES (2016c).
2 Institutions work by changing the structure of incentives individuals face in situations in which they need to cooperate and by creating shared expectations about the behavior of others (North, 1990). Ostrom (2005) observes that “a major problem in understanding institutions relates to the diversity of situations of contemporary life.”
reveal a patchwork of outcomes whereby some labor markets appear to work better than others. Putting this information in context, Chapter 3 shows the great variation in policies and governance systems covering education and training across the nation’s states and localities. And as seen in Chapter 4, this variation is reflected in the diversity of workforce education and training programs across the nation.
This chapter examines the various impediments and disincentives that pose challenges for fully leveraging the nation’s investments in a skilled technical workforce. Section 5.1 looks at the factors that affect the perceived returns on investment in training and education for students, workers, and employers. Section 5.2 explains how some current social and policy frameworks, funding mechanisms, and data gaps impede better outcomes. Section 5.3 then explores the particular challenges of training the skilled technical workforce in the allied health and manufacturing sectors. It also takes up the unique challenges faced by about 200,000 service members who transition each year from active military duty to skilled technical occupations in civilian life. The final section presents conclusions. Chapter 6 addresses some of the policy proposals that might help overcome the challenges discussed in this chapter.
Americans share in the ideals of equal opportunity; fair competition; and the ability to capture a reasonable return on initiative, effort, and other investments. Individuals who invest in skill development should be able to generate enough income from a job or business to cover the associated costs and realize an equal or better return relative to an alternative use of the money. Similarly, enterprises that invest in training their workers should have a fair chance of generating enough revenue to realize a return on their investment. And taxpayers should receive a fair return on their tax dollars in the form of high-quality, reliable public education and training services. Yet limitations in both data and analyses make it difficult to determine whether and how individuals, employers, and the public are investing in skilled technical education and training, and whether they are realizing sufficient returns to sustain or increase their investments.
The returns on investment in acquiring technical skills are both tangible and intangible in nature. Some of these returns—such as higher wages or better benefits for workers; higher profits or greater worker productivity for firms; and additional tax revenues for local, state, and federal governments—can be estimated. Other returns, such as improved civic engagement (see Chapter 2), may be more difficult to measure but may be of equal or even greater importance. In some cases, intrinsic motivation and nonmonetary incentives that
signify social recognition and respect can play a greater role in individual motivation than monetary incentives alone.3 Potential nonmonetary rewards from education and training include the joy of doing something one likes to do or is particularly good at doing; the personal satisfaction derived from accomplishment, social recognition, or respect; and the opportunity to advance.
How students, workers, and employers estimate their return on investment depends on the local context. Education and training for skilled technical jobs are influenced primarily by local policy and practice, and there can be substantial differences across locations, institutions, and industries even within the same state.
5.1.1 Evidence on Returns on Postsecondary Education and Training
Overall, there is strong evidence that postsecondary education yields high positive returns.4 Prior research using national longitudinal survey data has shown that individuals with an associate’s degree earn more than those with only a high school diploma (Belfield and Bailey, 2011). Bailey and Belfield (2013, Chapter 6) estimate that an additional year of schooling raises yearly earnings by 5 to 10 percent, or more than 10 percent after controlling for other factors. Likewise, using data from Washington State, Dadgar and Weiss (2012) observed increases in wages with an associate’s degree, although the size of the increase varied depending on the field of study. A review by the California Community Colleges system estimated average returns of 10 percent in additional earnings for short-term vocational certificates and more than 25 percent for associate’s degrees (AA/AS) (Stevens et al., 2015). Similar results were seen in a study in Florida focused on minorities and disadvantaged workers (Backes et al., 2014), in which students’ decisions to study general humanities and failure to complete those programs had a significant negative impact on their return on postsecondary education. Another study using data from Arkansas, Colorado, Tennessee, Texas, and Virginia found that the returns on associate’s degrees or certificates for career and technical programs of study were higher than those on associate’s degrees for academic programs of study. In four of these five states, moreover, average first-year earnings were higher for associate’s degree holders than for bachelor’s degree holders (Schneider, 2013). Interestingly, researchers using data from Kentucky found that the returns on degrees or diplomas differed for men and women, with women receiving
4 For evidence on positive returns to postsecondary education, see, for example, several of the papers commissioned for this study, including Heinrich (2015), Karp (2015), Rothwell (2015), and Stern (2015).
approximately $2,400 in higher quarterly earnings compared with $1,500 for men (Jepsen et al., 2012).
Finally, completing a program and receiving a degree appear to be valued more highly by employers than acquiring credits or skills (Kane and Rouse, 1995). Using data from Kentucky, Jepsen and colleagues (2012) found that the higher earnings associated with certificates or diplomas were due to completion (the so-called “sheepskin effect”) rather than the number of credits taken. However, they note that the effects of completing diplomas and certificates were mixed and varied by field of study and gender (Jepsen et al., 2012).
5.1.2 Challenges in Completing Postsecondary Programs
The large returns on investment in obtaining postsecondary education despite stagnant wage premiums over the last few decades appear still to make this a good investment, for both 2- and 4-year degrees (The White House, 2016). However, college completion rates are not rising commensurately with either costs or earnings, suggesting that there is room to improve the outcomes of college investments (see, for example, Baum and Ma, 2013; Baum et al., 2013). As of October 2014, 68.4 percent of high school graduates were enrolled in colleges or universities (BLS, 2016a). As described in Chapter 4, the data suggest that a large proportion of these students, particularly those in community colleges, will drop out without earning a college credential. Although completion rates vary considerably across colleges, comprehensive analyses of the differences are not available. Researchers have shown, for example, that even after controlling for K-12 achievement, students who attend 4-year private colleges have much higher completion rates than those who attend 2-year colleges (Stern, 2015). Other research has shown that completion rates for students enrolled in 4-year colleges are higher for those enrolled in colleges with selective admission requirements (Bound et al., 2010). Low student completion rates also have been documented among for-profit colleges (Cellini, 2012; Deming et al., 2013). Some of the factors affecting college completion are reviewed below.
The percentages of Americans with low literacy and numeracy skills (see Table 5-1) suggest that inadequate elementary and secondary education may be a large part of the reason for students failing to complete their education.
High Costs of Remediation
Students without adequate preparation for postsecondary education and training are often required to complete remedial programs before they can
|Subject Area||Grade 4 (%)||Grade 8 (%)||Grade 12 (%)|
|Civics||27 (2010)||23 (2014)||24 (2010)|
|Geography||21 (2010)||27 (2014)||20 (2010)|
|Mathematics||40 (2015)||33 (2015)||26 (2013)|
|Reading||36 (2015)||34 (2015)||38 (2013)|
|U.S. History||20 (2010)||18 (2014)||12 (2010)|
|Writing||—||27 (2011)||27 (2011)|
SOURCE: NAEP, n.d.
participate in more advanced programs, which increases the costs of the education and training and reduces potential returns on that investment (Bettinger et al., 2013; Long et al., 2014). Remediation is costly not only for students but also for states and employers, with some estimates reaching as high as $2.3 billion per year (NCSL, n.d.). Students who are required to take remedial courses are also more likely to drop out of postsecondary programs without a degree: 9 of 10 students subject to remediation requirements when they enter their postsecondary program of study never complete the program, and only 1 in 4 students in remedial classes will eventually earn a degree from a community college.5
Rising Tuition Costs
Between 2008-2009 and 2013-2014, revenues from tuition and fees per full-time equivalent (FTE) student increased by 17 percent at public institutions (from $5,681 to $6,639, in constant 2014-2015 dollars) and by 6 percent at private nonprofit institutions (from $19,206 to $20,293). At private for-profit institutions, revenues from tuition and fees per FTE student were 34 percent higher in 2013-2014 than in 2008-2009 ($19,480 versus $14,515) (NCES, 2016b). Tuition increases are particularly burdensome for students with limited financial resources (Lovenheim, 2011). The high costs of tuition, as well as the opportunity costs of time spent away from work or of child care, represent another impediment to college completion.
5 Data from the 31 Complete College America partner states indicate that roughly 22 percent of developmental education students at community colleges complete remediation requirements and associated “gatekeeper courses” within 2 years (see Complete College America, 2017).
Educators and policy makers rely on students to make decisions that suit their interests, capabilities, and means and to work diligently to acquire the relevant education and training. They rely on families and employers to prepare and encourage students to learn and to reinforce education and training at home and in the workplace. David Stern of the University of California, Berkeley, notes several trends regarding investment by students and workers in skilled technical education and training (Stern, 2015):
- Public high school graduates are less likely to earn career and technical education (CTE) credits than they have been in the past.
- The percentage of Americans aged 18-24 who enroll in degree-granting postsecondary education is increasing.
- Even though most bachelor’s degrees awarded in the United States are in programs of study related explicitly to occupations, such as business, education, health professions, engineering, law enforcement, and agriculture, the number of students who complete a degree by age 30 is much smaller than the number of high school students who say they want to pursue a bachelor’s or advanced degree.
Stern observes that enrollment and completion patterns and trends can pose a dilemma for families, educators, and policy makers. If education policies reflect students’ aspirations, many students may not acquire the technical knowledge or skills needed to earn a living. However, if policies limit access to the courses required for high levels of academic success to those students who are deemed likely to succeed (an inexact assessment at best), they are likely to shortchange many talented young people, particularly those from low-income or historically underrepresented groups.
5.1.4 Are Employers Sufficiently Engaged?
Educators and policy makers rely on employers to help them understand changing skill requirements, contribute to policy and program design, assist with program delivery, and provide access to up-to-date equipment and facilities. They also rely on employers to provide graduates with jobs or business opportunities that will deliver a return on investment in education and training. Yet these expectations of employers may not be met.
Some educators and policy makers complain that employers are “free-riding” on public resources because they are not sufficiently engaged in producing and providing education or training or fail to invest their fair share in
workforce development processes. As discussed in Chapter 4, the evidence on employer investments in education and training and the return on these investments is mixed, in large part because these data are missing or incomplete.
Growth of Contingent Work Arrangements
From the perspective of some students, workers, and policy makers, employers in certain industries and locations may not be committed to hiring specific groups of workers, paying high enough wages, or providing sufficiently stable employment to ensure an adequate return on investment in education and training. It appears that when making hiring decisions, companies increasingly prefer to hire workers under alternative, contingent work arrangements, such as temporary, on-call, contract, freelance, or part-time.6
For example, results of a 2013-2014 Harvard Business School alumni survey indicate that employers are reluctant to hire full-time workers:
Forty-six percent of respondents agreed that their firms’ U.S. operations prefer to invest in technology to perform work rather than hire or retain employees, while only 25 percent disagreed. Forty-nine percent said that their firms preferred to rely on vendors to perform work that can be outsourced, while only 29 percent reported that their firms would rather hire additional workers and keep work in-house.…Companies that increased their reliance on part-time workers over the past 3 years outnumbered those that had reduced their proportion of part-time workers by two to one. (Accenture et al., 2014, p. 7)
Data show that a significant share of the workforce comprises contingent or temporary workers or those who work part-time for multiple employers, and some analysts believe that this trend is increasing at a rate that is not well captured with existing workforce measures.7 Recent data collected by the Department of Labor (DOL) show more than 30 percent of the workforce being defined as in a contingent work arrangement (BLS, 2005).8 Among this group,
8 Contingent workers are those who do not expect their jobs to last or who report that their jobs are temporary; they do not have an implicit or explicit contract for ongoing employment. Alternative employment arrangements include people employed as independent contractors, on-call workers, temporary help agency workers, and workers provided by contract firms.
44 percent reported working part-time, and 25 percent reported being independent contractors or freelance workers. Other contingent workers are either self-employed or employed on a temporary basis through agencies or on-call relationships. Analysis of results of a 2014 survey conducted by the Freelancers Union suggests that more than 34 percent of Americans are doing at least some freelance work (Horowitz and Rosati, 2014).
Employers who are reluctant to hire full-time and long-term workers may not be sufficiently motivated to invest in skilled technical workforce development. At the same time, contingent workers may not have sufficient incentives or resources to invest in developing and maintaining their skills, a phenomenon that in turn feeds employers’ reluctance and inability to hire qualified workers.
A similar dynamic exists for full-time workers. As reported by the Society for Human Resource Management (SHRM), few full-time employees utilize available tuition reimbursement programs that would permit them to enhance their skillset.9 If employers do not unambiguously encourage and support continued education and training and provide employment opportunities that offer a return on those investments, employees may not invest in skill development—a dynamic that limits all stakeholders.
The perceived costs and benefits of education and training are further influenced by a number of social, economic, and institutional impediments. These range from the challenges that students may face in pursuing an education while raising children and managing households, which increase perceived costs, to the lower social prestige some accord to CTE, which reduces the perceived value of its benefits. Funding formulas that incentivize the volume of enrollment over the quality of educational options, inadequate career guidance, inconsistent standards, and limited labor market data, among other factors, also make it more difficult to make informed decisions about investing in joining or growing the skilled technical workforce.
5.2.1 Limited Support for Students
As described in Chapter 4, nearly half of all Americans enrolled in postsecondary degree programs attend community colleges. Students who attend community colleges often face greater challenges than those who attend 4-year institutions. More than 50 percent of students enrolled in 2-year degree programs work either part- or full-time while pursuing their studies, and they are
more than twice as likely as students enrolled in 4-year degree programs to be enrolled in part-time study. They also are more likely to be older, to have lower socioeconomic status, or to belong to minority groups.10
Workers who pursue continuing education and training often have a wide range of responsibilities to coordinate along with their studies, such as establishing their households, raising their families, and caring for aging family members. These challenges can affect students’ return on investment in education and training and their incentives to complete postsecondary programs (Holzer and Nightingale, 2009).11 These factors highlight the need for a wide range of services to support students who pursue skilled technical education and training and for schools to devise effective strategies for helping them access support services that meet their particular needs (Purnell and Blank, 2004). The nature of these services is locally specific and needs to be tailored to the requirements of the local population.
Most students struggle to adjust to the expectations of college coursework and campus life. However, the evidence suggests that students without strong social support networks have more difficulty accessing support services relative to those who have such networks (Karp, 2011). Some research on access to services at community colleges, for example, has found that although support services are available to all students, few low-income students find them accessible. Typically, students who take full advantage of these services have preexisting knowledge of the available resources or social relationships that help them find the information and resources they need to be successful (Karp et al., 2008).
Most American students and workers who pursue postsecondary education and training have difficulty identifying appropriate programs and courses of study, funding their costs, and developing supportive educational and career networks. Many secondary and postsecondary students receive limited career guidance. Colleges typically operate under a “cafeteria-style” model of education in which students select courses from a bewildering array of choices, with limited guidance (Bailey et al., 2015). As a result, students enroll in skilled
11 The authors note that “wraparound services,” such as career and educational guidance, stipends, child care, and transportation, can boost program completion rates.
technical programs at relatively low rates, even after controlling for academic ability (see, for example, Holzer and Nightingale, 2009).
Recent White House and congressional reviews have sought to address these challenges. Vice President Biden’s committee on workforce development reported that Americans often are not sure which education and training programs to pursue or whether jobs will be available for them when they finish (The White House, 2014). Congress also has sought to address this challenge; one of the express purposes of the 2014 Workforce Investment Opportunity Act (WIOA) is to improve coordination to increase access to opportunities and support services.12
5.2.2 Perceptions of Career and Technical Education
As discussed in Chapter 4, for most of the 20th century and up to the present time, federal and state policies have encouraged schools to develop curriculum for skilled technical work, known today as CTE. Prior to the 1960s, high school vocational programs provided basic job training for students that included school-based learning, work-based learning, and connections between activities and employers. An evaluation of the School to Work program of the 1990s, for example, found that these types of programs were particularly beneficial for students who were less likely to go to college, especially men (Neumark and Rothstein, 2005). However, several analysts have pointed out that as the focus in education has shifted toward preparing most students for college, the occupationally oriented CTE infrastructure in secondary schools has deteriorated (Klein and Green, 2011).
Stern (2015) observes that although policy makers have encouraged improvements in the quality of high school CTE curricula, most students and parents continue to perceive CTE as a poor substitute for college preparation rather than as a complementary or alternative pathway to a good job (see also Hoffman and Reindl, 2011; Symonds et al., 2011). This sentiment is supported by a recent report of the Independent Advisory Panel of the National Assessment of Career and Technical Education, which states that “exemplary CTE programs are seen as exceptions to mainstream options. CTE is still perceived by many as an alternative to rigorous academics—a separate track for students who are not college bound” (Independent Advisory Panel of the National Assessment of Career and Technical Education, 2014, as cited by Stern, 2015, p. 3). This diminished image of CTE and preparation for skilled technical work among students and parents inhibits efforts to improve CTE offerings, enrollment, and completion rates.
Stern notes further that experience and research over the past several decades have demonstrated the necessity and the feasibility of designing high
12 WIOA, Public Law 113-128, Section 2.
school programs that prepare students for both college and careers.13 In many schools in the United States, however, CTE tends to be course-specific and geared to preparation for particular jobs. This tendency may be reinforced by federal and state policies that have created bureaucratic rules and funding strategies that tend to isolate CTE from mainstream education reform.
As discussed above, perceptions about skilled technical work may lead parents and students to avoid training for these jobs. In addition, there is evidence that the additional earnings from obtaining postsecondary education are smaller for individuals seeking skilled technical credentials versus 4-year degrees. Since 2000, the wage premium for a bachelor’s degree compared with a high school diploma has leveled off at approximately 80 percent, meaning that college graduates make 80 percent more over their lifetime than those with only a high school diploma (Autor, 2014). While the wage premium for an associate’s degree compared with a high school diploma has increased over time, it remains much smaller, at around 30 percent (Modestino, 2010). In addition, research by Bailey and Dynarski (2011) suggests that high-income students have stronger incentives than low-income students to enroll in 4-year colleges and obtain 4-year degrees.
These considerations suggest that federal, state, and local policy makers and education professionals need to improve perceptions and beliefs about skilled technical work and better align standards for CTE with academic subjects so that students will be prepared for both work and postsecondary education that leads to a full range of credentialing options.
5.2.3 Standards for Certificate and Certification Programs
Questions exist about the overall value of alternative credentials such as certificates and certifications. Reviewing the evidence on the value of these credentials in preparing for and obtaining skilled technical work, Karp (2015) notes that this evidence is inconclusive. She calls for better data and research focused on creating coherent local systems of workforce development rather than per se developing additional credentials. Karp identifies several issues with respect to certificates and certifications.
The first is variability in value. The available research on certificates and certifications indicates that outcomes are highly context dependent. While Karp observes a positive labor market return to certificates in the aggregate as compared with earning only a high school diploma or completing some college without earning a certificate, different types of certificates have different value. Certificates that require a long period of study (long-term certificates) and are aligned with industry certifications provide more benefits to their holders
relative to those that require short periods of study (short-term certificates). The value of certificates also varies by field and subfield: long-term certificates provide value in such fields as health care, but not in education and child care, and certificates provide differential benefits even within a single field, such as information technology (IT). It also appears that the value of certificates may vary by geographic region, gender, and race or ethnicity, and they may not be transferable across locations or related industry segments. Karp discovered few rigorous studies of certifications, from either an individual or systems perspective. The Lumina Foundation (2015) estimates that fewer than 10 percent of certifications are subject to third-party review or accreditation.14 Community colleges’ involvement in preparing students for certifications is highly variable and includes aligning curricula with certification exams, creating certification exam preparation programs, and/or becoming a certification testing center. Karp explains that variability may be responsive to employer needs and provide multiple access points for students, but whether it conveys value in labor markets remains an unanswered question.
The second issue identified by Karp is stackability of credentials. Many experts are promoting “stackable credentials” as a way to prepare for skilled technical jobs. A stackable credential is one of a sequence of credentials that can be accumulated over time to build qualifications, advance in one’s career, and improve earning opportunities. Karp reveals some evidence that employers value this concept, but notes that certificates and certifications are potentially associated more closely with higher wages when they are stacked on a widely understood, lower-level credential, such as a general education diploma or high school diploma. Moreover, the timing of obtaining certificates and certifications may be important: they may have a positive effect on wages only when individuals earn them before obtaining an associate’s or bachelor’s degree. However, existing data suggest that most certificate/certification completers do not transfer to degree programs, which in turn suggests that many of these credentials serve as terminal credentials rather than stepping stones to degrees.
Finally, Karp cites the issue of inconsistent standards and limited data. Results of her survey reveal that inconsistent standards for credentialing and limited data on the role of credentials in career outcomes make it difficult to rigorously assess the value and impacts of certificates and certifications. In turn, this difficulty points to a need to adopt consistent credentialing terminology, ensure that credentials are aligned with employer and labor market requirements, and ensure that these requirements are consistently implemented and understood. This difficulty also points to a need to collect more longitudinal data on credentials and career experience and conduct more research on the
14 In her review, Karp found only one study that examined the labor market impact of certifications—a study of Microsoft certification holders using data from 2000, which has limited generalizability.
value and impacts of particular credentials. While efforts are under way to address terminology, measurement, and data collection issues, evidence on the desirability and feasibility of articulating specific career pathways or pipelines in many occupations is difficult to find. A 2014 review of federal workforce development programs advocates using a wider range of research strategies to understand the value of credentials (DOL et al., 2014). To date, studies on credentials have used mainly nonexperimental evaluation designs. However, several experimental demonstrations are currently using random assignment designs, which will help build a more rigorous evidence-based understanding of the impact of industry-driven credentials.
5.2.4 Quality of Curricula and Pedagogy
Improving links between K-16 education and training and work is an area of further concern. Effecting these improvements includes updating curricula to reflect advances in science and technology and changing occupational requirements; focusing on an experiential learning approach that includes work-based learning rather than on a theoretical, lecture-oriented approach to teaching; growing the number of technically qualified teachers; and providing adequate and up-to-date training equipment and facilities. Training institutions, particularly community colleges, need to develop new ways to structure and deliver education and training for both traditional and nontraditional students. Although evidence on the effectiveness of some of the emerging instructional models discussed in this report is not yet available, the face validity of several new strategies appears promising for students who are also working, workers seeking to retrain for a new occupation, and low-skilled workers with little prior postsecondary education experience (DOL et al., 2014).15 Addressing quality issues will require attention to the framework within which educators, students, workers, and other actors in the ecosystem address a number of coordination and information issues.
5.2.5 Incentives in Institutional Funding
The role of community colleges in many communities has expanded from providing relatively easy access to college studies to providing a wide range of credentials, custom education and training, programs in English as a second language, and remedial courses. Their mission, as well as the evidence on effective education strategies discussed above, requires the formation of strong employer partnerships in strategically important economic sectors in their communities. Advances in science and technology that create new occupations
with high technical and problem-solving skill requirements create additional pressure for community college faculty and administrators to develop new curricula and acquire new equipment and facilities (Autor, 2010).
There is growing concern, however, that community colleges have insufficient resources or funding incentives to provide programs of study that meet changing economic development needs and have potentially high labor market returns (Carnevale and Hanson, 2015). For example, the funding of many community colleges is based on formulas that emphasize the volume of enrollment rather than the value or quality of the services provided (Miao, 2012). This emphasis can create a financial incentive to focus investments in faculty, equipment, facilities, and support services in widely understood, high-volume education and training programs that easily generate high enrollment rather than on innovation and organizational change.
An additional challenge for community colleges is the level and stability of funding. Unlike 4-year public institutions, community colleges serve almost exclusively local students who typically continue to live and work in the community after graduation (Mellow and Heelan, 2015). This characteristic of community colleges would suggest that they potentially offer a competitive return on public investment. Yet the data displayed in Figure 5-1 indicate that in the two time periods shown (2007-2008 and 2012-2013), annual state and local revenues per FTE student were on average $3,000 lower for 2-year than for 4-year public institutions. Moreover, annual state and local resources at public 2-year institutions declined by $1,456 per FTE student between 2007-2008 and 2012-2013 (or 18 percent) (NCES, 2015c).
Similarly, even though the majority of undergraduate students are enrolled in community colleges, these 2-year Title IV institutions provide fewer resources per pupil relative to public 4-year universities. These data suggest a need to investigate two competing hypotheses: (1) community colleges are not appropriating enough money to postsecondary skilled technical education, or (2) postsecondary education funds are adequate, but they are not being allocated effectively to achieve the objectives of skilled technical workforce development.
5.2.6 Limited Data and Information
A lack of information about skill requirements makes it difficult for policy makers, educators, employers, and workers to coordinate to address quality issues in workforce development policies, processes, and programs. Policy makers and labor market participants need better data and more useful tools with which to analyze labor market dynamics, returns on investments in workforce development, and the effectiveness of alternative strategies for making adjustments to improve outcomes. Students and workers, guidance counselors, and educators need better information on skill requirements, career pathways, and potential returns on their investments in education and training. Employers
The information system that supports U.S. labor market functions is funded and administered primarily by DOL. The states, which own and often collect the data, depend on DOL for supplemental data, technical support, and grants to support data capture, analysis, and research, while federal agencies depend on the states to share good-quality data in a timely and accurate manner. Reviewing the evidence on workforce labor market information issues, Reamer (2015) notes that although there are high returns to investment in the workforce labor market information system, funding has been stagnant over the past decade, and Congress has provided limited oversight. The result is a system that is less effective than it could be:
- Funding. Despite the potential for very high returns to investing in workforce information, Reamer observes that funding is insufficient. In his opinion, federal departments and the Office of Management and Budget tend to ask for too little funding; Congress appropriates too few funds for the President’s requests; and Congress rarely approves new information budget initiatives, which typically range from $1 million to $5 million. In addition, Reamer notes a lack of federal funding for state labor market information agencies. Annual funding
- for Bureau of Labor Statistics grants to state agencies has remained at about $72 million for well over a decade. Similarly, the Employment Training Administration’s state workforce information grant funding has been flat at $32 million. With too little money to begin with, the utility of these grants has eroded with inflation. Therefore, state agencies have been unable to provide the level of information services needed by labor market participants.
- Data and research methods. Advances in information and data sciences make it possible to increase significantly the volume of data collected, analyzed, and disseminated to users. However, statisticians and other labor market researchers are in many cases using out-of-date data; methods, particularly surveys; and approaches. For example, federal statistical agencies continue to rely on legacy methods for collecting and producing data, resulting in lagged data that do not reflect the true state of the labor market. New data analysis techniques offer opportunities to improve information through greater use of federal administrative records and through commercial data vendors and service providers. However, labor market information specialists have yet to determine how to work with data and statistics that fail to meet the standards of reliability defined by the Office of Management and Budget and the National Academies of Sciences, Engineering, and Medicine. Key issues include privacy and data commodification concerns related to providing data to commercial vendors and bias associated with big data analytics.16
- Tools. Current tools for sharing and understanding the meaning of labor market data are difficult to use. Reamer argues that federal economic statistics agencies lack sufficient understanding of how participants in labor markets at the local level use information to make decisions, nor do they tend to have the relationships with data users that would provide that understanding. Moreover, the federal government does not adequately assess the performance and impacts of its investments in workforce information. There is a need to develop user-friendly tools that provide direct access to information about options and performance to assist labor market participants in their decision-making processes.
WIOA mandates provision of most of the information resources that Reamer identifies as needed by labor market participants, and the law aims to address many of the systemic problems he discusses. As Reamer observes,
16 For a comprehensive review of the issues, see the paper by Reamer (2015) commissioned for this study. The paper and additional material related to the topic can be found at http://nas.edu/SkilledTechnicalWorkforce.
however, the predecessor act (the Workforce Investment Act [WIA] of 1998) gave the secretary of labor a similar mandate to create and maintain a national employment statistics system, but succeeding secretaries did not pursue this mandate (Reamer, 2015). Full implementation of the information system aspects of WIOA is needed to address these data issues.
This section turns to the special challenges of educating and training the skilled technical workforce in two major sectors of the economy—health care and manufacturing. This is followed by a review of the challenges of transitioning the skills acquired by service members over the course of their military careers to the civilian workplace.
5.3.1 Training the Health Care Workforce
As described in Chapter 3, several changes in technology and practice, social demographics, and public law and regulations are creating the need for ongoing education and training in skilled technical health care occupations. In their paper commissioned for this study, Frogner and Skillman (2015) identify data challenges with respect to analysis of health occupations and then offer the following observations that are relevant to the coordination of workforce development in this sector:
- Confusing education and training requirements. Frogner and Skillman see evidence that education and training requirements for skilled technical health occupations are confusing, and that not all programs succeed in opening up pathways to well-paying careers. In their opinion, the career ladder in health care is more like a lattice because of the many options faced by entrants. Career transitions also are unclear. Credentialing through exams, certificates, and licenses is necessary to ensure quality care in some health occupations. In addition, scope-of-practice laws dictate what a worker can and cannot do. These requirements add to the confusion of occupational choice, education, and training. They also impose out-of-pocket and mobility costs for qualified workers, as credentials and scope-of-practice requirements vary by state and by occupation. In addition, obtaining credentials often requires payment of fees. Frogner and Skillman aver that, without a capable mentor or role model to serve as a guide, potential workers may find these complexities too daunting to enter the field.
- Need for training relationships. Public–private partnerships, including practical training partnerships between employers and educational institutions, are one way to develop necessary competencies—particularly as practice models and technologies evolve. Although clinical practicums are common in the health care industry, more formal training relationships, such as registered apprenticeships, are not, and several scale and coordination issues are associated with building sustained relationships between educators and employers.
- Cost as a barrier to education and training. The cost of education and training can be a barrier to entry in health care occupations for many people, and funding sources for this education and training can be unstable. Among all skilled technical jobs in allied health, nursing is the fastest-growing and offers the highest return over the span of one’s career. In Frogner and Skillman’s opinion, however, the cost of postsecondary education may be a barrier to entry in the nursing field for many potential workers. Other health care occupations in high demand, such as personal care assistants, have low educational entry requirements; however, they are among the lowest-paid. Many workers in these high-demand positions experience high turnover, are likely to live at the federal poverty level or below, and suffer from high rates of disability. Although the Carl D. Perkins Career and Technical Education Act’s Tech-Prep programs could address funding issues, Congress has not appropriated funds for these programs since 2011, and subsidized loan programs are quickly disappearing as financial aid options shrink and the Pell Grant program struggles to meet the financial needs of eligible students amid funding cuts.
Health care goods and services are significant components of the U.S. economy in every state. Frogner and Skillman’s analysis suggests that policy makers may need to create incentives for health care employers to assume a more active role in designing and funding education and training; improving job quality to recognize skill attainment; and collaborating with educators and workers to improve educational, training, and career guidance programs. In addition to better aligning workforce development incentives and programs, federal and state policy makers could work across state lines with health care industries to rationalize and harmonize health care credentialing and scope-of-practice requirements.
5.3.2 Training the Manufacturing Workforce
A recent report by Deloitte and the National Manufacturing Institute (NMI) claims that U.S. manufacturers are struggling to support their strategic, business, and production plans as a direct consequence of a shortage of skilled
production workers (Deloitte and NMI, 2015). As Figure 5-2 shows, the executives surveyed by Deloitte indicated that a talent shortage affects critical aspects of business activity and expansion. Given the mixed evidence of a significant skills gap reviewed in Chapter 2, however, it is important to examine why manufacturing companies are unable to recruit and retain workers who have the necessary skills:
- Public perception. Public perception is a major issue for this industry: manufacturing firms often are perceived to be unstable, dirty, and dangerous workplaces with few opportunities for workers to grow and develop their skills (Panchak, 2016). Similarly, in a paper commissioned for this study, Messing-Mathie (2015) observes that students, parents, and their advisors often overlook manufacturing apprenticeship programs in favor of academic education, reflecting a bias against skilled technical education and training that tends to exist in most advanced economies. Although negative perceptions could make it difficult to attract a new generation of workers to skilled technical occupations in manufacturing, other factors also contribute to this situation.
- Skill deficiencies. More than half of the executives surveyed by Deloitte and NMI (2015) described difficulty in finding qualified workers and serious deficiencies in their current employees related to technology and computer, problem-solving, math, and technical knowledge. Some data show that the average manufacturing worker
- earned $77,506 in 2013—20 percent more than what an average worker earned in other industries—and that four of five manufacturing executives are willing to pay more than the current market rate to hire and retain capable workers. However, other data suggest that skill demands have risen modestly and at a pace that makes most jobs accessible to most people. In a national survey, Osterman and Weaver (2014a) found that three-quarters of establishments can find the workforce they need but that a significant minority are experiencing difficulty. They note further that a retirement wave could exacerbate this problem. In their opinion, effective policies and institutions exist to address this problem, but they are scattered and not well supported. As discussed in more detail in Chapter 2, skill requirements for technical occupations change over time and are likely to continue to do so with advances in science and technology. These data suggest that workers who are interested in and willing to work in manufacturing are not investing in, do not have access to, or are not receiving the continuing education and training they need to be successful throughout their careers.
- Perverse incentives. Data collected in a recent employer survey by Accenture, Burning Glass, and the Harvard Business School suggest that manufacturing executives understand the importance of investing in employee training, development, and certification programs (Accenture et al., 2014). However, 58 percent of those surveyed reported that they use overtime to mitigate the effects of being unable to attract and retain skilled workers.17 Yet working overtime makes it difficult for workers to find time to invest in additional education and training. In addition, loss of income due to the ability to hire a sufficiently skilled workforce makes it difficult for firms to invest in education and training, potentially creating a vicious circle of competitive decline.
- Investments in automation and outsourcing. Market forces also may provide an incentive for firms to invest in capital rather than labor. Over the past several decades, firms have increasingly automated or outsourced tasks and jobs previously held by skilled technical workers, resulting in job losses in specific industries and occupations. In a recent survey of Harvard Business School alumni, for example, nearly half of respondents agreed that their firms’ U.S. operations preferred to invest in technology to perform work instead of hiring or retaining employees, while only 25 percent disagreed.
17 See Accenture et al. (2014, Figure 16) for an overview of manufacturing executives’ response to the question, “How important are the following techniques to mitigate the effects of existing skills shortages for a skilled production workforce?”
- Forty-nine percent said their firms preferred to rely on vendors to perform work that could be outsourced, while only 29 percent reported that their firms would rather hire additional workers and keep work in house. When choosing to hire, moreover, companies also showed a distinct preference for relying on part-time workers (Accenture et al., 2014).
- Program design. An additional set of coordination challenges in the manufacturing sector is related to how education and training programs are designed and implemented. Messing-Mathie (2015) notes that apprenticeship programs typically require a diverse range of social partners: if key partners are excluded or if roles are not clearly specified, confusion and poor coordination will result (see also Wolter and Ryan, 2011). However, manufacturing apprenticeships and other types of education and training programs do not always offer a final certification or a portable credential upon completion, which makes them less valuable across firms and industry segments. There is a growing view that some employers, particularly small and medium-sized businesses—the predominant manufacturing firm size in the United States—are concerned about their ability to earn a return on the costs of employing and supporting apprentices. In addition, apprenticeships often pay a relatively lower wage in the early phases of the program, which may make them less competitive than other job opportunities and may create a sense of inequity that can be a disincentive for participation.
These observations suggest that the policy objectives of manufacturing-driven growth and competitiveness are more likely to be realized if policy makers better align the incentives of workers and employers to improve the returns on investment in education and training in this sector.
5.3.3 The Military Transition Challenge
18 The total number of military personnel in fiscal year 2014 (3.5 million) includes 1,101,939 Department of Defense (DoD) Ready Reserve and Department of Homeland Security (DHS) Coast Guard Reserve members, 214,784 Retired Reserve personnel, 13,700 Standby Reserve personnel, and 856,484 appropriated and nonappropriated fund civilian personnel. The DoD active duty and DHS Coast Guard active duty members make up the largest portion of the military forces (38.4 percent). This section focuses on skill transfer possibilities with respect to active duty military personnel in the Armed Services; therefore, the discussion does not include Reserve or civilian personnel.
7 years on average. In 1979, the United States eliminated the draft and created an all-volunteer military force. The Office of the Deputy Assistant Secretary of Defense (Military Community and Family Policy) reported that in fiscal year 2014, the military community included 1.3 million Department of Defense (DoD) active duty military personnel and 39,454 Department of Homeland Security (DHS) active duty Coast Guard members (DoD, 2015). Although the size of the armed forces is declining as the engagements in Afghanistan and Iraq wind down, Table 5-2 shows that there were still about 1.3 million women and men working in the Armed Services at the end of January 2016.
Many Americans join the armed forces as a pathway to education, training, and preparation for life and careers. As Figure 5-3 shows, 43.2 percent of all uniformed active duty personnel are aged 25 or younger. Military personnel receive excellent training in the competencies associated with skilled technical work.
Jobs in the military vary by type of work and level of responsibility. The majority of active duty jobs in the Armed Services are enlisted jobs (82.3 percent), which require a high school diploma and typically involve technical skills that could transfer to jobs in civilian occupations such as those in maintenance, health care, or IT. Some enlisted members (7.0 percent) have a bachelor’s degree or higher, while most (92.1 percent) have a high school diploma and/or some college experience but less than a bachelor’s degree. Officer positions, which represent 17.7 percent of active duty members of the Armed Services, involve planning, directing operations, and making critical decisions (DoD, 2015). These jobs require a minimum of a 4-year degree, and some careers, such as law and medicine, require advanced degrees. Relative to 1995, the percentage of total active duty members with an advanced degree has increased for both enlisted personnel (from 7.3 percent in 1995 to 10.1 percent in 2014) and officers (from 79.9 percent in 1995 to 86.5 percent in 2014) (DoD, 2015).
|01/31/16||12/31/15||Change from Previous Month||01/31/15|
|Total Armed Services||1,347,232||1,344,258||2,974||0.22||1,360,406|
NOTE: DoD = Department of Defense.
SOURCE: DoD, 2016.
Military–Civilian Transition Issues
In 2014, 184,444 enlisted members and 20,112 officers left active military duty and returned to civilian life (DoD, 2015). The most common reason for leaving for enlisted members was voluntary separation/early release, while the most common reason for officers was non-disability-related retirement (DoD, 2015).19 These workers have received excellent technical education and training, and they have developed the “soft skills” required to work well with others. Even though they receive comprehensive transition services, however, the Department of Veterans Affairs reports that, compared with civilians with similar demographics, veterans have an 8 percent higher total unemployment rate, and their median income is slightly below the nonveteran median of $42,317 (VA, 2015). Veterans participating in the GI Bill program complete
Veteran unemployment rates, earnings rates, and completion rates can vary across the services and base locations, as well as across skills, age, reason for separation, and other demographic factors. For example, the Department of Veterans Affairs reports that over the period 2005-2013, veterans aged 18-34 had a 20 percent higher unemployment rate than nonveterans, but median earnings for the same veteran age cohort were about 8 percent higher than those for nonveterans. Moreover, veterans under the age of 30 who used the GI Bill had 8 percent higher completion rates than traditional students in the general population. However, it is possible that these statistics misestimate the veteran–nonveteran difference for a wide range of reasons, including the way in which the experience of mobilized Reservists is treated.20
A 2014 U.S. Congressional Joint Economic Committee staff analysis indicates that more than 50 percent of veterans work in industries, such as transportation and utilities, manufacturing, construction, information, wholesale trade, and professional and business trades, that employ skilled technical workers (U.S. Congress Joint Economic Committee Democratic Staff, 2014). However, the analysis suggests that veterans are underrepresented in the fast-growing sectors of the economy, including two of the three industries that have fared the best during the past several years (in terms of job gains as a percentage of employment in the industry): the education and health services industry and the leisure and hospitality industry.21
To develop a better understanding of the contradictions in veteran transitions, the committee asked Susan Payne Carter and Brian J. Miller of the U.S. Military Academy to investigate the transitions of U.S. Army soldiers, who
20 For example, the rates for the 18- to 24-year-old cohort likely reflect the experience of those who enlisted in their early 20s and left the military relatively early because it was not a good fit for them. Those who enlist at a young age and leave early are atypical because most enlistments are for 3 years or more. The proportion of those who leave early in older age cohorts may be much smaller. Additionally, the period 2005-2013 saw relatively high Reserve mobilization. Once demobilized, Reservists will appear in the data as “veterans.” However, Reservists, particularly younger Reservists, may have a weaker connection to the labor force before mobilization (e.g., they may have been students or new to the workforce). Similarly, veteran–nonveteran comparative per capita income statistics may be misleading. For example, veterans who retire with an annuity or have other financial resources or support or who have fewer financial responsibilities may choose lower-paying jobs than veterans and nonveterans who have fewer financial resources or less financial support or greater financial responsibilities.
21 The report notes that veterans represent only 4.3 percent and 3.3 percent of employees in these industries, respectively. Since December 2007, employment in the education and health services industry has expanded by 14.7 percent and in the leisure and hospitality industry by 8.9 percent.
represent the largest uniformed workforce in the Armed Services (Carter and Miller, 2015). Unlike civilian workers, departing military personnel may collect unemployment benefits even if they leave the service voluntarily. In many states, moreover, veterans may collect both GI Bill and unemployment benefits. Carter and Miller report that in 2013, DoD spent $829 million on Unemployment Compensation for Ex-servicemembers (UCX), more than one-half ($463 million) of which went to Army veterans. In 2010 and 2011, 53 percent of all transitioning veterans applied for benefits. On average, veterans drew benefits for 20 weeks during this time; 95 percent did not exhaust their benefits.
Between 2011 and 2013, 59 percent of departing Army soldiers who were eligible for UCX benefits applied, and most who did so ended up using the benefits. When data on soldiers are disaggregated across demographic and military service dimensions, however, average rates vary considerably. Surprisingly, soldiers with seemingly transferable skills, such as logisticians, apply for UCX at the highest rates, a result that holds when Carter and Miller control for education, cognitive ability, and other demographic differences. The authors speculate that some individuals in this population do not, do not wish to, or cannot directly transfer their Army skills to civilian jobs.
Similarly, Frogner and Skillman (2015) observe that military personnel have a unique career transition in health care. They point out that military skills and experience may not translate to the competencies needed or follow the path required by accreditation or licensing bodies in the civilian sector. Preparing and sitting for exams to obtain licenses despite years of experience can take more time than veterans can afford to be out of the workforce.
To explain differences in UCX application rates among demographic groups, Carter and Miller (2015) look at a wide range of demographic factors. They document evidence suggesting that a complex set of life factors can affect Army soldiers’ UCX application rates (see Box 5-1). Groups more likely to apply include soldiers with 2 to 3 years of service upon separation and lower pay grades; female, black, and Hispanic soldiers relative to white male soldiers; divorced soldiers and soldiers with children relative to single soldiers; and soldiers with lower levels of educational attainment and lower cognitive ability scores on their qualification test.
Carter and Miller note further that the highest Army UCX application rates are associated primarily with service and support specialties. Veterans in the following specialties (listed in descending order) apply at higher rates: quartermaster, finance, chemical, adjutant general, air defense, and transportation. Although 62 percent of soldiers who separate because of satisfactory completion of their service contract apply for benefits, those who
separate for other reasons are more likely to apply.22 Finally, application rates vary across base locations, suggesting differences in transition assistance programs.
Actions to Improve Veterans’ Transitions
Several programs, including federal and state government programs, private-sector initiatives, and public–private partnerships, have been implemented to improve transitions for returning veterans (U.S. Congress Joint Economic Committee Democratic Staff, 2014). The United Services Military Apprenticeship Program (USMAP), for example, is a formal military training program implemented by the Center for Personal and Professional Development that gives active duty Coast Guard, Marine Corps, and Navy service members the opportunity to improve their job skills and complete their civilian apprenticeship requirements while they are on active duty. The objective of the National Apprenticeship Standards for USMAP is to provide registered certification of the training of individual service members and to achieve recognition for them equal to that of their civilian counterparts.23 Lerman and his colleagues at the Urban Institute report that currently, about 1 in 4 enlisted
22 The data show that soldiers with disability ratings are more likely to apply for UCX benefits. However, soldiers who have 100 percent disability ratings and are discharged for medical reasons are less likely to apply.
23 DOL provides the nationally recognized “Certificate of Completion” upon program completion. Originally established in 1976 as a Navy program, USMAP became a single program registered with DOL (N-93063) for the three Sea Services in April 2000. USMAP allows active duty service members to complete a DOL apprenticeship program while serving their country (see DOL, 2012).
Sailors and 1 in 14 Marines participate in USMAP (Lerman et al., 2015). In their opinion, one of the program’s major accomplishments is that it has registered about 100 occupations with DOL’s Office of Apprenticeship that are related to civilian fields.
The VOW to Hire Heroes (VOW) Act (P.L. 112-56) strengthened and expanded tax credits for employers who hire unemployed veterans, increasing the tax credits for hiring veterans who have been seeking a job for more than 6 months, as well as those eligible for disability compensation from the Department of Veterans Affairs. Through the Hiring Our Heroes program, the U.S. Chamber of Commerce Foundation is working with businesses to obtain commitments to hire veterans and military spouses (Eversole, 2016).
Additional action is needed, however. Suggestions solicited by the Joint Economic Committee include streamlining credentialing and providing better-targeted transition assistance (U.S. Congress Joint Economic Committee Democratic Staff, 2014). Streamlining state occupational licensing, for example, helps veterans with equivalent military skills and abilities meet credentialing requirements for training and education. Veterans in jobs in health care and transportation occupations, such as emergency medical technicians, licensed practical nurses, and bus and truck drivers, often have skills that overlap with civilian occupational requirements. In addition, spouses of returning veterans often move with them across state lines and may need to obtain a certification or license in their new state. Today all 50 states and the District of Columbia allow veterans with relevant experience to receive waivers of the skills test to obtain a commercial driver’s license, and dozens of states have acted in other ways to facilitate credentialing and licensing for military veterans transitioning to civilian employment and their spouses (U.S. Congress Joint Economic Committee Democratic Staff, 2014). Further legislation could facilitate the transition of additional veterans with specific skill sets into in-demand civilian occupations.
As Carter and Miller (2015) report, challenges in moving from active duty to civilian life often extend beyond simply having the right skills for available jobs. Veterans also may need assistance with more general skills that are helpful in the workplace and throughout the job search process (U.S. Congress Joint Economic Committee Democratic Staff, 2014). The VOW Act requires transitioning service members to participate in the Transition Assistance Program, which teaches them how to approach a job search, write a resume, and apply their military skills to civilian jobs (Orvis, 2016). The Veterans Employment Center, which lists private- and public-sector jobs, also helps veterans build profiles and translate their military skills into plain language. Carter and Miller argue that the Army should collect more data on soldiers’ transition experiences, better analyze the efficacy of transition assistance classes, and tailor the classes to better address the challenges of transitioning from a military to a civilian career.
The information presented above suggests that DoD and the Department of Veterans Affairs could do a better job of assessing the transition risks for military personnel and of designing and delivering services to effectively mitigate these risks, including ensuring that the military culture encourages service members to transition without drawing UCX assistance. In addition, it appears to the committee that DoD could coordinate better with civilian policy makers, regulators, and educators to improve the transferability of military education, training, and certification.24 A reevaluation of the Post-9/11 Veterans Educational Assistance Act may also be required.
Efforts to improve veterans’ transitions into the civilian workforce have focused on actions taken immediately prior to their leaving the military at the point of transition into the civilian workforce. A seamless transition may require that planning begin earlier in—perhaps even at the start of—service members’ military careers. Doing so would help ensure that service members are trained in fields in which they wish to continue after leaving the military and that these skills will be valued in both the military and civilian workforces. As discussed at a workshop convened by the committee on “The Military as a Pathway to Skilled Technical Jobs,” DoD is seeking to pursue more fluid employment models that meet its workforce needs while allowing for easier transitions between the civilian and military workforces (see Box 5-2).
Given the sheer number of former service members transitioning out of the military each year (about 200,000) and the costs of UCX benefits (nearly $1 billion annually), better information on these transitions is needed, including return to military service after voluntary separation. For example, information on how often transitioning service members switch occupations and how they make occupation decisions is needed to help inform efforts of policy makers and DoD leadership to improve these transitions.
Policy makers, educators, employers, and workers need to better integrate economic development and workforce development objectives, undertake initiatives across and within industry sectors to align incentives, and invest in education and training. They also need to design and deliver relevant programs that meet the changing requirements of workers and employers at the local level. Addressing quality issues in skilled technical workforce development requires
This chapter has considered a variety of current challenges to identifying and better aligning returns on investment in education and training on the part of students, workers, and employers. The next chapter reviews experiments under way across the United States to address many of the challenges identified in this and preceding chapters.
This page intentionally left blank.