Human Resource Management
O*NET was initially conceived of as a useful tool for a multitude of human resource management (HRM) applications (Peterson et al., 2001). Both descriptions of O*NET development and information on the O*NET websites—including the O*NET Academy and the O*NET Knowledge Site—note how O*NET might be used in designing systems to select, train, evaluate, and support workers, as well as to reconfigure jobs (Peterson et al., 1999). This chapter reviews the evidence regarding the extent of O*NET usage in HRM applications and evaluates its value for these purposes. The final section presents the panel’s conclusions and recommendations.
The chapter has two major sections. The first describes the major categories of O*NET use: job analysis (which, as the most widespread use, is subdivided according to the specific aims of the effort), person-job matching systems in organizations, human resource strategic planning, and worker health and safety programs. We review both actual uses of O*NET as well as potential, undocumented uses. The second major section discusses user views of O*NET’s strengths and limitations.
The evidence reviewed in this chapter was gathered from a number of sources. The panel reviewed information about O*NET in the scholarly HRM literature, searching for mention of O*NET not only in titles and abstracts but also in the body of papers. We also reviewed supporting materials provided by the National Center for O*NET Development (the O*NET Center) regarding uses in HRM. Second, experts invited to the panel’s workshops provided examples of applications of O*NET in HRM.
Third, the panel conducted a short qualitative survey of HRM professionals, asking about possible users of O*NET (Ryan and Pearlman, 2009).
Although these information sources were valuable, they cannot be presumed to comprehensively represent the entire user community of O*NET for HRM. Time and resource constraints precluded us from conducting systematic, in-depth, or long-term fact-finding or data-gathering efforts. The panel faced the problem that “we did not know what we didn’t know.” It is not possible to determine how much additional information on uses of O*NET data for HRM purposes might have been uncovered had time permitted additional efforts. Because there is no single, readily available repository of O*NET user information and feedback to shed additional light on typical O*NET applications and data needs, the information that follows must be considered suggestive. Furthermore, because of these limits, it was not possible to evaluate the effectiveness or appropriateness of the specific applications of O*NET reported to us by human resource managers and consultants.
Despite these caveats, the panel judged the data gathered sufficient to reach conclusions and recommendations about the use of O*NET data for HRM purposes. This judgment is based on intensive and diligent efforts to solicit and receive input from a relatively wide range of O*NET users. It is also based on the observation that there is some degree of repetition or redundancy among the comments made and issues that surfaced across workshop presentations and papers and other user input. This can be considered a sign that most of the relevant issues or information has been captured.
USES OF O*NET IN HUMAN RESOURCE MANAGEMENT
A major use of O*NET is for job analysis. Note that the terms “job analysis” and “occupational analysis” are not synonymous. An occupational analysis looks at all those holding jobs in a given occupational category (e.g., all firefighters), whereas a job analysis focuses specifically on those holding jobs in an organization (e.g., all firefighters in the city of Detroit). O*NET provides an occupational analysis, but job analysts are typically not interested in all jobs in an occupation, but only those job holders in their own organizations.
Organizations conduct job analyses to describe the nature of work to be performed and to identify worker requirements for accomplishing that work. O*NET provides four essential elements of a job analysis that can serve as input to various HRM applications: (1) O*NET can inform job descriptions for use in designing and implementing selection systems, train-
ing and development programs, and performance management systems; (2) O*NET descriptor information may be used in job redesign efforts; (3) O*NET information might play a role in job clustering for various HRM purposes; and (4) O*NET information might serve to supplement internal, multipurpose organizational job analysis efforts. Each of these uses is discussed below.
Job descriptions are used in many ways in organizations. In this section, we discuss six applications of job description information for which there is some evidence that O*NET has been used.
Organizations use job descriptions generated from O*NET data for designing selection systems. O*NET’s role in selection system design may take several forms. O*NET job descriptions are used to determine minimum and “preferred” qualifications for a position (e.g., a minimal level of knowledge required). For example, one survey respondent described mapping reading requirements for O*NET occupations onto the National Assessment of Adult Literacy reading scale using a Census Bureau crosswalk to derive literacy requirements for occupations synthetically. In a related effort, O*NET has been used in developing competency models that underlie organizational selection systems (e.g., Jeanneret, 2009, describes this use in the refining and insurance industries). O*NET also has been used for checking competency models already developed by an organization against O*NET descriptor profiles.
O*NET also has been used in evaluations of the job-relatedness of existing tools. For example, a test or interview process is examined to determine what knowledge, skills, and abilities are assessed relative to what O*NET descriptions suggest are the required characteristics (Human Resources Research Organization, 2009). O*NET job information also has served as an input to efforts to develop new instruments, such as designing experience inventories for job applicants (e.g., Anderson, 2009) and developing or documenting question content for structured interviews (Ryan and Pearlman, 2009). O*NET taxonomies have been used specifically in supporting content validation efforts. For example, Jeanneret (2009) describes linking interview questions to O*NET general work abilities to assist a client in the hospitality industry.
Numerous publications report the use of O*NET in job component validation (e.g., Coaster and Christiansen, 2009; D’Egidio, 2002; Jeanneret and Strong, 2003; Johnson et al., 2003; LaPolice, Carter, and Johnson, 2008). Job component validation is a methodology used by organizations for identifying selection tools that are appropriate for a specific job component that may be common to multiple jobs. This process is often used
for jobs with few incumbents when conducting criterion-related validation studies at the job level is infeasible. Organizations also use it as an efficient way to develop selection procedures for large numbers of jobs efficiently. For example, LaPolice, Carter, and Johnson (2008) successfully used O*NET knowledge, skills, ability, and generalized work activity data to predict literacy requirements for occupations and to thus support the use of assessments of literacy in selecting employees.
Most of these selection efforts describe using O*NET as a starting point, followed by collection of job analysis data for a specific organization and job, rather than using O*NET as the sole source of legally defensible job-analytic information. However, there are anecdotal examples from the panel’s survey of HRM professionals in which O*NET data were used by themselves to support the legal defense of selection tools.
O*NET has also played a role in organizational recruitment efforts. Organizations have used O*NET descriptions in developing job information materials that are used to inform and attract job candidates. For example, the employment services company Manpower, Inc., uses O*NET job descriptions as a basis for developing its standard job descriptions, which are then used for recruiting and job advertisement (Dorman, 2009). DeLuca and Hirsh (2009) describe using O*NET data for identifying alternative sourcing opportunities when recruiting for occupations with a limited supply of workers.
Another possible use of job analysis information by human resource professionals is in designing compensation systems. O*NET information could be used to identify compensable factors (i.e., determinants of pay). O*NET might also provide input to comparisons of jobs in wage and salary system design. The National Center for O*NET Development (2009b) describes one compensation analyst from the banking firm Trustmark Corporation using O*NET data to help managers develop job descriptions that then served as input into determining salary ranges. DeLuca and Hirsh (2009) also note the use of O*NET in investigating whether organizations are retaining and rewarding the right skills and competencies through studies on turnover and raises. Overall, there was little documentation of using O*NET in any extensive way for compensation system design, although anecdotal evidence suggests that it can serve in such a capacity.
Another major category of job description usage is in performance management system design. That is, organizations create processes to evaluate job incumbents’ performance, to provide developmental feedback, and to motivate future performance. O*NET can serve as input for such system design by providing information on competencies and tasks to be evaluated. The U.S. Department of Labor (2008) mentions this use and Jeanneret (2009) and Anderson (2009) both provided examples of using O*NET taxonomies as organizing frameworks in developing performance
evaluation systems. Overall, there was only a small amount of evidence of O*NET data being used for this purpose.
O*NET information has been used for training and development system input. Specifically, it might be used to design job-specific training programs. For example, O*NET was used by one organization to structure electronic technician training programs (National Center for O*NET Development, 2009a). It also has been used in identifying the training needs of an individual employee for a specific job (Ryan and Pearlman, 2009) or the relevance of existing training courses for workers in specific jobs. For example, Dorman (2009) describes how Manpower uses O*NET descriptors in matching individuals with specific courses in their training programs. O*NET information might indicate trends in worker requirements for future training program development. O*NET information might also be used to identify jobs in which cross-training is feasible (i.e., other jobs with similar competency requirements). Overall, the evidence suggests that O*NET data are used more frequently to inform state and local workforce development programs (see Chapter 6) than they are used to inform internal training and development programs created by private employers. Nevertheless, there is some documentation of employers using O*NET data to inform their training and development programs.
A final use of O*NET in a job analysis capacity is to define occupational training, education, and experience requirements of jobs in order to be in compliance with government regulations or to determine eligibility for various government programs. For example, the U.S. Department of Labor (DOL) might use O*NET data in its process of making determinations on permanent labor certification; such certification allows an employer to hire a foreign worker to work permanently in the United States. Similarly, decisions about the “essential job functions” that must be defined to comply with the Americans with Disabilities Act might be informed by O*NET information. The State Department’s Diversity Visa Lottery program, which provides visas to randomly selected applicants who meet strict eligibility requirements from countries with low rates of immigration to the United States uses O*NET data on the education/training and experience requirements of occupations. As another example, organizations have used O*NET data to align job descriptions for employee visa sponsorships with the database used by the U.S. Customs and Immigration Service (Ryan and Pearlman, 2009). Although some survey respondents described these sorts of uses, there are also clear examples of O*NET’s being considered not useful for eligibility determinations. For example, as Chapter 8 describes, the Social Security Administration has found that O*NET is not currently useful for the purpose of determining disability eligibility.
In sum, O*NET information has been used in developing job descriptions for a wide variety of HRM purposes. Indeed, as discussed in
Chapter 5, the HR-XML Consortium has used the O*NET database to identify common elements of human resource systems and develop standard definitions, suggesting that O*NET may play a larger role in these systems in the future.
O*NET data could provide useful information for organizational job redesign efforts—specifically, in deciding whether jobs are similar enough to be treated the same for HRM purposes. Conversely, O*NET information might support the splitting of an internal organizational job category into two or more jobs with different worker requirements to better utilize worker skill sets or to make pay systems or training programs more efficient. O*NET data might be used to suggest how an organization could adjust specific job duties or requirements in response to substantive changes in operations, procedures, or equipment used. Although these uses are possible, little documentation was available documenting actual use of O*NET for these purposes.
O*NET information can be used when planning for downsizing or outsourcing of work. For example, the Boeing Corporation used O*NET data to evaluate the transferability of individual workers’ skills when planning for an anticipated plant closing (National Center for O*NET Development, 2009b).
O*NET data can inform redesign efforts that change the way work is structured to improve productivity, worker well-being, and worker health. For example, O*NET might serve to identify jobs in an organization that are candidates for increased flexibility (in time or place of work), nonstandard work arrangements (e.g., job sharing), team-based structures, and greater self-management. Anderson (2009) provided an example of how O*NET taxonomies and rating tools might be used to identify which jobs are candidates for “greening,” that is, redesign to reduce energy consumption and waste. Thus, there is potential for O*NET usage in assisting organizations in changing the nature of work to meet with environmental, social, and economic changes, although there is currently little documented use for this purpose.
O*NET has been used to identify various types of similarities across jobs (such as in tasks or worker requirements). Such information can be used to cluster jobs for specific HRM purposes. For example, organizations use O*NET to cluster jobs according to various worker requirements (e.g., skills, abilities) when designing selection systems so as to make more
efficient use of selection tools that assess those requirements (Jeanneret, 2009; Ryan and Pearlman, 2009). In addition, when organizations wish to extend the use of a selection instrument across jobs (Jeanneret, 2009) or to determine if an instrument shown to be valid for a particular job category might demonstrate generalizability across organizations, units, or cultures, O*NET information has been useful. For example, Taylor et al. (2008) found that O*NET descriptors are meaningful in other countries as well as in the United States. Jeanneret (2009) provided the panel with multiple examples of the use of O*NET for job grouping, including a grouping of over 100 different titles for first-line supervisory jobs at a large telecommunications firm, a grouping of 63 jobs in the hospitality industry into 8 families, and a grouping of over 900 jobs in a municipal government into smaller job families. Thus, there is documentation of O*NET use in job clustering for selection-related purposes.
Jobs also are clustered for use in compensation programs, unified performance management systems, and common training programs, and O*NET descriptors can be used to determine appropriate grouping for these purposes. For example, DeLuca and Hirsh (2009) describe using O*NET to identify similarities in occupations for their compensation clients and to create company-specific job groupings. Job clustering can provide a means to determine career ladders and provide employees with information on career development opportunities, as well as inform organizational succession planning. Finally, human resource managers and researchers in organizations might use O*NET information to compute various occupational statistics for internal workforce planning purposes.
A final use of O*NET descriptor information in a job analysis capacity would be as supplemental or starter information for internal job analysis efforts. O*NET information has been consulted to structure job analysis interviews (DeLuca and Hirsh, 2009), to develop preliminary content for job analysis questions (e.g., Reiter-Palmon et al., 2006), and to generate job descriptions for subsequent review and editing (Ryan and Pearlman, 2009). Among those who provided information to the panel, this was the use of O*NET data mentioned most frequently. In one of these many examples, Reiter-Palmon et al. (2006) described how the U.S. Navy used O*NET Generalized Work Activities as the basis of a web-based job analysis process developed for internal use. The evidence suggests that O*NET data are frequently used, not as a standalone source of data for organizations, but as a starting point to be supplemented by further collection of data tailored to the individual organization’s context and needs.
While the use of O*NET for person-job matching may be more widely employed by career and workforce development experts (see Chapter 6), individual organizations and organizational units also make use of O*NET for internal person-job matching purposes. Organizations may have internal career development programs that have been built using O*NET information. O*NET has played a role in developing self-assessment and exploration tools specific to a particular organization’s job set, or the O*NET interface itself has been promoted for employees’ use in their own career exploration (Ryan and Pearlman, 2009). For example, although the U.S. military’s Armed Services Vocational Aptitude Battery Career Exploration Program did not use O*NET ratings directly for developing its career match information, O*NET data were used as a starting point in generating skill importance ratings for various occupations. As another example, Dorman (2009) described the use of O*NET in developing a transferable skills index used by Manpower, Inc.
Organizations also use O*NET information in employee development programs (Ryan and Pearlman, 2009). For example, Converse et al. (2004) describe using O*NET data to match individuals to occupations on the basis of abilities using a multiple-aptitude test battery. As mentioned earlier, O*NET information might be used to develop career ladders for employees that show possible paths in an organization for those with certain skill sets, although we did not find documented examples of that specific use.
As Chapter 6 describes, O*NET data also can play a role in outplacement activities of organizations. It is also used by the military and by veterans groups to identify civilian jobs and career paths for transitioning military personnel (National Center for O*NET Development, 2009b).
In sum, O*NET information is widely used for individual career planning and career decision systems development by workforce development agencies and career development professionals. Human resource managers in individual establishments also appear to use it, both for internal career management purposes and for outplacement.
O*NET has the capacity to be used by organizations in strategic HRM, although the panel found only a few documented examples of such use. For example, organizations can use O*NET in determining recruitment needs, identifying where retention incentives might be needed, and selecting locations for facilities by forecasting available labor supply in specific geographic regions. Using O*NET for such projections, organizations can align recruitment strategies with available supplies of workers. They might
mine O*NET data to identify trends in the availability of skilled workers, in order to increase or decrease recruitment activities as appropriate. Organizations might also examine O*NET data to identify potential “feeder jobs” and target recruitment activities toward individuals in these jobs to fill high-demand skilled positions. For example, Anderson et al. (2007) used O*NET data to analyze recruitment needs in the transportation industry. Similarly, O*NET data might inform succession planning—the process of preparing for the retirement of top managers by identifying and developing younger people who might replace them in the future.
O*NET is often used by states and the federal government to analyze skill gaps and identify appropriate education and training programs to fill these gaps. Similarly, it might be used by an individual organization’s human resource managers to better understand the capabilities of the internal workforce. O*NET information on trends can point to jobs that will require fewer or lower levels of skills in the future, as well as jobs that will be upgraded in skills or require different ones as a result of technological advances (National Research Council, 1999). Such information can help organizations better anticipate training needs and shifts in the workforce and to plan accordingly.
Although this discussion has identified many different ways O*NET data could potentially be used to support strategic human resource planning, the panel found only a few documented instances of such use of O*NET data. The available evidence did not point to any specific barriers to this use of O*NET information.
Worker Health and Safety Programs
A final area of use of O*NET by those in HRM is in the context of worker health, stress, and safety programs. O*NET data might be used to identify safety needs for a given job, as well as common safety issues across jobs, and thereby inform worker selection and training for those jobs. O*NET data can be mined to identify similarities in health concerns across jobs and inform the development of educational or informational programs and materials appropriate for those jobs. The O*NET database provides information on similarity in job stressors (e.g., emotional demands), allowing for the clustering of jobs to provide appropriate programming and training materials (Liu, Spector, and Jex, 2005). Anderson et al. (2004) used O*NET as a basis for making expert ratings regarding the effects of seasonal allergies on job performance.
Although the panel did not find wide documentation of uses of O*NET for workforce health and safety purposes, the research literature on occupational health issues appears to increasingly report the use of O*NET data in establishing links between occupational conditions and activities and
worker health and safety (see Chapter 9). This suggests that O*NET data may be increasingly used for workplace health and safety research.
The panel found evidence that O*NET data are used for a variety of HRM purposes by public- and private-sector organizations. However, we did not uncover concrete evidence of the use of O*NET data for some potential purposes, such as for strategic human resource planning. This may indicate that O*NET data are not useful for such purposes. Alternatively, they may indeed be used for these purposes, but they are difficult or impossible to document or verify. Another interpretation of the lack of concrete evidence of some uses is that the costs of understanding and using O*NET data prevent their wider use in HRM applications.
STRENGTHS AND LIMITATIONS FOR HUMAN RESOURCE MANAGEMENT
Most of the comments about O*NET from the HRM community were oriented toward taking a system that is useful and making it better. Descriptions of use often came with significant caveats, reflecting some users’ frustration with O*NET’s unrealized potential. For example, a number of users holding favorable views of O*NET’s content and data quality said that deficiencies in the O*NET websites created barriers to use of the data or hindered access to content and data. More broadly, different types of human resource specialists, such as practitioners and researchers, focused on different aspects of the O*NET system when identifying strengths, limitations, and opportunities for improvement. Even within these categories, there was a range of opinions. For example, some viewed the use of analysts to provide certain descriptor ratings as an asset, and others viewed it as a liability.
Human Resource Management Views of O*NET Strengths
The aspects of O*NET frequently cited as strengths or advantages by HRM users can be roughly categorized into those involving O*NET content, O*NET data, and the general O*NET system. For the most part, these correspond to many of the capabilities for which O*NET was explicitly designed, as described by Peterson et al. (1999).
Features of O*NET content cited favorably by users include the following (Anderson, 2009; DeLuca and Hirsh, 2009; Jeanneret, 2009; Morgeson, 2009; Ryan and Pearlman, 2009):
The comprehensiveness and theoretical basis (e.g., Fleishman’s taxonomy) of the O*NET content model and its various component descriptor taxonomies. The breadth and variety of O*NET descriptors enable both work and worker requirements to be described in multiple ways (e.g., in terms of skills, knowledge, abilities, tasks, work styles, work context, and the education or training required). This in turn permits substantial flexibility and versatility in how the system can be used.
The hierarchical organization of many of O*NET’s descriptors, allowing for occupational description at different levels of analysis.
The standardized descriptors used to collect and report data for all occupations. The use of descriptor taxonomies facilitates cross-job comparisons both within and across organizations, industries, and economic sectors.
Features of O*NET data cited favorably by users include the following (Handel, 2009; Morgeson, 2009; Ryan and Pearlman, 2009):
The degree of rigor in the design and execution of the data collection operation as a whole (planning, execution, data cleaning, and quality control).
The multifaceted sampling strategy underlying the collection of descriptor rating data, involving different rater types (analysts, incumbents, occupational experts) and multiple respondents from multiple establishments, yielding high cooperation rates and reasonable survey response rates, thereby resulting in useful information about occupations.
The ratings relate to other data (for example, wages) in meaningful and expected ways, resulting in logical within- and across-occupation descriptor rankings.
The ability to significantly reduce the cost of occupational information gathering when used to provide starter job analysis, interview, and questionnaire development information.
O*NET system or general features cited favorably by users include the following (Anderson, 2009; Morgeson, 2009; Ryan and Pearlman, 2009):
The ability to link to multiple types and sources of data and information (e.g., the provision of Bureau of Labor Statistics wage and labor market information by occupation).
The availability of and easy access to the O*NET OnLine website, especially its search facilities, which allow for quick searches of the database for occupational information presented at different levels of detail or customized in different ways for different needs.
The provision of extensive specific job titles as part of the summary page for each occupation, as well as links to lay titles commonly associated with each occupation.
Human Resource Management User Views of O*NET Limitations
The aspects of O*NET frequently cited as weaknesses or limitations by HRM users primarily involve either O*NET content or O*NET data.
Features of O*NET content cited unfavorably by users include the following (Anderson, 2009; Dierdorff, 2009; Handel, 2009; Harvey, 2009; Morgeson, 2009; Ryan and Pearlman, 2009):
Many O*NET descriptors are viewed as too generic—or not sufficiently specific—for some applications (e.g., defining training needs). In conjunction with the brief occupational description and limited associated task information provided, occupations are primarily defined in O*NET by cross-job descriptors rather than by job-specific content, making it difficult to readily understand how a job is performed.
Occupational information is not customized for jobs in a particular organization. This inability to describe a specific job in detail can limit O*NET’s utility for the legal defensibility of personnel selection procedures. It also means that O*NET realistically cannot serve as an organization’s only source of information about its own jobs for many applications. This issue may be exacerbated by the fact that the National Center for O*NET Development does not clearly describe or market O*NET as an input
or source of data, rather than a ready-made solution for human resource management applications requiring organization-specific job information.
Some specific descriptor elements appear to be redundant or overlapping both within and across descriptor taxonomies or domains (see Chapter 2 for examples).
Some of the descriptors, at least as labeled, do not have obvious meaning or relevance to many HRM professionals (e.g., the Extent Flexibility and Static Strength abilities), do not readily lend themselves to measurement, and may hence be of limited practical value.
Some descriptors of potential relevance or value are not represented in any domains of the content model. For example, certain medical and health-related abilities, such as finger, hand, arm, or leg strength or flexibility, and some technology-related skills and knowledge (such as advanced knowledge of information technology, IT) are missing. Some work context variables, especially those that may be useful in the characterization of “high-involvement” or “high-performance” workplaces, are missing. Although some such variables are included in the Organizational Context domain of the content model, data are not currently collected in this domain. These omissions can limit O*NET’s utility for HRM applications in organizations for which such attributes or variables are relevant.
Some content model descriptors (for example, in the Work Styles and Work Context domains) may be too organizationally specific to make sense as stable, appropriate, or generalizable occupational descriptors (for example, Work Context items concerning the effects of one’s decisions on others).
Some occupational areas are not adequately represented or are not well differentiated. Frequently mentioned examples of this are professions related to health care and IT, in which superficially similar jobs may have very different knowledge and training requirements. The absence of military-specific occupations also was mentioned as limiting the use of O*NET by the various military service branches and components (Styer, 2009).
Features of O*NET data cited unfavorably by users include the following (Anderson, 2009; Handel, 2009; Harvey, 2009; Jeanneret, 2009; Karman, 2009; Morgeson, 2009; Ryan and Pearlman, 2009):
O*NET’s occupations represent too broad a level of aggregation for some applications (e.g., personnel selection). In other words, these occupations represent too heterogeneous a collection of specific jobs or job titles. As a result, O*NET’s occupational-level data—and, consequently, inferences based on such data—may not be valid for some or most of the more specific jobs or titles encompassed by the occupation. This could be problematic for HRM applications that involve establishing linkages between O*NET data and jobs in specific organizations, such as job analysis/job description applications.
Various aspects of the quality of O*NET descriptor ratings are questionable, based on such issues as:
the possibility of rating inflation because of use of incumbents as raters for many descriptor domains;
the possibility of inaccurate ratings because of the use of analysts who do not perform the job;
low interrater reliability for some descriptors;
lack of descriptor or scale validity, especially for more abstract attributes, such as abilities, because of the use of single-item scales, occasionally unclear or jargon-laden item or anchor wording, and questionable validity of rating scale value intervals between some descriptor anchors;
potentially useful types of rating data that are not collected, such as frequency or time spent (for tasks or generalized work activities), “needed-at-entry” (often very important for personnel selection applications), consequences of error, and depth of required knowledge or degree of required skill/expertise (for tools and technology items); and
a variety of concerns about the utility of the “level” rating scales used for some descriptor domains (see Chapter 3).
Some of these user concerns comport with the panel’s evaluation of the content model in Chapter 2. Although some of these concerns have been disputed (Tsacoumis, 2009), the panel thinks that they warrant further systematic examination and evaluation. Such examination would serve the long-term interests of both DOL and the O*NET user community. The areas the panel views as most in need of such evaluation are outlined in the following section.
CONCLUSIONS AND RECOMMENDATIONS
The panel has identified gaps in the available information on the uses of O*NET data for HRM purposes.
Recommendation: To address the lack of systematic information on uses of O*NET, the Department of Labor should, with advice and guidance from the technical advisory board recommended in Chapter 2 and the user advisory board recommended in Chapter 6, establish and execute a framework for evaluating uses of O*NET that includes
Development of evaluation metrics aligned with various uses of O*NET.
Review of the usefulness and accuracy of existing information on O*NET uses.
Development of methods to systematically evaluate the adequacy of existing methods for obtaining feedback from O*NET users.
Development of new methods to systematically and continuously obtain information about who uses O*NET, how and how frequently it is used, reasons it is not used or might not be appropriate for certain suggested uses, user community awareness of O*NET, the specific applications it is used for, user satisfaction, and objective measures of effectiveness or success in meeting user needs.
The panel recognizes that the O*NET Center currently collects and analyzes a variety of user input and feedback, such as O*NET OnLine site visit and search frequency data, customer service request/inquiry data, and user e-mails. However, these appear to primarily serve relatively narrow internal program feedback and day-to-day operational needs rather than broader system review/analysis needs and issues involving a longer term or strategic focus. Nonetheless, as noted above, we encourage evaluation of these existing user feedback mechanisms in terms of their potential utility in contributing to a more broadly targeted, and publicly available, O*NET usage and evaluation base of data and information.
The panel received a great deal of feedback from the HRM community focusing on perceived weaknesses of the O*NET content model, specifically on the domain taxonomies, such as Abilities, Skills, Knowledge, Work Styles, Generalized Work Activities, and Tasks. These concerns, combined with questions about some of the descriptors, imply that the content model is not as parsimonious as it could be, and that the content of some domains could be reduced or simplified. This community has expressed concerns
about factors that cast doubt on the reliability and validity of existing descriptor ratings, rating data that are not currently collected but could be useful, and issues specific to the “level” ratings completed for some descriptor domains.
The HRM community echoes the panel’s conclusion that there is a need for research on the content model.
Recommendation: The Department of Labor should, as part of its research on the content model and with advice and guidance from the technical advisory board recommended in Chapter 2 and the user advisory board recommended in Chapter 6, commission research and analysis directed to either mitigate or dismiss concerns raised by the human resource management community. Among other concerns, this research should explore the potential need for:
Descriptor modernization and updating, particularly in domains (such as Knowledge and Work Context) directly affected by technological change and other ongoing changes in workplace and workforce dynamics.
More emphasis on descriptors reflecting the cognitive, social/interpersonal, and other changing requirements of work in an economy that has continued to shift from a manufacturing and agricultural base to a knowledge and service base.
Improved linkages to broader world-of-work information that would expand on the labor market data information currently provided for each occupation—such as industry trends and forecasts and technological, demographic, and geographic trends affecting occupations. This type of information could enhance O*NET’s value for human resource strategic planning applications.
O*NET users in the HRM community have raised questions about the ability of the O*NET occupational classification system to accurately reflect the changing nature of jobs and employment relationships. Ten years ago, a National Research Council review of work and occupational classification systems noted that the boundaries between jobs were becoming more fluid and the range of choices around how to structure work was increasing (National Research Council, 1999). These trends, which have continued since that time, should be considered in the study of the O*NET occupational classification system recommended in this report.
Recommendation: As part of the research on the occupational classification system recommended in Chapter 3 and to meet the needs of human resource managers and promote seamless integration of O*NET
with industry competency models, the Department of Labor should commission research on:
Methods to describe or represent hybrid jobs (i.e., jobs effectively comprised of the work of two or more separately defined occupations);
Methods to expand the current sampling frame to better represent smaller establishments, as well as self-employed, part-time and contract employees;
Other potential sampling frames to better represent the changing labor market; and
Methods to appropriately capture or otherwise represent the increasing variability in how work is done and the increasing fluidity in job boundaries.
The available data overwhelmingly point to the need for improved communication and outreach about O*NET to the HRM community. There is a lack of awareness in this community about O*NET’s existence and capabilities. The current O*NET Toolkit for Business contains few of the specifics, examples, or illustrations that would be of practical value to HRM practitioners.
Recommendation: The user advisory board should advise the Department of Labor on the development of new materials and tools for knowledge dissemination about O*NET to the human resource management community. These materials and tools might include an instructional resource depository of business cases or exercises, lecture overheads for use in higher education and training, and toolkits, checklists, interactive tools, and illustrated user guides that clearly and simply demonstrate how O*NET can be used for a variety of human resource management applications.
In keeping with the panel’s recommendation that most development of O*NET tools and applications be conducted by external developers, these materials should be developed in collaboration with, and marketed by, professional associations, such as the American Society for Training and Development, the Society for Human Resource Management, and the Society for Industrial and Organizational Psychology.
In carrying out this recommendation, DOL and the O*NET Center can draw on their existing procedures for making O*NET data and information freely available for use by public and private developers, described in the previous chapter. These procedures, including a formal user agreement and voluntary registration system, provide models for similar procedures
to govern the process of providing information about O*NET to professional associations for use in creating and marketing educational materials and tools.
Anderson, L. (2009). Uses of O*NET in human resource management. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Anderson%20Power%20point.pdf [accessed July 2009].
Anderson, L., Cronins, C., Martin, C., Swetharanyan, S., and Weingart, E. (2007). Entry-level transportation construction workforce shortages. Report provided to Florida Department of Transportation, FDOT Contract # BDF05. Available: http://18.104.22.168/Main/Public/Blurbs/159590.aspx [accessed October 2009].
Anderson, L., Morath, R., Light, E., and Wilken, J.A. (2004). Estimating the dollar utility of changes in job performance due to seasonal allergic rhinitis and its treatment. Human Performance, 17, 43-69.
Andreassi, J.K., and Thompson, C.A. (2007). Dispositional and situational sources of control: Relative impact on work-family conflict and positive spillover. Journal of Managerial Psychology, 22, 722-740.
Coaster, J.A., and Christiansen, N.D. (2009). Effects of occupational requirements on the validity of personality tests. Presented at the Annual Conference of SIOP, April, New Orleans.
Converse, P.D., Oswald, F.L., Gillespie, M.A., Field, K.A., and Bizot, E.B. (2004). Matching individuals to occupations using abilities and the O*NET: Issues and an application in career guidance. Personnel Psychology, 57, 451-487.
D’Egidio, E.L. (2002). Building a job component validity model using job analysis data from the occupational information network. Dissertation Abstracts International: Section B: The Sciences and Engineering, 62(11-B), 5416.
DeLuca, D., and Hirsh, W. (2009, March 5). Comments on the Occupational Information Network (O*NET). Letter from the Workforce Sciences Group at Mercer to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Mercer%20Associates%20Comments%20on%20ONET.pdf [accessed July 2009].
Dierdorff, E. (2009). Improving O*NET: Perspectives on the content model. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/O_NET_Erich_Dierdorff_Presentation.pdf [accessed July 2009].
Dorman, R. (2009). Manpower’s use of O*NET. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Dorman%20Power%20point.pdf [accessed July 2009].
Handel, M. (2009). The O*NET content model: Strengths and limitations. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/O_NET_Michael_Handel_Presentation.pdf [accessed July 2009].
Harvey, R.J. (2009, April). The O*NET: Do too-abstract titles + unverifiable holistic ratings + questionable raters + low agreement + inadequate sampling + aggregation bias = (a) validity, (b) reliability, (c) utility, or (d) none of the above? Paper prepared for the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/O_NET_RJHArvey_Paper1.pdf [accessed July 2009].
Human Resources Research Organization. (2009, February). Status update: Develop/modify ASVAB Content Specifications Based on Military Job Analytic Information and Training Curricula. Alexandria, VA: Author.
Jeanneret, P.R. (2009). Use of O*NET in job analysis. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Jeanneret%20PP.pdf [accessed July 2009].
Jeanneret, P.R., and Strong, M.H. (2003). Linking O*NET job analysis information to job requirement predictors: An O*NET application. Personnel Psychology, 56, 465-492.
Johnson, J.W., Carter, G.W., and Dorsey, D.W. (2003, April). Linking O*NET descriptors to occupational aptitudes using job component validation. Paper presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
Karman, S. (2009). Definition of disability: What compels SSA to use DOT. Presentation to the NRC Panel to Review the Occupational Information Network (O*NET). Workshop on Uses of O*NET, March 26. Available: http://www7.nationalacademies.org/cfe/Karman%20Power%20point.pdf [accessed June 2009].
LaPolice, C.C., Carter, G.W., and Johnson, J.W. (2008). Linking O*NET descriptors to occupational literacy requirements using job component validation. Personnel Psychology, 61, 405-442.
Liu, C., Spector, P.E., and Jex, S.M. (2005). The relation of job control with job strains: A comparison of multiple data sources. Journal of Occupational and Organizational Psychology, 78, 325-337.
Morgeson, F. (2009). Thoughts about O*NET. Presentation to the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/O_NET_Fred_Morgeson_Presentation.pdf [accessed July 2009].
National Center for O*NET Development. (2009a). O*NET products at work 2009. Raleigh, NC: Author.
National Center for O*NET Development. (2009b). Sample spotlight: Profiles of O*NET users. Raleigh, NC: Author.
National Research Council. (1999). The changing nature of work: Implications for occupational analysis. Washington, DC: National Academy Press.
Peterson, N.G., Mumford, M.D., Borman, W.C., Jeanneret, P.R., and Fleishman, E.A. (1999). An occupational information system for the 21st century: The development of O*NET. Washington, DC: American Psychological Association.
Peterson, N.G., Mumford, M.D., Borman, W.C., Jeanneret, P.R., Fleishman, E.A., Levin, K.Y., Campion, M.A., Mayfield, M.S., Morgeson, F.P., Pearlman, K., Gowing, M.K., Lancaster, A.R., Silver, M.B., and Dye, D.M. (2001). Understanding work using the occupational information network (O*NET): Implications for practice and research. Personnel Psychology, 54, 451-492.
Reiter-Palmon, R., Brown, M., Sandall, D.L., Buboltz, C., and Nimps, T. (2006). Development of an O*NET web-based job analysis and its implementation in the U.S. Navy: Lessons learned. Human Resource Management Review, 16,294-309.
Ryan, A.M., and Pearlman, K. (2009). Summary of responses to survey on use of O*NET in human resource management and job analysis. Paper prepared for the Panel to Review the Occupational Information Network (O*NET). Available: http://nrc51/xpedio/groups/dbasse/documents/websupport/051750.pdf [accessed July 2009].
Styer, J. (2009). 2008 OCCU-Find in the ASVAB Career Exploration Program. Presentation to the Panel to Review the Occupational Information Network (O*NET). Workshop on Uses of O*NET, March 26. Available: http://www7.nationalacademies.org/cfe/Styer.ASVAB%20Action%20%20Story.pdf [accessed July 2009].
Taylor, P.J., Li, W., Shi, K., and Borman, W.C. (2008).The transportability of job information across countries. Personnel Psychology, 61, 69-111.
Tsacoumis, S. (2009). Responses to Harvey’s criticisms of HumRRO’s analysis of the O*NET analysts’ ratings. Paper prepared for the Panel to Review the Occupational Information Network (O*NET). Available: http://www7.nationalacademies.org/cfe/Response%20to %20RJ%20Harvey%20Criticism.pdf [accessed July 2009].
U.S. Department of Labor. (2008). O*NET data collection program, Office of Management and Budget clearance package supporting statement, Volume 1. Raleigh, NC: Author. Available: http://www.onetcenter.org/dl_files/omb2008/Supporting_Statement2.pdf [accessed June 2009].