National Academies Press: OpenBook

Pay for Performance: Evaluating Performance Appraisal and Merit Pay (1991)

Chapter: 6 Private-Sector Practice and Perspectives

« Previous: 5 Pay for Performance: Perspectives and Research
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

6
Private-Sector Practice and Perspectives

This chapter offers a broad-brush picture of contemporary organization performance appraisal, merit pay, and individual and group incentive pay practices as they apply to the managerial and professional jobs that are the focus of the committee's review. As the previous chapters demonstrate, the research on performance appraisal and pay for performance is limited in its ability to offer federal policy makers specific guidelines. A review of contemporary private-sector practice that covers the types of appraisal and pay for performance plans organizations use, the way they design and administer these plans, the contexts in which plans are operating, and the criteria many use to judge the effectiveness of these plans may offer some additional insights. Our aim is to identify points of convergence between sometimes tentative research findings and predominant private-sector practices.

The committee reviewed several sources of information on private practice: major proprietary surveys, a special survey of Conference Board firms conducted at the committee's request, and invited interviews with the personnel managers of five Fortune 100 firms whose appraisal and merit plans are generally regarded as successful. More details on the surveys reviewed by the committee appear in Appendix A. The Fortune 100 firms we consulted represented large, financially successful manufacturing firms in high-technology industries with employee populations of more than 50,000. Three of the five operated with U.S. unions. Most of them had been using their current performance appraisal systems for 20 years without major changes. For all our surveys and interviews, the respondents were predominately personnel managers who may be presumed to have some interest in presenting a favorable picture of their organization's practices—a

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

point noted in our earlier chapter on the nature of the evidence, and one to keep in mind throughout this chapter.

The chapter is organized into two major sections. The first reviews performance appraisal practices: the predominant types of appraisal used, the typical objectives of performance appraisal, common performance appraisal design and administrative characteristics, and measures of plan effectiveness. The second provides a similar review of merit pay and individual and group incentives. We focused on the individual and group incentive plans that do not add pay increases into base salaries, and we have labeled these variable pay plans. In each section we describe general trends concerning performance appraisal and merit and variable pay plans to provide a profile of "average" practice; we then use information from our interviews with the personnel managers of the five Fortune 100 firms to provide richer detail about performance appraisal, merit, and variable pay plan practices that are generally considered successful. Such details are not available in survey reports of performance appraisal practice. Each section ends with a brief discussion of the convergence or divergence between practice and the research findings presented in earlier chapters.

PERFORMANCE APPRAISAL: CURRENT PRACTICE AND EMERGING TRENDS

Our review of research in the previous chapter made it clear that performance evaluation is thought critical to the success of pay for performance plans in achieving performance improvements, in being accepted and thought fair and equitable by employees and other organization stakeholders, and in helping the organization to regulate costs wisely. Performance appraisals that focus on individual performance and typically use a combination of quantitative and qualitative performance objectives are the type of performance evaluations most often associated with merit pay plans.

General Trends in Performance Appraisal

Prevalence, Distribution, and Objectives

Between 93 and 99 percent of private-sector organizations use performance appraisal plans for their exempt and nonexempt salaried employees (Bretz and Milkovich, 1989; HayGroup, Inc., 1989; Hewitt Associates, 1989; Wyatt Company, 1989b). Larger organizations (> 1,000 employees) are slightly more likely than smaller ones to use performance appraisal (Bretz and Milkovich, 1989; Hewitt Associates, 1989). Hourly employees—especially unionized hourly—are less likely to be covered by performance appraisal plans; even so, over half the organizations surveyed in the last few years had performance appraisal plans for hourly employees (Bureau of National Affairs, 1981; Hewitt Associates,

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

1985, 1989; Bretz and Milkovich, 1989). The prevalence and distribution of performance appraisal plans appears to have increased since the mid-1970s. In particular, small companies are now more likely to use these plans, and executive and hourly employees are more likely, in all companies, to be covered by them (Bureau of National Affairs, 1974; Conference Board, 1977; Bretz and Milkovich, 1989).

Organizations have historically used performance appraisal to accomplish multiple organization objectives (Conference Board, 1977; Bretz and Milkovich, 1989; Wyatt Company, 1989b). Improvements to work performance, tying pay to performance (via merit plans), and communicating work expectations to employees are the three objectives that were consistently rated as the highest priorities among the surveys we reviewed. There was more interest in using performance appraisal results to validate selection and promotion decisions in the late 1970s—especially for hourly and nonexempt salaried employees—but this interest is not a high priority today (Bureau of National Affairs, 1974; Conference Board, 1977; Bretz and Milkovich, 1989).

Design Characteristics

The Conference Board (1977) reported that, despite the fact that most personnel managers believe job analysis, description, and evaluation provide necessary foundations to effective performance appraisal plans, less than half the companies in its survey even reviewed job descriptions prior to plan development or revision. Only about one-fourth of the larger organizations in the Conference Board sample had conducted any sort of pilot testing of performance appraisal plans prior to their implementation.

Management by objective (MBO) or "objective" work standard approaches were the performance appraisal formats most commonly reported for executives, managers, and professionals (Bretz and Milkovich, 1989; Wyatt Company, 1989b). The MBO format is a very loosely defined one and thus difficult to compare across organizations. MBO is really both a planning and an appraisal process in which the organization's strategic plans are supposed to shape broad goals that are passed down to employees through the management hierarchy. Both employees and their supervisors then participate in setting individual performance objectives against these goals. For work standard appraisals, management defines important job factors or dimensions that may be applied uniformly throughout a major job group, such as managers, or may be customized for particular jobs. Factors may be either qualitative (such as "provides group leadership") or more quantitative (such as "finishes projects within days assigned"), and they are scaled to denote different levels of performance ("well above average" to "well below average''). Both these performance appraisal approaches require raters to assess an employee against performance objectives or factors. The employee's ratings on these factors are then combined into one

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

overall rating. The typical appraisal rating includes three to five performance intervals or "buckets," ranging from "below expected standards" to ''meets expected standards" to "far exceeds expected standards" (Conference Board, 1977; Wyatt Company, 1987, 1989b; Bretz and Milkovich, 1989).

Most organizations reported skewing in their performance appraisal ratings—that is, ratings that do not follow the normal distribution; most employees are rated as fully satisfactory or above (Wyatt Company, 1987; Bretz and Milkovich, 1989). About 20 to 25 percent of the organizations in the Bretz and Milkovich survey required that summary appraisal ratings be either ranked (that is, individual employees in similar jobs are ranked top to bottom) or forced to approximate a normal distribution; others may suggest informally that managers rank or force distributions (Conference Board, 1984).

Administration Characteristics

The typical organization, as reflected in our survey review, used different performance appraisal plans for different employee groups (executives, managers/professionals, clerical employees, etc.) and has used the same plan, without major revisions, for about nine years (Bretz and Milkovich, 1989). Most organizations also reported that policy guidelines for performance appraisal design and administration were centralized (Hewitt Associates, 1989; committee's survey of Conference Board firms, 1990).

The Bretz and Milkovich survey (1989) reported that most organizations required an employee's immediate supervisor to conduct performance appraisals annually. Appraisals for managers and professionals were likely to be reviewed by a second level of management. There was no evidence that organizations are making increasing use of peer, subordinate, or self-review for performance appraisals, despite the reported popularity of these practices in the business press (Kiechal, 1989:201). Formal evaluations of managers' use of performance appraisal and penalties for poor use were rarely reported. The average time that managers spent on annual appraisals per employee was four to six hours.

Several surveys (Bretz and Milkovich, 1989; Wyatt Company, 1989b; committee's survey of Conference Board firms, 1990) reported that employee participation in performance appraisal design and administration was mostly limited to personnel staff; line managers were involved in administration only via actual assessments of their employees; employees were involved only if there was joint manager-employee setting of performance objectives for the appraisal and, in some cases, the appeals process. Only about one-fourth of the organizations in the Bretz and Milkovich survey (1989) had a formal employee appeals process.

The Wyatt Company (1989b) reported that most organizations did not provide managers and employees with much assistance in understanding and using performance appraisal. When assistance was provided, it was to the managers

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

who are expected to conduct appraisals, not to the employees being appraised. Only a small proportion of companies have written objectives for their performance appraisal systems or provide written instructions for supervisors about how to use performance appraisal plans. Performance appraisal training was typically conducted for managers only when a new performance appraisal plan was first implemented. Training focused on using forms, measuring performance, conducting interviews, providing feedback, and setting performance objectives. There was more training emphasis on avoiding bias (perceptual, memory, and racial/ethnic types of bias) in the 1970s than there is today.

Measures of Success

Fewer than half the organizations participating in the surveys the committee reviewed reported any formal measurement of performance appraisal success. Among those who did measure, managerial and employee opinion surveys were typical measurement approaches. These surveys ask personnel managers, other managers who administer the plans, and employees covered by the plans about how effective they perceive the plan to be both overall and in accomplishing specific plan objectives (improving performance, tying pay to performance, and communicating work expectations). Personnel managers (the designers of performance appraisal plans) were the most likely employees to be questioned in opinion surveys, as well as the most likely to view plans as "very effective" or "partially effective," but even they recognized problems. In general, less than 20 percent of personnel managers polled in recent surveys gave their performance appraisal plans an overall rating of "very effective"; another 60 to 70 percent, however, rated their plans "partially effective." Other managers and employees were similarly unenthusiastic. On average, less than one-third rate their organization's performance appraisal plans as ''effective" in tying pay to performance or in communicating organizational expectations about work (HayGroup, Inc., 1989; Wyatt Company, 1989b).

Richer Detail on Performance Appraisal Practices

These survey statistics on performance appraisal success paint a fairly bleak picture. It appears that most employees do not believe their organizations do a very good job of managing performance appraisal. However, the personnel managers we interviewed from the five Fortune 100 firms reported that they used similar measures of performance appraisal effectiveness, and that between 40 and 70 percent of all employees surveyed believed that performance appraisals were meeting objectives. In this section we discuss some of the detail these personnel managers provided about their firms' performance appraisal practices. To organize the discussion, we use the distinctions between effective and ineffective performance appraisal practices drawn from a 1989 Wyatt survey of performance management. This survey defined effective and ineffective

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

TABLE 6-1 Elements of Performance Appraisal Plans

Appraisal Plan Elements

Percentage of All Companies Surveyed n = 3,052

Percentage of Companies With Effective Plans n = 427

Percentage of Companies With Ineffective Plans n = 518

Written goals

66

84

39

Supervisor instructions

69

77

47

Supervisory guides

42

54

26

Annual training

18

32

6

Senior management training

64

79

45

Training covers:

objective setting

79

88

62

providing feedback

71

78

53

Joint supervisor-employee objective setting

34

54

14

Measurement of supervisors' plan use

10

21

2

Integration with the pay system

65

74

48

Note: Table reports survey results for a sample of 3,052 companies, which are also broken out for companies with effective and ineffective plans. Percentages in each cell are based on total number of companies in the column. Of the personnel managers responding, 14 percent (427) considered their firm's plan effective; 17 percent (518) considered their firm's plan ineffective; and 66 percent considered it partly effective.

Source: The Wyatt Communicator: Results of the 1989 Wyatt Performance Management Survey, Fourth Quarter, 1989 (Chicago: The Wyatt Company) pp. 7-8.

according to personnel managers' opinions about whether their performance appraisal plans met objectives. Effective and ineffective performance appraisals were distinguished along the elements shown in Table 6-1. These elements include process factors such as written goals, manager training, joint objective setting, and a structural factor, integration with the pay system. Each of the elements is more likely to be associated with companies that consider their performance appraisal systems effective, than with those that judge them ineffective.

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
The Performance Appraisal Process

The Wyatt performance management survey reported that performance appraisal plans involving annual training for managers—especially on how to set objectives and provide feedback—and encouraging joint participation of supervisors and employees in developing appraisal objectives and standards were more likely to be considered effective. So too were plans including written policy guidelines and instructions to managers about administration—especially with regard to merit increases and those requiring that managers be evaluated on their use of performance appraisal.

The performance appraisal details provided to the committee by the five personnel managers we interviewed follow effective practice as reported by Wyatt. Four of the five Fortune 100 firms had MBO performance appraisal plans covering exempt employees; the fifth had several types of performance appraisal plans for exempt employees, and one of these was an MBO plan. (Only one of the plans described covered unionized employees; it was not an MBO format.) In most cases, employees and their managers jointly developed the plan objectives against which performance would be measured, typically using a set of corporation-wide factors (e.g., customer satisfaction, affirmative action responsibilities, people development) and a set of more specific job-related responsibilities as guides. In one of the five firms, employees and their managers actually developed job descriptions together and could revise these descriptions in preparation for setting objectives. In two firms, there were formal interim reviews in which managers and employees could discuss the need for changes in performance appraisal objectives due to changes in priorities, working conditions, and so forth.

One firm demonstrated an array of communication tools (written and audiovisual) for instructing both managers and the individual employee about their roles in the performance appraisal process and the organization's goals for performance appraisal.

All five firms held managers accountable for their management of performance appraisal—primarily by reviews of performance appraisal results by the next higher level of management and by making effective performance appraisal management one of a manager's own performance appraisal objectives. Two firms had automated several aspects of the performance appraisal process and routinely notified managers about upcoming performance appraisals for their employees and any delinquencies in completing them.

The training efforts of most of these firms did not appear to go beyond those described earlier as average. Most offered two-day training sessions to introduce managers to the performance appraisal process. At a minimum, training covered objective-setting and how to provide feedback.

These process details from the interviews and the Wyatt results suggest that many personnel managers believe the process surrounding performance appraisal

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

design and administration to be at least as critical to employee acceptance as the appraisal format used, the number of intervals used in summary ratings, or whether the distribution of ratings is ranked or forced. It also suggests that these managers believe that employee acceptance or perceptions about performance appraisals as fair are important measures of plan success. Indeed, other sources (Bretz and Milkovich, 1989, the committee's survey of major Conference Board firms, 1990) confirm that the emphasis on process represents the latest thinking among personnel managers about how performance appraisal should be managed. These views are generally described as a shift away from what has been viewed as traditional performance appraisal toward "performance management." Bretz and Milkovich (1989) also suggested that this emphasis on process is consistent with current trends in performance appraisal research.

How Performance Appraisal Ratings Are Used for Pay Allocations

The Wyatt Company's comparison of effective and ineffective plans (see Table 6-1) also illustrated that organizations with effective performance appraisal plans were more likely to integrate them with their pay systems. Presumably, a rationale for how performance appraisal ratings influence merit increases and a clear statement of the place of merit increases in the organization's overall pay plan increases the likelihood that both employees and managers will understand the connection. The research reviewed in the previous chapter suggests that the better employees understand this connection, the more likely that performance will be improved. Since our profile of the typical organization's performance appraisal plan indicates that most organizations do use performance ratings for merit pay allocations, the integration of performance appraisal and the pay system that is characteristic of more effective plans makes sense.

However, a traditional rule of thumb among managers of performance appraisal has also suggested the wisdom of decoupling the appraisal process from merit pay. The rationale for this has been that both employees and their managers will be too focused on the money involved with the appraisal rating to attend to its developmental objectives. In particular, the concern has been that managers will deliberately inflate performance appraisal ratings to distribute merit pay, thus decreasing the chances that employees with real training needs will be identified or increasing the chances that overrated employees will be promoted beyond their capabilities. In this regard, it is interesting to note that four of the five firms we interviewed used performance appraisal ratings as one of the inputs to a ranking process. The rankings were then used to support merit increase, promotion, and reduction-in-force or dismissal decisions. This practice contrasts with our summary of general organization trends, which showed that fewer than 25 percent of the organizations surveyed used ranking schemes in addition to their performance appraisal summaries.

The ranking schemes presented by the four firms were similar. Managers

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

within the same functional responsibilities (for example, marketing) or divisional areas (for example, small appliances) gather at least annually, bringing the performance appraisal summaries of their employees with them. They are joined by their own managers. They emerge with a relative ranking for each employee that reflects joint decisions, negotiations, and shared goals regarding the group's norms for employee performance. The role of the higher-level managers is to help shape the group's definition of employee performance norms in a fashion consistent with other groups throughout the organization. Employees are told whether they are in the top, middle, or bottom ranks. In one of the four firms, ranking was used throughout the corporation; in the other three, ranking was used only in specific divisions.

The personnel managers in these firms believed that ranking helped to separate the performance appraisal process from the decisions about merit pay and promotion, thus strengthening managers' and employees' association of the appraisal process with counseling and development. They also believed that the joint management meetings involved in the ranking process helped managers calibrate and define their expectations about individual employee performance. This was viewed as especially important in middle-management ranks, where there is high mobility and thus a less shared sense of employee performance norms for a specific function or division. They believed that joint meetings also provide an added incentive for managers to do a careful job of performance appraisal because they would have to defend their rating decisions before their peers and superiors. Two other benefits of these meetings were noted: (1) they give managers familiarity with a wider set of employees and (2) they permit managers to attribute low rankings to the group.

Fit With Organization Culture and Personnel Practices

Surveys do not convey a sense of how an organization's performance appraisal plans are wedded to its culture, its work force, and its other personnel practices. The five firms that presented their performance appraisal plans to the committee all believed their plans were successful because they fit firm culture and personnel philosophy. In most cases, personnel philosophies were essentially meritocratic—that is, these firms hire, place, develop, reward, promote, and dismiss employees according to their contributions to a range of organizationally defined and ranked positions. Employees are continually made aware of this philosophy, from their first employment interview to their retirement. It would seem strange under a meritocratic philosophy if performance appraisal was not closely tied to other personnel programs. Indeed, all five firms had used performance appraisal plans for at least 20 years. The personnel managers also said they believed that meritocratic beliefs were fundamental to U.S. culture,

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

and that employees would perceive personnel practices with no reference to merit to be unfair.

Consistent with this meritocratic personnel philosophy, the five personnel managers we interviewed emphasized that their firms were perceived as good, even elite, places to work. Both formal organization communications and informal social norms reinforced these perceptions. Indeed, all the personnel managers we interviewed considered the identification of employees far above or below acceptable performance norms as a primary performance management objective of their plans.

This private-sector view of a meritocratic personnel philosophy and the role that performance appraisal plays in it appear to differ from the federal government's meritocracy, especially in practice. For example, the importance of identifying top and bottom performers in order to sustain high levels of work force contributions is accompanied in the private sector by relative discretion of managers to promote top performers and dismiss employees with consistently poor performance. Federal managers have more limited discretion to make such decisions. This lack of discretion may reduce the potential organizational benefits of performance appraisal and make its role in the federal meritocracy less clear.

All five of the personnel managers we interviewed indicated that their firms regularly canvassed employee opinions regarding performance appraisal plans; they were most concerned with indicators related to specific objectives—such as "the plan helps communicate work expectations" or "the plan links pay to performance"—than to overall satisfaction ratings. They believed these more specific indicators provide a better yardstick against which personnel managers can judge whether employees perceive that performance appraisal plans are operating as intended. They believed that employees' sense of consistency between what the organization says performance appraisal is supposed to do, and what it does, is basic to their perceptions about its fairness and that employees' sense of fairness about personnel programs in general is basic to a meritocratic personnel philosophy.

Four of the five firms had a centrally developed performance appraisal plan for exempt employees. The fifth, itself a major division (45,000 employees) of a larger corporation, had traditionally decentralized all performance appraisal decision making within the division, but had recently proposed a common MBO-type plan for all the division's exempt employees. In all cases, centralization meant that headquarters personnel staff provided managers with sample communications defining the firm's performance appraisal philosophy and its relationship to other personnel practices, a set of broadly defined performance areas (such as people management and development or customer satisfaction), a set of administrative guidelines, and training materials. Managers then had considerable discretion to adapt these to their own departments.

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Convergence/Divergence Between Research and Practice

There are two major points on which research and practice converge and diverge. The major point of convergence involves the emphasis on process in performance appraisal design and implementation. The details of performance appraisal practice provided by the five personnel managers we interviewed, as well as the distinctions drawn between effective and ineffective performance appraisal plans in the Wyatt report (1989b), both suggest the importance of performance appraisal process and its fit with the organization's culture and personnel philosophy. This emphasis is consistent with some emerging research trends in industrial-organizational psychology and human resource management.

However, it is also important to understand that this emphasis on process and fit comes mostly from the managers of performance appraisal plans and from their beliefs about how process investments will enhance employees' sense of the fairness of performance appraisal. We do not know whether line managers and other employees are equally enthusiastic about the process aspects of performance appraisal, and there are no generally available surveys that frame questions about performance appraisal in ways that would allow us to judge employees' beliefs about its fairness.

The major point of divergence between research and practice is in the area of measurement. Certainly the opinion survey measurement reported by most organizations does not exhaust potential measurement of the objectives typically reported for performance appraisal—work improvements, communication of work expectations, and tying pay to performance. For example, in the area of work or performance improvement, surveys indicate that most organizations do not attempt to validate their performance appraisals, or even to revise job analyses and descriptions prior to performance appraisal plan changes. Yet some validation efforts and good job information appear important to improving performance via appraisals and to enhancing employee perceptions about an appraisal's fairness. The research on performance appraisal focused for many years almost exclusively on psychometric properties but, as we noted in our review of performance appraisal research, the research focus has now given way to a much greater interest in operational aspects of appraisal.

The lack of measurement of performance appraisal effectiveness contributes to problems for policy makers. For example, performance appraisal does have serious detractors, such as Deming (1986) and his interpreters (Scholtes, 1987), who view the appraisal of individual employees as a "deadly disease" (Scholtes, 1987:1). In particular, they argue that individual performance appraisals cannot lead to significant improvements in organization productivity and quality. In fact, Deming believes that any use of individual performance appraisal is deadly; that organizations should focus on system-level, not individual-level, performance. While this view may represent an extreme, the evidence from

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

research and practice on performance appraisal is not sufficient to either confirm or deny it.

MERIT AND VARIABLE PAY PLANS: CURRENT PRACTICE AND EMERGING TRENDS

The committee's review of practice covered both merit and variable plans and included many of the same proprietary surveys covered in the review of performance appraisal practice, as well as interviews with the personnel managers of the five Fortune 100 firms.

General Trends in Merit and Variable Pay Practice

Prevalence, Distribution, and Objectives

Recent surveys report that merit pay plans cover exempt employees in 95 percent of private-sector organizations (Wyatt Company, 1987b; HayGroup, Inc., 1989; Hewitt Associates, 1989). Executives and hourly employees, especially unionized hourly employees, are less likely to be covered by merit pay plans, and larger organizations are slightly more likely than smaller ones to have merit pay programs (Bureau of National Affairs, 1984; Hewitt Associates, 1989). There is no evidence of a decline in the use of merit pay programs, despite some predictions to the contrary (O'Dell, 1987; Hewitt Associates, 1989).

Merit pay objectives are related to an organization's compensation objectives. We observed in the previous chapter that most organizations report at least three objectives for their pay systems and their pay for performance plans: attracting and retaining high-performing employees and sustaining or improving the performance of these employees, ensuring that pay for performance plans are fair and equitable, and regulating costs.

Most surveys, however, do not directly address the question of why merit pay plans are used. Paying for performance is a top-ranked compensation objective for over 80 percent of the organizations responding to a 1984 Conference Board survey. When asked why, many top managers stated that U.S. managers and employees believe pay increases should be related to performance (Wm. M. Mercer, Inc., 1983; Conference Board, 1984). Because merit pay plans are so prevalent, these survey statistics suggest that organizations view merit pay as a means of at least sustaining employee performance in a way that will be viewed as fair or equitable by the majority of employees. More explicitly, merit pay plans appear, by design, to help regulate payments consistently according to performance ratings—that is, everyone with the same rating and position in the salary range receives the same payment. This could help regulate labor costs and enhance employees' perceptions of the fairness of merit pay. Merit pay plan design practices are discussed in the next section.

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

The prevalence and distribution of variable pay plans are difficult to gauge via surveys. There is such a variety of plans that it is difficult to tell exactly which plans are being counted in any given survey. A recent survey (O'Dell, 1987) reported phenomenal growth in organizational interest in variable pay plans, but such growth must be assessed against a 40- to 50-year history of scant use of or interest in variable pay in U.S. industry (Mitchell et al., 1990). There is no doubt that variable pay plans are much less prevalent and less widely distributed across employee groups than merit plans. For example, O'Dell's 1987 survey of incentives (conducted by the American Compensation Association and the American Productivity Center) indicated that 13 percent of the firms they surveyed (n = 1,598 private-sector firms) were using gainsharing plans. The Hewitt Associates 1989 compensation survey reported that 16 percent of their survey respondents (n = 705 private-sector firms) were using gainsharing; the Conference Board's 1990 survey of variable pay (n = 435 private-sector firms) reported 13 percent. Hewitt noted that two-thirds of the gainsharing plans in their survey had been in place less than three years. Similarly, Hewitt reported that 16 percent of the firms they surveyed reported using cash profit-sharing plans; the Conference Board reported 19 percent. Hewitt also reported that half of the cash profit-sharing plans in their survey had been in place for less than three years.

Executives have traditionally had profit-sharing and bonus plans, sales people are often on commission plans, and a limited number of hourly employees work on piece rate plans (such as in the garment industry); however, the vast majority of employees have not been covered by variable plans. The 1989 Hewitt survey suggests that variable pay plans may now be covering some nontraditional employee groups. For example, 35 percent of the organizations in the survey (n = 435) reported using gainsharing plans for exempt employees, although these plans have been more commonly used for nonexempt employees. Profit-sharing, traditionally used for executives, covered nonunion hourly employees in 47 percent of the organizations surveyed. TPF & C/Towers Perrin (1990) reported that variable pay plans are less likely to be found in union environments.

The objectives claimed for variable pay plans are legion. The 1990 TPF & C report on group incentives indicates that 73 percent of the organizations they surveyed (n = 144) gave "supports personnel strategy as it relates to competitive or revitalization business strategies" as their most important reason for adopting variable pay plans. They noted that this objective encompassed other goals: encouraging employee participation, increasing organization productivity and quality, increasing employees' sense of ownership in the organization, and moving employees away from a sense of entitlement to automatic annual pay increases. O'Dell's 1987 report (700 organizations in their sample used gainsharing or profit-sharing plans) listed increasing organization productivity and financial performance as one of the most important reasons for organizations'

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

adoption of variable plans. Controlling costs was another important reason given. The Conference Board (1990) reported that, of the 57 organizations with gainsharing plans in their survey, over half thought organization productivity and quality improvements were the most important reasons for adopting the plans; between 25 and 30 percent indicated that they used gainsharing to increase employee involvement and promote teamwork; 19 percent reported controlling labor costs as important. Taken as a whole, these reports suggest that organizations adopt variable pay plans to improve organization performance, increase employee acceptance and involvement in organization goals, and regulate costs.

Plan Design Characteristics

Merit pay plans are typically implemented via a merit grid (see Figure 6-1). Hewitt Associates (1989) reports that 58 percent of the organizations in their survey used merit grids that determined individual merit payments according to appraisal performance ratings, positions in the pay range, and the size of the merit budget. Typically, the distribution of pay increase percentages in a merit grid is based on assumptions about the percentage of employees at each performance level and position in the pay range. (Approximately 70 percent of organizations employing 10,000 or more use merit grids. Another 20 percent use merit grids that allocate payments according to individual ratings and the size of the merit budget.) Merit payments are usually expressed as a percentage of each employee's base salary and distributed annually. For budget and cost regulation, merit grids are often designed so that the higher an employee is in a pay range, the lower the recommended merit payment for a given performance appraisal rating. When organizations are in growth periods, higher-performing employees in the top of their ranges can be promoted into the next pay range; in low-growth periods, such moves are less likely. Under today's lean staffing policies, some organizations are considering dropping this cap on merit payments and offering lump sum bonuses. These latter are essentially merit payments that are not depressed by an employee's position in the range. In contrast to conventional merit pay plans, some organizations do not add lump sum bonuses to the employee's base pay. Hewitt (1989) reports that 15 percent of the organizations in their survey were using lump sum bonuses for exempt employees.

Merit budget setting is centralized in most organizations (79 percent) (Hewitt, 1989). Top managers report that their organizations' "ability to pay" or its profitability and the pay offered by its labor market competitors are the most important factors determining budgets (Wyatt Company, 1989a). Merit budgets are typically allocated to each business unit or department as a percentage of its payroll (Conference Board, 1984).

Just as there are many types of variable pay plans, there are many variations on plan design. Although it is beyond the scope of our review to describe the

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

 

 

Position in Salary Range

Performance Rating

Percentage of Employees*

Low

->

High

A

B

C

D

E

Outstanding

6%

9%

8%

7%

6.5%

5%

Exceeds expectations

20%

7%

6%

5.5%

5%

4%

Fully satisfactory

70%

5%

4.5%

4%

3.5%

3%

Needs some improvement

3%

1%

1%

1%

1%

No increase

Unsatisfactory

1%

No increase

No increase

No increase

No increase

No increase

* Employees distributed across performance ratings

FIGURE 6-1 Sample merit grid.

spectrum of variable pay plan designs, there are some common design issues that must be addressed in all such plans: determining the performance measure to be used, identifying employee eligibility, specifying the payout distribution rules, and setting payout form and frequency (Milkovich and Newman, 1990). As under merit plans, the distribution rules help to regulate costs or the distribution of the plan funds and ensure that individual employees are treated consistently.

An example will suffice to illustrate these issues. Under a gainsharing plan, determining the performance measure involves deciding what level of system performance (work group, department, plant) is best and what type of measurement (ratios of labor costs to hours or of production value to volume) should be used. Employee eligibility might include only hourly employees, hourly plus nonexempt salaried, or all employees in a work group, plant, or department. Decisions about payout distributions in gainsharing might involve both potential splits of any gains between the company and the employees and whether to distribute the gains to individual employees as a percentage of base salary or across the board. Finally, decisions must be made about how often to assess and distribute gains (monthly, quarterly, twice a year, and in what form) as part of the regular check, as a separate check, as cash, or as some form of deferred payment. (For an overview of variable pay plan design and a source of further references, see Milkovich and Newman, 1990.)

Plan Administration Characteristics

We reported earlier that top managers tend to talk about merit pay as an important part of their overall compensation systems (Conference Board,

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

1984). This suggests that organizations define the role of merit pay in their pay communications to employees. Yet we found no recent surveys detailing what information organizations provide their employees about pay. A 1976 Conference Board survey indicated that up to 70 percent of the organizations in their survey had a policy of telling employees their pay range, but fewer than 20 percent discussed the organization's overall pay structure, let employees know what other organizations were used as a comparison group for determining salary market competitiveness, or told employees the size (percentage) of the average merit increase. This is certainly in direct contrast to the federal government, where this pay information is available to employees.

There is no average set of administrative guidelines for variable pay plans. Unlike merit plans, however, variable pay plan administration—or perhaps the better word here is implementation—often goes hand in hand with much broader organization changes such as job redesign, team development, changes in management style, increased investments in employee participation, major communication efforts, more sharing of information with employees, more explicit provisions for job security, training for plan administration, and so forth (Conference Board, 1990; TPF & C, 1990; Wallace, 1990). (For more information on variable pay plan implementation and further references, see Milkovich and Newman, 1990.)

Measures of Plan Effectiveness

There is no direct survey evidence of the effectiveness of merit pay programs in improving individual performance, enhancing employee perceptions of pay fairness, or regulating labor costs. The Bureau of National Affairs (1984) reports that fewer than 6 percent of the organizations in their pay survey attempt any formal assessment of their merit plans. The little evidence that does exist comes from opinion or attitude surveys of managers and employees. This evidence suggests that most do not see clear links between their performance and their merit increases. For example, the Wyatt Company's report on employee attitudes (1989b) found that only about 28 percent of the employees they surveyed saw a link between their pay and their job performance.

The surveys on variable pay plans also tend to report variable plan effectiveness in terms of managers' opinions about a plan's success in meeting the organization's objectives. TPF & C's 1990 report, for example, indicates that the managers involved in plan design and administration believed that their plans yielded quantifiable improvements in group or organization measures of quality and productivity and in employee involvement, communications, and commitment. All recent surveys of variable pay plans (O'Dell, 1987; TPF & C, 1990; Wallace, 1990) suggest that plan design variables and the organizational context in which the plan operates are critical to a plan's effectiveness. Careful specification of performance measures, the distribution of gains, the information

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

exchanged with employees, the steps taken to enhance employee involvement, and an emphasis on how the plans fit the organization's broader mission were all considered important to plan effectiveness.

At first glance these measures of variable pay plan effectiveness appear more positive than those for merit pay. They must be placed, however, in perspective. There are few survey measures of the specific effects of merit plans. We also know very little about how employees react to variable pay plans—whether they view them as fair, whether they view them as linking pay to performance, and so forth. There is, after all, a downside to variable pay that is not usually emphasized in these survey reports, namely that variable plans do not pay out when there are no performance improvements. Wallace (1990) also reports that, based on his study of 46 firms, the costs of implementing variable pay plans are significant.

Richer Detail on Merit Pay Plan Practices

Our review of survey results on merit pay suggests that most private-sector organizations use merit pay plans for their exempt employees, that there is little variation in plan design and administration, and that most top managers and personnel managers report that merit pay is an important part of their pay systems—in part because they believe that U.S. social values support the rightness or fairness of tying pay to individual performance.

The personnel managers (representing five Fortune 100 firms) interviewed by the committee all reinforced the importance of viewing merit pay as part of a broader pay system that, in turn, supports meritocratic personnel practices. Merit plan design and administration in their organizations do not depart from the typical company described in our survey review but, as was true in their discussion of performance appraisal, all the personnel managers emphasized the efforts they make to place merit pay in a broader meritocratic context. They noted that pay communications, in particular, are designed to demonstrate to employees that their base pay is competitive with a relatively elite group of corporations, that the organization plans to continue to provide base pay that at least meets competition, that each employee is doing an important job, and that merit plans allow the organization to provide returns to individual contributions. In general, the tone was ''This is a good place to work, we pay competitively, we expect a great deal from employees, and thus each one of you who meets those expectations is one of an elite group." These managers also indicated that they share little specific pay information with employees, which is consistent with the survey information we reviewed.

As noted in our discussion of performance appraisal and its role in a meritocracy, there appear to be differences between the private-sector and federal meritocracies that may influence the role of merit pay. For example, in the federal government, managers have little control over the pay information

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

available to employees. This may make it more difficult to persuade employees that they are paid competitively overall and that merit plans provide returns for individual contributions and long-term salary growth. Instead, employees may simply view merit pay as a means of making their base salaries more (or less) competitive.

The five personnel managers we interviewed believed that merit pay plans helped their organizations regulate the distribution of the annual merit payments so that employees were treated consistently and payments were within budget.

The committee did not interview the five personnel managers regarding variable pay plans in their organizations.

Convergence Between Research and Practice

There appear to be two major points of convergence between research on merit pay and variable plans and current private-sector practices. The first is the lack of measures of effectiveness for merit plans in meeting any pay for performance plan objectives. In comparison, the measures of effectiveness of variable pay plans seem prolific. Despite the prevalence of merit pay plans, we cannot determine their effects. And the relative scarcity, the recency, and the air of advocacy surrounding variable pay plans also means that the existing survey measures of their effectiveness must be taken with more than a few grains of salt.

The second is that both research and practice consider context important to organization decisions about adopting and implementing merit or variable pay plans. The five personnel executives we interviewed stressed that merit pay plans reflected their firms' meritocratic personnel philosophies and that merit principles were an important part of their cultures. The survey literature on variable pay plans reveals that firms with some experience in implementing these plans stress the importance of management support for the plan, work cultures that reflect good employee-management relations and participative management styles, and training for those involved in administering the plan. The research literature also suggests the importance of context variables such as job or task structure, the organization's technology, occupational diversity, labor intensity, and personnel practices, as well as environmental variables such as unionization and rate of economic growth.

PRIVATE-SECTOR PRACTICE: CONCLUSIONS AND THEIR IMPLICATIONS FOR THE FEDERAL GOVERNMENT

Our review of practice indicates that performance appraisal and merit plans are extensively used for professional and managerial employees in private-sector

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

firms. However, surveys report little formal measurement of the effects of performance appraisal and merit plans on individual performance, on employee perceptions of the fairness of these plans, or of the direct and indirect labor costs associated with plan development and administration. The measures that are reported involve opinion surveys of either employee perceptions about the success of appraisal and merit plans in linking pay to performance, or of personnel managers' assessments of how well these plans work. In general, neither employees nor personnel managers are overly enthusiastic. These opinion survey trends appear consistent with the federal government's experience, yet they have not been accompanied by any decline in the private sector's use of performance appraisal and merit pay.

Our interviews with the personnel executives of five Fortune 100 firms that considered their performance appraisal and merit plans successful provided the committee with richer details. These details may offer one explanation for the persistence of performance appraisal and merit pay plans in the face of less than universal enthusiasm about them. These five personnel managers all stressed that their performance appraisal and merit plans are embedded in meritocratic personnel philosophies and work cultures that support merit pay. They believed that the majority of their employees would find a pay system with no connections to their performance unfair, and that no plan for pay distribution would meet with unqualified employee approval and satisfaction. These beliefs are consistent with surveys of managerial and professional employees that report support for merit pay.

These interviews also pointed out some differences between meritocratic personnel practices in the five Fortune 100 firms and in the federal system, differences that have implications for federal performance appraisal and merit plans. For example, the personnel managers noted that a major benefit of performance appraisal is the identification of top and bottom performers. However, private-sector managers have more flexibility (and incentive) in promoting top performers and dismissing consistently poor performers than is typical in the federal government. Similarly, the personnel managers stressed the importance of communicating merit pay as one element of a broader, competitive pay system that recognizes a variety of employee contributions and needs. These managers, however, also have more control over the pay information that employees receive than is typical in the federal government. Finally, the personnel managers we interviewed stressed the importance of process in managing performance appraisal and merit plans. However, in at least one process area—that of procedural protections—the private sector appears to have less formal, more flexible procedures in place to handle employee appeals than is true of the federal government.

These examples suggest that, at least in these five firms, a sizeable degree of management flexibility accompanies a meritocratic personnel philosophy and performance appraisal and merit plan administration.

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Management flexibility is just one of many context factors that may influence the federal government's merit pay reforms. We have listed a number of other potentially influential context factors throughout this report. In the next chapter, we discuss these factors and their implications for the federal government in more detail.

Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 102
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 103
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 104
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 105
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 106
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 107
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 108
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 109
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 110
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 111
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 112
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 113
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 114
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 115
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 116
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 117
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 118
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 119
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 120
Suggested Citation:"6 Private-Sector Practice and Perspectives." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 121
Next: 7 The Importance of Context »
Pay for Performance: Evaluating Performance Appraisal and Merit Pay Get This Book
×
Buy Paperback | $44.95
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

"Pay for performance" has become a buzzword for the 1990s, as U.S. organizations seek ways to boost employee productivity. The new emphasis on performance appraisal and merit pay calls for a thorough examination of their effectiveness. Pay for Performance is the best resource to date on the issues of whether these concepts work and how they can be applied most effectively in the workplace.

This important book looks at performance appraisal and pay practices in the private sector and describes whether—and how—private industry experience is relevant to federal pay reform. It focuses on the needs of the federal government, exploring how the federal pay system evolved; available evidence on federal employee attitudes toward their work, their pay, and their reputation with the public; and the complicating and pervasive factor of politics.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!