National Academies Press: OpenBook

Pay for Performance: Evaluating Performance Appraisal and Merit Pay (1991)

Chapter: 2 The History of Civil Service Reform

« Previous: 1 Introduction
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

2
The History of Civil Service Reform

For nearly 50 years, the federal government has operated with some performance appraisal procedures whose purposes have been to strengthen the link between pay and performance. Since 1978, specific pay for performance programs have been in place for mid- and upper-level federal managers. There is general agreement that these programs have not attained the desired objectives; their troubled history has included a series of adjustments and changes, differing levels of financial support, and little evidence of success. The ability to demonstrate a link between performance and pay—to both the employee and the public—remains problematic for the federal government.

As we approach the year 2000, the questions surrounding pay for performance in the public sector have assumed a new importance—indeed, a central position—in new proposals for federal civil service reform. Many of the questions raised in the debate about the 1978 reforms are being raised again. Why is this so? What accounts for the intransigence of problems surrounding effective pay for performance systems in the federal government? Is there evidence to support the validity of the effort, despite its problems?

In the federal government, the answers to these questions are made more difficult by the nature of the federal personnel system, by the intermingling of issues of political responsiveness with issues of effective management, and by the need to marshal very scarce resources for a policy activity that never ranks very high on the national agenda. It is our intent in this chapter to provide the historical and contextual information necessary to understand these constraints and their implications for performance-based pay schemes in the federal government.

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

THE CONTEXT AND THEORY OF CIVIL SERVICE REFORM

The Civil Service Reform Act of 1978 and its outcomes can be best understood within the context of the historic and institutional influences that led to its creation. Two aspects of this context are discussed here: (1) the historical evolution of merit principles in the federal sector and (2) the evolution of federal management strategy.

Evolution of the Federal Merit System

The passage of the Pendleton Act in 1883 marked the origin of the merit system and the classified civil service in the federal government. This landmark legislation was intended to create a system that not only protected federal employment and employees from the excesses of partisan politics, but also provided the federal government with a competent and politically neutral work force. The Pendleton Act contained three fundamental merit principles: fair and open competition for federal jobs, admission to the competitive service only on the basis of neutral examination, and protection of those in the service from political influence and coercion (Ingraham and Rosenbloom, 1990). At the time the Pendleton Act was passed, the spoils system had thoroughly politicized the federal service. The electoral success of candidates supported by civil service leagues in the 1882 congressional elections put civil service reform on the national agenda. Public attention to the problem was galvanized by the assassination of President Garfield by a demented campaign worker who sought federal office.

Despite the clarity of the problem and fairly widespread consensus on the need for action, the reality of the Pendleton Act was modest: only 10 percent of the federal work force at that time was covered by the initial legislation. Congress granted the President authority to add federal employees to the merit system as he saw fit. Van Riper (1958) notes that the act permitted "… an orderly retreat of parties from their prerogatives of plunder. …"

The Pendleton Act created decentralized Boards of Examiners to administer entrance examinations. Unlike the British system, which had served as the ideal for many reformers, the U.S. system did not rely on elite formal academic training. Rather, it emphasized common sense, practical information, and general skills. Neutrality was a primary value in the merit system. One observer wrote that "… the civil service was like a hammer or a saw; it would do nothing at all by itself, but it would serve any purpose, wise or unwise, good or bad, to which any user put it" (Kaufman, 1954).

The fledgling merit system, intended to remove politics from the federal service, developed and grew as politics allowed. Its history is not, therefore, one of coherent development; it reflects shifting and changing political priorities and cycles. Both Congress and the President retained a keen interest in patronage

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

issues long after passage of the Pendleton Act. President McKinley, for example, included 1,700 new positions in the classified service, but exempted 9,000 that had previously been covered (Skrowonek, 1982). Congress excluded entire agencies from the classified service. In the New Deal years, Franklin D. Roosevelt successfully urged that the new agencies created be staffed by persons with policy expertise congruent with the President's interests, rather than the ''neutral competents" produced by the civil service examination system. When Roosevelt assumed the presidency in 1932, about 80 percent of federal employees were in the competitive civil service. By 1936, that proportion was about 60 percent (U.S. Civil Service Commission, 1974).

Of equal significance, new provisions and procedures were layered on incrementally as the system grew. Until the time of the New Deal, most of the new provisions, with their emphasis on economy, efficiency, and standardization, reflected the scientific management principles in vogue in the business and public administration communities. One such effort was the creation, in 1912, of the skeleton of a performance appraisal system. In that year, the Civil Service Commission (CSC) was directed by Congress to establish a uniform efficiency rating system for all federal agencies. The commission established a Division of Efficiency to carry out this task (U.S. Civil Service Commission, 1974).

The passage of the Classification Act in 1923 represented a more ambitious attempt to bring scientific management principles to the federal merit system. In words that have a familiar ring, the Joint Commission on Reclassification of Salaries had concluded in 1920 that the United States government, the largest employer in the world, needed a "modern classification of positions to serve as a basis for just standardization of compensation" (quoted in Gerber, 1988). The Classification Act established in law the principle of nationally uniform compensation levels, providing for the standard classification of duties and responsibilities by occupations and positions with salary levels assigned to the resulting positions.

In addition, the Classification Act legalized the principle of rank in position. Unlike the more common European practice of rank in person, the U.S. system provided that wages and/or salary for each position were to be determined solely by the position description and the qualifications for it, not by the personal qualifications of the person who would occupy the position. Finally, the Classification Act of 1923 led to the creation of a standard rating scale, which required supervisors to rate employees for each "service rendered." This was the first government-wide effort to describe job requirements and employee performance.

The Classification Act came under almost immediate attack. Evaluations in 1929 and 1935 found major problems with the classification system that it established. Primary criticisms focused on the extremely narrow and complex nature of the classification process. The 1935 inquiry noted, for example, that "what seem to be the most trifling differences in function or difficulty are

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

formally recognized and duly defined …" (Wilmerding, 1935). Nonetheless, the Classification Act was not reformed until 1949, following the release of the first Hoover Commission report. That report had been blunt about the state of the federal merit system:

Probably no problem in the management of the Government is more important than that of obtaining a capable and conscientious body of public servants. Unfortunately, personnel practices in the federal government give little room for optimism that these needs are being met.

Although not universally considered an improvement (see Gerber, 1988), the Classification Act of 1949 simplified the classification system by reducing the number of pay categories from five to two: the 18-grade General Schedule for white-collar employees and another schedule for blue-collar employees. It created the "supergrade" system (GS 16–18), which was in many ways the predecessor of the Senior Executive Service (a version of which had been recommended by the first Hoover Commission). The 1949 act also marked an early point on what has come to be a centralization-decentralization cycle in federal personnel policy, when it delegated some classification authority back to the agencies (Ingraham and Rosenbloom, 1990). Classification of managerial jobs in the federal merit system has not been reformed since the passage of the 1949 act.

There have been other initiatives related to performance appraisal, however, that are worth noting in this brief overview. The Ramspeck Act created efficiency rating boards of review in 1940. The uniform efficiency rating system that resulted was in place until 1950, when it was replaced by the provisions of the Performance Rating Act of 1950. The Performance Rating Act required agencies to establish a performance appraisal system with the prior approval of the CSC. This system established three summary rating levels: "Outstanding," "Satisfactory," and ''Unsatisfactory." Employees were permitted to appeal ratings to a statutory board of three members consisting of representatives from the agency, one selected by employees, and a chairperson from the CSC. The act required a 90-day written warning of an unsatisfactory rating and opportunity for employees to improve.

Financial incentives to accompany performance were introduced by the Incentive Awards Act of 1954, which authorized recognition and cash payments for superior accomplishment, suggestions, inventions, or other personal efforts. The intent of the Incentive Awards Act was reinforced by passage of the Salary Reform Act of 1962. This act established an "acceptable level of competence" determination for granting General Schedule within-grade increases. Within-grade increases could be withheld when performance dropped below an acceptable level, but the agency was obliged to prove that performance was not acceptable. Employees were permitted to appeal to both the agency and, if denied at the agency level, to the Civil Service Commission. The Salary Reform

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Act also authorized an additional step increase or quality step increase (QSI) for "high-quality performance." This system guided performance management in the federal government until the passage of Civil Service Reform Act in 1978.

These incremental changes and 100 years' accretion of laws and procedures have resulted in an enormously complex federal merit system. Entrance to the system can now be through "competitive," "noncompetitive," or "excepted" authority. Veterans have preference in hiring and, until 1953, did not have to pass an examination to be considered for employment. There are direct hiring authorities for hard-to-hire and specialized occupations, for outstanding scholars, for returned Peace Corps Volunteers, for Vietnam-era veterans, and many others. Examinations are not required in these cases. There is extensive use of temporary and part-time hiring; there are 35 different ways to hire temporary employees alone (for additional discussion, see Ingraham and Rosenbloom, 1990).

At the time the Civil Service Reform Act of 1978 was passed, over 6,000 pages of civil service law, procedure, and regulation governed the federal merit system. There were at least 30 different pay systems in place; there were over 900 occupations in the federal civil service. This complexity was one of the problems addressed by civil service reform; the history and development of the complexity profoundly influenced the reform's potential for success. It is significant that the 1978 act did not for the most part address basic entrance procedures, the classification system, or the basic federal compensation systems. In many respects, it reformed at the fringes of the system.

Federal Management Strategies and Civil Service Reform

Federal management strategies provide another set of influences that were important to the context and development of civil service reform. At least since 1937, when the Brownlow Commission issued its report on the Executive Office of the President, appropriate theories and structures for federal management have been debated by academic analysts and elected officials. The remarkable growth of government in Franklin Roosevelt's first term created a management problem unknown to previous presidents. The steady expansion of the civil service system in the years prior to the New Deal had created a large permanent bureaucracy founded on the neutral competence model. President Roosevelt wished, however, to have bureaucracies and bureaucrats more responsive to his policy agenda.

The Brownlow recommendations, while continuing to argue for neutral competence, firmly articulated the concept of the President as manager of the executive branch. The Federal Reorganization Act of 1939 was the cornerstone for the development of that presidential capacity. An emphasis on structural change, such as that found in President Carter's Reorganization Plan No. 2, has

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

been a consistent emphasis of most presidential management initiatives since that time.

The evolution of those management efforts has been characterized by a shift from the basic question "How should government be managed?" to a new query: "Who should manage government?" The answer from the White House has been consistent and predictable: the President (and therefore not the Congress) is responsible for the coordination and direction of the executive branch. This view has grown more explicit in the past 25 years. Particularly since the Nixon presidency, it has been an aggressively pursued ideal.

There have been three basic components to the presidential control strategies that have emerged: structural change, governmental reorganization, and larger numbers of political appointees to direct the career bureaucracy (see Ingraham, 1987; Pfiffner, 1988). President Nixon essentially created the model for future presidents by combining all of these strategies into an overall vision of presidential management. The "administrative presidency" that he attempted to create was cut short by Watergate; the lessons from it, however, were quickly adopted by the presidents who followed. (For a complete discussion of the Nixon strategy, see Nathan, 1983.)

President Carter, for example, agreed with the intent of the administrative presidency—better management and coordination and greater accountability of the career bureaucracy to elected officials. Carter, in fact, used the Schedule C political appointment authority more heavily than had any president since its creation in 1956 (Ingraham, 1987). The Civil Service Reform Act of 1978 was one part of this larger strategy. Carter's primary interest was in improving the managerial and technical competence of the presidential office; Alan Campbell, Carter's director of the Office of Personnel Management, observed in a 10-year retrospective on the design of the Civil Service Reform Act that its structural changes were intended to work "… no matter who was in office" (Campbell, 1988). In retrospect, however, many observers feel that the emergence of the administrative presidency has politicized the bureaucracy and placed the ideal of a politically neutral and protected civil service under stress. This in turn has implications for compensation policy and the efficacy of performance appraisal.

THE CIVIL SERVICE REFORM ACT OF 1978

Civil service reform was central to President Carter's election campaign and he selected an adviser to spearhead the effort shortly after his announcement to seek the office. One of Carter's first acts as President was to create the President's Personnel Management Project (PMP) to assist him and his staff in the design of the promised reform.

The structure of the PMP was purposefully comprehensive: there were nine task forces, an assistant secretary's advisory group, several other more informal advisory groups, and a number of public hearings. From these activities, the

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

PMP produced a two-volume report of problem analysis and recommendations. The report contained well over 100 specific recommendations for reform; it was released in December 1977. From that report, members of the Inter-Agency Advisory Group drafted the legislation (for a complete discussion of the design, see Ingraham, 1989).

President Carter first introduced the broad outlines of the Civil Service Reform Act in his State of the Union message on January 19, 1978. At that time, he called the reforms "absolutely vital." It was the first time that a U.S. president had included civil service reform among his major legislative proposals. President Carter's ultimate objective, he said, was to create "… a government that is efficient, open and truly worthy … of understanding and respect."

Carter's reforms came in two parts. Reorganization Plan No. 2 preceded the actual reform legislation: it abolished the Civil Service Commission and replaced it with the Office of Personnel Management, the Merit Systems Protection Board (including the Office of Special Counsel), and the Federal Labor Relations Authority. The Office of Personnel Management would oversee the human resource management activities of the federal government. Those responsibilities would include implementation of the other reforms. The Merit Systems Protection Board would serve as guardian of the merit system and merit principles and as an appeals body for personnel actions brought by federal employees. During congressional consideration of Reorganization Plan No. 2, Carter administration officials argued that the board and the special counsel would protect the merit system from any abuse resulting from reform provisions regarding pay for performance, discipline, or the senior civil service (Vaughn, 1989).

The Civil Service Reform Act itself contained a number of provisions intended to improve the performance of the federal civil service. Major provisions included the creation of the Senior Executive Service, a rank-in-person system for top executives, performance appraisals for all employees, merit pay for middle managers, delegations of specified personnel management authorities to the line agencies, formalization of the federal labor management relations program, and modifications in procedures for dealing with poor performers.

The Senior Executive Service

The Senior Executive Service (SES) was conceived by the designers of the reform as the centerpiece of the Civil Service Reform Act. The Senior Executive Service was a multipurpose reform. Its members were to be the federal government's managerial elite. They were to participate in policy making activities as well as the management activities reserved for the traditional career civil service. The structure of the SES, and the removal of some civil service protections from its members, also ensured that the link between

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

political executives and senior career managers would be strengthened. The inclusion of some political appointees in the Senior Executive Service itself further emphasized the objective of political responsiveness of the reform. This emphasis on responsiveness to political direction can be read, in the context of pay for performance, as an effort to more closely link individual managerial activity to organizational objectives.

Performance appraisal and pay for performance were important parts of the concept of a Senior Executive Service. The act required the establishment of a performance appraisal system designed to permit the accurate evaluation of performance in any SES position. Performance criteria were to be position-specific and were to identify critical elements of the position. Performance appraisals were intended to encourage excellence in performance by senior executives. They were to provide a basis for performance awards and for promotions and other executive development opportunities, as well as for retention decisions.

SES performance appraisals were to be based on both individual and agency performance, and were to include such factors as improvements in efficiency, productivity, quality of service, cost efficiency, timeliness of performance, and the achievement of equal employment opportunity requirements. SES performance appraisals were required on an annual basis, with performance described according to one of several standard summary ratings. Final appraisals could be made only upon review by an agency-level Performance Review Board, which was required by the act.

The SES pay system included strong pay for performance elements. It did not provide for any type of annual pay increases, except the general "comparability" increases. Instead, incentives were offered in the form of awards and bonuses. Thus, the only way for an SES employee to move up in pay is to receive a change in rank to a higher level. The act created two levels of SES awards: Meritorious Executive and Distinguished Executive. Subject to the congressionally mandated limitations, the President would designate career appointees to either of these two ranks. A designation as a Meritorious Executive carried with it a cash award of $10,000; receipt of a Distinguished Executive award provided the recipient with a lump sum award of $20,000. A minimum of a fully successful rating (equivalent to a satisfactory rating) was required for nomination to one of the ranks.

To provide incentives for excellent performance, Congress also created a bonus system for SES incumbents. Fully satisfactory performance was established as a baseline for eligibility for bonuses. The number of bonuses awarded within any agency was limited to less than 50 percent of the SES positions allocated to the agency. The act further stipulated that individual awards could not exceed 20 percent of the career appointee's rate of basic pay.

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Performance Appraisal

The general logic of the SES performance appraisal provisions was applied to non-SES employees as well. But while the primary emphasis of the SES system appeared to be on linking individual performance to organizational objectives, the program for mid-level managers (GS 13–15 supervisors and management officials) emphasized the link between individual performance and pay. Under the performance appraisal provisions of the Civil Service Reform Act, each agency was required to develop performance appraisal systems that "(1) provide for periodic appraisals of job performance of employees; (2) encourage employee participation in establishing performance standards; and (3) use the results of performance appraisals as a basis for training, rewarding, reassigning, promoting, reducing in grade, retaining and removing employees." These systems were required to meet criteria prescribed in OPM regulations and were required to be implemented by October 1, 1981, three years after the act was passed. The designers of the act believed that this time lag would permit the SES reforms to become institutionalized before other pay for performance reforms were implemented.

The OPM regulations were intended to develop job-related and objective performance appraisal systems consistent with the dictates of the statute. The regulations required that performance standards and critical elements be consistent with the duties and responsibilities covered in an employee's position description. OPM guidance suggested that performance standards be based on a job analysis to identify critical elements of a position, and that each agency develop a method for evaluating its system to ensure its validity.

This identification of critical elements of a job was a key component of the performance appraisal reforms. A critical element was defined by OPM as "any requirement of the job which is sufficiently important that inadequate performance of it outweighs acceptable or better performance in other aspects of the job." Employees who failed to perform at a satisfactory level on a critical element were to be subject to performance-based actions, including dismissal if performance did not improve.

Merit Pay

The Civil Service Reform Act also created a new pay for performance system for middle managers, GS 13–15. The merit pay provisions represented a break from the long tradition of essentially automatic salary increases based on length of service. Borrowing from private-sector practices, Title V of the Civil Service Reform Act contained provisions intended to motivate mid-level managers to perform at higher levels by tying performance to financial incentives.

The Merit Pay System (MPS), which became mandatory on October 1,

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

1981, altered the pattern of annual incremental adjustments to salary. Under MPS, employees received only half of the comparability adjustment automatically. The nonautomatic portion of the comparability adjustment, plus the within-grade and quality step increase monies were pooled and distributed according to performance ratings (U.S. Office of Personnel Management, 1981). A crucial point is that the legislation provided that the Merit Pay System would be revenue neutral, so that if some employees benefited, others would of necessity be less well off than they would have been under the General Schedule.

Federal Employee Expectations About the Reform

It is difficult to identify the attitudes of federal employees toward performance appraisal, pay for performance, and other reform provisions prior to the Civil Service Reform Act because of the lack of baseline data. One survey, conducted by Lynn and Vaden, questioned a random sample of about 2,000 federal employees about their attitudes toward the reform in general. Lynn and Vaden (1979) reported fairly high levels of skepticism and distrust about the reforms, including frequent references to them as a "return to the spoils system."

The most comprehensive source of data about employee attitudes prior to reform is the Federal Employee Attitude Survey, Phase I (FEAS I), conducted by OPM. This survey, which preceded implementation of the act but followed its passage, yielded about 14,500 responses. A second survey, which used the same questionnaire, was administered to GS 13–15 employees at four Navy research and development laboratories. The Navy survey produced 2,068 valid responses. Data from these surveys are reproduced in Tables 2-1 and 2-2.

Based on the Federal Employee Attitude Survey, Nigro (1982) reported that employee responses revealed a widespread lack of satisfaction with the pre- 1978 performance appraisal system. Nigro argued that this created a favorable climate for the performance appraisal reforms and that a system that promoted the developmental aspects of performance appraisal stood a good chance of success. He found that employees considered performance appraisals consequential, but that there were problems with the critical link between performance and reward. He concluded that there was potentially large support for the merit pay provisions of the Civil Service Reform Act. Significantly, Nigro also noted that trust in the organization was relatively low and could create serious problems if implementation was conducted in a top-down fashion. From the same data, Bann and Johnson (1984:79) concluded that "… there was neither a wholesale rejection of the old system, nor unqualified support. … [T]hose most dissatisfied with the old system of performance appraisal … were also unhappy with their jobs and with the organization in general. … Those who were relatively content with the old performance appraisal system also placed

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

more trust in the organization and demonstrated greater satisfaction with their jobs."

It is also important to note, though with less empirical foundation, that the political rhetoric surrounding the Civil Service Reform Act influenced expectations and created both positive and negative perspectives on its likely outcomes. The positive expectations are reflected in the objectives for performance appraisal and merit pay contained in OPM's evaluation plan:

Performance Appraisal

Short-Term Objectives:

  1. Increase employees' understanding of performance standards.

  2. Ensure effective appraisal of performance.

  3. Ensure equitable appraisal of performance.

  4. Link performance to personnel actions through the performance appraisal process.

Long-Term Objectives:

  1. Increase the effectiveness of employees and supervisors.

  2. Improve the quality of federal working life.

  3. Contribute to agency productivity.

Merit Pay

Short-Term Objectives:

  1. Relate pay to performance.

  2. Provide flexibility in recognizing and rewarding good performance with cash awards.

Long-Term Objectives:

  1. Motivate merit pay employees by making pay increases contingent on performance; clarifying job expectations, i.e., defining goals and objectives, increasing competition for recognition and rewards.

  2. Improve the productivity, timeliness, and quality of work in the federal government through better management and more effective programs.

The negative expectations resulted from the punitive tone—the "bureaucrat bashing," as it came to be known—that accompanied descriptions of the need for reform. New whistleblower protections were said to be necessary to ferret out waste and fraud; greater managerial flexibilities were needed to eliminate deadwood; performance appraisal and pay for performance were necessary because federal employees were not productive and did not measure up to their private-sector counterparts (see Ingraham and Barrilleaux, 1983). This, coupled with the characterization of the federal bureaucracy as the "giant Washington

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

TABLE 2-1 Opinions of Federal Workers Concerning Established Performance Appraisal Systems and Pay-Performance Linkages (1979)

 

Response (%)

 

OPM Survey Respondents

Navy R&D Laboratories Respondents

Item

D

DK

A

D

DK

A

Performance appraisals do influence personnel actions taken in this organization.

22

17

62

28

24

48

This organization considers performance appraisal to be an important part of a supervisor's duties.

18

21

61

25

29

45

My job performance is carefully evaluated by my supervisor.

22

19

59

23

23

53

The standards used to evaluate my performance have been fair and objective.

17

26

57

15

32

52

My performance rating presents a fair and accurate picture of my actual job performance.

26

23

51

30

27

44

In the past I have been aware of what standards have been used to evaluate my performance.

23

16

61

27

20

54

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Did your last performance appraisal help you to:

NH

SH

H

NH

SH

H

a. Assess you strengths and weaknesses?

44

33

23

47

33

19

b. Establish a plan for training and development?

60

25

15

60

27

13

c. Determine your contribution to the organization?

42

30

27

47

29

24

d. Improve your performance?

46

31

23

51

31

18

 

NI

SI

I

NI

SI

I

How important is the quality of your performance in determining your pay?*

15

23

62

16

29

53

How important should the quality of your job performance be in determining your pay?*

0

4

95

0

3

97

* GS 13+ only.

D = Disagree; DK = Don't Know; A = Agree; NH = Not Helpful; SH = Somewhat Helpful; H = Helpful; NI = Not Important; SI = Somewhat Important; I = Important

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

TABLE 2-2 Federal Employee Trust and Confidence in Their Organizations, Supervisors, and Coworkers

 

Response (%)

 

OPM Survey Respondents

Navy R&D Laboratories Respondents

Item

D

DK

A

D

DK

A

When changes are made in this organization, the employees usually lose out in the end.

46

19

36

38

25

38

Employees here feel you can't trust this organization.

41

16

42

36

21

44

My supervisor deals with subordinates well.

24

15

61

23

16

62

I have confidence and trust in my coworkers.

10

9

82

7

8

85

D = Disagree; DK = Don't Know; A = Agree

Due to rounding, row percentages may not add up to 100 percent.

marshmallow'' during the presidential campaign created a negative aura around the reforms for many federal employees.

Finally, it is significant that employee expectations about the reform were strongly influenced by the federal pay situation. For senior career managers, the link of federal employees' pay to that of members of Congress created a situation in which they had "topped out," that is, reached the top statutory pay level. Many career executives had been at this level for several years prior to the passage of the act. Absent fundamental pay reform, the pay for performance provisions in the SES were the only means available for escaping the pay cap. In this regard, both the stakes and the expectations were very high.

The Record

The record of the Civil Service Reform Act has been turbulent. The orderly implementation of the act envisioned by the Carter administration was interrupted by the election of Ronald Reagan in 1980. President Reagan was not a supporter of the civil service; cutting back the size and cost of government was high on the Reagan policy agenda. OPM's human resource function was redefined; most planning, evaluation, and research activities were eliminated; the organization was downsized and restructured. Because political control of key components of executive branch agencies was considered critical to policy success, a specifically political role emerged for OPM.

Donald Devine, the director of OPM for the first Reagan term, explicitly

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

espoused the Weberian view of organizations; under his direction, OPM emerged as a political management arm of the White House, rather than an agency concerned with broader human resource management issues (Newland, 1983). The organization was not so overtly political in the second Reagan term, and serious efforts were made to address some of the most pressing federal personnel and management problems. Nonetheless, many of the reforms created by the Civil Service Reform Act had been deferred, eliminated, or redefined. Many observers have noted that the reforms were simply overwhelmed by the dramatically changed political environment in which federal agencies existed in the 1980s.

Pay for performance and performance appraisal were also affected by the turbulence of implementation. The experience of the Senior Executive Service is notable in a number of respects. Because it was the first to be implemented, the SES performance appraisal and bonus system was carefully watched by most federal employees. It did not serve as a positive model.

The first SES payouts occurred in the year following passage of the reform. The first agency to complete the process paid out the full amount allowable under the law; not only was the number who received bonuses considered excessive in the view of Congress and some other external observers, the proportion of Performance Review Board members who themselves received a bonus was much too high. As a result, six months into the implementation of the SES system, Congress altered the provisions of the act. Under the new provisions, the percentage of SES positions in the agency eligible for a bonus was reduced from 50 to 25 percent. OPM, using its rulemaking authority in an effort to demonstrate its good faith to Congress, further lowered that percentage to 20 percent of the total approved positions.

This dramatic change in the SES pay for performance system had an immediate and negative impact. Members of the SES, who had viewed the bonus system as an escape from the federal pay cap, were disillusioned with the new system. The formation of the Senior Executive Association to lobby Congress for the interests of the SES was one indicator of the disenchantment and dissatisfaction with the reform very early in the implementation process.

The Merit Pay System

If pay for performance was less than triumphant in the Senior Executive Service, how successful was the Merit Pay System (MPS) in rejuvenating the mid-level managerial work force? Its clearest shortcoming was its failure to establish a demonstrable relationship between pay and performance. This failure is attributable to a variety of causes. One of the chief ones was a lack of adequate funding for merit pay. Agencies were required by law to spend no more on the Merit Pay System than they had under the previous General Schedule system. This problem was exacerbated by implementation

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

difficulties. For example, a dispute between OPM and the General Accounting Office concerning the permissible size of payout led, in September 1981 (one month before payout), to a determination that the OPM formula for calculating the merit pay fund was not in conformance with the statute. The ruling resulted in a modified payout that provided only small differentials among the mid-level managers covered, again undercutting pay for performance principles and diminishing the incentives for supervisors to differentiate among employees.

Because the Merit Pay System was not perceived as fair in some fundamental ways, it failed to establish credible links between pay and performance. Managers who performed satisfactorily often found themselves receiving lesser rewards than their nonmanagerial counterparts at grades 13-15, whose pay was set under the General Schedule. The perceptions of employees that nonperformance factors (e.g., the composition of the pay pool) affected payout and that ratings were arbitrarily modified also diminished the effectiveness of the pay for performance aspects of the system. Employees in most agencies perceived no greater likelihood that their performance would be recognized with a cash award after the establishment of the Merit Pay System than had previously been the case (U.S. General Accounting Office, 1984).

The reported successes of the Merit Pay System in motivating employees emanated primarily from the performance appraisal requirements of the Civil Service Reform Act. Gaertner and Gaertner (1984) reported that developmental appraisals—those that focused on planning for the coming year and clarifying expectations—were more effective than appraisals that focused only on past performance. However, developmental appraisal strategies were seldom used, and the pay administration role for appraisals tended to undermine this function. In fact, one study reported a significant drop in the organizational commitment of employees who received satisfactory, but not outstanding, ratings (Pearce and Porter, 1986).

The Performance Management and Recognition System

Although the Merit Pay System did not take effect for most federal managers until 1981, it very quickly became apparent that it performed poorly when judged by the objectives established for it. Relief was sought in legislation, introduced in 1984, that proposed the Performance Management and Recognition System (PMRS). PMRS was enacted on November 8, 1984, but the first payout was made retroactive to the fiscal 1984 performance cycle. Retroactive application created a number of short-term implementation problems (U.S. General Accounting Office, 1987).

The drafters of the legislation sought to retain pay for performance principles but to eliminate the dysfunctions of the original system. Under PMRS, employees are rated at one of five summary rating levels: two levels below fully successful, fully successful, and two levels above fully successful. The system

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

has three monetary components: (1) employees who are rated fully successful or better are assured of receiving the full general pay or comparability increase. (2) They are also eligible for merit increases, which are equivalent to within-grade increases. The size of the merit increase depends on an employee's position in the pay range and performance rating. (3) In addition to these monies, employees rated fully successful or above also qualify for performance awards or bonuses. Beginning in fiscal 1986, performance awards of no less than 2 percent and no more than 10 percent became mandatory for employees rated two levels above fully successful. Moreover, an agency may give a performance award of up to 20 percent of base salary for unusually outstanding performance. An upper limit of 1.5 percent-of-payroll for all performance awards was placed on agency payout under the system.

PMRS also created Performance Standards Review Boards, modeled after the Performance Review Boards in the Senior Executive Service, to review performance standards within an agency to ensure their validity and to perform other oversight functions. At least half of each board is required to be made up of employees eligible for merit pay. Although the number and functioning of these boards was left to agency discretion, they are required to report annually to the agency head.

Although the evidence is thin, there are some indications that PMRS has functioned better than the Merit Pay System. The Merit Systems Protection Board (MSPB) conducted surveys of employee attitudes at three-year intervals beginning in 1983. The report of the most recent survey (Merit Systems Protection Board, 1990) says that, in 1986 and 1989, 32 and 36 percent, respectively, of the federal employees surveyed believed they would receive more pay for performing better. This represents a substantial increase over the 17 percent of employees surveyed in 1983 who perceived a link between pay and performance and provides an interesting comparison to the Wyatt Company's 1989 report on employee attitudes in private-sector firms that about 28 percent of those surveyed saw a link between their pay and their job performance.

It nevertheless remains true that the conceptual support of pay for performance remains far stronger among federal employees—the report of the 1989 MSPB survey says that 72 percent of respondents endorse the proposition—than their support of existing pay for performance systems. Only 42 percent indicated that they would choose to be under a pay for performance system if given the choice; about the same proportion of respondents indicated that they would not so choose, many of them citing the shortcomings of the present system as the grounds for their disinclination. The most commonly registered reservations involved (1) the ability and freedom of managers to make meaningful distinctions among levels of performance and (2) the availability of enough money to reward the best performers. The monetary concern coincides with a more general dissatisfaction with pay expressed by 60 percent of respondents to the 1989 MSPB survey.

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

It is not clear that PMRS has provided the hoped-for motivational stimuli. It is unlikely that pay for performance devices such as merit increases, bonuses, and awards would produce performance effects in the context of a deep, generalized dissatisfaction with pay levels of the kind reported in each of the three MSPB surveys. In addition, even though most merit system employees have received performance awards (U.S. Office of Personnel Management, 1989), the General Accounting Office found that 50 percent of the employees surveyed in the first year of PMRS felt the size of the awards was inadequate. Insofar as performance may be affected by the communication of performance standards, the Performance Management and Recognition System appears to be functioning well. Nine out of ten respondents to the 1989 survey said that they understand the performance standards for their jobs.

A somewhat more negative picture of PMRS emerges from informal surveys of their membership conducted recently by two federal managers' associations. Most of the managers responding to the surveys indicated support for the concept of basing pay on performance. Only 3 percent, however, felt that PMRS should be maintained in its current form and approximately 40 percent said that PMRS should be completely abolished. More than 75 percent of the managers indicated that they believed that their ratings were influenced by officials above their supervisors, that their performance evaluations were of little guidance for development purposes, and that insufficient funds have resulted in meaningless performance awards. Given that the current system is viewed as so unfair and ineffective, there is a concern over whether any new pay for performance system could function effectively.

The evaluations of PMRS to date have been silent with respect to the influence of PMRS on agency effectiveness. The Merit Systems Protection Board has identified a tentative relationship between turnover and performance ratings that suggests that poor performers are more likely than good performers to leave federal service (Merit Systems Protection Board, 1988). However, no such relationship was found between turnover and performance ratings in an earlier study by the General Services Administration (Perry and Petrakis, 1987).

IMPLICATIONS

This brief account of civil service reform is a record of modest changes and frequently conflicting objectives, accompanied perhaps by unrealistic expectations about the effects of the reforms on the performance and productivity of federal personnel. Neither the Merit Pay System nor the Performance Management and Recognition System has been able to counteract what, since at least the early 1980s, has come to be called the "quiet crisis" in the federal government. That crisis, according to the National Commission on the Public Service and others, is marked by below-market public-sector salaries, an inability to recruit new employees for many federal occupations, an inability to retain

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

seasoned federal managers, and a perceived decline in the overall quality of the federal work force (National Commission on the Public Service, 1990).

The uncompetitiveness of the Civil Service is particularly noticeable in certain fields, for example, law and the scientific and engineering professions. A recent National Research Council report noted that recruitment of scientific and engineering personnel was a problem for the National Institutes of Health, the Environmental Protection Agency, the National Institute of Standards and Technology, the Department of Health and Human Services, the Social Security Administration, and the National Science Foundation, among many others (National Research Council, 1990). However, the overall problem of recruiting and retaining a well-qualified work force is being felt throughout the federal government.

While there is no reason to believe that the present malaise cannot be reversed, there are important tensions between the potential benefits of pay for performance and the reality of the federal personnel and compensation systems. We describe these tensions below.

  1. The tension between the principle of neutral competence and pay for performance. We have described the centrality of the principle of neutral competence to the modern civil service. In turning away from the spoils system, the founders of the merit system in the late nineteenth century envisioned federal employees as dispassionate servants to the body politic who, to function properly, needed to be shielded from invidious political influences. Many of the most characteristic elements of the merit system—entry by competitive examination, retention rights, limitations on partisan activities—derive from this vision of neutral competence. Efforts to ensure that political neutrality could be maintained for the career service, however, have created an extremely complex system of constraints that have come to place severe limits on the discretion of career managers, in addition to controlling partisanship. Two outcomes of a merit system built on the concept of neutral competence are directly related to the potential success of pay for performance. First, the managerial constraints and legalistic environment that have come to characterize federal management are antithetical to the managerial discretion necessary for effective pay for performance processes (National Academy of Public Administration, 1983). Second, merit pay carries far more meaning in the context of the civil service than in the private sector. The objective of any merit pay system is to relate pay to individual or group contributions to organizational purposes. But in the public sector, the possible impact of political influence on ratings of individual performance will inevitably be of concern. Moreover, the definition of organizational purpose will always be complicated in the public sector because of the frequent turnover of the political leadership. These considerations at the very least raise questions about the transferability of private-sector practice.

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

Perhaps the fundamental problem is with the concept of neutral competence itself. It has not been articulated in a way that distinguishes sufficiently between politics and policy. The elaboration of merit protections during the twentieth century has tended to make the bureaucracy unresponsive to presidential leadership. Hence the recent spiral of presidential efforts to better manage and control executive branch employees, which has raised the flag of partisan intrusion to new levels.

Two conditions appear to be necessary if pay for performance is to be compatible with a merit system based on the ideal of neutral competence. First, a fundamental rethinking of the concept of neutral competence is needed that will offer a more appropriate balance between merit protections and the effective implementation of the administration's policy. Second, a much broader view and acceptance of career managerial discretion is critical. Managers must have the authority and the support to manage employees effectively if the necessary conditions for pay for performance are to be present.

  1. The tension between the promise of pay for performance and the reality of the federal record. As the following chapters demonstrate, there is an association between levels of organizational trust and shared values on one hand and perceptions of the fairness and effectiveness of pay for performance systems on the other. Of course, in both the public and private sectors, there will always be some dissatisfaction with merit pay because not everybody gains from such a system. Nonetheless, in the federal government, the absence of organizational trust and shared values and objectives may be an obstacle to effective pay for performance. Expectations have been high for previous reforms; in most cases, the reality has not approached the expectations. At the same time, increased efforts at political control and rhetoric that has devalued the public service have created high levels of dissatisfaction and demoralization among members of the career civil service. In such a setting, common goals and objectives, consensus, and trust may be difficult to achieve.

  2. The tension between inadequate resources and pay for performance. If pay for performance is to contribute to perceptions of equity in compensation systems, the base from which it builds should be perceived as equitable and fair. At the present time, there is nearly unanimous agreement that federal pay is not competitive with private-sector pay in many regions. Every major survey of federal employees in the last 10 years has documented dissatisfaction with pay. Simply put, at the present time, base federal pay levels are not perceived to be equitable.

Moreover, revenue-neutral provisions for merit pay programs have resulted in small or modest bonuses. The General Accounting Office and the Merit Systems Protection Board report that employees do not perceive the link between performance and reward to be strong—or even present in some cases (U.S. General Accounting Office, 1988; Merit Systems Protection Board, 1990). In

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×

this context, the utility of pay for performance plans in contributing to equitable compensation systems appears to be very limited.

Our purpose in this chapter was to provide a general flavor of the complexities of the federal sector and to introduce some of the more salient issues to be considered as policy makers turn to the redesign of the merit pay system in the federal government. We turn now to an examination of the scientific and clinical evidence on performance appraisal and pay for performance and an assessment of the implications of this evidence for a merit pay system for federal managers.

Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 13
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 14
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 15
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 16
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 17
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 18
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 19
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 20
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 21
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 22
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 23
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 24
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 25
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 26
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 27
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 28
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 29
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 30
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 31
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 32
Suggested Citation:"2 The History of Civil Service Reform." National Research Council. 1991. Pay for Performance: Evaluating Performance Appraisal and Merit Pay. Washington, DC: The National Academies Press. doi: 10.17226/1751.
×
Page 33
Next: 3 The Nature of the Evidence »
Pay for Performance: Evaluating Performance Appraisal and Merit Pay Get This Book
×
Buy Paperback | $44.95
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

"Pay for performance" has become a buzzword for the 1990s, as U.S. organizations seek ways to boost employee productivity. The new emphasis on performance appraisal and merit pay calls for a thorough examination of their effectiveness. Pay for Performance is the best resource to date on the issues of whether these concepts work and how they can be applied most effectively in the workplace.

This important book looks at performance appraisal and pay practices in the private sector and describes whether—and how—private industry experience is relevant to federal pay reform. It focuses on the needs of the federal government, exploring how the federal pay system evolved; available evidence on federal employee attitudes toward their work, their pay, and their reputation with the public; and the complicating and pervasive factor of politics.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!