To develop, implement, and improve effective interventions, it is critical to have accurate information about the needs of the target populations. The November 2011 workshop included a session that focused on the various tools and considerations for measuring stress, employees’ experience of work, and individual and organizational measures of resilience.
Bengt Arnetz is a professor in the Division of Occupational and Environmental Health in the Department of Family Medicine and Public Health Sciences at the Wayne State University School of Medicine. His department works with operational personnel including first responders, as well as private-industry employees. Arnetz discussed his research in measuring individual and organizational stress, and how stress can be linked to organization performance.
Nancy Rothbard is the David Pottruck Associate Professor of Management at the Wharton School, University of Pennsylvania. Rothbard discussed the various measures used to understand employees’ motivation and engagement in their work and how they influence the work experience.
Dennis Reber is the managing director of Global Learning and Development at FedEx Express and is responsible for leadership development programs, succession planning, talent management, change management, and the annual climate survey. Reber discussed the lessons learned from FedEx’s more than 30 years of ongoing assessment. Several themes emerged from the presentations and panel discussion (see Box 7-1).
Themes from Individual Speakers on Performance Measures
- Role of evidence and performance measurement in developing and improving interventions
- Variety of measures available to assess resilience and different aspects of the employees’ experience of work
- Participant engagement is a component of effective assessments
- Performance measurement is possible at multiple levels, including individual, group/team, and agency/organization
- Performance measurement is an effective aspect of program design when it is:
- Driven by the leadership
- Focused on key issues of interest
- Provides real benefits to all participants
- Actively used to improve processes
Dr. Arnetz’s research was partially supported by the National Institute of Mental Health (NIMH) and the National Institutes of Health (NIH).1
Stress and the Resilience Model
Arnetz and his colleagues have developed a four-level model to look at the different aspects of stress and resilience: (1) the individual, (2) the group or team, (3) the agency or organization, and (4) the society. Using this model as a base, the next key question was to consider how best to measure resilience at each of these levels. Additionally, what are the benefits of resilience within each of these components, and how do they relate to each other?
At the individual level, it is important to understand that everyone faces challenges in their lives and that not all stress is bad. Challenges
1The content of Arnetz’s presentation is solely his responsibility and does not necessarily represent the official views of the NIMH or the NIH. Furthermore, Dr. Arnetz’s research was, in part, supported by a research award from the Swedish Royal Foundation (Kungafonden), which exclusively focuses on research and services aimed at enhancing health and well-being among first responders.
are positive when the individual has some control over the situation, their skills are used, and they have a positive expectation about the outcome. The experience becomes negative when the individual feels he or she has little or no control and is unable to predict the outcome, and the experience is perceived as a threat.
These challenges and stressors generally activate the normal human stress response, which includes both emotional and physical reactions. In dealing with challenges and stressors, people usually go to an elevated psychophysiological level where they sleep less and are more alert and operationally ready. This response helps them prepare for the task or to address the perceived threat. Once the task is complete or the threat has been addressed, individuals go back to a lower level in order to recover. The normal human stress response is designed to address short-term and immediate stressors and is not well designed to deal with long-term stressors. Increasingly, people maintain the elevated psychophysiological level in response to chronic stressors that in the long term lead to fatigue. At some point people become too fatigued and “hit the wall.” DHS has real issues to address, and the question becomes how to increase employees’ sense of control and predictability, as well as how to increase the use of their skills and positive expectancy.
Measuring Resilience and Program Performance
Within his work, Arnetz has used a variety of measures to assess resilience and the effects of various interventions. Measures such as psychophysiological exhaustion, fatigue, vital exhaustion, biological resilience, mental energy, and mental well-being can provide insight on a number of factors that influence resilience and possible interventions.
Recovering and “Hitting the Wall”
Arnetz and his colleagues have been looking at the effects of long-term chronic stressors on health and well-being as well as possible interventions to mitigate the adverse effects of such. This work includes an intervention with individuals in management and the effects of that intervention on performance and well-being in the private-sector corporation Ericsson. Ericsson is a large international telecommunications company. Because of an organizational restructuring, the corporation experienced a significant reduction in the workforce, dropping to 60,000 from 120,000 employees. The company’s welfare was at risk owing to the competitive
market, and there was significant pressure to develop new products. The study looked at one of the major research and development units. This unit was interesting because its members were white-collar workers of which approximately 60 percent had a Ph.D. The reality was the corporation was in jeopardy. The question became how to reframe the employees’ perception of Ericsson’s situation from a threat to a challenge, so their reaction to it was not negative but positive. As a result of such a shift, employees could possibly be more cognitively and psychologically efficient, as well as more flexible and productive (Arnetz, 1996).
Another example of sustained stress and its impact on health and performance is a patient enrolled at Dr. Arnetz’s stress center. She is a professional with cognitively demanding job functions. The patient tracked herself using a free web-based stress and performance system. The system tracks individual and organizational determinants of sustained health and performance. As the person was tracking herself with the system, she “hit the wall” and was not able to keep up her typical high-performance levels. An intervention was delivered to help her cope with the organizational restructuring and concomitant work-life challenges. After the intervention there were improvements across all the measures. However, the measures never returned to their original levels. This finding indicates that although people can recover from high levels of psychophysiological fatigue, it is difficult, and when they do, they may not regain past levels of performance. This potential long-term loss in performance can have significant effects on the organization and emphasizes the importance of early interventions (Arnetz et al., 2011).
Another study looked at the effectiveness of a comprehensive recovery program for individuals that had already burned out in their job. The intervention included a broad range of services such as stress management, nutrition, exercise, and sleep. The study used a measure of self-reported mental energy as an indication of resilience. After the intervention participants’ self-rated mental energy level was back in the desirable range.
Biological resilience is the ability to recover after an initial stressor. Measures such as stress levels, anabolic hormones, sleep, and the ability to concentrate were used to evaluate the biological resilience of media and high-performance information technology (IT) personnel. The study design included a control group. The study found that participants’ biological
resilience increased in relation to a targeted personalized intervention. The control group’s levels remained stagnate. Six months into the study, the control group was also given access to the intervention, and its resilience increased as well. However, the positive effects of the intervention were not sustained over the entire 12-month study period, suggesting a need for additional interventions (Hasson et al., 2005; Schell et al., 2008).
Types of Stressors
The Royal Foundation of Sweden’s (Kungafonden) mission is to research and assist first responders and their families and includes all the top first-responders agencies in Sweden. The foundation funded a study looking at the effects of low-impact stress on first responders. The study includes a series of focus groups with different operational first responders. The study found that threats to the first responders’ physical, psychological, and occupational security were the primary concerns in these populations (see Figure 7-1). Interestingly, across the various first-responder agencies, personnel responded that changes in their work and profession were key areas of concern. These concerns focused on shifts in how performance was measured, expansion of responsibilities, and requests to take on tasks they were not trained to perform (Ventimiglia et al., 2011).
Training for Stress
An intervention with police cadets that provided psychological training in dealing with crisis events such as high-speed car chases or a domestic violence cases found both short-term and long-term health effects. On top of the normal basic training, the study participants received psychological training similar to that used in military special forces. The training included emotional control and regulating stress levels as well as imagery-based performance enhancement training. In the short-term the study participants had higher levels of self-reported well-being than the controls. After the cadets entered the police force, the study followed them for 2 years. After 2 years the cadets reported higher levels of mental well-being than the control group, as well as less physiological stress
FIGURE 7-1 Relationships between stressors and resilience in first responders.
SOURCE: Nevedal et al., 2009.
and stress-related adverse behavioral effects, including measures of sleep (Arnetz et al, 2009; Backman et al., 1997).
Performance as a Measure of Resilience
For organizations the primary resilience outcome of interest is performance. A study that assessed the use of mental imaging training and the reenactment of crisis situations found that officers who participated in the training had a lower heart rate than the control. The officers’ performance was also evaluated by police experts in the reenactments. The trained group performed 40 percent better than the nontrained group. However, it should be noted that the highest performing group performed at 50 percent of the highest possible levels, indicating that in very stressful situations only half of the maximum performance was achievable.
A study of an inner-city health care center found that fatigue directly relates to more physical symptoms and lower self-rated health. Socioeconomic
factors such as living in a high-stress environment influenced fatigue, and personality factors interrelate to affect how individuals cope with the environment (Arnetz et al., 2009; Maghout-Juratli et al., 2010).
Organizational Efficiency’s Relationship with Resilience
A study looking at performance in positions that demand high cognitive ability found that organizational efficiency is very important to resilience. Low efficiency means one has to work harder to achieve the same results. The study developed systems to trace individuals over time and looked at determinants of exhaustion, as well as what can be done to decrease exhaustion. The factors that decrease exhaustion tended to link back to management.
For the past 15 years, Arnetz’s team has been following a hospital as it experiences significant change. Initially the hospital was well positioned with approximately 20 percent more resources than average. The perceived workload among physicians and nurses was between 60 and 70 percent, and other personnel’s perceived workload was less than 60 percent. Then there was a 20 percent reduction of the staff, and the perceived workload increased to more than 80 percent for physicians, to about 75 percent for certified nurses, and to almost 70 percent for other staff members. During this same time frame (Arnetz et al., 2011) measured mental energy, which is another measure of resilience. Arnetz noted that in healthy populations mental energy scores are usually 70 percent or higher. Although there was a decrease in mental energy scores for all the staffing types, all groups except the physicians scored between 72 and 74 percent. The physicians dropped below 68 percent. The highest-skilled group lost the most mental energy, but all groups lost some. Multiply this loss of mental energy by the 5,000 people employed by the hospital, and the implications for productivity are significant.
While researchers might say that the way to address these issues is to not create stress, in the real world that is not generally realistic. Different units within the hospital responded differently to the cutbacks with various outcomes. Arnetz and his collaborators looked at the characteristics of the units to see what effect they had on the unit’s performance— whether it improved or worsened.
The key factors for sustained health and high organizational performance were leadership, feedback on performance, participatory management, and the work climate. Leadership was an important determinant in how employees rated their skills development. Units that felt their
skills were being used and developed tended to cope well and had higher resilience. Employees who reported improved performance feedback rated leadership higher and were more resilient. Where management involved the employees in the decision-making process, employees reported improved skills development.2
Based on several studies, Arnetz asserts that in terms of resilience the most important determinant is organizational efficiency. Efficiency is determined by work climate, participatory management, performance feedback, and leadership. Improved efficiency is underestimated as an intervention to enhance resilience in an organization. Improved organizational well-being can also improve biological markers of resilience overtime.
In conclusion, Arnetz suggested measures for DHS to consider at both the individual and organization levels. At an individual level, he suggested self-reported measures of mental energy, concentration ability, self-rated health, control, and positive expectancy. At the organizational level, he advised assessing leadership, performance feedback and goals, skills utilization and development, organizational efficiency, and work climate.
Dr. Nancy Rothbard noted that there are a multitude of measures that look at the employee’s experience of work. It is important to understand what the measures actually tell us and how they relate to each other.
The construct of performance is a multifaceted one. Also, organizations may be interested in either objective performance criteria such as productivity and efficiency, or in subjective criteria such as the quality of an employee’s work. Often, measures of performance include an overall evaluation of an employee by a supervisor along with multiple subdimensions of evaluation. Typically these types of measures of performance (e.g., productivity, efficiency, quality, and supervisor evaluations of whether the employee meets or exceeds job expectations) are referred to as in-role performance. Some measures of performance also look at the additional work people do that is not part of their job per se
2Also see Dunn, et al., 2007. Meeting the imperative to improve physician well-being: Assessment of an innovative program. Journal of General Internal Medicine 22:1544-1552.
but that helps support the organization, and this is referred to as extra-role performance.
Satisfaction is an important measure of the employee experience and captures an individual’s attitude toward his or her job. Satisfaction is related to a number of other employee experience attitudes. An employee’s commitment to the organization, its mission, and his or her coworkers is an important representation of the employee’s loyalty and attachment to the organization. Likewise, turnover is an important aspect of employee experience. Does the employee intend to stay in the position or quit? Additionally, there are employee experience measures that look at the employees’ health and well-being.
Choosing which aspect of the employee experience to focus on depends on the organizational problem that needs to be solved. If a company has a problem related to stress and burnout and the need to hire and train new people is great, the focus may be on what can be done about turnover. The ability to be hyper-vigilant and to hone in on the tasks at hand is particularly important within DHS. In this situation, measures of in-role performance may be critical.
Before looking at what measures to use, it is necessary to identify the problem that needs to be solved. Although everything is important, some aspects of the job outweigh others. For instance, DHS may be concerned about burnout but feels that vigilance is the more important issue for the mission. Rothbard focused her presentation on discussing two types of measures she feels might be the most useful to DHS. These measures look at the experiences of burnout and engagement.
Based on the previous presentations, Rothbard noted that burnout appears to be a concern for DHS. There is long-standing, well-established research on burnout that focuses on the more negative aspects of employees’ experiences. This research, which comes out of the stress and coping literature, is also focused on preventing burnout. The idea is that when people experience burnout, there is psychological depletion, which can have other long-term implications including mental health concerns. The problem with burnout is that it can also affect an employee’s current quality of work because of fatigue and the inability to focus on the work at hand and do it well. The classic conceptualization of burnout from the Maslach Burnout Inventory includes three components: emotional exhaustion, depersonalization, and personal accomplishment. Emotional exhaustion is linked to fatigue but is not just about physical
fatigue. It also includes emotional fatigue, which goes beyond just the physiological experience.
Depersonalization is related to responses to stressful circumstances where people try to keep matters at arm’s length such that the circumstances are not as personal and do not matter as much to the focal individual. As a result, people engaging in depersonalization can become callous and distant. Depending on the situation, this can be an appropriate coping response. If the person becomes too depersonalized, however, it can lead to negative outcomes in terms of the ability to care and to continue to bring energy and focus to the task.
The other side of the burnout construct is the notion of personal accomplishment. However, the research on this component of burnout has not had as much validity and reliability. Because the results are mixed, Rothbard focused on the elements with the stronger predictive validity— emotional exhaustion and depersonalization.
Measures of Burnout
Rothbard cited the Maslach Burnout Inventory, which includes multiple questions for each of the three dimensions mentioned above (Maslach et al., 1996). The questions for emotional exhaustion include asking whether the person is feeling drained by work, fatigued, and at the end of his or her rope. The depersonalization scale asks whether the person treats some people as if they were impersonal objects, whether the job is emotionally hardening to the person, and whether the person cares about work or people anymore. The questions for personal accomplishment are a mixture of different issues.
The concept of employee engagement, in contrast to burnout, originated out of positive organizational psychology and looks prospectively at how people are focused, vigilant, and involved in their work. When people are engaged, work can be an enriching experience. There is a great deal of debate about the definition of engagement, however. Rothbard defines engagement as people’s psychological presence in the role. Are they mentally there when they are supposed to be? There are different aspects of engagement:
- Attention: Are you cognitively focused on the task at hand?
- Absorption: Are you completely absorbed and engrossed when working, or are you distracted?
- Energy: How much energy and vigor do you bring to the job?
Measures of Engagement
Various scales for measuring engagement are available, a number of them being practitioner measures of engagement. The Gallup Q12 is a popular practitioner measure of engagement. Its questions are proprietary so it is not possible to share them exactly; however, it is possible to discuss them in general. Some of the questions ask about overall satisfaction with the company, knowing what is expected, having the resources needed to do the job, having the opportunity to do what the person does best, and receiving recognition or praise for work. The questions also ask about the whether the person feels that people at work care about them, encourage their development, take their opinions into account, and talk to them about their progress. Questions also address the socio-emotional side, whether the person feels that coworkers are also committed to doing quality work and whether the person has a best friend in the work environment. Questions also address the values of the individual—whether there is a feeling of doing something important and whether there are opportunities for learning and growth (Harter et al., 2009).
The Gallup Q12 groups together a number of different dimensions of work experience. This is because the primary focus is predictive validity. By combining these items Gallup finds a substantial positive correlation between this measure and performance. This can be useful when deciding which things about the workplace—climate, leadership, or context—to consider changing. The grouped items can also help identify some of the factors that predict engagement.
The Gallup Q12 items tap into important aspects of the workplace by measuring inputs to the employee’s experience, or, potentially, outputs of the employee experience, but the items do not really measure what the employees are experiencing themselves. From a diagnosis and research standpoint, Rothbard is concerned that the Gallup Q12 confounds antecedents and outcomes. It does not accurately determine which of these drive engagement and good performance, and it is not possible to tease apart what aspects of the measure are driving the relationship they have found.
There are several research-based measures of engagement in the literature. Rothbard presented two, the Utrecht Work Engagement Scale and the Rothbard & Patil Engagement Scale (Rothbard, 2001; Schaufeli and Bakker, 2003). The two scales are similar and include both an absorption and an energy component. Utrecht uses the term “vigor,” and Rothbard & Patil use the term “energy” to capture this concept (Crawford et al., 2010).
The difference between the scales is that Utrecht looks at “dedication,” which includes items such as “I find that the work is full of meaning and purpose,” and “I am enthusiastic about my job.” One of Rothbard’s concerns with the dedication subcomponent is that it confounds emotion and cognition. Both are important, but emotion affects cognition in interesting ways, and she chose to use separate scales to look at cognition and emotion in her work. Another difference between the scales is that Rothbard and Patil focused more explicitly on attention, which is defined as the duration of focus or degree of the vigilant attention that people use to focus on their work.
Relationship Between Burnout and Engagement
How do these concepts relate to each other? Burnout and engagement have a negative correlation of about 0.45, which means they are negatively related to each other but they are not on opposite ends of the continuum. They are tapping into slightly different things. The meta-analytic research shows that engagement is positively related to all of the outcome measures except for turnover intentions. The depersonalization and emotional exhaustion components of burnout are negatively related to pretty much all of these outcomes and are positively related to turnover intentions.
When Rothbard reverse-coded depersonalization and emotional exhaustion to compare the effect sizes, she made some interesting observations. She found that, compared to burnout, engagement is a much stronger predictor of in-role performance, which looks at how you are doing in your job, and of extra-role performance, which looks at how you do in aspects of your job that are not in your job description. By contrast, the burnout measures are much stronger predictors of turnover intentions than engagement. Therefore, burnout is much more predictive of turnover, and engagement is much more predictive of performance.
Burnout and engagement both relate to job satisfaction and organizational commitment at similar levels, but in opposite directions. Burnout
is negatively associated with these factors, and engagement is positively associated with them. Job involvement is how much an employee identifies with and cares about his or her job. Interestingly, engagement is an incredibly strong predictor of job involvement, whereas burnout is pretty much unrelated to this aspect of the employee experience (Christian et al., 2011; Halbesleben, 2010; Lee and Ashforth, 1996; Taris, 2006).
Because it is clear from the meta-analytic results that burnout and engagement are related to these outcomes, Rothbard next discussed what kinds of factors predict burnout and engagement. Meta-analytic findings suggest that the strongest predictors of burnout are such factors as role conflict, role stress, and stressful events. Workload and work pressure predict emotional exhaustion but do not predict depersonalization as much. Role ambiguity is also predictive, leading to greater burnout.
What helps reduce burnout? One of the biggest factors that helps reduce burnout is role clarity. Role clarity entails being very clear about what the employee is supposed to do and what is expected of them.
What helps with engagement? The meta-analysis results indicate there are a number of job characteristics that influence engagement. The following factors increase engagement: autonomy, task variety, task significance, feedback, and problem solving. Autonomy is how much control employees have over how they do their work. Task variety is about doing a lot of different things versus repeating monotonous tasks. Task significance is about assigning meaningfulness to the task. Feedback is about getting reliable information about how you are doing on your job. Problem solving is interesting because framing work as a problem-solving exercise can also be more engaging. Job complexity also increases engagement, and this is interesting because even though there are more demands in complex jobs, it is more energizing. This finding is probably conditional on the fact that people have the ability to meet those demands. Social support also increases engagement. Not surprisingly, physical demands and stressful work conditions decrease engagement.
Leadership is also an important predictor of engagement. The research found that transformational leadership and leader-member exchange are strongly related to employee engagement. Transformational
leadership is about motivating and inspiring people. Leader-member exchange is about having a good relationship between a subordinate and a leader.
The meta-analytic results also indicate that conscientious people and people who have higher positive affect than others are more engaged. People who have a proactive personality, meaning they take initiative and are go-getters, are also more engaged.
Demands and Resources
While burnout and engagement are associated with demands and resources, demands are not necessarily depleting to engagement. Demands can increase engagement. However, if the demands are physically and psychologically taxing, or if people do not have the resources to meet the demands, that can lead to burnout or a lack of engagement.
Rothbard asserts that it is important to understand that emotion matters a great deal. She found in some of her research that when people were engaged, they had positive affect as a result of engagement and were able to devote themselves to other tasks. The engagement enriched them. However, when they were engaged and that engagement led to negative emotion, it was depleting, and they were less engaged in another role. This enrichment and depletion effect depends in large part on the emotional component of it.
Person-environment fit (P–E fit) is the compatibility that exists when individual and work environment characteristics are well matched. P-E fit includes a whole host of dimensions and is another factor that influences burnout and engagement measures. The term is used as an umbrella for research on various aspects of the work environment. P-E fit can exist at the organization, team, job, and vocational levels. For example:
- Organization—“I work for DHS.”
- Team—“I’m a part of the X team.”
- Job—“I spend my day poring over data.”
- Vocation/Profession—“I’m a security analyst.”
Rothbard noted that what is interesting about P–E fit is that it can be easier to see a bad fit than it is to predict what would be a good fit. The meta-analysis results illustrate why P–E fit matters. The findings show that P–E fit is important for job satisfaction and turnover intent. When there is a good fit, people are less likely to quit. P–E fit also has a large effect on stress and an effect on whether people actually leave their jobs. However, fit has a smaller effect on performance in the job (Kristof-Brown et al., 2005).
In her research, Rothbard has found that personal preferences play a part in P–E fit as well. For example, policies can reflect the organization’s values and may not be congruent with the employee’s values or preferences. For instance, policies such as on-site child care are generally perceived to be a bonus for everyone. However, there are people who prefer to keep their work and their personal lives separate, while others like to integrate them. Employees who prefer to integrate the different aspects of their lives tend to see this as a positive policy, and it improves their job satisfaction. However, job satisfaction goes down when there is higher access to on-site child care for people who prefer to keep work and home separate. Rothbard found the same effect for commitment. People who wanted to keep the two worlds separate were less committed when they had access to that integrating policy, whereas the integrators were more committed (Rothbard et al., 2005).
For a segmenting policy like flextime, people who prefer to integrate were less satisfied when they had more access to flextime, whereas those who prefer to segment were more satisfied. However, it is worth noting that those who preferred to segment were never as satisfied as those who preferred to integrate (Rothbard et al., 2005).
Consequences of Misfit
There are a number of consequences related to a bad fit or misfit including lower satisfaction and commitment, higher turnover intentions, and higher stress. Although more autonomy and separation of work and family than desired can decrease outcomes such as satisfaction and commitment, Rothbard pointed out that misfit is not always bad. For instance having more job security and good relationships at work do not negatively affect satisfaction and commitment (Edwards and Rothbard, 1999).
Rothbard concluded by iterating that P–E fit is an important factor influencing an employee’s experience of work. Burnout and engagement are two important processes that relate to performance and psychological outcomes at work, and they are negatively related to one another. Engagement has a stronger effect on performance, whereas burnout has a stronger effect on turnover intentions. When considering what measures to select in a study it is important to keep in mind the differences in what each measurement explains and how the measurements relate to the aspects of the employee experience with which you are most concerned.
Reber started his presentation by noting that FedEx was founded in 1971 and is the world’s largest transportation company serving more than 220 countries and territories. FedEx employs approximately 142,000 people worldwide. From FedEx’s beginning the leadership has stated that the key to the organization’s success is the employees. FedEx’s operations are centered on the idea that if the company takes care of its people, then the people in turn will deliver impeccable service to the customers, who will reward the company with more work, which in turn leads to a profitable company. FedEx has coined the term P-S-P (People-Service-Profit) Philosophy to describe this concept. Measuring service is fairly easy. How well do our couriers deliver the packages? Measuring the financial side is also very clear cut. But the people side is more challenging. In order to get at this issue, FedEx has been using a process called survey-feedback-action since 1978.
The annual climate survey is designed to be a continuous improvement tool, and the process is broken down into three steps: survey, feedback, and action (SFA). The survey measures employee satisfaction and engagement. Reber noted that while these issues are different from resilience, they are related to it. The objectives of the survey are the following:
- Support the P-S-P Philosophy.
- Assess the FedEx Express climate.
- Identify concern areas.
- Facilitate work group problem solving and continuous improvement.
- Ensure that upper management is aware of employees’ concerns.
- Provide a means for management to review outcomes.
- Increase employee satisfaction, motivations, and effectiveness.
The annual climate survey is fielded to all FedEx employees around the globe and is translated into 22 different languages. The questionnaire is standardized, anonymous, and given to management and nonmanagement personnel online. The questions are intended to gather information about what helps and hinders employees in their work environment. There are 34 items on the survey that are rated on a 5-point scale. The items look at the employee’s
- immediate managers;
- corporate leadership;
- employee identification with the company;
- fairness and adequacy of the pay and benefits;
- cooperation inside and between work groups;
- issues that affect job conditions such as rules, resources, and safety; and
- local and company-wide concerns.
The survey goes from being very specific such as “how is my boss doing” to broader concerns such as benefits, cooperation, rules, and procedures. On average, the response rate is 97 percent.
Results are processed and analyzed in less than 48 hours for each work group, and then more than 10,000 reports are returned to each work group’s manager electronically. A meeting is held between the manager and all members of the work group to discuss the results. The goal of the meeting is to identify specific concerns, examine causes, and devise action plans to address those concerns moving forward.
The outcome of the feedback meeting is then developed into a formal action plan. That plan is implemented and monitored, and action is taken to address work group concerns. This leads to the next SFA cycle where the process is repeated. The survey results are analyzed to develop three outcome measures. The Corporate Identification Average looks at how strongly employees identify with the company and includes attributes that research shows to be critical to employee commitment and retention. The SFA average is a percentage of favorable responses for core survey items and provides an indication of overall morale of the work group. The last measure is the Leadership Index. This index is the average percentage of favorable responses for immediate leadership items and is intended to provide an indication of how well a work group’s immediate manager is perceived as providing effective leadership.
In describing the survey Reber quoted FedEx’s president and chief executive David J. Bronczek:
Successful organizations adapt to change. If they don’t, they will atrophy and eventually fail. For over 32 years, our survey-feedback-action process has allowed us to listen to our people and make the changes necessary to keep our workforce motivated, satisfied, and effective even as the environment around us changes rapidly.
Reber noted that one of the reasons for the high response rate to the survey is that employees see its value and how it can improve their work. Measurement without action will lead to decreased response rates as employees lose faith in the process.
What Employees Need
The frequency of the survey allows the corporation to assess the climate, identify work group issues, and develop an action plan to address problems as they arise and before they become widespread. The corporation has also been able to watch for trends and themes in the results, and over the years it has seen the same things repeatedly come up in good and bad ways:
- Employees want to be heard, and they want to have a voice in the design of their work. If you ignore your employees, then they will become disengaged, and the quality of the work will suffer. If they are dialed into the decision-making process, they are invested in making it work.
- Employees need to have the resources to do their job well. Reber cited an example where the company tightened down on the budgeting for supplies that couriers give to customers. The couriers raised serious concerns about the reduction, and it was immediately changed.
- Employees want challenging work that allows for personal growth.
- Employees want to build mature and positive relationships with their coworkers.
- Employees want problems addressed quickly and to not let the issues linger. An unaddressed problem in a work group can have serious consequences. It is better to identify the problem and deal with it quickly.
- Employees want clear direction and communications from the immediate management as well as from the top of the corporation.
- Employees want to grow and develop, and the corporation needs to provide career development opportunities on an ongoing basis.
- Employees want to be rewarded for their work. Reber noted that this is not just about money. Small gestures like being praised in front of the work group can have a big impact on the employee’s sense of accomplishment.
- Make room for celebration. Every day is not just churning out the next project or dealing with the next issue. It is important to take time periodically to celebrate the successes.
- Employees want space to be able to structure and create their work environment. The more rigid the structure, the less room there is for change.
- Employees want balance in terms of their home and work life.
Lesson Learned from Managing the Process
It is important that employees at all levels take the feedback from the survey and apply it every day. The process is not intended to be take the survey once a year, have the meeting, and then forget about it until the next survey. Employees have to work hard to make the SFA process
work. To ensure the SFA process has meaningful results, it is essential to have support from the top of the corporation. That requires that top management support the process, take action when necessary, communicate with all aspects of the organization, and hold management accountable. If the survey results are going to be seen as credible, individual results are not shared with management. No matter how important the results of the survey may be, if they are not made available in a timely fashion they will be useless. Therefore it is important that the reports are accurate and delivered when promised. The results have to be accurate, and the numbers must be explainable to the readers. Reber’s final lesson is that the process has to be evaluated and updated over time. Any program will get rusty over time, so the corporation updates some of the survey questions, changes the way results are communicated, and looks for ways to improve the process in general.
The three speakers participated in a panel discussion and addressed questions from workshop participants. Planning committee member Karen Sexton moderated the discussion.
Sexton began the discussion by noting that there were several measures that could help DHS better understand the issue of resilience, including satisfaction, productivity, efficiency, and burnout and turnover rates. She asked the speakers if they had insights about how DHS should go forward given all the options available.
Arnetz replied that measurement is critical for any successful change. DHS is a huge and complex organization. The first step of implementing an effective intervention is understanding what is actually happening in the organization, which means putting a sound measurement system in place. Unfortunately, this is where organizations often fail. He noted that there are tools that already exist that could be quickly modified to work for DHS. Additionally, he feels that it is important to establish communication between peripheral supervisors and employees because that allows change to happen faster. Reber agreed that measurement
is critical and noted that it can be a challenge to get top management’s support. A variety of issues around fielding a measurement tool requires leaderships’ buy-in, such as implementation logistics and communicating the intent of the survey. Reber noted that there might also be concerns with bringing data about their organization to the forefront. Rothbard noted that it is clear that all the speakers feel measurement is critical for assessing and diagnosing in order to figure out the problem that needs to be solved and how to develop an appropriate intervention. She added that the presentations illustrate how interrelated the problems and subsequent measures are to each other. Teasing the issues apart can be important in determining how best to use limited resources.
Rothbard commented that combining or “lumping” measures together can be powerful in terms of predictive validity. However, it also makes it difficult to then pinpoint the core problem. Breaking measures out or “splitting” them can be more complicated, but it can also be helpful in diagnosing the core issues and targeting interventions.
Kathryn Brinsfield from DHS commented that the presentations indicated that resilience is not only about how people feel about their jobs, but also includes their family and how they are perceived by their community, and so on. It is also possible that these perceptions are inaccurate. How do these issues relate to measures such as engagement and burnout?
Arnetz replied that there is an increasing amount of work on work-life balance and the importance of partners. Most of the focus is really on how work spills over on family, however. An added complication for security-sensitive positions is that employees are not able to communicate work concerns with their spouse, family members, or other support groups. That is why using a “diffusing” group or having peer support holds promise.
He also added that spare time or leisure activities are important for recovery from stress. Almost all first responders that he has interviewed indicate that spending time with their family is their number one choice for recovering from stress. Is it possible to integrate that into the management process for some of these groups?
Rothbard added that her work looks at the spillover between work and family in both directions. Some researchers use measures where partners and family members are asked about spillover. This adds a valuable perspective and helps determine what is going on. Is work-related stress simply the person’s perception? Is there a true measure of what is experienced at work? Is it all in the person’s perception? Or is there
something about the relationship between what the coworkers perceive, what the person perceives, and what the family perceives?
Planning committee chair James Peake noted that in order for a survey to be useful, the process must be trusted and relevant to the organization. He asked about FedEx’s organization structure and how it relates to the survey.
Reber responded that the frontline workers in FedEx are generally divided into two groups: operations managers oversee the couriers and service managers oversee the service agents who answer customer calls. That span of control for the frontline operations is typically 1 manager to about 24 employees. It is fairly large, but it is still small enough that you can get the employees’ feedback and figure out what is going on with them. On the service side, there is a smaller span of control, the work is more complex, and there are generally 10-15 employees per manager. The individual manager’s report gives you a good idea of what is going on in that group. The individual reports are then passed to the senior manager, vice president, and senior vice president levels. At this point the information is a little more diffused, but it still reflects the opinions of the employees. The key report is at the manager level because that is what is needed to develop action plans.
Planning committee member Joseph Barbera asked the speakers to discuss the segmentation between work and home and the effect of shift work. Rothbard noted that in a study of firefighters who work 24-hour or 48-hour shifts, she and her co-author found that the firefighters use segmentation as a coping strategy. In general they have to deal with some difficult and graphic situations, and they tend to not want to discuss that with their families. The exception interestingly was when their spouses were emergency nurses. In any case, they chose to segment as a deliberate decision. Firefighters in her study were about 98 percent male. They identified work-life balance as one of their number one issues. The study also found that suppressing the experience had negative implications for health outcomes and risk-taking behavior off the job. The culture and the organizational setting of compassion and caring within the firehouse made these outcomes easier to deal with.
Reber asked if training workers on the need to recover in their off hours has been studied. Arnetz noted that talking to workers about the importance of recovery at home could be beneficial. However, in many
cases it is difficult for people to implement given the demands of home life. Many workers use their off hours to deal with errands, take care of children, and so on.
Arnetz commented that in many lines of work, not just the operational areas, there is a need to quickly shift from tedious work to high-intensity demands. That shift takes its toll. Many workers, however, particularly firefighters and law enforcement, will say that those high-intensity moments are why they went into this field.
Planning Effective Performance Measurement
Planning committee member Scott Mugno asked the speakers if they had suggestions about where DHS should start as it moves forward on these issues. Reber commented that DHS must clearly identify what the vision and goals of a program should be. The process can be built around key pieces. He added that most companies have good measures of their financial and service components. At FedEx, the leadership asserted that it was essential to find a way to measure the people side. During the first year of the survey they used a tool from the University of Michigan, which had about 200 items. The problem was no one wanted to deal with the survey because it was too long and took too much time. The lesson is that the key measures should be decided up front in order to streamline the tool so it is manageable. He added that it may also help in getting support from the top management.
Rothbard stated that ensuring people are answering the survey honestly is important. She added that what FedEx has been able to do internally is impressive. If DHS is going to do this work internally, it must be very conscious about the need to engender that trust in order to get honest answers. In her work she has found that employees are sometimes more willing to answer questions honestly when the survey is managed by people outside the organization. People are concerned about anonymity, and outside researchers are often perceived as independent and less likely to have an agenda.
Arnetz asserted that there are three important factors that must be addressed up front, leadership buy-in, ensuring the validity of the measures, and building a process where the results are used. Getting management and leadership buy-in is critical. Arnetz noted that in the business world the first step is getting top management to link the survey and its results to the corporation’s performance. Making this connection ensures that management is engaged and sees the effort as relevant. To
build support for this effort in DHS, it will need to show that this work is directly linked to the key task of defending the homeland. How the survey is designed is also important and can determine whether it is psychometrically proper. Finally, if the results of the survey are not used then they are useless and people will stop participating. The survey must be seen as a vehicle for improvement and not a tool for punishment.
Given the success of the FedEx SFA process, Mugno asked how FedEx is staffed to take on such a large task. What kind of staff and information technology (IT) infrastructure is needed? Reber replied that there are three full-time industrial psychologists on staff to manage the survey process. That does not include the separate IT piece, which is part of the larger FedEx system. It is a logistically complex process because all employees have to be assigned a manager, and the survey results must be disseminated to the senior manager level and director level. There is a lot of work up front to make sure the information is accurate.
Surveys are great if they are used properly, noted Alisa Green from DHS. She commented that DHS has a reputation problem owing to (1) the consistently low ranking from the Office of Personnel Management’s (OPM’s) employee satisfaction annual survey and (2) the current climate of public criticism of federal workers. The response rate from the OPM survey is relatively low, but the results are published everywhere. She is concerned that the idea that DHS is a bad place to work might become a self-fulfilling prophecy.
Rothbard iterated the important distinction between surveys and how the information is used. The type of survey the speakers are discussing is intended to diagnose problems internally and then figure out what to do about them. Peake noted that there is also an issue with restrictions on surveys. These restrictions can become a barrier to getting meaningful information to the appropriate levels.
A good deal of the discussion concerned important issues such as a healthy culture and organization. Although these are important, James Schwartz from the Arlington Virginia Fire Department noted that the other side of the coin could be having the right people in the right positions. He asked if there are tools that can help predict if a person is the right “fit” for the job, culture, and organization. He also wanted to know if prior exposure to trauma or excessive stress can predict potential issues.
There are measures of fit that people use, noted Rothbard. However, it is not always clear until people get into the job which experiences are actually going to be relevant, nor is it known whether these measures are valid for use as a selection tool. Similarly, identifying prior trauma as a predictor of suitability is not necessarily useful either. For instance, a person can experience trauma and find a way to cope with it, and it may help them and give them experiences to draw upon. However, someone else with the same type of trauma may be immobilized by it.
Arnetz, B. B. 1996. Techno-stress: A prospective psychophysiological study of the impact of a controlled stress-reduction program in advanced telecommunication system design work. Journal of Occupational and Environmental Medicine 38:53-65.
Arnetz, B. B., D. C. Nevedal, M. Lumley, L. Backman, and A. Lublin. 2009. Trauma resilience training for police: Psychophysiological and performance effects. Journal of Police Criminal Psychology 24:1-9.
Arnetz, B. B., T. Lucas, and J. E. Arnetz. 2011. Organizational climate, occupational stress, and employee mental health: Mediating effects of organizational efficiency. Journal of Occupational and Environmental Health 53:34-42.
Backman, L., B. B. Arnetz, D. Levin, and A. Lublin. 1997. Psychophysiological effects of mental imaging training for police trainees. Stress Medicine 13:43-48.
Christian, M. S., A. S. Garza, and J. E. Slaughter. 2011. Work engagement: A quantitative review and test of its relations with task and contextual performance. Personnel Psychology 64:89-136.
Crawford, E. R., J. A. LePine, and B. L. Rich. 2010. Linking job demands and resources to employee engagement and burnout: A theoretical extension and meta-analytic test. Journal of Applied Psychology 95:834-848.
Dunn, P. M., et al., 2007. Meeting the imperative to improve physician well-being: Assessment of an innovative program. Journal of General Internal Medicine 22:1544-1552.
Edwards, J. R., and N. P. Rothbard. 1999. Work and family stress and well-being: An examination of person-environment fit in the work and family domains. Organizational Behavior and Human Decision Processes 77:85-129.
Halbesleben, J. R. B. 2010. A meta-analysis of work engagement: Relationships with burnout, demands, resources, and consequences. In A. B. Bakker (Ed.), Work engagement: A handbook of essential theory and research. New York: Psychology Press.
Harter, J. K., F. L. Schmidt, E. A. Killham, and S. Agrawal. 2009. Gallup Q12 meta-analysis: The relationship between engagement at work and organizational outcomes. Omaha, NE: Gallup.
Hasson, D., U. M. Anderberg, T. Thoerell, and B. B. Arnetz. 2005. Psychophysiological effects of a web-based stress management system: A prospective randomized controlled intervention of IT and media workers. BioMed Central Public Health 25:5-78.
Kristof-Brown, A. L., R. D. Zimmerman, and E. C. Johnson. 2005. Consequences of individuals’ fit at work: A meta-analysis of person–job, person– organization, person–group, and person–supervisor fit. Personnel Psychology 58:281-342.
Lee, R. T., and B. E. Ashforth. 1996. A meta-analytic examination of the correlates of the three dimensions of burnout. Journal of Applied Psychology 81:123-133.
Maghout-Juratli, S., J. Janisse, K. Schwartz, and B. B. Arnetz. 2010. The causal role of fatigue in the stress-perceived health relationship: A MetroNet study. Journal of the American Board of Family Medicine 23:212-219.
Maslach, C., S. E. Jackson, and M. P. Leiter. 1996. Maslach burnout inventory manual, 3rd ed. Palo Alto, CA: Consultant Psychology Press.
Nevedal, D., L. Backman, A. Lublin, and B. B. Arnetz. 2009. Sources of low-intensity professionally induced stress in first responders. Presented at the NIOSH/APA Work, Stress, and Health Conference. San Juan, Puerto Rico, November 5-8.
Rothbard, N. P. 2001. Enriching or depleting? The dynamics of engagement in work and family roles. Administrative Science Quarterly 46:655-684.
Rothbard, N. P., K. W. Phillips, and T. L. Dumas. 2005. Managing multiple roles: Work-family policies and individuals’ desires for segmentation. Organization Science 16:243-258.
Schaufeli, W. B., and A. B. Bakker. 2003. UWES—Utrech Work Engagement Scale: Test Manual. Utrecht, The Netherlands: Department of Psychology, Utrecht University.
Schell, E., T. Theorel, D. Hasson, B. B. Arnetz, and H. Saraste. 2008. Impact of a web-based stress management and health promotion program on neck-shoulder-back pain in knowledge workers: 12-month prospective controlled follow-up. Journal of Occupational and Environmental Medicine 50:667-676.
Taris, T. W. 2006. Is there a relationship between burnout and objective performance? A critical review of 16 studies. Work & Stress 20:316-334.
Ventimiglia, M., S. Thomsen, D. Nevedal, and B. B. Arnetz. 2011. Sources of chronic organizational stress in first responders. Presented at APA/NIOSH Work, Stress, and Health Conference. Orlando, FL, November 19-21.