Post-Occupancy Evaluations and Organizational Learning1

Craig Zimring, Ph.D., Georgia Institute of Technology Thierry Rosenheck, Office of Overseas Buildings Operations, U.S. Department of State

Federal building delivery organizations face intense pressures. Not only must they provide buildings on time and within budget, but they have increased demands. They are called on to deliver buildings that are better: more sustainable, accessible, maintainable, responsive to customer needs, capable of improving customer productivity, and safer. In many cases, they must achieve these goals with fewer staff.

Some organizations have faced these pressures proactively, by creating formal processes and cultural changes that make their own organizational learning more effective. In this chapter we adopt the approach to organizational learning of Argyris (1992a), Huber (1991) and others. We mean that organizations are able to constantly improve the ways in which they operate under routine conditions, and they are able to respond to change quickly and effectively when needed (Argyris, 1992a). Learning is “organizational” if it is about the core mission of the organization and is infused through the organization rather than residing in a few individuals. More simply, in the words of Dennis Dunne, chief deputy director for California’s Department of General Services, they “get it right the second or third time rather than the seventh or eighth.” By being more systematic about assessing the impact of decisions and being able to use this assessment in future decision-making, building delivery organizations are able to reduce the time and cost to deliver buildings and increase their quality.

Some of the best models come from private sector organizations. For example, Disney evaluates everything it does and has been doing so since the 1970s. Disney has at least three evaluation programs and three corresponding databases: (1) Disney tracks the performance of materials and equipment and records the findings in a technical database. (2) Guest services staff members interview guests about facilities and services, focusing on predictors of Disney’s key business driver: the intention of the customer to return. (3) A 40-person industrial engineering team conducts continuous research aimed at refining programming guidelines and rules of thumb. The industrial engineering team explores optimal conditions: What is the visitor flow for a given street width when Main Street feels pleasantly crowded but not oppressive? When are gift shops most productive? This research allows Disney to make direct links between “inputs” such as the proposed number of people entering the gates and “outputs” such as the width of Main Street.

The Disney databases are not formally linked together but are used extensively during design and renovation projects. They have been so effective that the senior industrial engineer works as a coequal with the “Imagineering” project manager during the programming of major new projects.

Disney is a rare example. It uses an evaluation program to do the key processes that organizational learning theorists argue are key to organizational learning (Huber, 1991):

  1. monitoring changes in the internal and external business environment,


For their generous and thoughtful input we would like to thank Stephan Castellanos, Dennis Dunne, Gerald Thacker, Lynda Stanley, Polly Welch and Richard Wener.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 42
5 1 Post-Occupancy Evaluations and Organizational Learning Craig Zimring, Ph.D., Georgia Institute of Technology Thierry Rosenheck, Office of Overseas Buildings Operations, U.S. Department of State Federal building delivery organizations face intense Some of the best models come from private sector pressures. Not only must they provide buildings on organizations. For example, Disney evaluates every- time and within budget, but they have increased thing it does and has been doing so since the 1970s. demands. They are called on to deliver buildings that Disney has at least three evaluation programs and three are better: more sustainable, accessible, maintainable, corresponding databases: (1) Disney tracks the perfor- responsive to customer needs, capable of improving mance of materials and equipment and records the find- customer productivity, and safer. In many cases, they ings in a technical database. (2) Guest services staff must achieve these goals with fewer staff. members interview guests about facilities and services, Some organizations have faced these pressures focusing on predictors of Disney’s key business driver: proactively, by creating formal processes and cultural the intention of the customer to return. (3) A 40-person changes that make their own organizational learning industrial engineering team conducts continuous more effective. In this chapter we adopt the approach research aimed at refining programming guidelines and to organizational learning of Argyris (1992a), Huber rules of thumb. The industrial engineering team (1991) and others. We mean that organizations are able explores optimal conditions: What is the visitor flow to constantly improve the ways in which they operate for a given street width when Main Street feels pleas- under routine conditions, and they are able to respond antly crowded but not oppressive? When are gift shops to change quickly and effectively when needed most productive? This research allows Disney to make (Argyris, 1992a). Learning is “organizational” if it is direct links between “inputs” such as the proposed about the core mission of the organization and is number of people entering the gates and “outputs” such infused through the organization rather than residing in as the width of Main Street. a few individuals. More simply, in the words of Dennis The Disney databases are not formally linked Dunne, chief deputy director for California’s Depart- together but are used extensively during design and ment of General Services, they “get it right the second renovation projects. They have been so effective that or third time rather than the seventh or eighth.” By the senior industrial engineer works as a coequal with being more systematic about assessing the impact of the “Imagineering” project manager during the pro- decisions and being able to use this assessment in future gramming of major new projects. decision-making, building delivery organizations are Disney is a rare example. It uses an evaluation pro- able to reduce the time and cost to deliver buildings gram to do the key processes that organizational learn- and increase their quality. ing theorists argue are key to organizational learning (Huber, 1991): 1. monitoring changes in the internal and external 1For their generous and thoughtful input we would like to thank business environment, Stephan Castellanos, Dennis Dunne, Gerald Thacker, Lynda Stanley, Polly Welch and Richard Wener. 42

OCR for page 42
43 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING 2. establishing performance goals based on internal • Are participants in building projects, including and external influences, internal project managers, consultants, and clients, 3. assessing performance, aware of POEs or POE results, either from per- 4. interpreting and discussing the implications of re- sonal participation or from written results? sults, • If so, were POE results consciously used in 5. consolidating results into an organizational decision-making about buildings? For example, memory, are they used for programming, planning, design, 6. widely distributing findings and conclusions, construction, and facilities management? 7. creating a culture that allows the organization to • Can we see evidence that POE results are part of take action on the results, reflection and discussions about how to do a good 8. taking action based on organizational learning. job, among peers and with supervisors? • Are POE results consciously used to refine pro- Post-occupancy evaluation (POE) practice cesses for delivering buildings in terms of either focuses mostly on individual project support and formal process reflected in manuals or informal analysis rather than on lessons-learned. Although rules of thumb and customs? POE potentially provides a methodology for all of these • Are people who make policy about buildings, processes, POE practice has historically had a more such as policy directives, design guidelines, and narrow focus on assessing performance and interpret- specifications, aware of POEs? ing results. POE has often been used as a methodology • If so, is POE explicitly used in formulating aimed at assessing specific cases, while the other pro- policy? cesses are seen as part of strategic business planning. Even when evaluators have been able to create data- Were the Conditions for Organizational bases of findings, they have often been used to bench- Learning Present? mark single cases rather than to develop more general conclusions. As we attempted to document organizational learn- Structured organizational learning is difficult. It ing we were trying to understand the conditions that requires the will to collect data about performance and foster or thwart it: the time to interpret and draw conclusions from the data. More fundamentally, learning involves risk and • Does the organization have an infrastructure for change. Learning exposes mistakes that allow improve- learning? For example, are results from POEs ment but most organizations do not reward exposing consolidated in some way, such as in reports or shortcomings. Learning brings change and organiza- databases? Is this consolidated information dis- tions are usually better at trying to ensure stability than tributed, either internally or to consultants or the at supporting change. public? In this chapter, we explore how a variety of public • Is there a mechanism for ensuring that this infor- agencies and some private ones have used POE suc- mation is kept current? cessfully for organizational learning. We discuss the • If lessons are made available, do they support the “lessons-learned” role of evaluation rather than the kinds of decisions that are made by the organiza- project support and analysis role. We have examined tion? Are they likely to seem authentic and written materials from 18 POE programs and inter- important to decision-makers? Are the implica- viewed participants wherever possible. We explored tions of results made clear, or do busy decision- whether POE-based organizational learning appeared makers need to make translations between results to be going on, whether the organizations had estab- and their needs? lished support for learning, and the nature of the • Are there incentives for accessing the data, using learning. the results, and contributing to the lessons-learned knowledge base? For example, are internal staff or consultants evaluated on use of lessons- Did POE-Enabled Organizational Learning Occur? learned? Are they rewarded in some way for par- For example, in looking for evidence of organiza- ticipation? Are consultants rewarded for partici- tional learning we asked the following questions:

OCR for page 42
44 LEARNING FROM OUR BUILDINGS BRIEF INTRODUCTION TO pation? Are they rewarded for good performance POST-OCCUPANCY EVALUATION as judged by the POE? • Are there disincentives for participating in lessons- Post-occupancy evaluation grew out of the extra- learned programs? If an innovative initiative ordinary confluence of interests among social scien- receives a negative evaluation, is it treated as an tists, designers, and planners in the 1960s and 1970s opportunity for organizational learning or as a (see, for example, Friedmann et al., 1978; Shibley, personal failure? 1982; Preiser, et al., 1988). Early POE researchers were • Is there a perception of high-level support for the strongly interested in understanding the experience of lessons-learned program? Many organizations building users and in representing the “nonpaying” create frequent new initiatives, and seasoned staff client (Zeisel, 1975). Many early POEs were conducted often perceive these as the management “infatua- by academics focusing on settings that were accessible tion du jour”: wait a day and it will change. to them, such as housing, college dormitories, and resi- dential institutions (Preiser, 1994). How Can Organizations Develop Useful During the 1980s, many large public agencies Learning Content? developed more formal processes to manage informa- tion and decisions in their building delivery processes. We also assessed the content of the organizational As planning, facilities programming, design review, knowledge. We asked the following: and value engineering became more structured, agen- cies such as Public Works Canada and the U.S. Postal • Has the organization produced a shared view of Service added building evaluation as a further step in what makes a good building, in terms of either gathering and managing information about buildings process or product? For example, has the organi- (Kantrowitz and Farbstein, 1996). zation been clear about key design and program- This growth of POE occurred while politicians and ming decisions and about how these decisions policy analysts were advocating the evaluation of link to the client’s needs? Are these contributed public programs more generally. Campbell and many to by POE? others had been arguing at least since the 1960s that • Has the organization created an organizational public programs could be treated as social experiments memory of significant precedents? Are these pre- and that rational, technical means could contribute to, cedents described, analyzed, or evaluated in or even replace, messier political decision-making meaningful ways? (Campbell, 1999). A similar argument was applied to • Is this view tested and refined through POE or POE: statements of expected performance could be similar processes? viewed as hypotheses that POE could test (Preiser et al., 1988). In this chapter, we briefly report our findings and The term post-occupancy evaluation was intended analysis. We discuss four topics: to reflect that assessment takes place after the client had taken occupancy of a building; this was in direct 1. What is post-occupancy evaluation? What is its contrast to some design competitions where completed history, and how has this contributed both to its buildings were disqualified from consideration or to potential for and difficulties in achieving organi- other kinds of assessment such as “value engineering” zational learning? that reviewed plans before construction. Some early 2. Do organizations do POE-enabled organizational descriptions focused on POE as a stand-alone practice learning? aimed at understanding building performance from the 3. How have organizations created the appropriate users’ perspectives. Some methodologists have advo- conditions for learning through POE? cated the development of different kinds of POEs, with 4. How have they created a knowledge base for different levels of activity and resource requirements building delivery and management? (Friedmann et al., 1978; Preiser et al., 1988). For example, Preiser advocated three levels of POE: brief indicative studies; more detailed investigative POEs; and diagnostic studies aimed at correlating environ-

OCR for page 42
45 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING mental measures with subjective user responses • It creates mechanisms for quality monitoring, (Preiser, 1994). Whereas there was little agreement similar to using student testing to identify under- about specific methods and goals, most early POEs performing schools, where decision-makers are focused on systematically assessing human response to notified when a building does not reach a given buildings and other designed spaces, using methods standard. such as questionnaires, interviews, and observation, • It supports fine-tuning, settling-in, and renovation and sometimes linking these to physical assessment of existing settings. (Zimring, 1988). • It provides data that inform specific future deci- Over the years, many theorists and practitioners have sions. grown uncomfortable with the term POE; it seems to • It supports the improvement of building delivery emphasize evaluation done at a single point in the pro- and facility management processes. cess. Friedmann et al. (1978) proposed the term “envi- • It supports development of policy as reflected in ronmental design evaluation.” Other researchers and design and planning guides. practitioners have suggested terms such as “environ- • It accelerates organizational learning by allowing mental audits” or “building-in-use assessment” decision-makers to build on successes and not (Vischer, 1996). More recently, “building evaluation” repeat failures. and “building performance evaluation” have been pro- posed (Baird et al., 1996). Nonetheless, for historical This chapter focuses primarily on the use of POE for reasons the term post-occupancy evaluation remains improving organizational learning. common, and we use it in this chapter for clarity. Other discussions of evaluation emphasized the DO ORGANIZATIONS DO POE-ENABLED importance of embedding POE in a broader program of ORGANIZATIONAL LEARNING? user-based programming, discussion, and design guide development, proposing terms such as “pre-occupancy As discussed above, we reviewed materials from evaluation” (Bechtel, 2000), “process architecture” some 18 organizations that are currently doing POEs (Horgen et al., 1999), and “placemaking” (Schneekloth or have done so in the past. In looking at organizations and Shibley, 1995). As early as the 1970s, the Army that have active POE programs, we found that members Corps of Engineers conducted an ambitious program of project teams, including project managers, consult- of user-based programming and evaluation that resulted ants, and clients, tend not to be aware of POEs, unless in some 19 design guides for facilities ranging from a special evaluation has been conducted to address a drama and music centers to barracks and military police problem that the team is facing. Where they are aware stations (Schneekloth and Shibley, 1995; Shibley, of the POEs, team members often do not have the 1982, 1985). More recently, POE has been seen as part reports from past POEs at hand and do not apparently of a spectrum of practices aimed at understanding use POE results in daily decision-making. design criteria, predicting the effectiveness of emerg- Mid-level staff tend to be more aware of POEs. In ing designs, reviewing completed designs, and support- particular, staff responsible for developing guidelines ing building activation and facilities management and standards are often aware of POE results. For (Preiser and Schramm, 1997). With growing concerns example, in the U.S. Postal Service, the staff who main- about health and sustainability, several programs have tain guidelines also administer POEs; the POEs con- also linked user response to the physical performance ducted by the Administrative Office of the U.S. Courts of buildings, such as energy performance (Bordass and are used directly by the Judicial Conference to test and Leaman, 1997; Cohen et al., 1996; Leaman et al., 1995) update the U.S. Courts Design Guide. or indoor air quality (Raw, 1995, 2001). We were not able to find situations where senior POE methodologists and practitioners have identi- management used POEs for strategic planning. POEs fied several potential benefits of POE (Friedmann et have the potential for supporting “double-loop learn- al., 1978; McLaughlin, 1997; Preiser et al., 1988; ing” (Argyris and Schon, 1978)—that is, not only to Zimring, 1981): evaluate how to achieve existing goals better but also to reflect on whether goals themselves need to be • A POE aids communications among stakeholders reconsidered. However, we were not able to find cases such as designers, clients, end users, and others. where this actually occurred.

OCR for page 42
46 LEARNING FROM OUR BUILDINGS We were not able to find many compilations of POE an evaluation can help decision-makers reflect on the findings, although several organizations such as the implications of the results and make links to their own U.S. Army Corps of Engineers, U.S. Postal Service, practice. Administrative Office of the U.S. Courts, General Ser- A group of evaluators in New Zealand developed a vices Administration, and others have incorporated “touring interview” methodology to allow decision- POEs into design guides. Disney and the U.S. Depart- makers to actively participate in evaluations with little ment of State have incorporated POE into databases of training and only modest commitment of time information. These are discussed in more detail below. (Kernohan et al., 1992; Watson, 1996; 1997). For It does appear that POEs are not used for their full example, in an active evaluation program with more potential for organizational learning. In particular, we than 80 completed evaluations, Bill Watson, a consult- were not able to find many circumstances where POE ant to public and private clients, takes building users was part of an active culture of testing decisions, learn- on a tour of the building and asks open-ended ques- ing from experience, and acting on that learning. There tions—for example, “What works here?”—as well as are two major reasons for this: more specific probes about the functions of spaces and systems. POE reports are mostly verbatim comments 1. Learning is fragile and difficult, and many organi- by users and are sorted into categories such as “action zations have not created the appropriate condi- for this building” or “change in guidelines for future tions for learning. If learning is to be genuinely buildings.” This approach is quite inexpensive and can “organizational,” a large number of staff must be completed with several person-days of effort. The have the opportunity to participate and to reflect experience is vivid for the participants and produces on the results in a way that enables them to incor- results that are imageable and articulate. It also allows porate the results into their own practice. Poten- participants to discuss relative priorities and values. tial participants must see the value for themselves: However, because each touring interview group varies, there must be incentives for participating. Also, it is more difficult to compare evaluations of different evaluation will sometimes reveal that building settings. performance does not reach a desired standard. This kind of participatory evaluation can be an This is, of course, the value of POE, but many extension of existing processes for receiving feedback organizations punish people when innovations do from customers. Project managers in Santa Clara not work. In addition, many organizations simply County, California, were tired of receiving a storm of do not make information available in a format that requests from users after they moved into a building. is clear and useful to decision-makers. These were difficult to direct to contractors, suppliers, and others. They contracted with consultants Cheryl 2. Many organizations have not created a body of Fuller and Craig Zimring to create a Quick Response knowledge that is valuable in the sense that it pro- Survey (QRS) aimed at organizing and prioritizing user vides a coherent, integrated body of knowledge needs about three months after buildings were occu- that is helpful in everyday decision-making. pied. All building users fill out a one-page question- Knowledge tends to be informal and individual. naire, and project managers follow up with a half-day walk-through interview of the building with the facility manager and staff representatives. The project man- WAYS TO CREATE THE APPROPRIATE agers then prioritize requests and meet with the client CONDITIONS FOR LEARNING THROUGH POE organizations. The State of California Department of General Services is further developing the QRS and Create Broad Opportunities for Participation will have evaluators enter results into a lessons-learned and Reflection database. Our research suggests that POE-based knowledge is A lessons-learned program initiated in 1997 for New not widely shared within most organizations. One way York City to examine the success of school projects in to achieve this sharing is through direct participation in the state was aimed at participation by consultants. The evaluations. Seeing how a facility works while hearing School Construction Authority (SCA), whose member- directly from users is a memorable experience. Also, ship is appointed by the governor, the mayor, and the the process of analyzing and writing up the results from New York City Board of Education, was charged with

OCR for page 42
47 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING the program. To get the program approved, SCA, under also assessed using physical measures; if the user the leadership of consultant Ralph Steinglass, adopted satisfaction measures did not reach criterion, physical a simple methodology—require the architect or engi- measures could be substituted (Gregerson, 1997). The neer of record to conduct the POE. The rationale was designers and contractors consulted the scientists that this would guarantee that designers would confront throughout the process, showing them alternatives for how users responded to their designs and force a the façade design and full-scale mockups of the range lessons-learned loop in the design process. About hoods. The building passed on all criteria except satis- 20 POEs have been completed. To ensure reliability, faction with the range hoods, which were modified SCA reviewed the results before approving the POEs. after the evaluation as a response to user input. Whereas In some cases, the architects or engineers had to some aspects of this testing process might be question- reschedule their interviews when they were suspected able—Should you evaluate an entire building on 14 of introducing a bias or continue their investigation if yes-no scales? Should maintenance and operating they failed to include critical areas required in the experience be included?—this process gained from par- study. ticipation throughout. The design firm, contractors, The three programs described above involve evalu- management, and scientists all participated in estab- ation by the people who designed and managed the lishing the criteria at the outset, and the financial project. As such, participatory evaluation is well suited incentive encouraged the contractor and designers to to supporting learning by in-house project managers consult the users at every step in the process. It is diffi- and consultants. Whereas the New Zealand projects are cult to document the learning benefit of this process, led by consultants, the quick-response projects and the but the contractor, Sordoni Skansa, has since used SCA projects are conducted entirely by the consultants POE-based incentives in several other projects and has or project managers. refined the way in which buildings are delivered. The California Department of General Services is planning to include the results of POEs as part of the Create Incentives for Participation review of qualifications when selecting consultants and Most building professionals are interested in doing a contractors. This has strongly increased the interest in good job and see value in POE. However, as personal POEs by participating firms. We are unaware of any time management consultant Stephen Covey has POE programs that provide incentives for internal staff argued, things that are merely important often lose out members to participate in evaluations, though several to things that are urgent: general benefits such as long- programs have discussed such incentives, such as pro- term learning often lose out when professionals are viding a free vacation day as a reward for adding data faced with the pressing matters of everyday life. When to the knowledge base or providing a mini-sabbatical more specific incentives are offered, it often increases for participating in evaluations or a lessons-learned participation in a POE program. program. Disney provides a powerful, if indirect, The drug company Ciba-Geigy has used direct mon- incentive: knowledge. Only the industrial engineers etary incentives. The architectural and engineering firm have access to key data and they then become valuable HLW and the contractor Sordoni Skansa Construction members of the design team. put their design and construction profits ($300,000 and $1.2 million, respectively) at risk based on performance Reduce Disincentives: Create Protected on schedule, cost, and user satisfaction for the new $39 Opportunities for Innovation and Evaluation million Ciba-Geigy Corporation’s Martin Dexter Labo- ratory in Tarrytown, New York. One-third of the profits Organizational learning consultants have long was based on user satisfaction responses to 14 survey pointed to an inherent contradiction in many organiza- questions: heating, ventilation, air conditioning, acous- tions. Whereas most organizations espouse innovation tics, odor control, vibration, lighting, fume-hood per- and learning, they behave in ways that limit it. We formance, quality of construction (finishes), building recently participated in a meeting where an organiza- appearance, and user-friendliness. The questions were tion had used an innovative building delivery strategy binary-choice (acceptable-not acceptable), and the with which it was not familiar. They had left out a key building had to reach 70 percent satisfaction to pass the review step. When this became clear, a senior manager test. Some aspects such as sound transmission were turned to the project manager and said: “We would

OCR for page 42
48 LEARNING FROM OUR BUILDINGS have expected someone at your level to do better.” The out innovations on a limited number of projects before message to everyone in the room was clear: avoid rolling out the innovation to the larger organization. innovation and avoid evaluation! This syndrome— This office has recently used building serviceability focusing on the individual rather than the performance, tools and methods (Davis and Szigeti, 1996) for pro- blaming the innovator rather than learning from the gramming and design review for the new embassies in innovation—is pervasive among organizations more Dar es Salaam and Nairobi. generally (Argyris, 1992b; Argyris and Schon, 1978). The State of Minnesota Department of Natural However, some building-delivery organizations have Resources has used POE to evaluate two innovative used POE to at least partially overcome it. regional centers. In each of these cases the organiza- Some organizations have done this by explicitly tional learning effort provided some additional sanctioning “research” with the attendant acknowledg- resources for data collection and reflection as well as ment that innovations might not succeed. For example, the clear designation that this was an innovative effort the General Services Administration’s (GSA) Public that might not be fully successful. Buildings Service has recently appointed a director of In many organizations it is risky to be the first one to research. The first director, Kevin Kampschroer, has a try an innovation. Massachusetts Institute of Technol- budget to conduct, synthesize, and distribute research, ogy organizational consultant Edgar Schein has pro- including POE. The use of the term “research” carries posed that while organizations may benefit greatly with it the understanding that not all efforts are suc- from consultants, they often find the experience of cessful, and the budget provides some time for reflec- peers more helpful when they actually move to imple- tion about findings. To date, much of the research is menting an innovation. Schein has called for “learning conducted by academic consultants who bring outside consortia” where people can get advice from peers in learning into GSA. However, GSA is also looking at other organizations and learn from their experience ways to broaden internal ownership of the research (Schein, 1995). He argues that although such learning program. consortia may be effective at all levels of an organiza- GSA has also created an active “officing” labora- tion, they are particularly effective among chief execu- tory in its own headquarters’ building. The lab, super- tive officers (CEOs) or upper to mid- level managers. vised by Kampschroer, is one floor of actual workspace Although the prototype, laboratory, and learning that includes an innovative raised floor heating, venti- consortium efforts are quite different, all reduce the lating, and air-conditioning system and several brands disincentives for innovation and evaluation by allow- of modular office furniture systems. It also explores ing innovation and evaluation at relatively low personal design to support teamwork, with many small confer- and organizational cost. ence rooms and meeting areas. The workers are frequently surveyed and observed, and the lab also Provide Access to Knowledge for Different Audiences becomes a place where clients can see alternative office layouts. The simplest barrier to using POE for organizational The U.S. Courts and the General Services Adminis- learning is when POE results are not available to tration Courthouse Management Group are consider- decision-makers. Many organizations produce POEs as ing developing a different kind of laboratory: a full- case study reports that are not widely distributed. Part scale courtroom mockup facility where new courtroom of this may be due to the history of POE, which has layouts and technologies can be tested and refined at focused on single case studies, and part may be because relatively low cost and risk. This facility, to be con- of the perceived disincentives to distributing informa- structed at the Georgia Institute of Technology, would tion that might be seen as critical of internal efforts or allow mock trials to be conducted and would provide individuals. Part of the problem is the simple technical training for judges, staff, and lawyers. difficulty of distributing printed information, and this Another way to reduce the personal and organiza- has become a lot easier with the Internet and intranet tional cost of experimentation is by starting small with and virtual private networks. The National Aeronautics projects that have an experimental component. The and Space Administration makes its lessons learned innovation can be evaluated and considered for broader database available to all authorized staff and contrac- adoption. For example the U.S. Department of State tors. In the United Kingdom, Adrian Leaman and Bill Office of Overseas Buildings Operations (OBO) tries Bordass have created an interactive Web site for the 18

OCR for page 42
49 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING buildings they have evaluated as part of the PROBE buildings more effectively. Given the large number of project. Funded by the Building Research Estab- POEs that have been completed—longtime researcher lishment and Building Services Journal, PROBE stands and Environment and Behavior Journal editor Robert for post-occupancy review of buildings and their engi- Bechtel estimates that more than 50,000 have been neering. completed—one would expect that there would have Some organizations have overcome some of these been many books and guides that synthesize the results issues by creating design guides and databases of POE of POEs and tie them into a coherent guide for key information. Agencies such as the Administrative programming and design decisions. However, such Office of the U.S. Courts, the U.S. Postal Service, and guides are relatively rare. In part, this is because of the the General Services Administration have created focus of POE researchers and consultants on case design guides that are widely distributed. studies. Much knowledge about buildings has been As we have suggested, the problem with organiza- built up incrementally through negotiation on indi- tional learning is only partially technical. The tools for vidual projects and programs, but organizations seldom creating Web sites and databases are now widely avail- take the time to identify the key strategic decisions that able and inexpensive. A useful Web site requires the most affect their clients. Efforts such as American initiative to collect the information, the time to make Society for Testing and Materials (ASTM) Building sense of it, and the will to share it. Serviceability Tools and Methods have begun to do this Part of the problem with building delivery organiza- for programming and portfolio management, but we tions and design projects is that they represent many have seldom done this for POE. different professional cultures. Engineers tend to take a In this section we examine several strategies that technical problem-solving approach. Architects are have proven successful for beginning to create this kind often interested in form. Clients might be interested in of knowledge base. In several cases, organizations have the usability and experience of the building. Senior built on POEs that have been initiated for different pur- managers might be searching for help in setting strate- poses (or seem to us to be able to do so reasonably gic directions, whereas project managers might be easily). interested in lessons learned about specific materials or POE can be particularly successful in organizational equipment. Part of the challenge in creating any data- learning if it links strategic facilities decisions to the base or report is translating between these different pro- “key business drivers” of the client organization. In the fessional cultures, and evaluators have not always been 1970s, the U.S. Army was shifting to an all-volunteer successful at doing this. army. Potential recruits said that the aging facilities were a significant impediment to recruiting and reten- tion, and the Army sought to renovate or rebuild many Reduce Uncertainty by Upper Management’s of its buildings. To help guide the multibillion dollar Commitment investment, the Army Corps of Engineers created a Participants in POE programs report that uncertainty large program of participatory programming and evalu- about senior management’s commitment to the pro- ation, resulting in some 19 design guides (Shibley, gram is a key disincentive to participation. Sometimes 1985). the lack of commitment is seen in lack of resources, but In the 1980s, the newly reorganized U.S. Postal Ser- it is often manifest in lack of visible endorsement for vice (USPS) was losing customers to private competi- the program and lack of commitment to the two- to tors such as FedEx and UPS (Kantrowitz and Farbstein, five-year time span necessary to see results in terms of 1996). Focusing initially on the customer experience organizational learning. Savvy staff have learned not with lobbies, the USPS contracted with Min Kantrowitz to genuinely commit to the management’s infatuation and Jay Farbstein and Associates to conduct focus du jour, knowing that it will change quickly. group evaluations. This has led to a large and con- tinuing program of evaluations and design guide development. New concepts of post office design are CREATING A KNOWLEDGE BASE FOR BUILDING developed such as the retail-focused “postal store,” DELIVERY AND MANAGEMENT innovative projects are designed, the projects are evalu- Most fundamentally, organizational learning for a ated, and the ideas are refined and then incorporated building delivery organization is producing better into design guides. This program has sustained an

OCR for page 42
50 LEARNING FROM OUR BUILDINGS ongoing process of testing and refining the design history in case studies of blast, earthquakes, and other guides through evaluation and experience. More building failures, these kinds of POEs carry special recently, the USPS has de-emphasized on-site evalua- risks of becoming ways to focus (or deflect) blame. tions. Most POEs now involve having facility managers fill out relatively brief mail-out surveys. The POE man- Capitalizing on Evaluations of Innovations ager has found that the open-ended responses to the questionnaire are often most valuable in refining the Evaluation can help decide whether innovative USPS design guidelines because they are more spe- buildings or building components should be considered cific than the scaled satisfaction responses. for additional capital investment. For example, as men- tioned above, the State of Minnesota Department of Natural Resources (DNR) has recently changed the BUILDING ON EXISTING EVALUATIONS way in which it manages the environment. Rather than An organization can begin to rationalize its knowl- organizing its staff by discipline, DNR now uses a edge base by building on evaluations that occur for matrix management system where decisions are made other reasons. As the experienced evaluator Bob by a multidisciplinary group organized by ecosystem. Shibley has said, evaluations are easiest to justify if The DNR is creating new regional centers that include they bring project support, analysis benefits, and wildlife biologists, air and water specialists, and others lessons-learned benefits (Shibley, 1985). concerned with a given area. The centers are intended to encourage multidisciplinary collaboration and to be very “green.” The DNR contracted with a university Building on Diagnoses of Troubled Settings team led by Julia Robinson to evaluate two of the ini- Sometimes a building is the subject of complaints or tial projects. The team made numerous recommenda- controversy; a POE can help to diagnose the source of tions. The POE was included as an appendix for the problems and prioritize solutions. For example, the new funding request for the third center. This was the first San Francisco central library was a landmark when it time in DNR’s history that a capital request was fully opened in 1996, but it faced immediate controversy. funded on the first attempt, and the DNR was told that Some of the initial programmatic assumptions con- the POE was a major reason: it showed a high level of tinued to be debated—such as the wisdom of moving understanding of the project. This result provided an books to closed stacks to create room for computers. important incentive for DNR as an organization. How- As a result, the mayor appointed an audit commission ever, the project also raised some issues about that recommended a POE, led by architect Cynthia sustainability, and the internal staff did not feel that Ripley and including the director of the Los Angeles they had been fully consulted in the POE process. An library system. After interviewing staff and users, additional team was hired to create design guidelines observing use, and analyzing records, the POE team in close consultation with staff (M. Wallace, personal highlighted problems with way-finding, flexibility, and communication, 2000). Issues such as sustainability, public access to books. The POE recommended which are undergoing rapid change, are particular can- detailed renovations to reorganize the stacks and col- didates for “double-loop” learning where both goals lection (Flagg, 1999; Ripley Architects, 2000). The and methods for achieving them are being developed, POE is quite thorough in suggesting detailed specific if appropriate conditions are established for discussion, changes, and the basis of these recommendations could reflection, and action. potentially be turned into planning principles. Most sig- nificantly, this raises issues of programming process Focus on “Learning Moments” where the (former) library director went against the recommendations of his planning committee to reduce The Administrative Office of the U.S. Courts (AO) access to books in favor of closed stacks. It can support conducts a POE program that informs guidelines (in broader reflection about the role of libraries and physi- the U.S. Courts design guide). However, the AO has cal structures in providing information in the age of achieved organizational learning by linking the design computers. guide to a strategic learning moment in the develop- Whereas the focus on understanding problems and ment of courthouses: the negotiation between judges failure provides a clear direction to POEs and has a and the building agent (the General Services Adminis-

OCR for page 42
51 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING LESSONS FROM POE PROGRAMS: ENHANCING tration) about the scope and quality level for new court- ORGANIZATIONAL LEARNING houses. In the early 1990s, the U.S. government initiated the largest civilian construction program since We have suggested that POE has a large potential the Second World War, projecting to spend more than for lessons learned as well as for project support and $10 billion on 160 new courthouses. (The creation of analysis. Because of the historic focus of much POE new judgeships in the 1980s, concerns for increased research, the difficulty of finding resources for organi- security, and new technologies all necessitated new zational learning, and sensitivities in exposing prob- courthouses or major renovations.) However, both the lems, relatively few organizations have created effec- judiciary and the GSA were being criticized by Con- tive POE-enabled organizational learning systems that gress for creating marble-clad “Taj Mahals.” The AO include: initiated the POE program to identify necessary changes to the standards in the first edition of the design 1. monitoring changes in the internal and external guide, to defend the judiciary against attack by docu- business environment, menting the efficacy of the design standards, and to 2. establishing performance goals based on internal inform the negotiation about issues such as the dimen- and external influences, sions and materials of courtrooms and chambers. 3. assessing performance, Information from POEs was also used in training work- 4. interpreting and discussing the implications of shops for judges and staff who were becoming involved results, in new courthouse design and construction. This pro- 5. consolidating results into an organizational gram is run by the AO, but the design guide is actually memory, created and vetted by a committee of the Judicial Con- 6. widely distributing findings and conclusions, ference, the group that sets broad policy within the fed- 7. creating a culture that allows the organization to eral judiciary. This program is quite unusual: it is the take action on the results, only case that we are aware of where a POE and design 8. taking action based on organizational learning. guide are developed by a client organization that does not build its own buildings. However, based on a number of successful exam- The Minnesota Department of Natural Resources ples, we suggest the following strategies for creating project also focuses on strategic moments, especially the conditions for learning: the approval of the funding package by the key legisla- tive committees. • Create opportunities for participation The focus on strategic learning moments is similar • Add incentives to Shibley’s reminder that information is most likely to • Remove disincentives be used when it is asked for (Shibley, 1985). A strate- • Provide access to information gic learning moment is a critical time when informa- • Provide upper-level management support tion or a POE can help resolve a problem or issue that is of considerable importance to the participants. The Successful organizations have also used several focus on learning moments can also be used in devel- strategies for creating knowledge: oping policy documents or targeting POEs toward decisions. • Clarify key strategic choices • Build on existing evaluations Creating Organizational Memory for Precedents • Focus on strategic moments • Record precedents A key part of organizational memory is simply knowing what the organization has done, but few POE The lessons from these 18 POE programs, which programs have been linked to recording and analysis of influence billions of dollars of construction, suggest precedent. There is a real opportunity to link evalua- that the solution to creating a lessons-learned program tion to a record of past projects. This record can include is partly technical, such as using information tech- simple plans and photos and some analyses of cost, nology to reduce costs in gathering and distributing size, and materials. These descriptions can be linked to information. At its core, however, the problem is orga- evaluations.

OCR for page 42
52 LEARNING FROM OUR BUILDINGS nizational—creating a setting in which decisions can ongoing projects at the Office of Overseas Buildings be evaluated, discussed, and learned from. Operations. Prior to working with the Department of State, Mr. Rosenheck was in private practice. He has worked for architectural firms, a construction firm, and ABOUT THE AUTHORS taught at the School of Architecture and Urban Plan- Craig Zimring is a professor of architecture and of ning at Howard University. He holds a bachelor of psychology at the Georgia Institute of Technology. In architecture degree from Howard University, a his teaching, writing, consulting, and research he has master’s degree in architecture and environment- developed methods, procedures, and concepts for the behavior studies from the University of Wisconsin- evaluation of buildings, including comprehensive Milwaukee, and is a licensed architect in the District of studies of building types such as healthcare facilities, Columbia. jails and prisons, courthouses, and specialized studies of wayfinding, security, stress, and other issues. REFERENCES Dr. Zimring has focused on how social, organizational, and behavioral information can be incorporated into Argyris, C. (1992a). On Organizational Learning. Cambridge, Mass: Blackwell. design and decision making at a variety of scales, from Argyris, C. (1992b). Teaching smart people how to learn. In: C. Argyris a freshman design studio to the $4.5 billion California (Ed.) On Organizational Learning (pp. 84-100). Cambridge, Mass: prison development program, the $6 billion French Blackwell Business. Argyris, C., and Schon, D. (1978). Organizational Learning. Reading, Universities 2000 program, and the $1 billion annual Mass: Addison-Wesley. construction budget of the California Department of Baird, G., Gray, J., Isaacs, N., Kernohan, D., & McIndoe, G. (Eds.). (1996). General Services. He has worked in the design studio, Building Evaluation Techniques. New York: McGraw-Hill. Bechtel, R. (2000). Personal Communication. lectured to facility managers, written in the popular and Bordass, W., and Leaman, A. (1997). Future buildings and their services: professional press, served as a consultant and directed strategic considerations for designers and clients. Building Research research projects for AT&T, U.S. Department of State, and Information 25(4): 190-195. U.S. General Services Administration, the Administra- Campbell, D.T. (1999). Social Experimentation. Thousand Oaks, Califor- nia: Sage Publications Inc. tive Office of the U.S. Courts, U.S. Department of Cohen, R., Bordass, W., and Leaman, A. (1996). Probe: A Method of Inves- Transportation, Ministry of Education of France, and tigation. Harrogate, United Kingdom: CIBSE/ASHRAE Joint National many others and served on the boards of several pro- Conference. Davis, G., and Szigeti, F. (1996). Serviceability tools and methods (STM): fessional organizations including the Environmental Matching occupant requirements and facilities. In: G. Baird, J. Gray, N. Design Research Association and the Justice Facilities Isaacs, D. Kernohan, G. McIndoe (Eds.) Building Evaluation Tech- Research Program. Dr. Zimring was a distinguished niques. New York: McGraw-Hill. Flagg, G. (1999). Study finds major flaws in San Francisco main library. senior visiting fellow at the Centre Scientific et Tech- American Libraries 30(9):16. nique du Batiment, Paris; he has received awards from Friedmann, A., Zimring, C., and Zube, E. (1978). Environmental Design the American Society of Interior Designers and the Evaluation. New York: Plenum Press. National Endowment for the Arts Design Research Gregerson, J. (1997). Fee not-so-simple. Building Design and Construction (August): 30-32. Recognition Program. He holds a bachelor of science Horgen, T.H., Joroff, M.L., Porter, W.L., and Schon, D.A. (1999). Excel- from the University of Michigan, and a master of sci- lence by Design: Transforming Workplace and Work Practice. New ence and a Ph.D. from the University of Massachusetts York: Wiley. Huber, G.P. (1991). Organizational learning: The contributing processes at Amherst. and the literature. Organization Science 2: 88-115. Kantrowitz, M., and Farbstein, J. (1996). POE delivers for the Post Office. Thierry Rosenheck is a project manager at the U.S. In G. Baird, J. Gray, N. Isaacs, D. Kernohan, G. McIndoe (Eds.) Build- ing Evaluation Techniques. New York: McGraw-Hill. Department of State, Office of Overseas Buildings Kernohan, D., Gray, J., and Daish, J. (1992). User Participation in Building Operations (OBO), and has been working on embassy Design and Management: A Generic Approach to Building Evaluation. rehabilitation projects in New Delhi, Beirut, Tel Aviv, Oxford: Butterworth Architecture. and Jerusalem since 1999. He has developed a service- Leaman, A., Cohen, R. and Jackman, P. (1995). Ventilation of office build- ings: Deciding the most appropriate system. Heating and Air Condi- ability profile of user generic requirements for new tioning (7/8): 16-18, 20, 22-24, 26-28. chancery office buildings using the ASTM Standard on McLaughlin, H. (1997). Post-occupancy evaluations: “They show us what Whole Building Functionality and Serviceability and works and what doesn’t.” Architectural Record 14. Preiser, W. F. E. (1994). Built environment evaluation: Conceptual basis, with the Centre for International Facilities. He has benefits and uses. Journal of Architectural and Planning Research coordinated POE and serviceability input with other 11(2), 92-107.

OCR for page 42
53 POST-OCCUPANCY EVALUATIONS AND ORGANIZATIONAL LEARNING Preiser, W.F.E., Rabinowitz, H.Z., and White, E.T. (1988). Post-Occupancy Shibley, R. (1982). Building evaluations services. Progressive Architecture Evaluation. New York: Van Nostrand Reinhold. 63(12): 64-67. Preiser, W.F.E., and Schramm, U. (1997). Building performance evaluation. Shibley, R. (1985). Building evaluation in the main stream. Environment In D.Watson et al. (Eds.) Time-Saver Standards (7 ed., pp. 233-238). and Behaviour 1985(1):7-24. New York: McGraw-Hill. Vischer, J. (1996). Workspace Strategies: Environment as a Tool for Work. Raw, G. (1995). A Questionnaire for Studies of Sick Building Syndrome. New York: Chapman and Hall. BRE Report. London: Construction Research Communications. Watson, C. (1996). Evolving design for changing values and ways of life. Raw, G. (2001). Assessing occupant reaction to indoor air quality. In: J. Paper presented at the IAPS14, Stockholm. Spengler, J. Samet, and J. McCarthey (Eds.) Indoor Air Quality Hand- Watson, C. (1997). Post occupancy evaluation of buildings and equipment book. New York: McGraw-Hill. for use in education. Journal of the Programme On Educational Build- Ripley Architects. (2000). San Francisco Public Library Post Occupancy ing (October). Evaluation Final Report. San Francisco: Ripley Architects. Zeisel, J. (1975). Sociology and Architectural Design. New York: Russell Schein, E.H. (1995). Learning Consortia: How to Create Parallel Learning Sage Foundation. Systems for Organization Sets (working paper). Cambridge, Mass: Zimring, C.M., and Reizenstein, J.E. (1981). A primer on post-occupancy Society for Organizational Learning. evaluation. Architecture (AIA Journal) 70(13): 52-59. Schneekloth, L.H., and Shibley, R.G. (1995). Placemaking: The Art and Zimring, C.M., and Welch, P. (1988). Learning from 20-20 Hindsight. Practice of Building Communities. New York: Wiley. Progressive Architecture (July), 55-62.