Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendix D How the Committee Conducted Its Study The Committee to Study the Impact of Information Technology on the Performance of Service Activities was composed of knowledgeable infor- mation technology (IT) executives from major service industries, nationally recognized macroeconomists concerned with productivity in services, ex- ecutives from hardware- and software-producing groups, and representa- tives of government, labor, industry, and research. In addition, the commit- tee sought input from chief executive officers, senior IT executives, and people with strong operating and financial evaluation experience in service industries, and it attempted to draw on the existing literature when appro- priate. The charge given to the committee was to examine the impact of IT on productivity in the service sector. However, after examining the macroeconomic data, existing surveys and publications on how IT has been used in services, the taxonomy of IT as it relates to performance, and the management practices of successful and less successful users of IT in services, the committee deter- mined that the proper issue for investigation was not determining the impact of IT on standard measures of productivity in services, but rather understand- ing the impact of IT on the overall performance of service activities. As discussed in the main text of this report, many dimensions of perfor- mance are difficult or impossible to quantify, let alone to collect reliable data on. Nonetheless, the committee chose to regard performance broadly, including such considerations as fulfilling mandated requirements, reducing costs, creating new products, increasing the quality or timeliness of output, 247
248 INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY avoiding loss of competitive position, developing new strategic capabilities, generating greater flexibility of infrastructure, improving capacity to handle complexity or customization, and so on. Furthermore, it included all as- pects of IT investments and expenses (hardware, software, training, support, and operating) in its thinking, despite the fact that data on overall industry expenditures for software, training, support, and systems operations are generally difficult to obtain in any systematic manner. The committee conducted analysis at the national, industry, enterprise, and activity levels. Thus it did not address international, individual, or group issues in any significant detail. Nor did it address small companies, although financial data on smaller companies were included in the data that supported the industry- and national-level analyses. To conduct its national (or macroeconomic) analysis, the committee in- vestigated the current data relevant to evaluating how IT has been exploited in the service sector. Much of this data is incomplete or inadequate in ways discussed in the text, and in any event such data cannot establish cause-and- effect relationships between investments in IT and the performance measured. Once the committee went beyond the national level, the lack of system- atically collected data became quite apparent. The committee therefore undertook to interview a large number of outside sources. The committee did not attempt to develop new databases, a task that would have been far beyond its expertise or resources to undertake. Instead, it relied on experts, both its own inside the committee (through committee deliberations) and others outside the committee (through interviews), as well as on other stud- ies when available. The industry-level analysis was primarily for context-setting. The com- mittee drew on secondary literature and its own experts in the air transport, communications, commercial banking, insurance, retail-wholesale trade, and health care industries. These industries were selected because they had a wide range of measurable improvements in productivity, high levels of in- vestment in IT, and different contexts in which to introduce IT; reflected important implications for employment and public policy; and exhibited extensive variety in the sophistication and scale of the IT installed. Good data were available for some of these industries as well. At the enterprise and activity levels, the committee sought to shed light on the causal relationships underlying observed trends rather than merely to analyze correlations of data. The enterprise and activity levels are crucial for understanding these relationships, because decisions about investment in IT and use of IT are usually not made at the industry or national level; most are made by executives within the enterprises that employ them. Moreover, resources and appraisal procedures for investing in IT are generally devel- oped at the enterprise and activity levels, and appraisals of investments, IT strategies, and system paybacks ultimately occur at these levels.
APPENDIX D 249 USE OF INTERVIEWS APPROACH AND LIMITATIONS To understand the causal relationships between IT investment and use and the performance of service activities, the committee drew extensively on a set of interviews it conducted with senior executives from firms in the service industries. The purpose of these interviews was to seek insights (both from the past and present) into patterns of behavior that could help explain why some companies or industries were or were not-realizing more payoffs from IT. Thus the committee conducted these interviews and analyzed the questionnaires in an exploratory vein, rather than searching for high statistical reliability. To select appropriate interviewees, the committee established a list of ten leading companies in each chosen service industry (financial services, transportation, retailing, wholesale distribution, communications, professional services, and health care). These companies were selected on the basis of size, rate of growth, profitability, recognition for innovation, and recogni- tion as major users of IT. These firms seemed most likely to have given systematic thought to the problems of assessing and managing IT systems and to have encountered the full range of problems in both activities. Based on the committee's access to senior management, four companies in each group were then selected for contact. Such access was critical in order to obtain reliable responses concerning each company's practices, experiences, and results. Executives from two of these four top-performing firms were inter- viewed in each industry, in all a total of 80 individuals at 46 companies. All together, these firms had nearly $500 billion in revenues in 1991 and employed nearly 3 million peopled Because the cooperation of key indi- viduals was crucial, access to the firm was sought at the highest possible level. In each company, interviewers attempted to contact the chief execu- tive officer or chief operating officer, the senior executives for IT, a major operating officer, and the chief financial officer. The individuals inter- viewed are listed in Appendix E. All quoted statements in this report were verified by those who made them as being correct at the time of the inter- views. These interviews were conducted in a semistructured manner, and they focused on the following concerns: 1. How companies decided to go ahead with a particular IT project; 2. How companies evaluated IT projects post-investment (if at all); 3. 4. 5. Performance measurement and changes in reward systems made to The results of such evaluations; The impact of IT's use on organizational structures; exploit IT effectively; 6. Causes of less-than-satisfactory returns from investments in IT;
250 INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY 7. How companies evaluated intangibles like quality, flexibility, re- sponse times, or complexity effects; and 8. What executives perceived to be the most important impacts of IT's use in their industries. The introductory information sent to interviewees prior to the interview is given in Box D.1, and the interview guide for the interviewers is pro- vided in Box D.2. To obtain more standardized data, a written question- naire (Figure D.1) was submitted to each participating company after the oral interviews. Seventy-nine percent of the questionnaires were returned. Summaries of the responses to the questionnaire appear in Question Boxes 1 through 7. In addition to the qualifications noted in Chapter 3, some further limita- tions of these interviews must be noted. Although each interview was conducted to gain understanding about both failures and successes in imple- menting IT in services, those interviewed understandably focused more readily on the successes of their firms, though most were willing to discuss some of their failures or problems. The selection of companies "near the top" of their industries and avowedly major users of IT resulted in a greater focus on the "winners" or "survivors." In addition, the focus on large firms- because of limits on committee resources- left a large gap in the comm~ttee's understanding of how small and medium-size firms use and manage IT. This is a gap that should be addressed in future research. Although the data that emerged from the committee's interviews are primarily qualitative, such data help to interpret the quantitative macroeconomic data that do not adequately represent the wide range of impacts that using IT has had on the performance of firms in the service sector. Thus, semistructured interviews conducted by the committee were useful in defining (not measur- ing) parameters, in interpreting correlation data, and in providing support for causal hypotheses. Since the data from the interviews were not col- lected utilizing a statistical sample, common themes are reported in the text using qualifiers such as "virtually all" (90 percent or more), "a great major- ity" (70 percent or more), "most" (50 percent or more), "less than half,' (30 to 40 percent), "a few" (10 to 30 percent), "a very few" (less than 10 percent), and so on. Conclusions are limited to the respondents' practices and views. NOTE 1Revenues and employment figures for 1991 were gathered from a search of the Dunn and Bradstreet Market Identifiers database available through DIALOG Information Services. For a small number of companies, the 1991 data were not contained in this database, and 1990 figures were used. Thus the $500 billion in revenues and 3 million people employed should be regarded as rough estimates.
APPENDIX D 251 Question Box I Does your company have an overall IT strategy? Yes No Only recently 70% 1 3°/0 ~ 7% What is the time horizon of the strategy' Year 2 Years 3 Years 4 Years 42% 4% 5 Years in Years 4% 12% 38% 0% Does the decision process far IT d'Rer from other investments? Yes No 37% 63% Do the payback criteria differ from those for other Investments? Yes N4 3 1 °% 69% Observations and Additionai Commend ~ Time horizons of IT strategy are around 3-S years, generally with updates annually through capital budgets and specific operational plans keyed In at six-month to l-year Intervals. · IT investment processes and payback criteria are essentially the same as those for other investments. ~ In discussion with interviewees, a majority described long-range plans for IT investment and insmIlation, rather than competitive position- ing strategies.
252 Question Box ~ Do you routinely make P~-proje" evaluations of IT Investments? Type of IT Investment Yes No Occasionally Cost Reducing 83%7% f0 low Product 6411 25 Basic Infrastructure 6717 17 Desktop IT 3340 27 ldanclawd Systems 4747 7 if evaluated, what Jo the returns tend to be? Type of IT investment INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY Equal to Comparable Indexers Negative Cost at Marginal High to Other manage Return Capital Return Return lovestments Cost Reducing 0 0 0 f3 48 39 Mew Product 5 0 5 16 42 32 Baste Infrastructure 11 16 16 11 16 32 Desktop IT 0 15 8 8 23 46 Mandated Systems 8 f7 17 17 17 25 N.B. Percentages are based on number of responses to individual questions. Rows Nor not add to 100% due to rounding.
APPENDIX D Question Box 3 I Do you routinely make POST-project evaluations of IT investments, Type of IT lovestment Yes No Occasionally Cost Reducing 39% 32% 29 New Product 36 36 28 Basic Infrastructure 22 52 26 Desktop IT 17 48 34 Mandated Systems 16 64 20 If evaluated, what do the returns tend to be? Type of IT investment Equal to Comparable Indeter- Negative Cost at Marginal High to Other magnate Return Capital Return Return lovestments Cost Reducing 0% 0%0% 28%22% 50% low Product 8 08 825 50 Basic Infrastructure t4 1414 ~t4 36 Desktop IT 15 l50 158 46 Mandated Systems I I I I0 2211 44 NIB. Percentages are based on number of responses to individual questions. Rows may not add to 100°% because of rounding. Observations and Additional Comments ~ Poster evaluations are less universally pursued than pre~pr~ect assessments. ~ Cost frequent post-investment eYaluat~on$ are performed on cost re- duction and new-product programs. · Post-pro~e~t audits tended to show higher audited returns than those pro}eeted, except in the case of desktop systems. 253
2~ OUTBOX BOX 4 How do you "count for different costs In your IT evaluations! Capitalized Expensed Hardware 89% 1 lo Software 34% 66% n~g ^~ i id ~ rug Cogs 17% 8]% Obsessions and Consequences A subs~ndal poodle ~ IT costs do not show up in 'investment" azures.
APPENDIX D Question Box 5 Overall, how, i! at all, has use of IT influenced the organization structure of your company' Increased [:3ecreased halo change 255 Spans of control 68% 14% 18% Levels of hierarchy 4% 63% 33% Centralization 46% 24% 30% t)ecentraTization 44°%O 26% 30% Ouch Some Slight None Did IT Facilitate organizational flattening, postprojact? 27% 50% 12% 12% Did IT facilitate significant use of self-directed teasels' 8% 58% 19% 15% Were performance measures changed? 11% 56% 4% 30% Were compensation systems changed? 4% 33% 15% 48% N.B. Rows may not add to 100% because of rounding. Observations and Additional Comments ~ The mix is about even between increased centralization (usually of data centers or databases3 arid Increased decentralization (usually of operations). ~ Most report increased spans of control, flatter organizations, more use of self-directed teams.
256 i INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY Question Box ~ Are users involved in the design of IT projects? Rarely Sometimes Usually Always 7% 7% 33% 53% How were they involved? Specifying In System Interactive Only Design in Design Interactive Throughc: ut 69% 8% 6% 16% Are external customers involved In design of IT? Specifying In System Interactive Only Design in Design 46% 24% 1 8% hI.B. Rows may not add to ~00% because of rounding. Observations and Additional Comments Interactive Throughout 12% · Users are involved in desigr~ of IT projects at design and ampler mentation levels. ~ External customers are often Involved, but less frequently
APPENDIX D Question Box 7 What performance metrics are used' Neasu red Neasu red at Internally Customer Level Response times to customers 64% 50% Quality of performance 57 50 financial payback to company 82 id Financial benefits to customers 43 43 internal engineering metrics 54 4 Other important metrics 7 11 N.B. Percentages are based on number of responses to individual ques- tions. Columns are not mutually exclusive. Observations and Additional Comments · About half of the respondents reported using format techniques to monitor perceived quality from their customers, viewpoint. ~ Formal metrics to measure the quality of service outputs are almost always engineering metrics (such as system response times, com- puter availability percentages, cycle times for operations, etc.) or cus- tomer survey data (expressing customers' view of the services of the company) that cannot be converted into financial measurements of out- put quality. 257
258 INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY BOX D. 1 Introductory Information Sent to Interviewee Prior to Interview 1. Purpose of interview and follow-up Questionnaire: Numerous publica- tions have suggested a seeming paradox. They contend that the relatively high levels of investment in information technologies (IT) in services have led to little or no measurable productivity increases. This may be a problem caused by difficulties in measurement. intended to understand better: (a) to what extent companies have. or have not, fared better and improved performance levels through use of fT, (b) how con~panies determine how much to spend on IT and set metrics for anticipated performance, (c) how companies build IT into strategic plans, (d) how companies evaluate benefits from their IT invest- ments, (e) how these evaluations compare to anticipations, (f) what man- agement techniques seem to work well or to cause problems in achieving productivity from IT, (g) what the most important impacts have been of fT on the organization's structure, management style, and industry com- petitiYe Position. This questionnaire is 2. Definitions used in Interview and Questionnaire: Formation technolo- gies (IT) include ~e hardware, software, and implementation processes I far a cluster of technologies that provide transactions-oriented and ~nter active computer-based capabilities for: collection, input, processing, eom munic~ion, storage, retrieval, and presentation of data and information, Respondents should consider the impacts of information technc>logies at several dine rent levels: (a) strategic uses which change the nature and purposes of the entire enterprise, (b) systems uses which coordinate orga nizational units and databases to perform overall enterprise functions more effectively or e~ciently$ (c) ~erat~anal uses which enable managing of single functions or units for greater efficiency or higher value-added, (d) unit applications which improve performance at a single workstation. Services include all items sold in trade which are neither products nor constructions. The services industries include: financial services, re- ta~ling, wholesaling, transportation, communications, public utilities (such as sewer, garbage collection, electric companies, etc.), health care and delivery systems, entertainment, ~rofes$ional services (such as account- ing, legal, consulting, design, research, etc. firms), computer software' enterta~nmer't, and so on. Services activities (which Occur both widen the "product industrics'$ and the "sewices industries") may include logistics, applied research, process design' product design, plant engineering, ware- housing, market research, marketing, advertising, sales, distribution, re- pair, post-sale services, legal, public relations, accounting, personnel, regulatory affairs, maintenance, basic research, or finance functions. Continues
APPENDIX D 259 3. Rules of the interview: All interviews will be taped for accuracy All information will be held confidential unless specifically released by re- spondent. All quotes or paraphrases will be cleared in context with the respondent before publication. All publications will be sent to respon- der~ts free at the earliest date Hey are released for distribution. Inter- views do not seek proprietary data, but useful patterns and insights haY- ing to do with measuring and improving the performance of IT in services activities. If copies of any relevant reports or company documents are available, they would help ensure accuracy. 4. Outline of Interview: We will be focusing on four central questions For each of these there will be some aera''eo questions And ~r'`e~-Yi':w=' will use to flesh out and make your responses compatible with those of other respondents. These questions include the following: , . . ~ .~ ~ _ _ _ 1_ _:~_1 ~ -lab_ ;~-t!~#iQ~ A. Has your company made any major studies of the performance of its information technology (IT) inYestments' Please describe the breadth of the studies, levels of IT included, or if performed on individual projects, which projects were included What were the results7 What methodologies and metrics did you use? B. How was the technology developed? In~rnaJly? Externally' Joint ventures, other approaches, How was the IT function orga- r~ized at the timer What were the most important problems en- countereJ In Introducing the technology? What have been, or are anticipated to be, the most important changes resulting from the new technology' C. How, If at all, have you used IT to affect Me organkadonal structure of your company? Spans of control' Levels of hierarchy, centralization or decentralization of decisions? How has the tech- nology affected your competitive edge in some aspect of your ~r~dus- try? Flow has it influenced the use of self-d~rected teams' Bound- aries of units? Horizontal relationships and communications' Strategic alliances, and so on? D. In light of the above what would be your overall evaluation of the way IT investment enabled your Lombardy to improve services performance? What are the most useful Insights or patterns that analysts of past projects offers for Azure IT investments?
260 INFORMATION TECHNOLOGY IN THE SERVICE SOCIETY BOX D.2 Interview Guide for Interviewers Has your company made any major studies of the performance of its mformat`on w~hno}ogy pT) investments? If SO$ when? Please describe the breadth of the study, levels of IT Included, or if performed on individual projects, which projects were included? Were they successful' What performance measures were used' What was the general level of IT investment Noised? in software, Hardware? Over what time periods were results measured? Why? What were the general conclusions con- cerning performance payout Could this be measured for discrete projects? Cumulative gains? What were the main sources of gain' What were the main causes of greater or less than anticipated benefits? What were the most important problems encountered in measurement? Idow did you deal with these' What were the major intangibles considered' How were they evaluated? Quality of outputs? Faster response times' Flex- ibility potentials' Greater knowledge about customers' Were software costs (development and licensing) Included' Training and personnel de- velopment? If so, what were theft What would you say was the preci- sic~n At/- °~0) of the measurements? Please pick two or three major projects (in terms of cost, impact, or geographical scope) concerning which you are most Familiar or for which such evaluations may have been undertaken. If possible, please choose some large-scale projects which forwent well'' and some which i'did not go well." Far each of these projects, what were the main goals of the project? What nutrias were used in making the decision? What is the decision process for IT investments and how does that differ from other types of investment decisions2 How is IT integrated into strategic plan- ning7 How was the actual evaluation carried out' By whine' Plow were various critical ~:tors balanced or weighted' What was the approximate size of the Investment' In software7 In hardware' What was the cost vs. other information technology investments then installed' Was this a new actiY'ty for the company? The replacement of earlier automation' Replacement of people? Was there a specific investment return (hurdle ramp the project had to meet? I-low did this compare to other IT and non-lT hurdle rates? Were other criteria stated explicitly in advancer FLOW well or badly did the project meet initial expectations, goalst 2. How was Me technology defied? Through internal groups? extent sive outsourcing? Off-the-shelf purchases? joint Ventures' Who was involved at each stage' When were users involved? What was the nature of the development' How was the IT Unction organized' Where are standards set? Who had final decision authority' What important changes in organization or technical infrastructure were necessary to develop, accommodate, or exploit the new technologies? What were the Continues
APPENDIX D 261 ; most important other problems encountered in introducing the technol- ; ogy' What have been, or are anticipated to be, the most important unforeseen changes resulting from the new technology? Did the technol ogy change economies of scale? Economies of scope? From the opera tions level to the systems level, or vice versa' How were the effects measured? What time span did it take to introduce the project (from specification to outcomes)? Was the project organized as a separate division' Cross~functional team effort? Outsourced? What portions were expenses or capitalized' Why? 3. How, if at all, has the use of information technology offered the organiza- tional structure your company? Spans of control' Levels of hierarchy? Locus of control? Centralization, decentralization of decisions? Have there been shifts in staffing away from certain Unctions and toward oth- ers? Has the technology enabled your enterprise to shorten its response times? How much? What are the organizational, control, and motiva- tional impl'~ations of these changes7 How have they affected the use of self-directed teams' Boundaries of units? Horizontal relationships and communications? Strategic alliances' How has use of the technology affected your methods of performance measurements Motivation sys- tems? Control systems' What are the most important (or likely) ~m" pacts of the new technologies on job skills, training requirements, wage structures? How does your company approach the issue of 'managing and developing people as in"lle~ual assets"' Does the technology allow you to obtain a sustainable competitive edge in some component of your industry' Plow? What have been the major spinoffs or totally new directions new information technologies have enabled for the company' To what extent were these anticipated' tJnanticipated? 4. In light of the above, what would be your overall evaluation of the way IT investments enabled you to improve services performance in your company' Why' What are the most useful ~r~s~ghts or patterns that analysis at past projects offers for Future IT ~nvestn~ents'
262 - o ~ 4) ~ E o lo IS Ct ILL US LIZ O - LL LLI O ~Z Zig Of _ it ME a, J to Liz go o - o IL 3 Lo Is .°6 B A: d - _ 8 .~ ~ o Cal O.> ~ (D .= - ID o (-D >I o U. :i: . ~[ I ~ / - | L L,~ 5 ~ , 3 i] 5~ ~ ;~ ; 98~ 3 .,o c ~ ~ E E ~ E| E Cal
263 3 ~ ~ ~ ~ ~ ~ ~ o ~ r 0 El ado 2 Ott ] ~ ~ ~ F° -~< ~ >I ~ ~ hi| ~ t3 E ~en= e e a, ~ ~ El EM = ~ ~ ~ ~ AL E 31 ~2 ° ~ ce 0 so E en i] IS E >- 1 3 ~A; a UD ~
264 Cat Cat CO Q o O (D c: 3 C 3 C C o ._ CS o TIC ED o 0 in - C' - - AD ED ~5 a, ED - 0 ~3 CD ~ 1 1 1 1 03 A== o2~- c c ~ ~ o o o o z z z z 1 1 1 1 c - ~= ~ in in cn in ~D E E E E ~n ~ ~ ~ I I I I E -1 -1 -1 ~1 , - , . i~ ~ o C, ._ :t 1 1 1 ~ -~- ~ ~E ra- 0 cn ~ .~ =8 C ~ - C" O _ ~ ~ ~ tt' o 3 ~ E O ~ ~ (D ~ o ~ ~ E ~ - o ~ ~ ~ ~8 t, ~at E $: ~ ~ Q 8 _ ~ 1~ ~ (D C _ ~ O - - ~ ~ O S - O O ~~ ~n cn ~ C cn ~ ~ - ~> CC', 0 .> C 0 030~g 03~g ct ~C.=Q 0 O E O ~o CO cn O CC O ~D O O ~ O cn cn ~- Q - o U) (D > o ~D 3 ~D C~ ~D ._ - ~D E o ~: ~D C~ - s - ~D 3o I
265 - ~_ I I I I I I E 1111111 111111 ~11111 ~11111 1~11111111 1,) 1111 1 '<>D ~ E . ~ ~ ~ = ~ ~ Jan i ~ ~ ~ ~ ~ it` ~c~cnO A moo,, O ~ O ~tD O 3 (D O · ~ CD a) ~