Cover Image

Not for Sale



View/Hide Left Panel
Click for next page ( 215


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 214
APPENDIX D: ASSESSING VALUE ADDED TO CUSTOMERS This appendix describes in economic terms procedures for calculating value added to customers of maintenance attributes. For the most part, economic value to maintenance customers can be conveniently grouped into the following types: Avoided user costs, Avoided life-cycle costs, and Avoided external costs. Customers are willing to pay to avoid user costs, life-cycle costs, and external costs. Hence, the willingness to pay is also an important measure of economic value. AVOIDED USER COSTS There is a well-established convention among transport economists that road user costs should be calculated by the following formula: User Costs = Travel Time Costs + Vehicle Operating Costs + Accident Costs. In customer-driven benchmarking, it may be desirable to employ avoided user costs or some component of user costs as a customer-driven outcome measure. Avoided Travel Time Costs In a project to develop a prototype decision support system for customer-driven benchmarking, Minnesota DOT (MnDOT) examined how its maintenance activity of removing obstructions in the roadway (e.g., spilled boxes, fallen branches) related to avoided travel time and accident costs. Consultants to MnDOT applied standard techniques of highway capacity analysis to calculate average travel time delay per vehicle experienced by a motorist as a function of the following: Capacity of the road in vehicles per lane per hour; The duration of the obstruction, which is equivalent to the time it takes maintenance personnel to remove the obstruction from the road; or The degree to which an obstruction reduces the highway capacity. 221

OCR for page 214
Appendix D: Assessing Value Added to Customers Figure D-1 shows the relationship between average delay per vehicle as a function of the capacity, q = 1,200 vehicles per lane per hour, and the percentage reduction in capacity of the road, R. Once the average delay per vehicle is obtained, the mix of cars and trucks is estimated, and the average occupancy rate is determined, then it is possible to apply an estimate of the value of travel time to each driver and passenger in order to estimate the total avoidable road user costs in economic terms. 4 Q = 1200 vphpl 3.5 R : To Capacity Reduction Average Delay per Vehicle (hrs) 3 0% 2.5 10% 20% 2 30% 1.5 50% 1 67% 90% 0.5 100% 0 0 0.5 1 1.5 2 2.5 3 3.5 4 Duration of Obstruction (hrs) Figure D-1. Average Delay per Vehicle versus Duration of Obstruction for Traffic Volume = 1,200 Vehicles per Hour per Lane1 Equations such as these can be used to calculate avoidable road user costs. Indeed, many management and decision-support systems include estimation of road user costs, and the algorithms in those systems potentially can be used to develop performance measures for benchmarking. The Pontis Bridge Management System calculates travel time costs, vehicle operating costs, and accident costs as a function of deficiencies in clear deck width, vertical clearance, and load capacity of bridges. Pontis calculates the 1 Alfelor, R. M., W.A. Hyman, and G.R. Niemi (1990). "Customer-Oriented Maintenance Decision Support System: Developing a Prototype," Transportation Research Record 1672, Transportation Research Board of the National Academies, Washington, DC, pp. 110. 222

OCR for page 214
reduction in these costs resulting from various improvement options, such as strengthening and widening bridges. Similarly, the Highway Economic Requirements System (HERS), which the federal government uses to estimate national highway needs for the U.S. Congress, calculates all three types of user costs as a part of a benefit-cost calculation. AVOIDED LIFE-CYCLE COSTS Customers who are conscious of the taxes and fees they pay to maintain and improve roads would prefer not to pay more taxes and fees if they can avoid it. Therefore, an important performance measure is avoided life-cycle costs of assets. Life-cycle costs are defined as the stream of future costs an agency incurs over the life of an asset: Initial or startup costs, Recurring or periodic costs, Sporadic or infrequent costs, and Salvage and disposal costs. For long-lived assets, under certain circumstances you can assume they will remain in service in perpetuity and you can ignore salvage and disposal costs. However, if you have reason to pay close attention to environmental ramifications of disposal and reuse, you may wish to account explicitly for these end-of-life costs. Life-cycle costs can be derived from a life-cycle activity profile in which you identify for each year into the future each maintenance, rehabilitation, replacement, or reconstruction activity that will occur. From this life-cycle activity profile, you can identify a future stream of life-cycle costs by determining the cost of each activity. If the life of a particular asset will end during the calculation horizon, you will have to determine the nature of the replacement asset, identify its life-cycle activity profile, and append it to the first one. AVOIDED EXTERNAL COSTS A good example of external costs that can be avoided by road maintenance is the infestation by noxious weeds of farmland adjacent to a highway. Certain types of noxious weeds are destructive to crop yields and can significantly reduce the income of farmers. In fact, there is literature one can draw upon to estimate the reduction in crop yield of different types of crops as a function of the infestation of different types of noxious weeds. 223

OCR for page 214
Appendix D: Assessing Value Added to Customers Noxious weed control helps avoid infestation of property on neighboring roads. To calculate the avoided costs of noxious weeds caused by noxious weed control, you need to make the following calculation: Determine what type of noxious weeds are currently in the right-of-way; Assess the extent and severity of the weed in the right-of-way; Assess the extent and severity of infestation of noxious weeds in adjacent farmland; Assess, based on the literature, the reduction in crop yield caused by the infestation, assuming no control of the noxious weed; Calculate the reduction of income caused by the predicted infestation of the noxious weed, assuming no control of the noxious weed; Determine what percentage of the reduction of income to farmers can be avoided by controlling the noxious weed in the right-of-way; and Apply the percentage to the estimated reduction in income to farmers to calculate the avoided loss in farm income. One could potentially make similar calculations regarding other external side effects-- for example, the effect of salt damage to vegetation outside the right-of-way. One can also, in theory, make estimates using well-known statistical methods (e.g., regression) for estimating how changes in certain types of maintenance affect property values. For example, failure to remove graffiti from noise barriers is likely to reduce property values of adjacent property. DISCOUNTING Three different types of economic costs have just been discussed: avoidable user costs, avoidable life-cycle costs, and avoidable external costs. Estimated avoidable costs do not all occur at the same time, but rather at different times in the future. Economists have a way to put benefits and costs that occur in the present and at different times in the future on an equal footing. The method is called "discounting." Discounting is based on the idea that $1 in your hand today is worth more than a $1 you receive a year from now. An important reason is that there is some amount less than $1 that you could put in a bank or in some other investment at the prevailing interest rate or rate of return and earn $1 dollar in the future. The prevailing rate of return you can earn on your money is called the discount rate, r. It is useful to think of the discount rate as the opportunity cost of investment--that is, the rate of return you can earn on your next best use of funds. 224

OCR for page 214
To determine the present worth of an amount of a cost or of a benefit that will be incurred n years in the future, you multiply that amount by the following discount factor: 1/(1+r)n . The following is an example of how to calculate the present worth--discounted costs--of a future stream of benefits involving a future avoidable cost of $1,000 per year by using a discount factor of r = 0.07. The example reveals that a stream of avoidable future costs of $1,000 per year totaling $10,000 over 10 years has a present worth or discounted present value of $7,023.58 Undiscounted Discount Discounted Year Costs ($) Factor Costs ($) 1 1000 0.9346 934.58 2 1000 0.8734 873.44 3 1000 0.8163 816.30 4 1000 0.7629 762.90 5 1000 0.7130 712.99 6 1000 0.6663 666.34 7 1000 0.6227 622.75 8 1000 0.5820 582.01 9 1000 0.5439 543.93 10 1000 0.5083 508.35 TOTAL 10000 7023.58 WILLINGNESS TO PAY Customers of road transport and, in turn, of road maintenance are willing to pay various amounts for different types of road maintenance. Road users and others do indeed pay gas taxes, property taxes, and other fees in order to support road maintenance costs. If the value they receive is less than the amount they pay, they seek tax reductions. If they perceive the value of maintenance is greater than what they currently pay, they may be willing to pay more. Often there is a difference between what people are willing to pay and what they actually pay. If the difference is positive, economists call this difference "consumer surplus." As a part of its market research, including both surveys and its effort to develop a decision-support system for benchmarking maintenance activities, MnDOT sought to estimate what customers are willing to pay for different types of maintenance activities. 225

OCR for page 214
Appendix D: Assessing Value Added to Customers In one of its customer surveys, MnDOT asked respondents to allocate $100 among its different products and services as an indication of what customers are willing to pay for each. In another market research study dealing with snow and ice control, MnDOT asked its customers how many miles travelers would be willing to go out of their way during a snow storm to drive a road that was maintained in different ways. Some of these ways included the following: Only the right lane plowed, All lanes plowed, Full road width plowed, and Only the right road edge visible. With information on what customers are willing to pay in terms of driving distance to obtain different levels of service, MnDOT can estimate the travel time during winter conditions and can figure out how much travel time customers are willing to pay. MnDOT can then go one step further, apply standard estimates of the value of travel time in dollars, and calculate what people are willing to pay in monetary terms. Another approach MnDOT has taken to estimate willingness to pay is to develop stated preference surveys and apply them in focus groups where alternative scenarios are displayed using digital imagery. Stated preference techniques involve estimating models of consumer choice based on data derived from an experimental design (a mathematical pattern) embedded in a survey or derived from a laboratory simulation or experiment. In a large number of the stated preference surveys conducted in the transportation field, the experiments are designed so that each factor influencing a choice is independent, thus allowing an independent estimate of the strength of each factor that influences a choice. Consultants assisting MnDOT developed stated preference survey instruments to assess willingness to pay for litter removal and various types of vegetation control. The basic idea was to pose to customers whether they would take Road A or Road B where Road A required less time to travel, but road B had more aesthetically attractive attributes, safer attributes, or both. 226

OCR for page 214
By systematically posing different scenarios regarding the attributes of Road A and Road B and recording the choice of survey respondents (i.e., focus group participants who represented the customers), one can apply statistical techniques (e.g., regression, logit estimation) to infer how much extra time people are willing to spend to drive a road having different levels of each attribute of interest. Figure D-2 shows the relationship between MnDOT's litter indicator value and the amount focus group participants were willing to pay in travel time to obtain a particular level of litter removal. 5 4.5 Willingness to Pay (min) 4 3.5 3 2.5 2 1.5 1 0.5 0 0 2 4 6 8 10 Litter Indicator Value Figure D-2. Relationship Between Litter Indicator Value and Willingness to Pay Once the travel time people are willing to pay is determined, one can calculate the monetary value of the travel time to obtain an estimate of the willingness to pay in terms of dollars. Figure D-3 shows an example of a survey instrument that was used to estimate willingness to pay for litter removal. Different levels of litter correspond to different levels of the litter indicator measure that MnDOT uses to assess performance of litter removal from the standpoint of the customer. The results obtained from administering the stated preference survey were eventually incorporated into MnDOT's prototype decision-support system for benchmarking. 227

OCR for page 214
Appendix D: Assessing Value Added to Customers On average, how long does it take you to travel from home to work? ________ minutes Suppose you have a choice of driving two roads (A or B) to work. Road A takes an average length of time to travel. Road B takes more time to travel than does Road A, but Road B has less litter. Assuming that the two roads are identical in other respects, please indicate in the scenarios below whether you would take Road A or Road B: Litter Factors Travel Time Factors Choose A or B Scenario Litter on Litter on Extra Travel Time (Please encircle) Road A Road B on Road B 1 Some Hardly any 1 minute A B 2 A lot Hardly any 1 minute A B 3 A lot Hardly any 5 minutes A B 4 Some Hardly any 5 minutes A B 5 A lot Hardly any 10 minutes A B 6 Some Hardly any 10 minutes A B 7 Some Hardly any 20 minutes A B 8 A Lot Hardly any 20 minutes A B Hardly any litter: average litter count per 500 ft. is less than 20 pieces. Some litter: average litter count per 500 ft. is more than 20 pieces, but less than 34 pieces. A lot of litter: average litter count per 500 ft. is more than 34 pieces. Figure D-3. Survey Instrument for Litter Control: Impact of Litter on Work Trips This survey was administered to focus groups of rural and urban residents. Each scenario was illustrated using digital photos of Roads A and B that were systematically altered in accordance with the experimental design to show focus group participants each level of each factor that affects their choice of taking Road A or Road B. Figure D-4 presents examples of two such photographs. 228

OCR for page 214
Road B (hardly any litter) Road A (a lot of litter) Figure D-4. Visual Graphics for Litter Control Surveys 229

OCR for page 214
Appendix D: Assessing Value Added to Customers DOUBLE COUNTING When calculating economic costs and benefits, it is important not to double count. Generally, the sum of avoidable user, life-cycle, and external costs exhausts all the benefits that might occur. You should not add to these other avoidable costs or to willingness to pay to get an estimate of total benefits. In fact, an estimate of the sum of the willingness to pay of each customer is an estimate of the total potential benefits all customers receive. It would be double counting to add to this to an estimate equal to the sum of avoidable user, life-cycle, and external costs to obtain an estimate of the increase in total benefits--that is, the total increase in value customers receive. However, an increase in consumer surplus--the difference between what people are willing to pay and what they actually pay--is legitimate to add to these benefits. An increase in consumer surplus occurs when the price or disutility of purchasing or of using a product or service declines. Suppose the delay associated with maintenance work zones declined substantially; the price each person pays in terms of travel time will have declined, thus increasing the difference between the travel time a person is willing to pay and the travel time the person actually pays. This difference is the change in consumer surplus for each person. The sum of the change in consumer surplus over all people involved--both existing users and additional users induced to travel through the work zone because of the lower delay--represents the total change in what people are willing to pay as a result of the reduction in delay. In certain cases, it would be legitimate to add this to avoidable user costs without it being considered double counting. Adding avoidable resource costs--labor, equipment, and material--is alright to do, provided you are not already accounting for them. Do not include these costs if they are already included in life-cycle costs. In the benchmarking procedure the project team advocates (i.e., Data Envelopment Analysis), resources are treated separately; therefore, to include avoidable resource costs among total avoidable costs would be double counting. It is also double counting to include resource costs among the benefits when resource costs are already accounted for in the denominator of a benefit-cost calculation. An accounting framework that exhausts all benefits including avoidable agency costs and changes in consumer surplus would consist of the following: Avoidable user costs, Avoidable life-cycle costs, 230

OCR for page 214
Avoidable external costs, and Change in consumer surplus. The maintenance actions that minimize the sum of these the avoidable costs and that maximize consumer surplus would be the optimal set of actions. Benefit and cost analysis can be confusing; it is easy to misstep. If you have any questions regarding how to proceed, consult an economist or someone who has had substantial experience doing highway benefit and cost estimation. OTHER ISSUES IN CALCULATING CUSTOMER VALUE There are three additional issues in attempting to assess customer value for purposes of benchmarking. The first issue involves procedures for estimating economic value. The procedures have been applied for decades in the transportation field and to maintenance overseas, but only recently have they been applied to the area of maintenance in the United States. Therefore, methods for assessing the monetary value customers receive from maintenance are still experimental. It is desirable for maintenance managers and researchers to continue performing research and, as reliable methods are developed, to introduce the methods into practice. Otherwise, it will not be possible to achieve the objective of assessing the change in customer value caused by maintenance. The second issue concerns the practicality of applying measures of customer value to benchmarking maintenance activities. In many respects, it is more appealing to be able to take physical measurements of customer-oriented outcomes than it is to assess changes in value received by customers. Physical measurements are easier to take, easier to interpret, lack the subjective component of value, and do not require making an imputation of monetary worth. The third issue involves taking advantage of other models and management systems that have built-in procedures for calculating economic value to customers. One of the keys to success may be to use various management systems, such as a bridge management system, that apply optimization procedures to determine the actions that minimize user and life-cycle costs. The optimization procedure determines the right actions for each asset or element of an asset at each point in time. Any deviation from these actions, assuming the selection of actions is optimal, increases road user and life-cycle costs. These are the avoidable costs of optimal maintenance actions, ignoring, of course, externalities and changes in consumer surplus. 231

OCR for page 214
Appendix D: Assessing Value Added to Customers MNDOT PROTOTYPE DECISION-SUPPORT SYSTEM This section concludes with a brief overview of MnDOT's prototype decision-support system, which was developed for purposes of customer-driven benchmarking of maintenance activities. The project was meant to lay the foundation for achieving the goal of allocating resources in accordance with the marginal increase in value to customers that is measured in monetary terms and is caused by an increase in input levels. Rather than be content with benchmarking output and outcome measures, MnDOT held firm to its conviction that above all, the value to the customer of road maintenance is the fundamental issue and should be the focus of any benchmarking effort. Accordingly, MnDOT contracted with a private firm for the development of a benchmarking process and prototype software to explore the relationships among inputs, outputs, outcomes, and the value added of maintenance products and services in a manner that adjusts for uncontrollable environmental factors such as weather, terrain, and road type. The MnDOT project focused on two of the seven products and services identified by the department: (1) clear roads and (2) attractive roadsides. The objectives of the MnDOT project were as follows: 1. Develop a decision-support system that permits maintenance managers to assess the resources deployed relative to the value delivered to customers, 2. Identify best practices in providing products and services considering the environmental factors impacting their delivery and the preferences of the customers, and 3. Support continuous improvement efforts through measurements and analysis of relative performance of work units in similar or related environments. The MnDOT project used a number of innovative techniques to establish the relationships among inputs, outputs, outcomes, value added, and uncontrollable variables. The methods allow one to analyze how changing inputs that are consistent with a particular level of service for a maintenance activity affect outputs, outcomes, and value added. 232

OCR for page 214
Production Functions The consultant team sought to develop two sets of production functions for each maintenance activity included in the two product and service areas that were the focus of the MnDOT effort. One set was to estimate outputs, and the other was to estimate outcomes. Production functions provide an estimate of the outcome or output with respect to changes in the level of one or more inputs (e.g., labor, equipment, or material) or uncontrollable variables. Focus groups and expert elicitation were used to explore with MnDOT staff the various factors that affect outputs and outcomes. Maintenance superintendents from throughout the state were gathered together to discuss in detail the factors that affect production. The fact that the production functions had constant outcome and output elasticities (which were defined as the percent change in production for a 1-percent change in an input factor) enabled the consultant team to simply ask experts in the department by what percent they expected the outcome (or output) variable to change given a 10- percent change in an input or uncontrollable variable; this allowed the consultant team to quickly and easily obtain a preliminary estimate of each coefficient and corresponding production elasticity. Based on these focus groups, hypotheses were formed regarding the relative importance of factors affecting production and regarding whether there was a direct or inverse relationship between outcome (or output variables) and labor, materials, equipment, and each uncontrollable variable (such as weather). Next, weather data from the National Weather Service and was merged with the standard maintenance activity, resource, and accomplishment data in the Operations Management System (OMS). Transportation Information System (TIS) data regarding roadway type, traffic volumes, and terrain was also merged with the OMS data. The combined OMS, weather, and other highway-related data were used to estimate production functions and to test hypotheses concerning the significance of the variables included in the production functions. Production functions for certain outcomes and outputs for selected activities in MnDOT's Clear Roads and Attractive Roadsides products and service areas were successfully estimated based upon the fact that their coefficients were found to be statistically significant. Value Added The benchmarking framework developed was carefully structured to permit an assessment of the additional economic value a customer receives because of some incremental change in resources or uncontrollable factors. Two approaches were used to assess the economic value of maintenance activities to customers: 233

OCR for page 214
Appendix D: Assessing Value Added to Customers 1. Assessment of avoidable road user costs: avoided travel time and accident costs were calculated using standard methods of highway capacity analysis and economic analysis (see above). 2. Assessment of willingness to pay: willingness to pay was estimated using a stated preference market research technique (see above). Prototype Software and Benchmarking Based on Differences In order to support benchmarking, prototype software was developed to permit various comparisons within and among districts, areas, and sub-areas for various maintenance activities. The software was designed to examine differences in the results of production, whether expressed in terms of outputs, outcomes, or economic value added. To the extent that suitable production functions and value-added functions are estimated, software can be used to identify or calculate the following differences: The difference between the best performer and the others within the state, a district, an area, or a sub-area based on average outputs, outcomes, or economic value with or without adjusting for uncontrollable factors such as presence of shoulder, terrain, precipitation, and traffic. The difference in the economic value (outcome or output) of an instance of an activity and the economic value (outcome or output) associated with estimated production based on average or other prescribed levels of resources with or without adjusting for uncontrollable factors. Figure D-5 shows an example output screen from the MnDOT decision-support software that illustrates a comparison among the output, outcome, and economic value of an activity instance involving plowing and sanding compared with estimated production using the same level of resources. 234

OCR for page 214
PRODUCTION FUNCTION Activity Number: 2406 District: 7 Activity Name: Plowing and Sanding Area: Windom Product Type: Clear Roadway Subarea : St. James Date Rte. Class Rte. No # Lanes EQ Code Reg. Hrs. OT Output Measure Outcome Measure 2/7/96 02 14 2 620 4.00 0.00 Lane-Mile Minutes/Lane-Mile Production Performance Output Outcome Activity Activity Activity Net Factors Measures Elasticity Elasticity Instance Actual Estimate Value Labor: 0.16 -0.11 4.00 Output: 43.94 36.72 7.22 Equipment: 3.00 Maint . Cost: $148.98 $29.28 Material: Outcome: 5.46 10.46 -5.00 Temperature: 0.72 -0.34 37 Delay Cost: $2,585.99 $4,951.71 $2,365.72 Shoulder: gravel Will. To Pay: Terrain: -0.24 0.58 rolling Accident Cost: $68.49 $131.15 $62.66 Precipitation: Indic . Value: 1.00 AADT/Lane 0.52 1410 Value of Time ($/hr) $8.00 Cost/ $75,000.00 Accident Return to Previous Menu Figure D-5. MnDOT Production Function The following is a description of various fields shown in the sample screen: Activity Number--activity code, Activity Name--name of activity, Product Type--MnDOT maintenance product category, District--district number, Area--area name, Subarea--sub-area name, Date--date work was performed, Rte. Class--route class, Rte. No--route number, # Lanes--number of lanes, 235

OCR for page 214
Appendix D: Assessing Value Added to Customers EQ Code--equipment code number, Reg. Hrs.--number of regular labor hours, OT--number of overtime hours, Output Measure--lane miles of plowing or sanding, and Outcome Measure--minutes/lane mile to restore bare pavement. The lower left quadrant of the screen shows the coefficients for two CobbDouglas production functions: one for the output production function and the other for the outcome production function. A CobbDouglas function has the property that the coefficients are equal to their elasticities: Output or Outcome Y = a0 X1a1 X2a2 X3a3, where X1 is a factor input and a1 is a coefficient. As stated above, elasticity is defined as the percent change in output (or outcome) for a percent change in the production factor. In this example, the following statistically significant coefficients, found in the first column of cells, were obtained for the output production function2: Labor (output elasticity is 0.16); Temperature (output elasticity is 0.72); and Terrain (output elasticity is -0.24). The following statistically significant coefficients, found in the second column of cells, were obtained for the outcome production function: Labor (outcome elasticity is -0.11); Temperature (outcome elasticity is -0.34); Terrain (outcome elasticity is -0.58); and Average annual daily traffic (AADT)/lane (outcome elasticity is 0.52). The third column of cells shows the actual values of each production factor that correspond to the road work actually performed: Labor (4 hours); Equipment (3 hours); 2 Intriligator, M.D. (1971). Mathematical Optimization and Economic Theory, Prentice-Hall, Englewood Cliffs, NJ. 236

OCR for page 214
Temperature (37); Shoulder (gravel); Terrain (rolling); Precipitation (none); and AADT/lane (1,410 vehicles). In the lower right quadrant of the screen are three columns of fields: the first concerns results for the actual activity, the second consists of estimated results, and the third is the difference or net value for the following: Output, Maintenance cost, Outcome, Delay cost, Willingness to pay, Accident cost, and Indicator value (bare pavement indicator). In this example, there is a net savings of $29.28 in terms of maintenance cost based on the difference between the actual and estimated maintenance cost. There is also a net savings in road user delay costs based on the difference between the calculated delay costs associated with the estimated outcome and the actual outcome. There is also a net savings in accident delay costs based on the difference between the estimated accident costs and the calculated accident costs associated with the actual performance. Delay costs were estimated at $8 per hour and accident costs at $75,000 per accident. Data Completeness and Quality The completeness and the quality of the data used were an issue throughout the project. Both MnDOT staff and the consultant team recognized that the quality and completeness of the data would need to be improved over time and that the production functions would need to be re-estimated using less restrictive functional forms than the CobbDouglas production function. However, the feasibility of estimating production functions and making comparisons based on outcomes, outputs, and value added after adjusting for uncontrollable variables such as weather and terrain was established. Enhancements to the data and estimation of the production functions were viewed as an integral part of the process of continuous improvement. 237