Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 221
Measurement Error in
Productivity Statistics
PAUL HOLLAND
Educational Testing Service
and
BENJAMIN KING
University of Washington
Chapter 4 of the Panel report reviews the methods and data sources used
by the Bureau of Labor Statistics (B[S) to construct the various measures
of output per hour that it publishes. In this paper we discuss a method of
assessing the sources and the effects of measurement error in published
productivity statistics. Final conclusions concerning the accuracy of
current measures would require a study that is beyond the time and re-
sources available to this Panel. In discussing a general analytical ap-
proach, however, our aim is to delineate the problem, to illustrate by
example the way in which the analysis can be carried out, and thus set the
stage for further research into these issues.
MEASUREMENT ERROR DEFINED
The use of the term "measurement error" is explained by the following:
A productivity measure that is produced by the BUS is a function of a
set of quantities that can, in principle, be empirically determined. Calling
the desired quantities z,, Z2, ~ Zm, denote the productivity measure
M = Mazy, z 2, ~ Zm).
This work is a collaborative effort and the order of authorship is alphabetical.
221
OCR for page 222
222
PAPERS
In this paper we will avoid the fundamental conceptual question of
whether the function M is the proper way to express productivity when
the inputs to the function are error free. Perhaps the function should be
of some other form with a different set of quantities as the argument, say,
Navy, v:, · · ·, vet or Pews, we, · · ·, wp), but in the following we take the
function M as a given, and we are interested in determining whether
reported values of M are measuring what they are supposed to measure
in terms of the particular formula that has been adopted. ~
If M deviates from its "true" value, it is because the inputs to M deviate
from the true zoo, Z2, · · ·, Zm. We therefore define an observed value
xi = zi + eizi i = 1, 2, · · ·, m, (1)
where ei is a relative error of measurement, i.e., a relative deviation from
the desired (correct, true) value Zi. What we observe each period and what
gets reported is Maxi, X2, ~ Xm), not M(zi, Z2, ·. ~ Zm), and un-
certainty in M is the result of uncertainty in the ei that causes the xj to
differ from the Zi. Thus the ei and the xi can be looked upon as random
variables. Sometimes the phrase "margin of error" is used in similar
discussions. In the present setting, the standard deviation of the distribu-
tion of ei would determine the margin of error in ei and hence the margin
of error in the observed quantity, xi. In our analysis we shall assume that
the distribution of ei does not depend on the magnitude of Zi. A more
general approach might relax this assumption.
In discussing measurement error we are classifying under one heading
errors from several possible sources: (1) First there are frame deficiencies.
For example, a component zi may denote hours worked in a particular
sector of the economy, say, business establishments reporting to State
employment security agencies. Suppose, however, that the list of estab-
lishments to which reporting forms are sent is incomplete and thus
certain establishments are not included in the coverage for that sector.
The error of underreporting in this case is due to a frame deficiency.
(2) If an establishment is covered, but for some reason does not report,
and failure to report is not corrected by a follow-up procedu. e, then the
resulting error is called an error of nonresponse. (3) Erroneous responses
due to faulty record keeping, misunderstanding of questionnaire items,
deliberate falsification, and the like are classified under the heading
of response effects. (4) One may have perfect frame coverage and perfect
response, and yet if, for reasons of economy, it is necessary to sample
reporting units instead of taking a complete census, there will be errors
~ An example of the difference between M and some other form. N. is that between labor
and total factor productivity.
OCR for page 223
Measurement Error in Productivity Statistics
223
due to sampling variability. (In most operations of this type, sampling
error is the least important source of total error. As is well-known,
information from samples is sometimes more accurate than total counts
because of the possibility of using skilled interviewers instead of un-
trained field workers and self-reporters. ~ (5) Certain observed com-
ponents xi in the productivity measure may be the result of ad hoc
estimation or judgment. An example is the use of data from trade
sources because they are the only data that are economically feasible to
obtain. Another example is the error resulting from the imputation of
a deflator to an area of the economy for which prices are not available.
Further classification is possible and may be useful for the purpose
of detailed assessment of the errors that enter at each stage of the
process. In this discussion we shall lump everything under the term
measurement error, regardless of the exact reason for the error. We
repeat, however, that errors due, for example, to failure to take quality
changes in products into account do not concern us in this paper. They
would be called conceptual or definitional errors resulting from the
misspecification of the function M, rather than measurement errors, in
our scheme.
OVERALL ERROR IN THE PRODUCTIVITY MEASURE
The exact value that the set of errors (e,, e2, · · ·, e,',) will assume at any
point of observation in the future is uncertain. This uncertainty can be
expressed in terms of a multivariate probability function for the errors.
ei. The assessment of the exact form of the probability distribution is a
complex subjective task for a single individual, and it might be difficult
to attain unanimity of assessment for a group of concerned decision
makers. The contemplation of such an assessment, however, is a neces-
sary ingredient in an analysis of the overall error in the function M.
If the observed M is written as
M = Maxi,
= M(`z ,,
· · ·' X,,2
· ·, Zn~)rLl
!)
+ E(z 1, · ·
- ~ znl; e l ~ evil)] ~(2)
then the function E can be interpreted as the relative overall error in
M. 2 In principle, the probability distribution of E can be derived from
the probability distribution for the errors ei.
2 The symbol E used for overall relative error here should not be confused with the
expectation operator for random variables.
OCR for page 224
224
PAPERS
The problem of assessment of the probability function is made easier
if the errors are assumed to be multivariate normal. In that case, their
distribution can be completely specified in terms of a set of means
(expected values) and variances and covariances. Although the judg-
mental assessment of these (m2 + 3m)/2 means, variances, and co-
variances is exceedingly difficult, the task can be simplified by the
reasonable assumption that many of the pairwise correlations between
errors from different sources are zero. After the assumption of normality
and the specification of the parameters of the distribution of the ej,
there are at least three reasonable lines of attack in deriving the distri-
bution of the overall relative error, E.
1. Derivation of exact distribution for E via normal probability theory.
2. Anoroximate solutions using Taylor series expansion methods on
the function E.
3. Building a computer model of the error structure of M and simula-
tion of the distribution of E by Monte Carlo methods.
Method 1, exact derivation, is possible in principle, but does not
necessarily lead to a distribution that is easily analyzed or evaluated
numerically. Method 2, Taylor series expansion, depends on the as-
sumption that the components ei are generally close to their expected
values, and then it only permits the approximation of the distribution of
interest. Method 3, computer simulation, as is well known, is never a
totally satisfactory substitute for the exact derivation of a distribution
it is difficult to manage and interpret without a thorough scanning of
the full ranges of the parameters of the components of the model.
Because exact derivation is complicated in this case, simulation is useful
for studying the effects of various assumptions concerning the com-
ponent errors ei on the overall error E. As we shall see below, using
simulation methods, it is possible to approximate the moments of the
overall error distribution and to study the shape of the distribution of
E as well.
MEASUREMENT ERROR AND BIAS
A normally distributed error of measurement has equal probability of
falling above or below its expected value. Thus, if the mean of a particu-
lar error ei is zero, we expect the observed value xi to underestimate,
as well as overestimate, the true value zi some of the time. Sampling
errors, in particular, would be thought of as having this form. On the
OCR for page 225
Measurement Error in Productivity Statistics
225
other hand, there are some components of error in productivity mea-
surement that are thought to lead to more systematic upward or down-
ward biases in the overall statistic. An example was discussed in Chapter
4, where it was stated that the data from the current employment sta-
tistics (CES) program are for hours paid for, not for hours actually
worked. Data on hours coming from the Current Population Survey
(cPs), however, measure something closer to actual hours in the work-
place. The bias resulting from systematic overestimation of hours
worked in the CES source can be represented in the model by specifying
an expected value greater than zero for the error in the CES data. If
one believed that this bias were the principal source of error from the
CES source, then the standard deviation for the error would be specified
to be very small. Case H of the computer simulation reported below
illustrates this point.
AN ERROR MODEL FOR PRODUCTIVITY FOR THE PRIVATE
BUSINESS SECTOR
In Chapter 4 a detailed description is given for the construction of a
productivity measure for the private business economy. It is shown that
the data required for the numerator and denominator of the output to
input ratio come from many different sources, each of which was de-
signed for some other purpose, often of higher priority than that of
productivity measurement. The errors coming from the various sources
in the makeup of the ratio play the role of the ei in the functional
representation above.
In order to illustrate an analytical approach to the assessment of
overall error, we shall, at the risk of some oversimplification, express
the annual private business productivity ratio as a function of its major
components. It was observed in Chapter 4 that, in the terminology of
the Bureau of Economic Analysis (BEA), the numerator of the BES
measure is equal to gross domestic business product in constant dollars
less the "residual" and less the imputed value (in constant dollars) of
the services of owner-occupied homes. 3
In a typical issue of the Survey of Current Business (e.g., Table 3 of
October 1977) one can find a decomposition of gross domestic business
product into the components nonfarm, farm, and residual. The constant-
dollar values for nonfarm and farm business product in 1976 are dis
3The residual is the discrepancy in constant dollars between GNP measured as the sum of
final products and GNP as gross product originating.
OCR for page 226
o
· -
so
in
is
.
·
c)
o
~4
-
o
in
o
'e
hi
an
- ~
c)
o
s~
CQ
c~
. -
c~
m
Ct
.>
50
._,
o
~:
o
._
._
o
~L
O
C) S~
Ct
S:
O
~L
m ~
-
~ O
E~ ~
C)
o
U)
O ~
~ .
O
-
e ~ ~3 ~ ~, ~ C v
C) ~
<~) ~ ~O ~ ~ ~ c:
O ~ C) ~(;_) ~O (t
- ~O ~C ~ tV
$-
Ct
=
o
- 0
C~ ,_
O ~
. _
._ ~
V,
0 =
O r-]
= r ~
._ ,> O
~ _ _
~O~ ~
O-O O
... .
OOO O
11 11 11 11 11
_ ~0 ~ r~
. .... . .
~ -c ~ ~
~ oo- ~ ~
~ ~- -
-
o ~
. ..
r~ ~r~
r~ 00
1a~
c)
. ~3 ~ c'
~o C o
o V~' Ce _ ~
~·- ~L) O
C ~ o
o ~O, _
_ _ ~t_ V~ -
O , _ ~ _
o o ~Z ~ ~ ~o
Z ~
226
OCR for page 227
Measurement Error in Productivity Statistics
227
played in Table 1. It can be seen that subtraction of the BEA estimate of
the imputed rental value of owner-occupied homes yields the same total
in constant dollars for 1976 that is shown in Chapter 4 of the Panel
report in Table 4-2 for "BES numerator (private business sector)".
Because of the equivalence between the income and the product sides
of the national accounts, it is possible, for the annual measure, to
decompose nonfarm business product into the major sources of informa-
tion in terms of payments to the factors of production (see footnote 8 of
Chapter 41. Thus, we have the secondary breakdown into (1) employee
compensation, (2) profit-type return, (3) net interest, (4) indirect business
taxes, etc., and (5) capital consumption allowances, with amounts and
weights for 1976 shown in Table 1.4 The five categories for the income
side, plus farm business, enable us to express the measured numerator
of annual productivity for the private business sector as
XN = ZN + ZN (W,e, + W2e2 + Whet + W4e4 + WseS + Wield, (3)
where the ei are the relative errors of measurement coming from the
various sources and the Wi are the weights shown in Table 1. Although
these weights are estimated from data and may change over time, in
our analysis we assume that they are constant. A more elaborate model
would relax this assumption. XN is the observed total value of the
numerator, and ZN is the unobserved or "true" value.
In a similar manner, we can write the denominator of the productivity
ratio as
XD = Zl) + ZD~ Weed, + When + Wince I`' + We ~ e ~ I), (4)
where ZD is the true value of total labor input and XD is the observed
value. The components, 8-1 1, of the denominator are described in
Table 2.
Another source of possible error in the construction of the productivity
measure is in the transformation of output in current dollars to an
amount in constant dollars. Since it is not possible to deflate the com-
ponents of national income individually, the total value in current
dollars of the numerator is adjusted by applying an implicit deflator
that is derived on the product side. The method of double deflation,
4The current-dollar figures shown in Table 1 have the amounts due to the rental value
of owner-occupied homes subtracted out. These figures are unpublished and were pro-
vided by BEA consultants to the Panel.
OCR for page 228
228
PAPERS
TABLE 2 Decomposition of Total Hours for Private Business Sector
(Denominator of Annual Productivity Measure) into Component Parts
Component
1976 Hours Component
(millions) Weight Data Source
Current employment 122,093 We = 0.92 CES program
statistics: All employees
covered by unemploy
ment insurance
Farm employees; farm
proprietors; farm un
paid family workers;
nonfarm proprietors;
nonfarm unpaid family
workers; private house
holds
20,743 Wg = 0.16 cPs
Government enterprise 3,156Woo = 0.02
Less not-for-profit institu- 13,425W'' = - 0.10
tions
TOTAL 132.5671 .00
Estimates from BEA data on
employment and cPs
average weekly hours
followed by calculation of the implicit deflator for gross product originat-
ing is described in Chapter 4.
The combination of price relatives for many individual products and
materials and services used in production, weighting information from
sEA input-output tables, and various other data that are required sug-
gests a complicated error structure and resulting uncertainty about
the true magnitude of the deflator. For our simplified model we define a
single source of error for this factor:
Xp=2p+Zpe7,
(5)
where Xp is the implicit deflator that is applied to XN the numerator
value, and Zp is the unknown true value. 5
Finally, the three factors are combined to yield the productivity
statistic
M = XN/(XPXD).
(6)
5To consider the deflation procedure in terms of a single factor, Xp is an approximation
of the actual procedure but satisfactory for illustrative purposes.
OCR for page 229
MeasurementError in Productivity Statistics
The relative error in measurement is
E = EM(ZPZD)/ZN] -1,
or equivalently,
(1 + Woes + + W6e6)
E= -
(l + e7) (1 + Wses + + W~ei~)
COMPUTER SIMULATION
229
(7)
A computer program that simulates the distribution of overall relative
error for the model discussed above is available from the authors. The
program is written in BASIC language for the Dartmouth College time-
sharing system (DTSS).
The user specifies the vector of means, standard deviations, and the
correlation matrix for the eleven errors, ei. Then the program generates
a succession of pseudo-random error vectors from the multivariate
normal distribution with the specified parameters. With each randomly
generated vector of errors, the overall relative error is computed. After
a sufficiently large number of trials, one will obtain a fairly accurate
picture of the distribution of overall error E as a function of the param-
eters of the component errors, ei.
The program was run for a number of cases, each simulating 1,000
trials. We present these simulation results for illustration only. A more
detailed presentation would give standard errors or confidence intervals
for all of the quantities estimated by the simulation. The use of 1,000
trials made the simulation estimates of the means (i.e., values denoted
below as x) accurate to three decimal places. The simulation estimates
of the standard deviations (i.e., values denoted below as s) appear to be
accurate to between two and three decimal places, but we have not
made a detailed analysis of their accuracy. The results are as follows.
CASE A
For case A the means and correlations are all set to zero. All standard
deviations are set at 0.01. The result is
x= 0.000 s = 0.015.
Comment This model assumes that the probability is approximately
0.95 that the error in each source lies within plus or minus two per
OCR for page 230
230
PAPERS
centage points from the true value, and that, furthermore, the errors are
mutually independent. The result shows that the overall error, although
expected to be zero, will fall with probability about 0.95 between plus or
minus three percentage points. In the general case of independence and
constant error variance the standard deviation of the overall error is
about 1.5 times the common value for component errors.
CASE B
Conditions for case B are the same as those for case A except that the
correlation between en, employee compensation, and en, CES hours, is
set at 0.99. The result is
x= 0.000 S = 0.011.
Comment Here en, and es are based on the same survey (for annual
figures, a total enumeration of employees covered by State unemploy-
ment insurance), and one might expect the errors from the same source
to be highly correlated. The simulation, however, shows that this correla-
tion has little effect on the variance of the overall error. This is some-
what surprising since the respective component weights are high. The
same result occurred when the correlation was increased to 0.9999.
CASE C
The means and correlations are zero. The common standard deviation is
set at 0.01, except standard deviations for en, en, es, e' are all set at
0.0001. The result is
x= 0.000 s = 0.010.
Comment In this model the two major components in the numerator
and denominator, respectively, are assumed to be almost error free (very
small standard deviations), and yet the standard deviation for total error
is reduced only to 0.010, the common value for the remaining com-
ponent errors.
CASE D
The means and correlations are zero. Common standard deviation is
0.01, except for en, the error in the price deflator. Its standard deviation
is set at 0.0001. The result is
OCR for page 231
Measurement Error in Productivity Statistics
x= 0.000 s = 0.012.
231
Comment Reduction of the error variance in the deflator alone has
little effect when results are compared with those of case A.
CASE E
Case E combines cases C and D with standard deviations for en, en, ea.,
e9, and en each set at 0.0001. The result is
x= 0.000 s = 0.002.
Comment In this model, with the standard deviations for the deflator
and the major components of both numerator and denominator all near
zero, the standard deviation of total error is considerably reduced. Prom
cases A, C, D, and E we may conclude that reduction of uncertainty is
required in both the implicit deflator and the major components before
reduction of uncertainty in the overall productivity measure can occur.
CASE F
For case F the means are set at zero. Common standard deviation of
errors is set at 0.01. Intercorrelations between pairs of errors 2-5 are
set at 0.90; the same value is set for correlations for pairs of errors 9-11.
The result is
x= 0.000 s = 0.016.
Comment Correlations between pairs of errors from {RS sources in the
numerator are set at high positive value, 0.90. Similarly, the errors in
the denominator related to the cPs are intercorrelated. The results,
however, show no effect on overall error variance. This is due to the
fact that the numerator and denominator remain independent.
CASE G
Case G is the same as case F. except that the correlation between e, and
e ~ is also set at 0.90, introducing a correlation between output and
input. The result is
x = 0.000 s
= 0.012.
OCR for page 232
232
PAPERS
Comment There is some reduction in total error variance-parallel to
case B. This is in agreement with our general conclusion that correla-
tion between numerator and denominator corresponding to real world
expectations has little overall effect.
CASE H
For case H the common standard deviation is set at 0.01. The same
intercorrelations are used here as in case G. including 0.9 between en and
en,. Means are all zero with the exception of the mean of en, set at 0.10
and the mean of e7 set at 0.05. The result is
x= - 0.128 s = 0.009.
Comment In Chapter 4 the discrepancy in CES employment data
between hours paid for and hours at the workplace is discussed. In an
attempt to make the model more realistic we introduce an upward bias
of 0.10 (i.e., 10%) in CES hours. Similarly, in order to simulate the
possibility that the price deflator is overadjusting for the effects of infla-
tion, we set the mean error for e7 to 0.05. The result shows dramatically
the effects on the overall productivity measure. It has a downward bias
of 12.8 percent.
With the BASIC program used in these simulation runs, it is possible
to store the first 200 observations of the simulated overall error E in a
file for subsequent analysis. As an example, we show the histogram and
normal probability plot for the results of case A. Figures 1 and 2 indi-
cate that the distribution of the overall error E is approximately normally
distributed when the variance of the component errors is small. (In
Figure 2 the proximity of the plotted points to the 45-degree line indi-
cates conformance to normality. Measures of skewness, kurtosis, and
studentized range are also in line with expectations under the hypothesis
of normality.
TAYLOR EXPANSION APPROXIMATION FOR OVERALL
ERROR
The formula given in (7) for the relative error of measurement for a
productivity statistic may be expressed as
E = E(U. V Y) = {~1 + U)/~1 + V)~1 + Yell -1 (8)
The statistical plots shown in Figures l and 2 were produced with IDA (the Interactive
Data Analysis programs) on the Dartmouth Time-Sharing System.
OCR for page 233
Measurement Error in Productivity Statistics
~ 1 5
UJ
3
LL
~ 10
11
LL
o 5
m
6
3
1
o
-2
233
20 _
.
.
Mean = -1 .1828E-03
· Std. Dev.= 1.4456E-02
.
·
·.
·e ·.
.
· - - ·
· - ~ · --
.
· -
· ~ ~-- - ~--
· - ~ ~ ~-- --
· - ~-- ~---
·.-- - ~-- - --
· - ~-- -
· ·-~ · - ~-- - - - - -
· - - - ..-- - - - - - - - ~-
·-.-.~----.--~--~-~-
. .i----i.~-.
t ~ ---1 .-.-i
Sample Size= 200
l
-6000. E-05 -3000. E-05 0000. E-04
ERROR
3000. E-05 6000. E-05
FIGURE 1 Frequency histogram of 200 observations of overall error. E.
Mean=-1.1828E-03 X
Std. Dev.= 1.4456E-02 1 X
Skewness = -5.3452E-03 1
Kurtosis = -3.0572E-01 1
2 Studentized Range= 5.0307E+00 1 1 1
Sample Size = 200 2 3 2
_
_ 2
3- x
~1 1 1 1 11 1
x
x
1
2 3
3
X 1
3
1
1 -
.
1 6 2
2 6
2 4
· 6
· 4
5 1
2 5 1
9
· 1
2 · 1
NOTE: Frequencies Over 9 Indicated by
X
X
X
-3 -2 -1 0 1 2 3
FIGURE 2 Normal probability plot of 200 observations of overall error. E.
OCR for page 234
234
where
PAPERS
U = We, + + W6e6
V= e7
Y = Whet + · ~ ~ + We,.
In the simulation study, the individual errors e = fee .. e,,) are
specified to have a joint normal distribution with mean vector m = (m I,
· · ·, m ,, ~ and covariance matrix C. In the various cases considered
there, m and C were given particular values. In all of those cases the
variances of the ei were assumed to be quite small, and it is under such
conditions that the Taylor expansion' (or "delta method") can be
expected to yield useful approximations to the distribution of E.
First observe that U. V, and Y are linear functions of the vector of
errors e. Define the (3 X 11) matrix:
/w} . . .
L= {0
W6 0
·-- 0 1
O O
Then we have the matrix equation
o
o
W8 . . .
/ U\
V ~ =Le
~YJ
. . .
... o
o
W.,
(9)
(10)
where e is the transpose of the vector of errors. It follows from standard
theory for the multivariate normal distribution that the U. V, and Y
are jointly normally distributed with mean vector
t/ v ~ = Lm
and covariance matrix
(11)
D =LCL'.
(12)
The Taylor expansion method proceeds by expanding the function
E(U, V, Y) about the means u, v, end y to a first-order (linear) approxi
7See Rao (1965, p. 321) for a more detailed discussion of this method.
OCR for page 235
Measurement Error in Productivity Statistics
235
mation in (U. V, Y) and ignoring higher-order terms. Thus we have
E(U, V, Y) = t(1 + u)/~1 + v) (1 + y)] ~-1
+ ~ U-u)/~( 1 + v) ~ 1 + y)]
-(V-v)~1 + u)/~1 + V)2~1 + y)]
-(Y-y)~1 + U)/~1 + v)~1 +y)2]
. . .
(13)
The Taylor expansion method approximates the distribution of E by a
distribution with mean value given by
E(u. v, y) = t(1 + u)/~1 + v) (1 + y)] ~-
and variance given by the variance of the linear part of the expansion in
(131. In the present case this is given by defining the (1 X 3) vector
T = t1/~1 + v) (1 + y)], - (1 + u)/~1 + v)2~1 + y)],
-(1 + U)/~1 + v) (1 + y)2] i.
(14)
Then the Taylor expansion approximation to the distribution of E has a
variance equal to
b2= TDT',
(15)
where D is the matrix given by (12~.
To be technically precise, we must observe that with (U. V Y' jointly
normal the moments of a ratio such as E; U. V, Y) do not exist. The
assumption of normality for the error components, however, is not
needed, and normal distributions would, of course, never be observed in
the real world. Distributions that are symmetric (roughly' but truncated
at some point in each tail will serve quite well to express one s probability
beliefs concerning the errors of measurement. These are in fact the types
of distributions that are used in the simulations above with ''pseudo-
normal" variates.
To illustrate these calculations, in Table 3 we summarize the values of
b and of E(u, v. yJ for the cases given for the simulation and compare
them to the corresponding simulation results, x and s. Fron~ Table 3 we
see that the Taylor expansion method and the simulation results are in
close agreement in every case.
The strength of the Taylor expansion method is the ease with which
it may be applied to this type of problem. The method, ho``ever is
OCR for page 236
236
PAPERS
TABLE 3 Summary of Simulation and Taylor
Expansion Results for the Eight Cases Considered
in the Text
Simulation
x s E(u,vy)
Taylor Expansion a
b
A0.000 0.015 0 0.015
B0.000 0.011 0 0.011
C0.000 0.010 0 0.010
D0.000 0.012 0 0.011
E0.000 0.002 0 0.002
F0.000 0.016 0 0.015
G0.000 0.012 0 0.012
H-0.128 0.009 - 0.128 0.010
UWe would like to thank David Saxe at Educational Testing Service
for making these calculations.
based on a linear approximation to a nonlinear function. In the present
case, this approximation is satisfactory. If the variances of the ej were
substantially larger, this would not necessarily be true.
IMPLICATIONS FOR MEASURES OF PRODUCTIVITY
CHANGE
In this paper we have illustrated an approach to the assessment of the
overall relative error for a measure of the level of productivity at a single
time period. It is also possible to extend this approach to assess the
overall error in the percentage change in a productivity measure over
two time periods. Such an extension will require the specification of the
joint multivariate distribution of the relative errors of the components
in both time periods. While this is a more complicated case than those
we have considered, it may, in principle, be studied by the same tech-
n~ques.
For example, if the productivity measures for successive time periods
are unbiased and the relative errors are uncorrelated, and have the same
standard deviations, then it can be shown that the error in the per-
centage change (i.e., the deviation of the measured change from the true
change) has a standard deviation that is roughly 1.5 times the standard
deviations of the respective relative errors. In other words, if relative
errors in the level of productivity in successive time periods are, in fact,
OCR for page 237
Measurement Error in Productivity Statistics
237
random (i.e., their direction cannot be predicted) then uncertainty
about the changes in productivity is compounded.
CONCLUSION
We have presented this analysis in order to show a direction in which
research into the problem of measurement error might proceed. Many
questions remain to be answered, such as, Who is best qualified to
assess the distributions of errors from the various sources?, What are
the parameters of these distributions?, and When is the model sufficiently
complex to be considered realistic? With respect to the subjective assess-
ment of error, a precedent may be found in the pioneering research of
Simon Kuznets (1954~. In Chapter 12 (part 3) of his three-volume work,
Kuznets meticulously reports in 12 pages of tables the estimates of
"margins of error" elicited from the three investigators (including him-
self) who were responsible for the totals in various industrial divisions
and income categories. It is interesting to note that all margins of error
were raised by one-half because it was ". . . found that all three investi-
gators tended to underestimate the errors attaching to the results of
their labors." In more recent research into revisions in national income
and product accounts, A. Young (1974) displays subjective rankings of
the reliability of various components of GNP and national income. We
see from these two previous studies that, at least, the idea of persuading
experts to make subjective evaluations of errors in reported statistics is
not foreign to the agencies involved.
It is our recommendation that the Bureau of Labor Statistics and
the Bureau of Economic Analysis explore further the possibility of using
techniques similar to the technique in our simplified example in order
to estimate the overall margin of error in productivity measurement.
The routine reporting of an expected value of productivity along with
its estimated standard error, instead of the currently published single
number, should be seriously considered for the future. At the least,
the agencies responsible for the construction of the various measures
should consider carrying out a limited type of "worst case" sensitivity
analysis, e.g., What are the highest and lowest possible values of pro-
ductivity change this year if such and such a component is assumed
to have X-percent relative error?
Users of statistics often depend, without verification, on intuitive
ideas of the effects of errors when a test of the implications of these
errors on simple models can provide better insight into their actual
effects. The statement, "Everything will wash out in the end" is often
more hope than reality.
OCR for page 238
238
REFERENCES
PAPERS
Kuzuets, S. (1954) National Income and Its Composition 1919-1938. New York: Na-
tional Bureau of Economic Research.
Rao, C. R. (1965) Linear Statistical Inference and Its Applications. New York: John
Wiley.
Young, A. H. (1974) Reliability of the quarterly national income and product accounts
of the U.S., 1947-71. Order no. COM-74-11538 (July). Springfield. Va.: National
Technical Information Service.
Representative terms from entire chapter:
measurement error
