Statistics and Physical Oceanography (1993)

Oceanographers often have available multiple independent estimates of the various geophysical quantities of interest (e.g., sea surface temperature, surface winds, surface humidity, sea level, velocity, etc.). The sources of such estimates might be in situ observations, satellite-based observations, numerical model simulations, or so-called analyzed fields. The latter may consist of regularly gridded estimates constructed by subjective (i.e., hand-drawn) or objective (i.e., computer generated by some objectively prescribed interpolation algorithm) analysis of irregularly spaced observations. Alternatively, analyzed fields may be constructed from a numerical model forecast, adjusted to be consistent in some least-squares sense with all available observations acquired since the previous “analysis time.” Independent estimates of the same quantity are never precisely the same, and small differences can sometimes have a profound influence on the scientific interpretation or application of the geophysical field. An important statistical problem in oceanography is therefore development of techniques for quantitatively evaluating the degree of similarity or difference between independent estimates of a multidimensional field. This includes cross-comparisons between different observational data sets (e.g., in situ vs. satellite), comparisons of model simulations with observations, and comparisons between different model simulations.

An example of a geophysical quantity that illustrates the kind of problems that can be encountered in comparisons of different observational data sets is sea surface temperature (SST). Temporal variations of SST are generally dominated by the seasonal cycle, which may have an annual range of 5° to 10° C or more at any particular geographical location. Interannual deviations from the local seasonal cycle typically have magnitudes of only about 0.5° C. Such small anomalies in SST can have a significant effect on climate. Even the El Niño phenomenon that affects weather patterns on a global scale can be initiated by an SST anomaly in the eastern tropical Pacific of only a degree or two. It is very difficult to estimate SST to an accuracy of 0.5° C by any of the means currently available. Since the actual SST is not known on ocean-basin scales, it is difficult to assess the accuracy of the several different estimates available. Attempts to determine the accuracy of satellite estimates of the SST field are often made by comparisons with in situ observations from ships and buoys or with other satellite-based estimates (e.g., Bernstein and Chelton, 1985). In the case of in situ observations, comparisons are complicated by the sample size and distribution. The data are not uniformly distributed geographically or temporally. Observations tend to be concentrated along standard shipping routes and are generally more sparse during severe wintertime weather conditions. Moreover, in situ observations can differ from satellite estimates because of measurement errors and because of smaller-scale variations that are spatially averaged in satellite measurements. Comparisons between two different satellite estimates of SST are complicated by a common source of error, atmospheric effects on the radiance emitted from the sea surface, which obscures the errors in both data sets.

Systematic errors, particularly in satellite data, create biases in the simplest statistical measures, be they spatial or temporal averages. In addition to the problem of limited sample size discussed above (see also Preisendorfer and Barnett, 1983), such gross statistics can obscure important characteristics of the differences such as geographical or temporal biases (see, e.g., Barnett and Jones, 1992). For the SST example above, such biases may arise from systematic errors in the algorithms applied to correct for atmospheric effects on satellite estimates of SST. As an example, volcanic aerosols injected into the atmosphere by the El Chichon volcano in 1982 contaminated infrared-based satellite estimates of SST within about 30° of the equator for a period of about 9 months. As another example, microwave-based satellite estimates of SST have been found to be biased upward in regions of high surface winds because of incomplete corrections for the effects of wind speed on ocean surface emissivity.

Evaluation of numerical model simulations, either through comparisons with observations or by comparisons with other model simulations, presents additional problems. Models produce a large number of output variables on a dense space-time grid. An ocean circulation model, for example, typically outputs current velocities, temperatures, and salinities at a number of different depths, as well as the sea surface elevation. It is not reasonable to expect present models to reproduce the details of the actual circulation, but one hopes that basic statistics such as the mean or variance of some characteristics of the actual circulation are well represented by the model. Assessing the strengths and weaknesses of a model is thus complicated by the large number of possible variables that can be considered. For example, present global ocean circulation models can reproduce the statistics of sea level variability with some accuracy but generally underestimate the surface eddy kinetic energy computed from surface velocities (e.g., see Morrow et al., 1992). A model that successfully represents the statistics of some geophysical quantity at one level may misrepresent the statistics of the same quantity at a different level. An even more stringent assessment of the performance of a model is how accurately it represents cross-covariances between different variables (which can be shown to be related to eddy fluxes of quantities such as heat, salt, or momentum). Some of these issues are discussed by Semtner and Chervin (1992) with regard to comparisons of numerical model output to satellite altimeter estimates of sea level variance and eddy kinetic energy. The overall goal of such comparisons is to guide further research in an effort to develop more accurate numerical models.

The types of questions that need to be addressed by techniques for comparing two different geophysical fields, whether they consist of observations or model simulations, are indicated by the following:

1. How, where, and when do the two independent estimates of a field differ?

2. Are the differences statistically significant? Addressing this question may lead to development of appropriate bootstrap techniques for estimating probability distributions.

3. What statistical comparisons are most appropriate for evaluating a model?