Minimizing Potential Errors in Assessing Group and Individual Intakes
This chapter presents information on ways to minimize errors in dietary assessments, including tailoring the Dietary Reference Intakes (DRIs) to the specific group or individual, ensuring that the intake data have the highest accuracy feasible, minimizing sampling errors when collecting intake data on groups, and determining standard deviations of prevalence estimates.
Dietary assessments involve comparing nutrient intakes of individuals or groups with the DRIs. Thus, there are two primary areas where potential measurement errors can influence assessment results: (1) determining nutrient requirements; and (2) measuring dietary intake, including using appropriate sampling strategies, and accurate nutrient composition for foods consumed.
Intake data need to be collected with the most accurate techniques available, with cost and feasibility of evaluations taken into account. Furthermore, the assessment must use appropriate DRIs, and consider the age, gender, physiological status, and other relevant characteristics (e.g., smoking status) of the individual or group being assessed. If estimates of intakes or requirements (or upper limits) are incorrect, the assessment of inadequate or excess nutrient intakes for the individual or the group will also be incorrect.
TAILORING REQUIREMENTS FOR SPECIFIC GROUPS AND INDIVIDUALS
The Dietary Reference Intakes (DRIs) can be adjusted to be more appropriate for specific individuals or groups. For example, adjust-
ments might be made for body size, energy intake, or physiological status. However, such adjustments are usually not necessary since the DRIs are assumed to apply to all healthy individuals in the specified life stage and gender group.
Are there situations when adjustments to the Estimated Average Requirement (EAR), and thus the RDA, should be made for certain individuals to ensure that they are at little or no risk of nutrient inadequacy?
In most cases, adjustments are not likely to be required because the EAR already accounts for normal individual variability. However, adjustments may be warranted for individuals who have unusually high or low body weight, experience physiological changes at unusual ages, experience unusual physiological changes, or have unusually high energy requirements. These situations are discussed below.
When nutrient recommendations are established in relation to body weight, the weight of a reference individual is often used to derive DRIs. (See Appendix A for reference weights used in developing the DRIs.) For example, the RDA for protein has traditionally been related to body weight and in the 10th edition of the RDAs (NRC, 1989) the RDA for protein was set at 0.8 g of protein per kg body weight. Summary tables list RDAs of 63 and 50 g/day of protein, respectively, for reference adult men and women weighing 79 and 63 kg (NRC, 1989). Recommendations for individuals above or below these reference weights would be modified accordingly. For example, the RDA for individuals weighing 45 and 100 kg would be 36 and 80 g/day of protein, respectively. When this adjustment is made the individuals are assumed to have relatively normal body composition because protein requirements are related more strongly to lean body mass than to adipose tissue mass. Thus, a protein intake of 160 g/day would not be recommended for an obese individual weighing 200 kg. None of the DRIs established at the time this report went to press have been expressed in relation to body weight.
Age and Physiological Stage
For some nutrients, requirements change across the lifespan in association with physiological changes that are assumed to occur at various average ages. For example, the AI for vitamin D is higher for adults older than 50 years than for those younger than 50 years, and the recommendation for vitamin B12 is that individuals older than 50 years obtain most of their vitamin B12 from fortified foods or supplements. For these nutrients, the changes in recommendations are associated with age-related changes in vitamin D metabolism and in gastric acidity, respectively. These changes do not occur abruptly at age 50 and it could reasonably be suggested that average dietary requirements would be increased at the upper end of the 51- through 70-year age range.
In other situations the physiological changes that result in different requirements occur over a shorter time or can be identified by individuals. An example would be iron requirements of women. The requirements for women ages 31 through 50 years are intended to cover losses associated with menstruation whereas for women older than 50 years it is assumed that menopause has occurred. Onset of menopause, then, rather than age, is the physiologically significant event.
Although the EARs for intake of thiamin, riboflavin, and niacin are not set based on energy intake (IOM, 1998b), it may be appropriate to evaluate intake of these vitamins as a ratio to energy intake for some populations.
The DRI report on the recommended intakes for the B vitamins (IOM, 1998b) notes that no studies were found that examined the effect of energy intake on the requirements for thiamin, riboflavin, or niacin and thus these EARs and RDAs were not based on energy intake. Despite this lack of experimental data, the known biochemical functions of these nutrients suggest that adjustments for energy intake may be appropriate, particularly for individuals with very high intakes (such as those engaged in physically demanding occupations or who spend much time training for active sports). Adjustments may also be appropriate for healthy people with low intakes due to physical inactivity or small body sizes.
For thiamin, riboflavin, and niacin, an energy-adjusted EAR may be calculated as the ratio of the EAR to the median energy requirement for an individual or population. Because DRIs have not been
set for energy as of the writing of this report, the requirements for energy recommended in the 10th edition of the RDAs (NRC, 1989) can be used. For example, the thiamin EAR for men 19 through 50 years is 1.0 mg/day and for women is 0.9 mg/day. The recommended median energy intake for men and women 24 through 50 years of age is 2,900 and 2,200 kcal/day, respectively (NRC, 1989). Thus, an energy-adjusted thiamin EAR for adults in this age group would be 0.34 mg/1,000 kcal for men and 0.41 mg/1,000 kcal for women. As was suggested in 1989, for adults with intakes below 2,000 kcal/day, the requirement should not be further reduced (i.e., 0.68 mg/day for men and 0.82 mg/day for women).
An energy-adjusted RDA can be calculated from the energy-adjusted EAR by adding two standard deviations of the requirement. For thiamin, the coefficient of variation of the requirement is 10 percent, so the energy-adjusted RDA would be 20 percent higher than the energy-adjusted EAR, or 0.41 mg/1,000 kcal for men and 0.49 mg/1,000 kcal for women.
MINIMIZING ERRORS IN MEASURING DIETARY INTAKES
Factors influencing food and nutrient intakes are often the same as those influencing requirements, such as life stage, body size, life-style, genetic determinants, environment, etc. Food availability and culture also influence intakes but are not related to individual biological requirements. Box 8-1 summarizes points to consider in minimizing error in collecting dietary intake data.
Dietary intakes are determined using a variety of research instruments (e.g., 24-hour recall questionnaires, food records, food-frequency questionnaires, diet histories) that elicit information on types and amounts of food and beverage items consumed. This information is used with values from a nutrient composition database to determine dietary nutrient intake. Contributions of nutrient supplements to dietary intakes are similarly assessed. Following are some techniques for intake measurement that apply to most dietary data collection processes and can help avoid bias and measurement error—and therefore help to ensure the accuracy of individual and group intake measurements. For a more complete review of these issues, see Cameron and Van Staveren (1988), LSRO (1986), NRC (1986), and Thompson and Byers (1994).
BOX 8-1 Key Steps in Measuring Dietary Intake
Select the Appropriate Methodology
Dietary intake data are commonly collected using one or more days of recall or records. However, collection of dietary intake data using methods other than a few days of direct reporting of all foods and amounts consumed (e.g., food-frequency questionnaires, diet histories, and household inventories) may appear to be attractive alternatives. Because of the ease of administration and entry of consumption data, semi-quantitative food-frequency questionnaires are widely available and often used in epidemiological studies. These types of questionnaires may be appropriate for ranking intakes in epidemiological studies, but, as noted below, are seldom accurate
enough to use to assess the adequacy of dietary intakes of either individuals or groups due to several limiting characteristics of semiquantitative food frequencies.
First, there is no direct quantitative assessment of individual amounts consumed (Kohlmeier and Bellach, 1995). Either an average portion for all individuals in a group is assumed or the options are limited to a few categories, such as small, medium, and large. Assessment requires a precise quantification of nutrient intakes, and for this, accurate portion sizes are needed. Frequencies of consumption are truncated in a limited number of categories (usually five or seven).
Second, a food-frequency questionnaire does not assess intakes of all available foods. Foods are limited to those that are considered major contributors to the nutrients under study (Block et al., 1986), or to the foods that contributed most to the variance in intake in a specific group at the time the questionnaire was designed (Willett et al., 1987). Food-frequency questionnaires do not attempt to capture all food sources of a nutrient quantitatively.
Third, because of the discrepancy between thousands of foods being offered in a supermarket and a set of questions limited to a few hundred at most, many foods are combined in one question. Food composition data are averaged in some way across these foods, and the individual who consumes only one or another of these or eats these in other proportions will be incorrectly assessed with the nutrient database being used. As a result intakes may be either over-or underestimated. Also often overlooked is that food-frequency questionnaires are only applicable to the population for which they are designed and are based on their consumption patterns at a specific time. Continually changing food consumption patterns and new food offerings require that periodic changes be made in food-frequency questionnaires.
Diet histories, like food frequencies, attempt to capture usual diet but, unlike food frequencies, include quantitative assessment of portions and include the assessment of all foods eaten in a cognitively supportive fashion (meal by meal) (Burke, 1947). Because they are quantitative and do not truncate information on frequency, amount, or the actual food items consumed, diet histories overcome many of the limitations of food-frequency questionnaires for assessment of the total nutrient intakes of individuals (Kohlmeier and Bellach, 1995). Diet histories have also been shown to capture total energy intake more accurately than other methods (Black et al., 1993). However, if conducted by an interviewer, rather than a preset com-
puter program, they may show between-interviewer differences in responses (Kohlmeier et al., 1997).
Household inventories are weak measures of total food intake because of food waste, food consumed by guests or pets, and the large amount of food consumed outside of the home. They also require assumptions about the distribution of food consumption among the people within a household when the household includes more than one person.
Maintaining weighed food records over multiple days can provide a solid basis for nutrient assessment as long as the recording of food intake does not influence usual intake behavior and as long as seasonality in nutrient intake, where it exists, is adequately captured.
In summary, intakes assessed by 24-hour recall, diet records, or quantitative diet histories remain the strongest bases for quantitative assessment of nutrient adequacy using the Dietary Reference Intakes (DRIs). Quantitative assessments require both accurate determination of the quantities of foods consumed by an individual and inclusion of all of the foods that contribute even modestly (more than 5 percent) to the total nutrient intake. Not all dietary intake instruments are designed to meet these requirements. Their use for this purpose is likely to result in inaccurate assessments.
Ascertain All Foods Consumed
Either because of poor memory or a reluctance to report foods felt to be inappropriate, people often omit, add, or substitute foods when recalling or reporting dietary data. On average, total energy intake tends to be underreported by about 20 percent, although the degree of underreporting varies with weight status, body mass index, etc. (Johnson et al., 1998; Lichtman et al., 1992; Mertz et al., 1991). The most common additional food items that were remembered after prompting in the U.S. Department of Agriculture's Continuing Survey of Food Intake by Individuals (1994–1996, Day 1) were beverages, including alcoholic beverages, and snack food, with 5 to 10 percent of nutrient totals being added after prompting (B. Perloff, U.S. Department of Agriculture, unpublished observations, 1998). If foods —and therefore nutrients—are underreported, then the prevalence of inadequate intakes for a population or the probability of inadequacy for an individual may be overestimated. Little is
known about the relative sizes of nutrient versus energy underreporting.
Various techniques may be used to encourage accurate reporting. Because many studies of dietary intake rely on subjects' memory of food, food ingredients, and portion sizes, dietary survey instruments often specify the use of memory probes and cues to improve accuracy (Domel, 1997). Those with poor memory, such as some elderly adults and young children, are not good candidates for dietary intake interviews (Van Staveren et al., 1994; Young, 1981).
Some retrospective diet studies depend on the individual's long-term recall of past food intake and rely on memory that is more generic than that for recent intake. Complete food lists and probes using specific circumstances of life are helpful in these studies (Dwyer and Coleman, 1997; Kuhnlein, 1992; Smith et al., 1991a). The interview atmosphere should be kept neutral so that respondents do not feel they must report (or not report) items because of their social desirability (Hebert et al., 1997).
When dietary intakes are assessed for individuals with strong cultural or ethnic identities, it is useful to employ interviewers from the same background who speak the language of the interviewees and can knowledgeably guide dietary information exchange about the food, its ingredients, and portion sizes. Food composition databases used should contain the appropriate culture-specific food items. Respondents must be literate if written survey instruments are used (Hankin and Wilkens, 1994; Kuhnlein et al., 1996; Teufel, 1997).
Accurately Determine Portion Sizes Consumed
To minimize portion size as a source of error, various kinds of food models, portion-size models, and household measures have been used to assist the respondent (Burk and Pao, 1976; Guthrie, 1984; Haraldsdottir et al., 1994; Thompson et al., 1987; Tsubono et al., 1997). Training the interviewer in use of portion-size models improves accuracy of reporting (Bolland et al., 1990).
Determine Nutrient Supplement Use
Supplement use needs to be determined, and quantified, to obtain accurate estimates of the prevalence of inadequate nutrient intakes for a group. Otherwise, the prevalence of inadequacy will be overestimated, as will the probability of inadequacy for an individual. However, the proportion of individuals with intakes above the
Tolerable Upper Intake Level (UL) may be underestimated. The extent of under- or overestimation will depend on the dosages and frequency of use, and for groups, on the percentage of the group using supplements. Currently, the only national surveys available which quantify supplement usage along with dietary nutrient intakes are the 1987 National Health Interview Survey and the Third National Health and Nutrition Examination Survey.
Merging two different databases—one dealing with food use and the other dealing with supplement use —to estimate the distribution of usual total intakes is complex because supplements provide relatively high doses of specific nutrients but may be taken intermittently. More accurate methods for measuring nutrient supplement intake are needed.
When assessing adequacy of intake, it may be helpful to average supplement intake over time when the supplement is consumed intermittently (e.g., once per week or month). This will mask or smooth out the high intake associated with the day the supplement was actually consumed. This smoothing effect might be appropriate when assessing for chronic high intakes using the UL. However, if acute effects on health are possible from excessive intake of a nutrient, then a different approach to combining food and supplement intake needs to be proposed. An additional drawback of smoothing supplement intakes is that the day-to-day variability in nutrient intake cannot be estimated. This creates a problem when estimating the usual nutrient intake distribution in a group.
Consider Whether Intakes May Vary Systematically
When dietary intakes of a population or a population subset (e.g., athletes in training) vary systematically, reasons for this variation must be understood and incorporated into data gathering. These techniques also are part of defining what is usual intake (for example, over a calendar year). If systematic variations are not considered, prevalence of inadequate intakes may be under- or overestimated.
Seasonality and Other Issues of Periodicity
Seasonal effects on dietary intakes are reflected in changing patterns of food availability and use. These effects are usually greater for food items than for energy or nutrients (Hartman et al., 1996; Joachim, 1997; Van Staveren et al., 1986). The season of collecting yearly dietary data may bias results because the data will selectively overemphasize items consumed during the season of the interview
(Subar et al., 1994). Seasonally available local cultural food may affect seasonal and yearly average nutrient intakes (Kuhnlein et al., 1996; Receveur et al., 1997). The effects of seasonality on estimated nutrient intakes can be alleviated by a well-designed data collection plan.
Within-person variability also may include other nonrandom components (Tarasuk and Beaton, 1992), some of which may be related to sociocultural factors (e.g., intakes may differ between weekdays and weekend days) (Beaton et al., 1979; Van Staveren et al., 1982) and some of which is physiological (e.g., women's energy intakes vary across the menstrual cycle) (Barr et al., 1995; Tarasuk and Beaton, 1991a).
Illness and Eating Practices
Chronic illness affecting intakes of a part of the population is reflected in group dietary intakes and may bias the prevalence of inadequate intakes in what is assumed to be a normal, healthy population (Kohlmeier et al., 1995; McDowell, 1994; Van Staveren et al., 1994). Parasitism, eating disorders, and dieting—which may be prevalent in segments of a population—may affect food intake. Unlike dieting, illness presents a problem not only with regard to intake data but also in the assumptions underpinning the assessment of adequacy because the DRIs were established for normal, healthy populations.
Rapid Dietary Transition Including Effects of Interventions
Data may be biased by individuals whose dietary intakes are affected by rapidly changing life circumstances (such as migration or refugee status) or by successfully implemented nutrition intervention programs. Thus, it is important to consider how many affected individuals are included in the data sample (Crane and Green, 1980; Immink et al., 1983; Kristal et al., 1990, 1997; Yang and Read, 1996).
Consider the Unit of Observation (Individual, Household, or Population)
Data on nutrient intakes are sometimes collected for households rather than for individuals. When this is the case, the level of aggregation of the dietary data must be matched with an appropriate level of aggregation for the requirements. Appendix E discusses how requirement data may be aggregated at the household level. It
is sometimes of interest to compare population-level consumption data (such as food disappearance data for a country) with a requirement estimate. Appropriate ways to make such comparisons are also discussed in Appendix E. However, the methods involve many assumptions, and errors may be large.
Use Accurate Food Composition Data
Deriving nutrient intake data from dietary intake data requires the use of a food composition database. Accuracy of the food composition data and the software to access it are critical for assessments of dietary adequacy. Nutrient databases need to be kept current and contain data on dietary supplements. In the United States and Canada the primary sources of nutrient composition data are the U.S. Department of Agriculture Nutrient Database for Standard Reference, Release 13 and its revisions (USDA, 1999; Watt et al., 1963).
Databases should be evaluated for the number of food items included that are relevant to the population under study (Kuhnlein and Soueida, 1992; Smith et al., 1991b). The currency of data for foods derived from recipes is important; they should reflect changes in fortification levels of primary ingredients. Ideally, the database should not have missing values, and values calculated from similar food items should be identified (Buzzard et al., 1991; Juni, 1996; Nieman et al., 1992).
Other considerations when evaluating databases include whether the values are for food as consumed (rather than as purchased); nutrient analytical methodology used, including extent of sampling required and feasibility of addressing variability in nutrient content; and conventions and modes of data expression (Greenfield and Southgate, 1992; Rand et al., 1991).
When accurate food consumption data are not available, it may be more meaningful to compare food intake to food-based dietary standards (such as the Food Guide Pyramid [USDA, 1992]) than to compare nutrient intake to the DRIs.
Other Factors to Consider
For nutrients with a wide range of biological availability in food, a population's prevalence of inadequate intakes will be inaccurately estimated if the average bioavailability for foods chosen by individuals in the population differs from the bioavailability assumed when setting the Estimated Average Requirement (EAR). The distribution
of nutrient intakes also may be inaccurate if bioavailability varies within the population but is not considered when nutrient intake is estimated for each individual. Zinc, niacin, iron, and provitamin A carotenoids are nutrients with well-known issues of bioavailability. Nutrient equivalents are sometimes used (e.g., niacin equivalents for assessing niacin intake and retinol equivalents for assessing intakes of provitamin A carotenoids) (IOM, 1998b, 2000). The use of dietary folate equivalents to reflect the bioavailability of supplemental folate in contrast to folate naturally present in food has been recommended for evaluating dietary data (IOM, 1998b).
ISSUES OF VARIANCE IN DIETARY ASSESSMENT
Selecting a Representative Subsample of a Group
For large groups of people, it is not usually practical to assess the intake of every individual. Thus, a representative subsample is selected and assessed and the findings are extended to the full population. The methods used for ensuring that a sample is truly representative can be complex, but the results of an assessment can be misleading if the individuals who are assessed differ from the rest of the group in either intakes or requirements. Errors can arise if the sample is nonrepresentative. For example, a telephone survey might select more high-income participants by missing families who are too poor to own a telephone. Alternatively, the people who refuse to participate are not a random subsample (e.g., working mothers might be much more likely to refuse than retired people). Therefore, assistance from a statistician or other expert in survey sampling and design should be obtained (Dwyer, 1999; Van Staveren et al., 1994).
Determining Standard Deviations of Prevalence Estimates
Is the estimated prevalence of nutrient inadequacy in a population significantly different from zero?
Answering this question requires estimating the standard deviations assodated with the prevalence estimates.
The prevalence estimates obtained from the application of either the probability approach or the Estimated Average Requirement (EAR) cut-point method are exactly that: estimates. As such, there
is uncertainty associated with them and this uncertainty can, in principle, be reflected in a standard deviation for the prevalence. Uncertainty in the prevalence estimates can come from three sources: sampling variability, variability associated with the EAR, and variability associated with collection of intake data.
Any time a sample of individuals is used to make inferences about a larger group, a statistical error (often called sampling variability) is incurred. In the case of dietary assessment, not only are the intake data obtained for just a sample of individuals in the group, but also the sample of intake days is small for each of those individuals. Therefore, two sources of sampling variability are immediately identifiable —one arising from not observing the entire population and one arising from not observing intake on all days for each individual.
Statistical techniques can be used to estimate the amount of sampling variability associated with prevalence estimates, although the computations are complex. When standard deviations can be calculated, it is appropriate to report not only the prevalence estimate but also its standard deviation. For example, for group X the prevalence of inadequate intake of nutrient Y was a percent ± b percent, where a is the estimated percent prevalence of nutrient inadequacy and b is the standard deviation of the prevalence estimate. When b is small relative to a, the prevalence has been estimated with a good degree of accuracy.
An additional consideration when determining the sampling variability is the effect of the survey design. Dietary intake data are typically collected in complex surveys, and thus the survey design must be taken into account when estimating standard deviations. Additional information on the estimation of standard deviations under complex survey designs, or in particular, about the estimation of standard deviations for prevalence estimates can be found in Nusser et al. (1996) and Wolter (1985).
Variability Associated with the EAR
Variability associated with the EAR may increase the uncertainty around prevalence estimates. Both the probability approach and the cut-point method use the EAR when estimating prevalence of inadequacy. However, the EAR is itself an estimate, and thus has its own uncertainty. Practical statistical approaches have not yet been developed for combining the two uncertainties—those around intake
estimates and those around requirement estimates—into a single value that reflects the uncertainty around the prevalence estimate.
Variability Associated with the Collection of Intake Data
Other characteristics of dietary studies complicate the matter even further. Dietary intake data suffer from inaccuracies due to underreporting of food, incorrect specification of portion sizes, incomplete or imprecise food composition tables, etc. These factors may have a compound effect on prevalence estimates. In addition, systematic errors in measurement (such as energy underreporting) may increase the bias of the prevalence estimate. All of these factors have an effect on how precisely (or imprecisely) the prevalence of nutrient adequacy in a group can be estimated, and it is difficult to quantify their effect with confidence.
The software developed at Iowa State University (called SIDE) (Dodd, 1996) to estimate usual intake distributions also produces prevalence estimates using the cut-point method and provides an estimate of the standard deviation associated with the prevalence estimate. However, it is important to remember that the standard deviations produced by the program are almost certainly an underestimate of the true standard deviations because they do not consider variability associated with the EAR or with the collection of intake data.
Why should standard deviations be a concern?
Standard deviations of prevalence estimates are needed to determine, for example, whether a prevalence estimate differs from zero or any other target value or to compare two prevalence estimates.
The evaluation of differences in intakes requires the estimation of standard deviations of quantities such as prevalence of nutrient inadequacy or excess (e.g., Application 3 in Chapter 7). As another example, suppose that prevalence of inadequate intake of a nutrient in a group was measured at one point in time as 45 percent. An intervention is applied to the group and then a new estimate of the prevalence of inadequate intake of the nutrient is found to be 38 percent, a decrease of 7 percent. However, to accurately assess the effectiveness of the intervention, the standard deviations around the 45 and 38 percent prevalence estimates are also needed. If the standard deviations are small (e.g., 1 percent), then one could con-
clude that the intervention was associated with a statistically significant decrease in the prevalence of inadequacy. If the standard deviations are large (e.g., 10 percent), then one could not conclude that the 7 percent decrease was significant or that the intervention worked.
Finally, the part of the intake distribution being assessed affects the error associated with the estimate. Values in the tail of the distribution are harder to estimate (i.e., estimates are less precise) than values in the center of a distribution (such as means or medians). Thus, estimating prevalence of inadequacy of a nutrient is expected to be less precise for nutrients for which prevalence of inadequacy in the group is very low or very high (e.g., 5 or 95 percent) compared with nutrients for which prevalence of inadequacy is towards the center of the distribution (e.g., 30 to 70 percent) for the same sampling design and same estimation method.
Users of the Dietary Reference Intakes (DRIs) have many opportunities to minimize errors when assessing group and individual intakes. This chapter has focused on ways to increase the accuracy of both the requirement estimates (by considering the specific characteristics of the individual or the population) and the intake estimates (by ensuring that dietary data are complete, portions are correctly specified, and food composition data are accurate) and the importance of an appropriate sampling plan for group intakes.
Although users of the DRIs should strive to minimize errors, perfection usually is not possible or necessary. Comparing high-quality intake data with tailored requirement data to assess intakes is a meaningful undertaking and can, at a minimum, identify nutrients likely to be either under- or overconsumed by the individual or the group of interest.