The National Academies Press

Currently Skimming:

Pages 191-221

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.

From page 191... ... 185 CHAPTER 10 10. Assessment of Quality 10.1 Q-1: COMPUTING RESPONSE RATES 10.1.1 Background Proper calculation of response rates is important because response rates are used by analysts to assess survey quality. Read the entire page →
From page 192... ... 186 In the literature on response rate calculations, a sample is divided first into two groups. The first group is called the "eligibility known" group, and the second group is called the "eligibility unknown" group. Read the entire page →
From page 193... ... 187 response rates are not the only indicators of survey quality, they are important indicators that are readily quoted by survey practitioners, reinforcing the need for this item to be standardized. Response rates have become more of an issue because response rates have been falling over recent years (Ezzati-Rice et al., 1999; Dillman and Carley-Baxter, 2000; Dillman et al., 2001; Kalfs and van Evert, 2003) Read the entire page →
From page 194... ... 188 eA = estimated proportion of cases of unknown eligibility that are eligible (AAPOR eligibility rate: the same formula for calculating the eligibility rate is used) Read the entire page →
From page 195... ... 189 encounter difficulties as long as disposition codes are correctly labeled in terms of known and unknown eligibility. This would allow for the overall response rate to be calculated directly as demonstrated in equation 4: ………(4) Read the entire page →
From page 196... ... 190 Figure 26: Two-Stage Survey Process: Recruitment and Retrieval Equation 6 may make agencies aware that recruiting methods and materials used were not suitable, if the response rate calculated for this stage of the survey is poor. This exercise is even more beneficial to agencies wishing to undertake follow-up studies to surveys that yielded very poor overall response rates. Read the entire page →
From page 197... ... 191 CALL HISTORY FILES. The prime purpose for undertaking the analyses of call history files is to determine eligibility rates of the eligibility unknown sample units. Read the entire page →
From page 198... ... 192 Hard refusals 2 E Over quota county 60 I Second refusals 3 E No answer 101 U Disconnected number 4 I Busy 102 U System default 6 I 1/2 Busy 103 U Business number 8 I Call back specific 104 E Second language barrier not Spanish 13 I Call back non-specific 105 E Second fax machine/ modem 14 I System default (live number) 110 U Terminated interview/ Q BR 18 E First fax machine/modem 127 U Terminated Q1 50 E All other reasons 128 U Terminate out of area 51 I First refusals 140 E Bad zip code 52 I Answering machine 141 U Terminate Q20 53 E First language barrier not Spanish 143 U Terminate Q21 – household count 54 E Wrong number but second attempt chain – live 144 U Refused to participate at invite 55 E Language barrier Spanish 191 U Refused address component(s) Read the entire page →
From page 199... ... 193 For example, in the documentation for the second call history file, call dispositions "All other reasons", "Wrong number but second attempt chain – live", and "System default (live number) " were considered as ineligible sample units, which in turn, indicated that these numbers should not have been called again; the numbers were resolved. Read the entire page →
From page 200... ... 194 0 5 10 15 20 25 30 35 40 45 50 call 1 call 2 call 3 call 4 call 5 call 6 call 7 call 8 call 9 call 10 w eighted av. Call Number El ig ib ili ty R at e (P er ce nt ) Read the entire page →
From page 201... ... 195 0 10 20 30 40 50 60 70 80 90 100 call 1 call 2 call 3 call 4 call 5 call 6 call 7 call 8 call 9 call10 Call number Pe rc en t Figure 29: Percent of the Total Calls Made on Each Call Attempt that Were Non-Contacts For the first call history file examined, the percentage of calls that remain unknown (unresolved in this case because no calls determined as having an eligible status were called back) increased across the ten call attempts. Read the entire page →
From page 202... ... 196 0 5 10 15 20 25 30 35 40 call 1 call 2 call 3 call 4 call 5 call 6 call 7 call 8 call 9 call10 Call Number Pe rc en t %unknow n Figure 31: Percentage of Calls that Remain Unresolved after Each Call, File 2 Table 86 shows the response rate for the first household travel survey, using the CASRO and AAPOR formulas. In this case, the CASRO formula yielded a higher response rate. Read the entire page →
From page 203... ... 197 The eligibility rates for the second file examined are higher. This may be the result of the survey being able to interview households that speak Spanish only; therefore, the eligibility status of these households could be determined. Read the entire page →
From page 204... ... 198 The changes in overall response rates, as a result of a five call limit, range from a 0.3 percent reduction to a 0.7 percent reduction. It appears as though the CASRO method is slightly more sensitive to the five call limit than the AAPOR method; the reduction in the response rate for the CASRO method after a five call limit is greater than the reduction in response rate after a five call limit for the AAPOR method. Read the entire page →
From page 205... ... 199 Primary Disposition Code Secondary Disposition Code Out of sample 4.10 Fax/data line 4.20 Non-working/disconnected number 4.30 Non-working number 4.31 Disconnected number 4.32 Temporarily out of service 4.33 Special technological circumstances 4.40 Number changed 4.41 Cell phone 4.42 Residence to residence 4.431 Call forwarding 4.43 Nonresidence to residence 4.432 Pagers 4.44 Nonresidence 4.50 Business, government office, other organization 4.51 Institution 4.52 Group quarters 4.53 Not Eligible 4.0 No eligible respondent 4.70 Table 89: Final Disposition Codes for In-Person, Household Surveys Primary Disposition Code Secondary and Tertiary Disposition Code Complete 1.1 Interview 1.0 Partial 1.2 Refusal and break off 2.10 Household level refusal 2.111 Refusal 2.11 Known respondent refusal 2.112 Break off 2.12 Non-contact 2.20 Unable to enter building/reach housing unit 2.23 No one at residence 2.24 Respondent away/unavailable 2.25 Other 2.30 Dead 2.31 Physically or mentally unable/incompetent 2.32 Household-level language problem 2.331 Respondent language problem 2.332 Language 2.33 No interviewer available for needed language 2.333 Eligible, NonInterview 2.0 Miscellaneous 2.35 Unknown if housing unit 3.10 Not attempted or worked 3.11 Unable to reach/unsafe area 3.17 Unable to locate address 3.18 Housing unit, unknown if eligible respondent 3.20 No screener completed 3.21 Unknown Eligibility, Non Interview 3.0 Other 3.90 Out of sample 4.10 Not a housing unit 4.50 Business, government office, other organization 4.51 Institution 4.52 Group quarters 4.53 Vacant housing unit 4.60 Regular, vacant residences 4.61 Seasonal/Vacation/Temporary residence 4.62 Not Eligible 4.0 Other 4.63 Read the entire page →
From page 206... ... 200 Primary Disposition Code Secondary and Tertiary Disposition Code No eligible respondent 4.70 Quota filled 4.80 Table 90: Final Disposition Codes for Mail Surveys of Specifically Named Person Primary Disposition Code Secondary Disposition Code Complete 1.1 Returned Questionnaire 1.0 Partial 1.2 Refusal and break off 2.10 Other person refusal 2.111 Refusal 2.11 Known respondent refusal 2.112 Blank questionnaire mailed back, "implicit refusal" 2.113 Break off questionnaire, too incomplete to process 2.12 Non-contact 2.20 Other notification that respondent was unavailable during field period 2.26 Completed questionnaire but not returned during field period 2.27 Other 2.30 Death (including USPS category: deceased) 2.31 Physically or mentally unable/incompetent 2.32 Respondent language problem 2.332 Language 2.33 Wrong language questionnaire sent for needed language 2.333 Literacy problems 2.34 Eligible, "NonInterview" 2.0 Miscellaneous 2.35 Nothing known about respondent or address 3.10 Not mailed 3.11 Nothing ever returned 3.19 Unknown if eligible respondent in unit 3.20 No screener completed 3.21 Refused to accept 3.231 USPS category: refused by addresses 3.23 Refused to pay postage 3.232 USPS category: returned to sender due to various USPS violations by addressee 3.24 USPS category: illegible address 3.251 USPS category: insufficient address on mail from one Post Office to another Post Office 3.252 USPS category: no mail receptacle 3.253 USPS category: cannot be delivered 3.25 USPS category: delivery suspended to commercial mailing agency 3.254 Unknown whereabouts, mailing returned undelivered 3.30 USPS category: attempted – addressee not known 3.311 USPS category: postal box closed 3.312 USPS category: no such number 3.3131 USPS category: no such office in state 3.3132 USPS category: no such street 3.3133 No such address 3.313 USPS category: vacant 3.3134 USPS category: unable to forward 3.3141 USPS category: outside delivery limits 3.3142 Not delivered as addressed 3.314 USPS category: returned for better address 3.3143 Not delivered as addressed 3.314 USPS category: unable to forward 3.3141 USPS category: outside delivery limits 3.3142 Cannot be delivered as addressed 3.31 USPS category: returned for better address 3.3143 USPS category: moved, left no address 3.32 Unknown Eligibility, "Non Interview" 3.0 USPS category: returned for postage 3.33 Read the entire page →
From page 207... ... 201 Primary Disposition Code Secondary Disposition Code USPS category: temporarily away, holding period expired, unclaimed 3.34 USPS category: unclaimed – failure to call for held mail 3.35 USPS category: no one signed 3.36 Returned with forwarding information 3.40 Returned unopened – address correction provided 3.41 Returned opened – address correction provided 3.42 Other 3.90 Not Eligible 4.0 Selected respondent screened out of sample 4.10 10.2 Q-2: TRANSPORTATION MEASURES OF QUALITY 10.2.1 Definition A variety of data quality measures have been proposed in this study but, in this section, we consider variables that have not been used elsewhere. The type of variables considered are specific to personal travel surveys and are those that are temporally and spatially stable and, therefore, should acquire similar values among surveys. Read the entire page →
From page 208... ... 202 • The average trip rate (overall) per person; and • The average trip rates per household and per person for specific trip purposes. Read the entire page →
From page 209... ... 203 What proportion of these workers would make no trips is not known but the statistic does show that the source of immobility due to illness is small relative to that due to disability. Activity rates, like non-mobility rates, are statistics that are seldom reported. Read the entire page →
From page 210... ... 204 51urban travel surveys 1990-1995 NCHRP 236 8.91 Home interview surveys 1956-1976 NCHRP 187 7.6-14.1 12 urban travel surveys & NPTS 90 1985-1990 NCHRP 365 8.5-9.2 Ohio, Kentucky, Indiana Survey 1990 - 10.03 Salt Lake City 1993 - 13.8 NPTS 95 1995 NPTS 95 9.73 Baton Rouge Personal Tr. Survey 1997 LTRC/LSU 9.69 Dallas/Fort Worth 1996 NCTCOG 9.47 Nashville, Memphis, Knoxville, TN 1998-2003 Everett, 2003 8.04 - 8.44 South East Florida 2000 - 7.19 Florida - Schiffer, 2003 7.31 - 9.80 Twin Cities (urban) Read the entire page →
From page 211... ... 205 A problem with measuring trip rates at the household level is the impact household size has on the results. The effect of household size can be eliminated by observing trip rates per person. Read the entire page →
From page 212... ... 206 sampling frame, or from having units in the sampling frame that do not belong there. Non-response is failing to obtain a response from units that are within the sampling frame. Read the entire page →
From page 213... ... 207 Fx = sample population multiplied by the inverse of the sampling rate X~ = population from an external source. If the sample consists of a disproportionate stratified sample, the population estimated from the sample (Fx) Read the entire page →
From page 214... ... 208 estimate the coverage error on the sample as is. However, for coverage error that can be reliably corrected with weighting, these corrections should be made to the sample before the estimation of coverage error using the equation above. Read the entire page →
From page 215... ... 209 To use the incidence of proxy reporting as a measure of data quality, the definition, measurement, and interpretation of proxy reporting must be standardized. That is, a common understanding of what proxy reporting is, how it is measured, and how the results are interpreted, must be formulated so that a consistent expression of this measure can be generated in each data set. Read the entire page →
From page 216... ... 210 prepared the information being reported, it would seem advantageous to include questions that allow identification of proxy reporting in all types of surveys. Third, there is a need to evaluate the levels of proxy reporting produced. Read the entire page →
From page 217... ... 211 relative to the subject of each questionnaire was provided. Recommendations on using proxy reporting as a data quality measure are provided in section 2.7.4 of the Final Report. Read the entire page →
From page 218... ... 212 It is suggested that the following set of core questions be included in every validation survey conducted: 1. Did you complete the initial survey? Read the entire page →
From page 219... ... 213 Current practice of detecting and correcting errors in travel survey data tends to vary from survey agency to survey agency. While it is common practice to call respondents back to retrieve missing data on critical data items, the feasibility and logic checks used by agencies, and the practice of calling respondents back on these items, varies from agency to agency. Read the entire page →
From page 220... ... 214 10.7 Q-8: NUMBER OF MISSING VALUES 10.7.1 Definition The number of missing values in a data set is a measure of how much information was not collected. If expressed as a proportion of the total number of data items in the data set, it serves as a measure of the relative information content of the data. Read the entire page →
From page 221... ... 215 10.8 Q-9: ADHERENCE TO QUALITY STANDARDS AND GUIDELINES 10.8.1 Background One of the ways to improve the quality of data is to have a checklist of actions that must be performed or standards that must be met in each survey. Such a checklist is not currently accepted or used in reporting on household and personal travel surveys. Read the entire page →

From page 191...

... 185 CHAPTER 10 10. Assessment of Quality 10.1 Q-1: COMPUTING RESPONSE RATES 10.1.1 Background Proper calculation of response rates is important because response rates are used by analysts to assess survey quality.