The Importance of Common Metrics for Advancing Social Science Theory and Research A Workshop Summary (2011) / Chapter Skim
Currently Skimming:
2 Measurement in the Social Sciences
2 Measurement in the Social Sciences
Pages 7-30
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 7... ...
He began by introducing measurement in the physical sciences and then discussed measurement approaches in the social sciences, touching in particular on seminal developments that have facilitated or impeded progress. He also introduced the topic of index construction, observing that indicators often turn out to be determinants of the construct rather than just reflecting it.
|
From page 8... ...
In social science disciplines, the lack of strong theories is often reflected in the lack of well-accepted common metrics. MEASUREMENT STANDARDIZATION IN THE SOCIAL SCIENCES According to Bohrnstedt, there are some clear, tangible measures in the social sciences -- such as birth, age, marital status, number of children -- but the picture becomes murkier when one considers such concepts as attitudes, values, and beliefs at the individual or organizational level, or such concepts as school climate and organizational learning, or societal-level concepts, such as anomie and social disorganization.
|
From page 9... ...
This is sometimes called a "formative" as opposed to a "reflective" model of index construction. Examples include an index of socioeconomic status, consisting of education, income, and occupation, and the consumer price index, which is based on a market basket of goods and services.
|
From page 10... ...
In Hauser's estima tion, the official poverty line has been overused in thousands of research papers and books, and perceptions about poverty and the poor would differ if a standard measure of greater validity were widely accepted. Academic achievement levels offer a more recent example of a • nominally social scientific, standardized measure that has become visible and influential in public discourse and policy.
|
From page 11... ...
to recommend standards for adult literacy that could be used in the NAAL and applied retroactively to the National Adult Literacy Study in order to compare literacy lev els across the decade among all adults and specific population groups. The NRC report Measuring Literacy: Performance Levels for Adults (National Research Council, 2005)
|
From page 12... ...
The problem is that it suppresses the main socioeconomic dimension that comes out so clearly in the Edwards scale. 3 The Census Bureau follows the Standard Occupation Classification system, which is devel oped by an interagency group and agreed on by the Office of Management and Budget.
|
From page 13... ...
Hauser closed this discussion by raising the broader problem with the use of any of the standard measures of "social class": the belief that these, or closely related measures of social standing, taken alone, fully represent the social and economic standing of a person, household, or family. In his view, this simplistic view fails to recognize the complexity of contemporary systems of social stratification, in which inequalities are created and maintained in a substantially but by no means highly correlated mix of psychological, educational, occupational, and economic dimensions.
|
From page 14... ...
The story of the Duncan SEI is a case history of the rise and fall of a standard sociological measure that became obsolete over time. There is now an international socioeconomic index developed by Treiman and colleagues that is well suited for comparative work.6 Normalization of Metrics Multiplicative scales and log transformations are analytic schemes for normalizing metrics to achieve comparability in levels or effects.
|
From page 15... ...
First, how healthy is measurement science in the social sciences? Understanding common metrics to advance social science theory as the focus of the workshop, Bachrach probed whether theory is actually advancing metrics, common or not, in an adequate fashion in the social sciences.
|
From page 16... ...
Thus, common measures alone are insufficient if there is a lack of common understanding as to what those measures represent. She identified the structure of peer review as yet another set of factors that influences the health of measurement science in the social sciences.
|
From page 17... ...
Another example is the Duncan socioeconomic index, a measure that has been extremely successful in advancing research on social mobility. It, too, has required adaptation because of changes in the occupational structure itself and because of changes in the labor force.
|
From page 18... ...
Willis discussed another approach, which is to complement socalled objective measures with more subjective ones. He also argued that established statistical agencies have had to apply economic theory in order to produce economic data that are useful and credible for science and for policy.
|
From page 19... ...
National income measurement is based on a close connection between economic theory and the definition of the measurement tasks. In the 1930s, the project moved to the newly formed Bureau of Economic Analysis (BEA)
|
From page 20... ...
• Real income and related measures provide meaningful, interperson ally and intertemporally comparable measures of welfare that can be compared across subgroups. Willis elaborated on the idea that real income, which is income adjusted for inflation, can be used for economic welfare analyses that are relevant to policy often without knowing very much about individual characteristics or preferences.
|
From page 21... ...
Another is to relax the economist's preference for objective data and revealed preference in favor of subjective measures. He sees very few measures based on implicit markets or subjective measurement ready for standardization in the sense of official statistics.
|
From page 22... ...
One level up are summary health status measures that proxy pointin-time summaries of a person's health, but with respect to a particular disease or organ. They are sensitive to changes in symptoms or functional impairment due to a particular disease process.
|
From page 23... ...
According to Fryback, the key to making meaningful comparisons over time and across populations is the systematic collection of standardized measures with sufficient sample sizes. To date in the United States, only a few data sets have suitable measures, and only one has committed to longitudinal data collection.
|
From page 24... ...
Perhaps the most contentious issue among the different indexes is where they place the dead. Three of the scales have health states worse than dead.
|
From page 25... ...
discussed the role of theory in economic measurement, but balanced it with a discussion of the limitations of economic theory as a guide to economic measurement. On the measurement of medical care in economics, he tied Fryback's presentation on medical outcomes measures to problems in the economic measurement of medical care prices and output to show how some measurement problems in economics require information from outside economics.
|
From page 26... ...
There is therefore a linkage between the basic macroeconomic theory, the macro structure of the ac counts, and macroeconomic analysis, which is based on the theory. Consumer price index: Triplett noted that the Bureau of Labor • Statistics (BLS)
|
From page 27... ...
distilled some of the main points from Fryback's presentation and focused on challenges and opportunities related to the measurement of HRQoL. Turning attention to the three classes of HRQoL measures -- generic health indices and profiles, disease-specific measures, and preference-based measures -- and their interplay, Cagney considered how generic and disease-specific measures focus on the presence, absence, severity, frequency, or duration of symptoms and how these are drawn from psychometric theory, whereas the preference-based measures relevant for assessing preferences of individuals for alternative health states or outcomes are drawn from economic theory and ideas of comprehensiveness and comparability.
|
From page 28... ...
She endorsed the idea of potentially triangulating survey data resources with clinical assessments that come from the hospital or from a physician's office. There is also opportunity to focus on a framework for the study of cultural comparisons, to consider the larger social context and the bridging of mental and physical components, and to operationalize the social component for inclusion in social surveys.
|
From page 29... ...
Fryback agreed that the tension between essentially descriptive detail and the ability to summarize aggregated higher levels with standard measures is very real for health measures. His interest has mostly been in the measures that aggregate rather than disaggregate for deep understanding of pathways to outcomes.
|
From page 30... ...
He elaborated that in comparisons of indicators from more than one country, it is important that observed differences be attributable to actual differences in behavior of the people in those countries and not to differences in measurement. Hauser added occupation-based measures of social class as a positive example of harmonization, for which it is relatively easy to obtain all the information needed to produce several different measures in a single survey operation.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.