Measuring Serious Emotional Disturbance in Children Workshop Summary (2016) / Chapter Skim
Currently Skimming:
4 Design and Estimation Options
4 Design and Estimation Options
Pages 39-66
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 39... ...
Both the NSCH and the NS-CSHCN were conducted in partnership with the Centers for Disease Control and Prevention and were fielded as a module of the State and Local Integrated Telephone Surveys. Both surveys began as random-digit dial surveys, limited to landlines, but recently began including a cell phone sampling frame, due to the challenges with landline-only telephone interviews.
|
From page 40... ...
The list of conditions includes anxiety problems, depression, behavioral or conduct problems, substance abuse disorder, developmental delay, intellectual disability, speech or other language disorder, learning disability, attention deficit disorder or attention deficit hyperactivity disorder (ADD/ ADHD) , autism and autism spectrum disorder, and Tourette syndrome, as well as a fill-in-the-blank question for any other mental health condi For more information on the Title V Maternal and Child Health Services Block Grant 1 Program, see http://mchb.hrsa.gov/programs/titlevgrants/ [October 2015]
|
From page 41... ...
discussed design and estimation considerations for multiphase studies of child serious emotional disturbance, primarily focusing on the statistical objectives, multiphase design choices, and measurement error. Earlier presentations and discussion had focused on the psychometric properties of some of the available measures, and Heeringa's talk focused on the statistical implications of the various levels of sensitivity and specificity associated with the measures.
|
From page 42... ...
is validated, calibrated outcome of interest; θ is population parameter. SOURCE: Workshop presentation by Steven Heeringa.
|
From page 43... ...
One is the expected prevalence rate in the target population. The expected prevalence rate for serious emotional disturbance will drive the sample size requirements to obtain certain levels of precision.
|
From page 44... ...
Variable subsampling and weighting of Step 1 positive and negative screens increases the variance of population prevalence estimates and inflates variances of estimates for analyses of true subpopulation cases. Finally, if a calibration is needed, there is a potential for classification bias.
|
From page 45... ...
Table 4-3 shows formulas for the expected disposition of Step 2 eligible cases in a two-phase design. When thinking about the second objective of trying to identify a subsample of individuals for in-depth study, one example would be a longitudinal study of children to determine whether at a particular developmental stage they meet the criteria of a serious emotional disturbance.
|
From page 46... ...
In this case, the variance is a function of both the subsampling rate for the Step 1 negative screens and of the specificity of the Step 1 screener. The table shows that there would be an increase of more than 50 percent in variance if the specificity was 0.8 and one in four of the negative Step 1 screens were sampled.
|
From page 47... ...
SOURCE: Workshop presentation by Steven Heeringa, June 2015. In some situations, more information may be suitable to incorporate into the survey design and inference from the survey data collection, which is often referred to as model-assisted survey sampling: this is shown in the second box of the pyramid.
|
From page 48... ...
Step 1 involved screening and stratified subsampling for follow-up of the HRS panel, which was based on a stratification that used an externally estimated model relating probability of dementia to age, education level, Langa, K.M., Plassman, B.L., Wallace, R.B., Herzog, A.R., Heeringa, S.G., Ofstedal, M.B., 3 Burke, J.R., Fisher, G.G., Fultz, N.H., Hurd, M.D., Potter. G.G., Rodgers.
|
From page 49... ...
Once the uncertainty associated with the predictor is incorporated, either through multiple imputation or some other method, these predicted probabilities can then be added to the data file and used not just as dependent variables, but also as independent variables. In the case of child serious emotional disturbance, once these probabilities are assigned to a child's data record, for example, in the National Survey on Drug Use and Health (NSDUH)
|
From page 50... ...
It would be important, however, to look at the distributional properties, in other words, at normality and censoring of the tails at 0 or 1 in the case of linear regression modeling. MODEL-BASED ESTIMATES OF PREVALENCE IN A NATIONAL SURVEY Heather Ringeisen and Jeremy Aldworth discussed research that RTI International conducted on behalf of SAMHSA to develop model-based methods for use in an existing national survey to estimate the prevalence of serious emotional disturbance in children and serious mental illness in adults.
|
From page 51... ...
Ringeisen explained that the pilot study method was based on developing predictive models in which the five-item SDQ and a sixth impact item from the SDQ were the independent variables that predicted serious emotional disturbance, which was then determined on the basis of "gold standard" clinical interview data. The sample for the pilot study included two groups, both taken from participants in the final three quarters of the 2011 NHIS and the first quarter of the 2012 NHIS: one group was the parents of children aged 4-17; the other was youth aged 12-17 who reported about themselves.
|
From page 52... ...
The negotiations were hindered by the lack of a clear precedent, Ringeisen noted. Once the data collection could commence, a "gold standard" clinical interview was used as part of the study to ascertain serious emotional disturbance status.
|
From page 53... ...
The Federal Register definition of child serious emotional disturbance requires the presence of a past year mental disorder (see Chapter 1) , and there are a number of diagnostic assessments that ask about the past year.
|
From page 54... ...
The steps in a model-based method to estimate child serious emotional disturbance can be described as follows: • Administer a predicting scale, such as the SDQ, to all eligible respondents in the main survey. • Select a subsample from among the eligible respondents in the
|
From page 55... ...
The methodology was similar to that used by RTI to estimate child serious emotional disturbance, but while the child study required three distinct serioius emotional disturbance measures that varied by age group and led to the decision to develop three different statistical models by age, the serious mental illness study involved a single measure for the entire adult age span. Aldworth said that one challenge with the adult project was the effect of the weights.
|
From page 56... ...
For the gold standard estimates, the measure is assumed to be truth for the selected respondents. However, Aldworth noted, if the subsample size is too small, the gold standard prevalence estimates may be subject to large design-based sampling error, particularly at the subgroup level.
|
From page 57... ...
Next Steps After Aldworth's presentation, Ringeisen summarized some of the next steps needed, particularly for prevalence estimates of child serious emotional disturbance. She said that a priority is to develop a welloperationalized definition of the concept, amenable to estimation in a national study.
|
From page 58... ...
Finally, it will be important to address challenges associated with the varying reference periods that exist between the Federal Register definition of child serious emotional disturbance, predictive tools embedded in candidate national surveys, and "gold standard" clinical assessment tools. Kalton asked whether it would be necessary to include the respondent's state of residence as one of the variables in the model, such as the one used in the MHSS, in order to accomplish the goal of producing statelevel estimates.
|
From page 59... ...
SMALL-AREA ESTIMATION OF PREVALENCE Alan Zaslavsky (Harvard Medical School) discussed a small-area estimation project to estimate the prevalence of serious emotional disturbance among children in schools.4 Small-area estimation is useful when sample sizes are inadequate to produce direct estimates.
|
From page 60... ...
Zaslavsky explained that he and his colleagues fitted a Bayesian bivariate multilevel regression model with correlated effects for the probability of serious emotional disturbance and the augmented K6 (K6+5) score at
|
From page 61... ...
Children who have a high score on the K6+5 will tend to have a high probability for serious emotional disturbance, but those are only related through a covariance. The covariate effects are shown on the sides of the diagram, Zaslavsky noted.
|
From page 62... ...
. This correlation means that although the school's overall mean for serious emotional disturbance and K6+5 scores are not perfectly related to each other, they are strongly correlated, which is a measure of the validity of the K6+5 scale as an approximation or proxy for serious emotional disturbance.
|
From page 63... ...
Figure 4-5 shows a naive individual-level prediction. If the school information was not available, it would be possible to use the individual K6+5 data and random effects to predict the serious emotional disturbance scores for the same children, ignoring the clustering.
|
From page 64... ...
NOTE: This scenario involves collecting K6+5 measures for a sample and predicting serious emotional disturbance scores for the same sample. The design with sampling within school combines in- and out-of-sample prediction.
|
From page 65... ...
The correlations will be mostly determined by the correlation between K6+5 and serious emotional disturbance at the school level, which was very high. The prediction for individuals, that is, predicting children's serious emotional disturbance scores on the basis of their K6+5 scores, is a special case of in-sample prediction.
|
From page 66... ...
Zaslavsky concluded by saying that it is technically possible to combine a short screening scale with a calibration survey, and the calibration survey can be something that is practical to administer. The short screening scale could be integrated with the collection of school health information or some other activity, and it could even be done online.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.