Overall Building Condition and Student Achievement
Professional organizations and governmental agencies have for many years been calling attention to the deteriorating condition of the nation’s schools. The importance of effective operations and maintenance practices to the satisfactory performance of a school building’s envelope, mechanical systems, and surfaces has been emphasized in previous chapters.
The committee identified 20 studies that investigated the relationships between overall building condition and student achievement and overall building functionality and student achievement. The identified studies primarily come from the field of educational research and investigate possible relationships between individuals and groups and their physical environment.
In the course of its review, the committee identified significant limitations in the methodologies used in these studies (the limitations are discussed in a section near the end of this chapter). Nonetheless, the committee believes there is value in describing these studies and their findings because they could contribute to elaborating models for future research on green schools and their health and productivity performance outcomes. Considerations for future research are the focus of Chapter 10.
BUILDING CONDITION AND STUDENT ACHIEVEMENT
The committee identified eight studies that investigated the relationship between the overall condition of school buildings and at least two
student variables. The one consistent variable was student achievement as measured by some form of standardized or normed test.
Berner (1993) investigated the relationship between parental involvement, school building condition, and student achievement in the public schools of Washington, D.C. She hypothesized that the condition of public school buildings is affected by parental involvement and that the condition of the school building in turn affects student achievement. Using a regression model that included variables for race and household income, she analyzed relationships among the size and condition of school buildings, the extent of parental involvement, and the amount of funds parents raised for the local school and compared the results with student achievement as measured using average test scores on the Comprehensive Test of Basic Skills (CTBS).
Berner found that school size, parental involvement, and building condition did have an effect on student achievement scores. The analysis indicated that student test scores increased an average of 5 percent as the condition category of school buildings improved from poor to fair condition and from fair to excellent condition. Thus, students in buildings that rated as poor had test scores that were, on average, 5 percent lower than students in school buildings categorized as fair and 10 percent lower than students in buildings categorized as excellent.
Cash (1993) investigated the relationship between certain school building conditions, student achievement, and student behavior in rural high schools in Virginia. Cash essentially used the same methodology as Berner, although in this study the condition of the building was the independent variable, and student achievement and behavior served as dependent variables. The condition of school buildings was evaluated by local school system personnel using a questionnaire which was derived from previous studies that showed a positive relationship between a particular building condition and student achievement and behavior. The factors that were looked at included air-conditioning, classroom illumination, temperature control, classroom color, graffiti, science equipment and utilities, paint schedules, roof adequacy, classroom windows, floor type, building age, supporting facilities, condition of school grounds, and furniture condition. The presence or absence of these factors or, in some cases, their quality or adequacy determined the condition category of the building: substandard, standard, and above standard.
Student achievement was measured by student test scores on the Test of Academic Proficiency (TAP), which was administered to all eleventh-graders in Virginia. The ratio of students receiving free and reduced lunches was used to control for socioeconomic status, and the Virginia Composite Index was used as a measure of local fiscal capacity, to control for the wealth of the school jurisdiction.
Cash found significant differences between the achievement scores of students in substandard buildings and those in above-standard buildings when the overall condition of the building was used as a measure. She also found that students were more affected by the cosmetic than the structural condition of a building. The difference between test scores of students in substandard and above-standard buildings ranged from 2 to 5 percentile points, depending on the subtest (i.e., mathematics, reading).
Earthman et al. (1996) used a similar methodology to conduct a study in North Dakota that included all 199 high school buildings, but used the CSTB to measure achievement. North Dakota was selected because its students traditionally score among the highest in the nation on the Scholastic Aptitude Test and the state has a relatively homogeneous, mostly rural population. Although the differences in the composite score were exactly the same as for the Cash study, there were some notable differences. The CTBS had additional subtests that the TAP did not have, such as reading, vocabulary, mathematics concepts, and spelling. In all but one subtest (social studies) of the CTBS, students in above-standard buildings outscored students in substandard buildings. The difference ranged from 1 to 9 percentile points.
Hines (1996) used the same methodology and data-gathering instrument as Cash to study large urban high schools in Virginia, and his results were basically the same: test scores for students in above-standard schools were 9 points higher for writing and science, 15 points higher for reading, and 17 points higher for mathematics compared with the same scores for students in substandard buildings.
Lanham (1999) studied the relationship between classroom conditions and student achievement in the elementary schools of Virginia that housed both third- and fifth-grade students, using the same general approach as Cash (1993). From a total of 989 elementary schools, a random sample of 299 schools was drawn. Responses were received from 197 schools, representing 66 percent participation. Lanham concluded that although certain school building and cosmetic components and features explained some of the variance in student achievement scores, the socioeconomic status of the student as represented by participation in the free and reduced-price lunch program explained most of the variance.
Schneider (2002) investigated the relationship between the condition of school buildings and student achievement scores in Washington, D.C., and Chicago, Illinois. The researcher used the reading and math scores on the Stanford Achievement Test in Washington, D.C., and the Iowa Test of Basic Skills in Chicago. After controlling for factors such as poverty, ethnicity, and school size, Schneider reported that the students in schools with good conditions were performing from 3 to 4 percentage points better on reading and math than students in buildings with poor conditions.
Lewis (2000) conducted a study based on 139 elementary, middle, and high school buildings in Milwaukee, Wisconsin. All buildings were evaluated for both condition and adequacy. The Wisconsin Student Assessment System (WSAS) was used to measure student achievement. Fourth, eighth, and tenth graders were assessed in reading, mathematics, language arts and writing, science, and social studies. Scores on these examinations were reported as a percentage of students in each school building who were achieving at or above the level “proficient.” Lewis (2000, p. 11) concluded that the “significant relationships for facility measurements typically explain about 10 to 15 percent of the differences in scores across schools when the influences of the other variables were statistically controlled.” When comparing student demographic indicators such as mobility rates, eligibility for free/reduced-price lunches, attendance, and suspensions, only 9 estimates out of 48 were found to be significant. Thus those indicators that were significant explained between 8 and 28 percent of the difference between test scores when other variables were controlled.
Picus et al. (2005) designed a study to examine whether higher quality buildings are related to student performance. The methodology used falls loosely into the tradition of large-scale econometric studies, which are discussed in Chapter 10. This study included approximately 300 public schools, accredited institutions, and accredited private schools in Wyoming. Building quality data were gathered in response to a court ruling related to the adequacy of the state’s school funding system. Building condition scores were determined by collectively assessing up to 22 building subsystems (e.g., foundations, floors) using individual rating tools consisting of 1 to 20 questions, the answers to which were agreed on by a school representative and a subcontractor to a consultant. The consultant weighted the subsystem assessments relative to the cost of bringing the affected components into as-new condition and then averaged all the subsystem scores together to produce an overall condition score for each building.
The suitability tool purported to measure the degree to which each school was suitable for its current use, for example, whether the school was designed specifically for the grades it currently served. Ratings for suitability were self-reported by district superintendents or their designees; the report authors noted that the “suitability tool possessed a higher degree of subjectivity than the building condition instrument” (Picus et al., p. 81).
Student achievement was measured using a set of tests administered to all fourth, eighth, and eleventh graders in Wyoming (WyCAS). Three years of WyCAS results (1999-2001) were used and approximately 60,000 students were involved. The WyCAS tests comprise both multiple-choice and open-ended questions in reading, writing, and mathematics.
Two different measures of each school’s achievement in each content area were used: the 3-year average of the percentage of students whose performance was “proficient” or “advanced” and the 3-year average of the scale scores. In both cases, averages for reading, writing, and mathematics were combined to arrive at an overall proxy for student achievement at a school. Correlations with building scores were computed separately for each grade level for each content area and year.
Multiple regression analyses were used to examine the relationships between building and WyCAS scores while factoring out the influence of socioeconomic status in elementary school students (as measured by the percentage of free and reduced-price lunches); there was no control for socioeconomic status in middle and high school students. After running a series of analyses, the authors found no relationship between building condition and student achievement. The “finding implies that as building condition improves, there is no likelihood that WyCAS scores will either improve or decline” (Picus et al., 2005, p. 84).
SCHOOL BUILDING FUNCTIONALITY AND STUDENT ACHIEVEMENT
Twelve studies (including Picus et al., 2005) were identified that investigated the relationship between school building functionality and student achievement. In eleven of the studies, the age of the school building was used as a surrogate for functionality. Although the age of a building might not in and of itself directly influence student achievement, an older building might not have qualities or facilities—such as thermal control, proper lighting, acoustical control, support facilities, proper laboratories, and pleasing appearance—that could affect student achievement. In the Picus study, a suitability index was calculated based on factors other than age of a building.
Using the variable of school building age, McGuffey (1982) reviewed seven studies (Thomas, 1962; Burkhead et al., 1967; Michelson, 1970; Guthrie et al., 1971; McGuffey and Brown, 1978; Plumley, 1978; and Chan, 1979). In all cases, as building age increased, student achievement decreased.
McGuffey and Brown (1978) studied 188 school districts in Georgia to explore the relationship between building age and student achievement. They used the scores on the Iowa Test of Basic Skills for fourth- and eighth-grade students and the TAP for eleventh-grade students. The statistical analyses indicated that building age could account for 0.5 percent to 2.6 percent of the variance in test scores among fourth-grade students, 0 percent to 2.6 percent of the variance among eighth-grade students, and 1.4 percent to 3.3 percent of the variance among eleventh graders.
Garrett (1981) hypothesized that when the socioeconomic status variable was statistically controlled for, the age of a facility would have a significant correlation with the achievement of students and that the achievement of students taught in unmodernized school facilities would be significantly lower than those taught in partially or fully modernized schools. When the variable for socioeconomic status was statistically controlled for, the age of the facility made a significant difference in student achievement in composition, reading, and mathematics scores on the TAP (.01). The achievement of students taught in unmodernized facilities was not significantly lower than that of those taught in partially modernized schools. However, the achievement of students taught in partially modernized schools was significantly lower than that of those taught in modern facilities.
Chan (1982) compared student attitudes toward a new school and an older school. The researcher had four hypotheses: (1) no significant difference between student attitudes toward a new building and attitudes toward an old building; (2) no difference between the attitudes of male and female students toward old and new buildings; (3) no significant difference in the attitudes of students of different races toward old and new buildings; and (4) no significant difference in attitudes between students who pay for school lunches and those who receive free and reduced-price lunches.
Chan’s study used a quasi-experimental, nonequivalent control group design. The control group consisted of the 119 students in the second, third, and fourth grades in a school built around 1936. The experimental group consisted of 96 students in those same grades in a 1923-constructed building who were transferred to a new school.
After statistically adjusting the post-test scores of the control group with the corresponding pre-test scores of the experimental group, students in the experimental group scored 19 points (on a 55-point scale) higher on average than students in the control group. The difference in attitude scores was indicated by an F-value of 19.71, which was significant at the .0001 level. Race and socioeconomic status had no effect on student attitudes toward their school buildings. However, female students in the control group scored significantly higher than males on both pre- and posttests. All were significant at the .05 level.
Bowers and Burkett (1989) investigated the differences in student achievement, health, attendance, and behavior between two groups of students in two elementary school buildings in rural Tennessee. One school had recently been opened and was a modern building in all respects. The other building had been constructed in 1939 and had experienced few improvements to the physical structure. Two hundred eighty randomly selected fourth- and sixth-grade students in the two facilities were the
subjects of the study. Principals, teachers, and socioeconomic levels of the communities were similar. The variable of age of the facility was the only major difference when comparing the achievement and behavior of the students.
Students in the modern building scored significantly higher in reading, listening, language, and arithmetic than students in the older facility (greater than .01). Discipline was needed less frequently in the new facility, even though the new school had a larger enrollment. The level of significance for analysis purposes was .01. Students in the new school building significantly outperformed students in the older building in reading, listening, language, and arithmetic. Faculty in the new building reported fewer disciplinary incidents and health issues than faculty in the old building. Attendance also was higher among students in the new building.
Phillips (1997), replicating an earlier study by Plumley (1978), found a relationship between the age of the school facility and student reading achievement scores as measured by the Iowa Test of Basic Skills and between student mathematics achievement scores and building age. The average mathematics scores for those students in new buildings increased 7.63 percentile ranks after moving into the new facility. He did not find any significant differences in attendance patterns of students enrolled in the old and new buildings.
As noted above, Picus et al. (2005) analyzed data based on a suitability tool that was purported to measure the degree to which each school building was suitable for its current use. The authors noted that this tool was more subjective than the building condition index used in the study. They found little evidence of a relationship between the suitability scores and WyCAS test scores.
LIMITATIONS OF THE CURRENT STUDIES
These studies, which found some correlation between some measure of overall building condition and student achievement, differed in several ways. Some used age as a surrogate for building condition, while others used a subjective rating of building condition. Most focused on average student achievement at the school level but a few looked at differences between individual student achievement in a modern facility considered to be in good condition and a school that was old and out of date. The studies also included urban and rural schools in several different states and in the District of Columbia. With 19 out of 20 studies showing increases in test scores for students in buildings in better condition, one might reasonably assume a relationship exists between building conditions and student achievement. In fact, the limitations of the methodologies and data used
in these studies may reflect a consistent underlying bias rather than a consistent, albeit undefined, cause-and-effect relationship.
Two specific limitations lead to this conclusion. The first is the issue of omitted variable bias. None of the 20 studies included a complete set of variables that are considered to be related to student achievement. For example, parental education, family structure, family income, and teacher quality have been shown to be related to student achievement, but they were not measured. Omitting them increases the likelihood of a large positive bias for the variables that were included in the correlational or regression analysis. Thus, the coefficients on the measures of overall building condition are likely to have been inflated and their statistical significance is likely to have been established based on omitted variable bias.
The second significant limitation is that the relationships between overall building condition and student achievement were likely confounded because students of different socioeconomic status were not randomly assigned to schools. Minority students and students from low-income households are more likely to attend schools that are older and in substandard condition. In other words, students may not end up in old and poorly maintained schools by chance. This could give rise to reverse causality and almost certainly to ambiguity about the direction of the relationship between student achievement and overall building condition.
Additional methodological shortcomings were also present. For example, the Picus et al. (2005) study uses correlations and regression analysis or school-level averages. The current state-of-the-art research estimates multilevel models to assess not only the differences in average achievement but also the differences at the individual student level. Thus, building condition could affect the relationship between prior achievement and current achievement for individual students, meaning the students perform differently in different quality environments controlling for prior levels of achievement. This factor was not tested directly, although the study did test change in average test scores. As will be discussed in Chapter 10, the power to detect effects is severely limited in a school-level study in a small state, and the efficiency of the estimates is reduced as well.
Similarly, combining scores for different achievement domains, as was done in several studies, is a dubious practice for educational research. It is common to find that educational reforms affect math achievement more than reading achievement: By averaging math, reading, and writing together, important variation could have been missed.
Although this particular set of studies has methodological limitations, the literature on indoor environmental conditions, including lighting, noise, air quality, dampness, surface contamination, and ventilation,
provides evidence that specific, as opposed to overall, building conditions adversely affect the indoor environments of schools and may hinder learning and impact the health of teachers and students. For example, the studies of Wargocki et al. (2005) were designed as crossover longitudinal studies intervening on specific elements (ventilation, temperature) while holding other conditions constant. These studies demonstrated modest improvements in student performance on routinely used weekly tests of verbal and math skills. Perhaps the research that attempts to relate overall building condition to student achievement is asking the wrong question. To understand how building conditions affect student and teacher performance, it would be better to measure one or more building performance characteristics, develop a theory linking the performance characteristics and student and/or teacher outcomes, and test the linkage using adequate measures of the outcomes of interest and fully specified regression models. Issues related to improving research are discussed in detail in Chapter 10.
CURRENT GREEN SCHOOL GUIDELINES
Current green school guidelines encourage attention to school maintenance through measures such as a computerized district maintenance plan that inventories all equipment—including HVAC, lighting, roofing, and control systems—and establishes annual tasks, with the labor and material required for their maintenance. In combination with an indoor environmental quality management plan, a computerized maintenance management plan is intended to ensure that the performance of a green school is maintained over its service life.
Finding 8: The methodologies used in studies correlating overall building condition with student achievement are not adequate to determine if there is a relationship between overall building condition and student test scores. This research tradition seems to address a more general and diffuse question and does not produce high-quality evidence relative to either school design or specific aspects of maintenance. Improved research for understanding how specific building conditions affect student and teacher performance would measure one or more building performance characteristics, develop a theory linking those characteristics and student and/or teacher outcomes, and test the linkage using adequate measures of the outcomes of interest and fully specified regression models.