in norms with respect to evidence, warrants, publication, and peer-review practices.
Conclusion 16: Evaluation reports on particular programs provide an important source of evidence that can inform practice and theory more generally. Other kinds of research and data are needed, however, to build and empirically shape a shared knowledge base.
Evaluations can be designed to support improvement during the design and implementation phase (formative) and to measure final impact of educational practice (summative). Although a substantial body of high-quality evaluation reports informs the knowledge base on learning in informal settings, important findings are not always widely disseminated and reports can be difficult to obtain. Also, evaluation is typically carried out by external evaluation consultants hired by the science learning institution. This arrangement can result in uncomfortable conflicts between the material interests of the institution to document successes and the interests of obtaining even-handed and theoretically oriented analysis. Many opportunities to learn from evaluations are lost as reports of outcomes are often not accompanied with careful description of practice or relevant comparisons to prior efforts and findings.
Conclusion 17: There is an interdisciplinary community of scholars and educators who share an interest in developing coherent theory and practice of learning science in informal environments. However, more widely shared language, values, assumptions, learning theories, and standards of evidence are needed to build a more cohesive and instructive body of knowledge and practice.
The literature reviewed in this report is derived from widely varied traditions, including researchers in different academic disciplines, evaluators and communities of inquiry and practice associated with informal learning institutions. Although their disciplinary and organizational affiliations vary, these scholars and educators share common interests in understanding, building, and supporting science learning in informal environments. Further development of common frameworks, standards of evidence, language and values will require new ways to share knowledge and expertise. Several leading thinkers have recognized this need. Journal special issues, the new Center for Advancement of Informal Science Education, and new guidelines from the National Science Foundation on evaluating the impact of informal science education have initiated and furthered this work, with the goal of contributing to better knowledge integration.