National Academies Press: OpenBook
« Previous: Appendix A: Information Provided and Not Provided in Response to Committee Requests
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×

Appendix B

Sample Interview Protocol

The committee and its contractors conducted a total of 44 interviews with a wide range of people who are involved with the school system in D.C., including officials and other employees in city education agencies and community members who have been active in public education-related work (see Chapter 1). They were asked a range of questions about their experiences and views of the Public Education Reform Amendment Act (PERAA), with somewhat different interview protocols, depending on the interviewees’ roles.

This appendix reproduces an example of the interview protocols used in the interviews conducted by the committee and staff with city leaders: see Chapter 1 for a discussion of the interviews.

Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×

INTERVIEW TOPICS

As you know, the National Research Council is conducting an evaluation of how D.C.’s schools are faring since PERAA was passed that draws on a variety of information. To understand how PERAA’s reforms are operating on the ground and to help us in interpreting the information we’ve been collecting, we are interviewing a sample of those most directly involved in implementing reforms. Your responses will be kept confidential: nothing you say will be attributed to you personally or to your office.

Although we have some familiarity with the responsibilities of your office, in our interview, we would like to get a better sense of the details of your work.

  1. Of all the functions for which your office is responsible, which ones have been the most central to the smooth implementation of PERAA?
  2. [Ask as appropriate, given responses to the first question.]

You’ve mentioned your office’s most important functions, now we would like to ask you in greater detail about some of those activities [and also about a few others that have come up in our research.]

a. Could you briefly describe the process by which the IMPACT system was developed, and the most important factors that influenced its design?

PROBES:

  • objectives that DCPS leadership hoped to accomplish with IMPACT
  • desired balance between evaluation and support/ professional development in the system’s design
  • sources for the ideas and assumptions about how IMPACT would work
  • research and other types of evidence used to inform its development
  • role of Mathematica in IMPACT’s development as compared with that of DCPS staff
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×
  • how IMPACT was validated and piloted, including who was asked to provide feedback on its initial results

b. To what extent has IMPACT lived up to your expectations as a strategy for ensuring effective teaching?

PROBES:

  • any problems with its implementation (e.g., resource and time requirements; level of observers’ preparation and expertise; other technical, administrative, or political challenges)
  • teachers’ responses to IMPACT
  • public and media reactions
  • effects on the overall quality of the teacher workforce in DCPS
  • relationship to student learning outcomes

c. Have any aspects of IMPACT been significantly modified since its initial implementation?

If yes: — what was the impetus for the modifications?

d. IMPACT seems to be one of the most controversial of the PERAA-related reforms, including being subject to considerable media scrutiny. Why do you think that has been the case?

To what extent are criticisms of either the technical quality of IMPACT or assertions that it is unfair to teachers valid?

3. a. What have been the main strategies in DCPS’ efforts to strengthen the quality of school leadership?

PROBES:

  • principal recruitment strategies and selection criteria
  • impetus for recent changes in the evaluation system for principals
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×

b. What would you describe as the main differences between the characteristics of school principals prior to PERAA and those currently leading DCPS schools?

c. Some commentators have questioned the relatively high level of turnover in DCPS principalships since PERAA’s enactment. How do you respond to that concern?

4. a. In considering all the changes that have been implemented in D.C. since PERAA, which ones do you think have been especially effective in accomplishing their intended purpose?

PROBE:

  • likely reasons for their success

b. Are there others that have fallen short of expectations or that have been particularly difficult to implement?

PROBE:

  • likely reasons for their shortfall

c. In deciding whether or not PERAA-related initiatives are effective, what criteria or yardsticks do you use?

e.g.,

  • —evidence that student outcome measures are moving in a positive direction
  • —that specific goals set by the city or the State Board are being met
  • —how D.C.’s performance compares with that of other urban districts

5. Finally, what are the biggest lessons that you take away from your work in DCPS thus far?

Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×
Page 243
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×
Page 244
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×
Page 245
Suggested Citation:"Appendix B: Sample Interview Protocol." National Research Council. 2015. An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape. Washington, DC: The National Academies Press. doi: 10.17226/21743.
×
Page 246
Next: Appendix C: The Public Education Reform Amendment Act and Relevant Amendments »
An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

An Evaluation of the Public Schools of the District of Columbia is a comprehensive five-year summative evaluation report for Phase Two of an initiative to evaluate the District of Columbia's public schools. Consistent with the recommendations in the 2011 report A Plan for Evaluating the District of Columbia's Public Schools, this new report describes changes in the public schools during the period from 2009 to 2013. An Evaluation of the Public Schools of the District of Columbia examines business practices, human resources operations and human capital strategies, academic plans, and student achievement. This report identifies what is working well seven years after legislation was enacted to give control of public schools to the mayor of the District of Columbia and which areas need additional attention.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!