National Academies Press: OpenBook

Learning About Assessment, Learning Through Assessment (1998)

Chapter: IV. A Framework for Addressing Concerns

« Previous: III. Planning and Organizing Professional Development
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

IV. A Framework for Addressing Concerns

By this point, it should be apparent that there is a particular complexity to assessment-based professional development. Participating teachers learn about new directions in assessment while they engage in repeated applications of the assessment process.

From the start in assessment-focused groups, teachers are challenged to sharpen their skills in gathering and using evidence to make valid inferences. However, the varied underlying concerns that teachers bring to their group work will have a bearing on their attention to the challenges and their capacities to meet the challenges. For example, it would be unwise for a group facilitator to concentrate on teachers' judgment about the appropriateness of tasks if the dominant concern in the group is to get more information about the different kinds of tasks being used in alternative-assessment initiatives.

Those who design assessment working groups need to account for the variety of concerns in their planning. This section describes some common teacher concerns about alternative assessment and a framework to use in listening to and responding to the concerns.

Assuming that there will be a group of teachers working together on assessment, it may be that experience with assessment, and therefore the concerns, will vary within the group. And, no matter what the variation, concerns in a group will change over the course of time. Planning for the group activities needs to take variation and change into account. Whether the driving purpose is to improve classroom instruction or to align what happens in

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

the classroom with externally administered testing, assessment-focused professional development will be experienced by many teachers as an innovation that is nudging, if not requiring, them to change their practice. There is a well-established framework (Loucks-Horsley and Stiegelbauer, 1991) for thinking about the stages of concern that adopters of innovations typically pass through, and that framework, the Concerns-Based Adoption Model (CBAM) is relevant here. Briefly, the framework's stages of concern span transitions from initial awareness of an innovation all the way to exploring how to refocus and improve the innovation. The stages are shown in Table 2.

To capture the flavor of concerns that teachers express about assessment, we interviewed ten people with extensive experience in designing and conducting assessment-focused staff development. The interviews yielded a set of concerns often heard by these group leaders, which we have sorted using the CBAM framework. (See Table 3.) Some of the concerns could sit in more than one category. For example, the bottom, italicized quote seems primarily to be concerned about consequence for the students, but also seems to foreshadow concerns about collaboration with other teachers. No concerns were expressed that seemed to fit under Refocusing. An example of a refocusing concern voiced by teachers once they have worked with innovations in assessment for a while might be, "Will the district support us in using grade-by-grade student work analyses to redefine the district's performance standards?"

Table 2. Stages of concern

Stage 0.

Awareness concerns: Basic awareness about the innovation.

Stage 1.

Informational concerns: Focus on learning more detail about the innovation.

Stage 2.

Personal concerns: Focus on individual's role, the demands of the innovation, and adequacy to meet demands.

Stage 3.

Management concerns: Focus on efficiency, organization, management, time, best use of resources.

Stage 4.

Consequence concerns: Focus on impact on students.

Stage 5.

Collaboration concerns: Coordination and cooperation with others in use of the innovation.

Stage 6.

Refocusing concerns: Exploration of more powerful alternatives.

From Loucks-Horsley and Stiegelbauer, 1991.

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

Stages 0 and 1. Awareness and Informational concerns

How might a group facilitator plan for addressing needs for awareness and information? One economical way is to organize the information around the four shifts in assessment mentioned earlier. These were shifts

  • away from basing inferences on single sources of evidence and toward basing inferences on multiple and balanced sources of evidence;

  • away from reliance on comparing students' performance with that of other students and toward reliance on comparing students' performance with established criteria;

  • away from relying on outside sources of evidence and toward a balance between these sources and evidence compiled by teachers; and

  • away from a preponderance of assessment items that are short, skill-focused, single-answer, and decontextualized, toward a greater use of tasks that are context-based; open to multiple approaches and, in some cases, to multiple solutions; complex in the responses they demand—e.g., in communication,

Table 3. Assessment-related concerns

0. Awareness

 

1. Informational

''What are rubrics?'' "Portfolios?" "What are examples of the kinds of alternative tasks that are being used?"

2. Personal

"What are the newspapers going to do with the information?"

"What conclusions are going to come out of this, and how and where is the information going to be used?"

"How do I talk with parents about the changes?"

"Will my administrators support me?"

"How do I deal with transition? I'm using fewer traditional tests, but don't yet have an adequate system to replace it."

3. Management

"Where can I find the time to fit these tasks in? to score using rubrics? to handle portfolios?"

"If a simple grade isn't sufficient, how do I report in a way that is clear, concise, but not backbreaking for me?"

"With all the open-ended tasks, how can I be sure what's a right answer?"

4. Consequence

"Will there be some continuity between grades, so that my students don't only see something different in my classroom and not in others?"

"Do the new assessments really tell us what we need to know?"

"I'm afraid we're de-emphasizing rigor in favor of student inventiveness."

"How can you score a piece 'proficient' if the student doesn't get the right answer?"

5. Collaboration

"Will there be some continuity between grades, so that my students don't only see something different in my classroom and not in others?"

6. Refocusing

 

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

representation, and level of generalization; and drawn from a wide spectrum of mathematics concepts and processes.

The first implies that attention should be given to defining and providing examples of different sources of evidence—e.g., portfolios, short-answer tasks, performances, student self-assessments, observations. The second implies that attention be given to understanding and reaching consensus about the criteria that are used to judge quality; therefore, examples of scoring rubrics and performance standards need to be provided and discussed. The third implies that information should be provided about efforts in several states and districts to complement test data with other data sources, such as portfolios. The fourth implies that teachers should see and discuss a variety of mathematics tasks.

"Another question teachers have is, where do the problems come from? Where can I find good performance tasks? What do I do if I don't have one that fits what I'm teaching? Often teachers don't feel they have access to the resources they need. That's the biggest hit in workshops, is giving people collections of problems for them to use as resources in getting started."

District leader of test-change efforts

Stage 2. Personal concerns

Once concerns for basic information are addressed, and as assessment-focused groups progress, teachers express other, more personal concerns related to the impact on their own experience and on the values they hold dear. The challenges to teachers' judgments about the quality of mathematics, discussed earlier, are particularly likely to raise personal concerns. Understandably, individual feelings of inadequacy about mathematical understandings or skills will raise personal concerns in settings where open discussion is invited and expected.

Drawn from the experience of the educators interviewed for this document, the teachers' expressed concerns reveal a particular set of worries about how vulnerable new assessments will make them. Concerns about support are common, and they include the administrative net underneath, the buy-in by teachers in grades before and after, and the availability of helpful resources. Concerns about smooth transition from an old system to a new one are strongly felt. This is true of individual teachers trying to make changes in their own practice, but it is especially true in settings where widespread assessment change is underway or pending, where concerns include the use of information in and by the community, the handling of community questions, and the use of information to improve programs.

"Designing alternative forms of assessment is a real tricky thing. If you believe, if you've been convinced, that these forms of as-

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

sessment are the way to go, then getting the skills to do that kind of assessment is very necessary and requires time. You can't just say, I believe this is a good way to go and so it's going to just happen. We didn't see these forms of assessment when we were in school, we didn't see them in teacher preparation programs, and we haven't ever used them in our classrooms before. Teachers need to not give up when at first an assessment doesn't succeed. A lot of the time, you try something, and it doesn't work at first and it turns out it wasn't assessing what you thought you were assessing. These assessments are new, and different, and teachers ought to have the right to work on it, and improve on it. They need the space to try it without being evaluated on it right away, for instance."

Teacher veteran of portfolio-scoring teams

"I find it important to talk about taking a long-range view of assessment change. I don't talk about it in the usual one-year time frame, but more in terms of three to five years. I tell teachers, If this stuff is really profound, it may not show up right away."

District associate superintendent

"Another question teachers have is what are parents going to say, related to how their students will do on standardized tests. If we don't test that way, when students get to standardized tests, how are they going to deal with the multiple choice questions? What if the scores aren't there? These are concerns of both parents and administrators."

High-school mathematics teacher

"One big question from teachers is, how do we handle concerns and questions from the parents and the community, especially in California or other states where testing is a big political issue?"

Urban district mathematics supervisor

"My biggest concern right now is how do we educate the public, especially when the assessment is a visible thing. When changes are happening in the classroom, it can be handled, especially through parent-teacher communication: work gets done, feedback is given, work goes home, and there's a gradual awareness that builds."

Urban district mathematics supervisor

For those facilitating assessment groups, there are a few important considerations in planning to address these personal concerns:

  • Provide regular opportunities for participating teachers to discuss their progress and concerns.

  • Use small-group explorations of the mathematics, with an emphasis on the building of knowledge in the groups, to alleviate individual feelings of inadequacy around the mathematics.

  • Plan to advocate for the longer view in setting the tone with the teachers, to advocate for a commitment to exploration and experiment, and to resist requests to "show us how."

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
  • Find ways to elicit concerns and make them explicit, but be candid and realistic about the possibilities of addressing them within the group.

  • Separate the teachers' capacity building—especially the sharpening of judgment about tasks, student work, and consequent actions—from techniques for public relations. The latter may be of interest to participants, but they are no substitute for the former. Give concrete reasons for becoming proficient in assessing students' learning.

  • Plan that, over time, the group will select illustrative examples of tasks and student work for purposes of communication about assessment changes. Good examples and informed commentary can be the most effective kind of public relations for assessment in mathematics.

Stage 3. Management concerns

Once teachers invest themselves in trying to incorporate alternative approaches to assessment in their own practice, management concerns arise, related to time management; the application of criteria; and reporting procedures.

"The question, and I hear this one all the time, is where do you get the time, as a classroom teacher, to score all these papers? And not only to score them, but read them and process what the kids are saying? To be honest, when I was a teacher way back when, most of the time, I'd just check the 'answer column.' I hardly ever looked at the student work. And I think that's what most teachers do. Where do you find the time to read responses, make comments, give feedback? Like using student journals is a terrific idea, but teachers want to know, when am I going to find time to do this?"

Urban district mathematics supervisor

"The most frequent question I hear from other teachers is, how is this going to impact everything else I have to do, since this is going to take longer? . . . The big issue is time, and time in the perspective of, if I spend time on this stuff, how am I going to be able to cover the curriculum? . . . The time issue usually comes up in terms of the coverage of the curriculum. And I usually have to answer that by saying, ultimately we're going to have to grab stuff out of the curriculum. Because it's a legitimate concern, how can you do this stuff, and still cover the same curriculum."

High-school mathematics teacher

In our experience and in the experience of those we consulted, one of the most effective ways to address management concerns, once they have been elicited, is to ask teachers to address the group who have found ways to address particular management concerns. This approach increases the number of effective strategies available to the group, going beyond the personal experience of the facilitator.

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

In addition, there are helpful print resources available, such as Petit (1992), Stenmark (1991), and Tsuruda (1994).

Stages 4 and 5. Consequence and collaboration concerns

Even as they grapple with management concerns, teachers who are invested in learning through assessment and adopting alternative approaches to assessment develop concerns related to how their changes in assessment will affect the quality of their classroom work and the learning of their students. Two prominent clusters of such concerns are content and equity.

Content.

An essential component of learning through assessment is exposure to a variety of mathematics tasks. As they become familiar with tasks that require student performance, or that are open or complex, teachers often express concerns about validity, rigor, and quality.

"As we move to using new forms of assessment, the question I hear most often from teachers, and from parents, is 'Where's the math?' I think people can often see how these open-ended problems do address the first four NCTM standards: communication, problem solving, reasoning, and connections. But a lot of the time, they have this question: How do you know that students know the basic skills or are able to compute? They have a sense that there's not a good balance."

Urban district mathematics supervisor

"Teachers ask, where do the basic skills get taught in relation to performance assessments? Before, during, after? Do performance assessments address the teaching of basic skills? In other words, and this is a question for a lot of parents, can we assume that a child that can perform on a high level on a performance assessment knows the

basic skills?"

Director of assessment-based teacher program

"This is a more subjective way of assessing than multiple choice, in the sense that it relies on a subjective judgment. So a question teachers have is, is that fair? And of course, students have that question too. I think teachers can get around that by talking about the rubric before the assessment. On the other hand, . . . I have yet to write a rubric that I didn't need to revise once I saw the students' papers."

Teacher veteran of portfolio-scoring teams

Here is an instance where a concern about assessment has aspects of more than one category to it: consequence (Will the students get shortchanged?) and informational (Where is the rigor in these new assessment systems?). Both aspects would need attention in the professional development.

Equity.

The simplicity of the 4-part, cyclical diagram for the assessment process belies the complexity of assessment as a continuum of practice. On one end of the continuum, teachers must

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

attend to students' individual interests and ways of thinking, and learn to incorporate portfolios and project work in such areas as data study, mathematical model of physical systems or phenomena, design of physical structures, management and planning analysis, pure mathematical investigations, and histories of mathematical ideas. (See, e.g., the performance standards categories developed by the New Standards Project, 1995.)

While portfolios and projects provide avenues for individual expression in assessment, there is another end to the continuum, where assessors must attend to things that society has deemed important and to standard ways of knowing. For example, society expects that, no matter their background, students will be able to perform well on tasks that relate to desired levels of literacy and numeracy in the citizenry, such as the task in Figure 12, administered to 17 year-olds in the National Assessment of Educational Progress (NAEP) and on which only three percent responded at the satisfactory level or better (Lindquist et al., 1995).

In between these two ends of the continuum are a range of individualized expressions of socially important ideas. This range appears to be growing in most classrooms, but it is especially wide in classrooms where teachers see a broad diversity in individual interests or, especially where many cultures are represented, they see a variety of socially important ideas. It becomes important for designers of professional development to look at assessment innovations through the lens of teachers' equity concerns. These concerns become especially relevant, and important to elicit, when judgments about task appropriateness and accessibility are at issue. For example, we have found it advantageous for teachers to discuss equity issues around the use of open tasks that invite students to explore and construct responses. Some teachers believe that some students require structured tasks all the time, and so should be protected from open tasks. This belief can, and should, be questioned and alternative perspectives should be considered.

"I'm an advocate of open-ended problems because I think they allow for more equity. It's still a new experience for all students to have problems that could have multiple entry points and multiple solutions, problems that allow for sophisticated response or a basic response. If the problems are presented correctly, then everyone can have a stab at it. In contrast, if a problem depends on a certain technique, then if a student can't do it, that's it?

Urban high-school mathematics teacher

Alternative assessment as an innovation is relatively young. Using the stages-of-concern lens on the concerns reported in our interviews, however, we can infer by the quantities of concerns in the management and consequences categories that the innovation is beginning to take root. At the same time, the educators we

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×

One plan for a state income tax requires those persons with incomes of $10,000 or less to pay no tax and those persons with income greater than $10,000 to pay a tax of 6 percent only on the part of their income that exceeds $10,000. A person's effective tax rate is defined as the percent of total income that is paid in tax. Based on this definition, could any person's effective tax rate be 5 percent? Could it be 6 percent? Explain your answer. Include examples if necessary to justify your conclusions.

Figure 12.

A problem from NAEP

From the National Assessment of Educational Progress, as cited in Lindquizt, Dossey, & Mullis, 1995.

interviewed report a range of concerns in the earlier categories, as well, implying that planning for assessment-based staff development should proceed from a concerns-based perspective, and allow participating teachers over time to begin reconsidering their perspectives on assessment, and to consider more powerful alternatives. The optimal staff-development event is one that addresses current concerns while it foreshadows future concerns.

In any case, it takes time to adopt and internalize new perspectives on assessment, and it takes considerable support. In the final section of the document, we advocate for actions that school administrators and others can take so that teachers get the necessary time and support.

Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 33
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 34
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 35
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 36
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 37
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 38
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 39
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 40
Suggested Citation:"IV. A Framework for Addressing Concerns." National Research Council. 1998. Learning About Assessment, Learning Through Assessment. Washington, DC: The National Academies Press. doi: 10.17226/6217.
×
Page 41
Next: V. Supporting and Extending Professional Development »
Learning About Assessment, Learning Through Assessment Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The MSEB, with generous support and encouragement from the Carnegie Corporation of New York, seeks to bring discussion of assessment to school-and district-based practitioners through an initiative called Assessment in Practice (AIP). Originally conceived as a series of "next steps" to follow the publication of Measuring Up and For Good Measure, the project, with assistance from an advisory board, developed a publication agenda to provide support to teachers and others directly involved with the teaching and assessment of children in mathematics classrooms at the elementary, middle, and high school levels.

In a series of three booklets, AIP presents an exploration of issues in assessment. The first booklet, Learning About Assessment, Learning Through Assessment discusses ways to assist teachers in learning about assessment and how student work can be a rich resource in professional development. The second, Assessment in Support of Instruction, makes a case for aligning assessments with state and district curriculum frameworks and examines ways in which states have shifted their curriculum frameworks and related state assessment programs to reflect the NCTM Standards and other perspectives. The third booklet, Keeping Score, discusses issues to be considered while developing high quality mathematics assessments. This series is specifically designed to be used at the school and school district level by teachers, principals, supervisors, and measurement specialists.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!