Friday, September 17, 2010

Analyzing a Common Assessment Non-Success


Well, it seems it was widespread. So let's look at what we found.
1. The students aced the straight vocabulary questions.
2. When the questions asked the students to apply material in an analytical way, they went
with a practiced, memorized answer from an activity rather than looking at the new
question and thinking through the question. (jump to a familiar answer, don't think)
3. We have some test-taking issues. How to cross-reference a test, and how to use context to
produce a higher level answer.
4. We have some GREAT test takers, who marked their thought processes on the tests in
ways that showed us what led to them the precise answer they needed.

Here is a resource that we have used for a few years to help us with common assessments that's based on Marzano's research. It also has a rubric that helps to analyze the questions that a teacher is asking.

Our responses:
1. We are going to break down the objectives and do the reteaching in varied ways so that
they practice analyzing. We'll give feedback on those smaller chunks.
2. We'll put it back together a piece at a time so that we can see when we lose students and
catch them before they fall.
3. Lots of personalization. The students did well when they had used the material in a
personal way (they were great with infinitives following timeo, which means "I fear",
because we had and activity where they shared what they feared to do, but when we
replaced timeo with volo "I want", they fell apart.

Other items:
Mr. K noted that I was flying through material, and I didn't listen. He was right. I should have recognized a red flag when I incorporated venn diagrams and the students could not use the vocabulary that they had mastered to describe the relationships in the diagram. Why? Because they were not at the analysis level, they still were just memorizing. That's the flag that should have stopped me, but didn't.

Lessons learned:
Slow down. Chunk more. Check for transfer, not just recall. Things I know, and thought that I was doing, but I needed more, and these students need more. Shifting gears. I hope the transmission can handle it.

2 comments:

Anonymous said...

Great analysis and follow up plan. Do you think all teachers analyze test/quiz results that way?

We sometimes mistake compliance with understanding.

Mark

Andrea Weis said...

Not sure if others do this. We analyzed each question's percentage of correct responses, then looked at which questions were the worst and what their errors were. "Why?" we asked. Then we looked at our activities for clues of how we practiced the items and what we had given positive feedback-were we looking for one pat answer or something more?

We spent a good 5 hours over 2 days looking at it together. So we met during common time, after school and online. No. I think that's kind of extraordinary, but then, it was an odd situation. Good practice, though and I hope it will improve the way we do this whole "teaching" thing for this group of students.
-Andrea