Too Many Reassessments, Just in Time for Summer

I posted this graph of cumulative reassessments versus the day of the semester on Twitter:

That, my friends, is a reassessment system gone wild. The appropriate title for that image, as one person pointed out, is Too Many Reassessments. The grand total for this semester was 711. There are obvious bunches of reassessments close to the ends of the quarters when the grade book closes.

Here is a histogram of the reassessment data for the semester. There is some discrepancy in the total number in the data here, but I haven’t figured out exactly where that is.

I committed to transplanting the system I have used in the past to my new school this year, and didn’t want to make a full change without seeing how this would play out. This semester I was much more consistent in the types of questions I gave students reassessing, changing grades based on a reassessment, and the choice I offered them for the level of reassessment. Some of this I wrote about at the beginning of the semester.

The most important observation I can make at this point is that this system is not sustainable as is. I cannot make my sign-up and credit system more efficient to manage the volume – that isn’t the issue. I’m satisfied with the quality of the questions I give students. I’ve developed a pretty nice bank of questions that span the spectrum of application, understanding, and transfer. The bigger issue is my capacity to give the sort of feedback I want to give to students throughout the semester. I have many conversations about learning, and many of them are great, but I cannot multiply myself to have as many of those conversations as I want.

Here’s a graph of the average learning standards grade for a sample of students compared with the number of reassessments:

This doesn’t support the expectation that more reassessments implies a higher grade. Students are not necessarily doing machine-gun style reassessment. They are working on specific skills and show my what they are doing. They are responding in a positive way to my feedback. Credits, which students earn by doing work and review of concepts, are still required for students to reassess. Students are for the most part using their credits. Expiring credits, as much as I thought it would make a difference, is not making that much of a difference for behavior (i.e. signing up for reassessments) or course grade. I need to dig into the data more to be able to explain why.

In terms of moving forward, I have many things to think about.

• The past three or four years have been an exercise in exploring a system that centers around student-initiated reassessment. I’m not sure it’s time for that to completely go away, but I wonder about shifting my focus to an assessment structure centered on teacher-initiated. I already do this on unit exams, but I wouldn’t say it is the focus of where I spend my time.
• I wonder if reducing the permitted number of reassessments per student to one per week would improve their effectiveness. This effectiveness increase could be based in higher quality feedback from me, more focused effort on the part of the student for improving understanding on a given learning standard, or something else entirely. This reduces the options for students to learn on their own timeline, which isn’t a good thing. While we’re being honest though, that exponential curve at the end of the assessment period is all the evidence I need to accept that the timeline is based on the grading-period structure, not learning.
• How do I most efficiently help the weak student that reassesses on the same standard multiple times and makes limited progress on each attempt?
• How do I give meaningful guidance to the student that aces everything on the first try? How do I get them more involved in finding learning that is meaningful, rather than waiting for me to tell them what to learn?
• What do the students think? I’ve collected all sorts of anecdotal evidence that students appreciate the opportunities to reassess, and not just in a superficial way related to their course grade. I’ve given students an end of year survey to complete, and those results are rolling in slowly as students complete their final exams.

These are the big picture questions that add one more reason to be thankful that the summer is ahead. Getting back to my main point, I am brought back to the idea that quality feedback is the main way we as teachers add value. This, like many things in education, is not easy to scale. This need for improving and scaling the transfer of feedback is really the only basis for innovation in the ed-tech realm that interests me at all these days. So far, despite the best intentions of many that are trying, machine learning is not the answer yet. Make it easy for me to organize and collect student thinking, respond to that thinking, and give helpful nudges to the resources needed to make progress, and then I’ll consider your product.

Final exam marking is ahead. Stay tuned.