Tag Archives: end-of-year

Too Many Reassessments, Just in Time for Summer

I posted this graph of cumulative reassessments versus the day of the semester on Twitter:

That, my friends, is a reassessment system gone wild. The appropriate title for that image, as one person pointed out, is Too Many Reassessments. The grand total for this semester was 711. There are obvious bunches of reassessments close to the ends of the quarters when the grade book closes.

Here is a histogram of the reassessment data for the semester. There is some discrepancy in the total number in the data here, but I haven't figured out exactly where that is.

I committed to transplanting the system I have used in the past to my new school this year, and didn't want to make a full change without seeing how this would play out. This semester I was much more consistent in the types of questions I gave students reassessing, changing grades based on a reassessment, and the choice I offered them for the level of reassessment. Some of this I wrote about at the beginning of the semester.

The most important observation I can make at this point is that this system is not sustainable as is. I cannot make my sign-up and credit system more efficient to manage the volume - that isn't the issue. I'm satisfied with the quality of the questions I give students. I've developed a pretty nice bank of questions that span the spectrum of application, understanding, and transfer. The bigger issue is my capacity to give the sort of feedback I want to give to students throughout the semester. I have many conversations about learning, and many of them are great, but I cannot multiply myself to have as many of those conversations as I want.

Here's a graph of the average learning standards grade for a sample of students compared with the number of reassessments:

This doesn't support the expectation that more reassessments implies a higher grade. Students are not necessarily doing machine-gun style reassessment. They are working on specific skills and show my what they are doing. They are responding in a positive way to my feedback. Credits, which students earn by doing work and review of concepts, are still required for students to reassess. Students are for the most part using their credits. Expiring credits, as much as I thought it would make a difference, is not making that much of a difference for behavior (i.e. signing up for reassessments) or course grade. I need to dig into the data more to be able to explain why.

In terms of moving forward, I have many things to think about.

  • The past three or four years have been an exercise in exploring a system that centers around student-initiated reassessment. I'm not sure it's time for that to completely go away, but I wonder about shifting my focus to an assessment structure centered on teacher-initiated. I already do this on unit exams, but I wouldn't say it is the focus of where I spend my time.
  • I wonder if reducing the permitted number of reassessments per student to one per week would improve their effectiveness. This effectiveness increase could be based in higher quality feedback from me, more focused effort on the part of the student for improving understanding on a given learning standard, or something else entirely. This reduces the options for students to learn on their own timeline, which isn't a good thing. While we're being honest though, that exponential curve at the end of the assessment period is all the evidence I need to accept that the timeline is based on the grading-period structure, not learning.
  • How do I most efficiently help the weak student that reassesses on the same standard multiple times and makes limited progress on each attempt?
  • How do I give meaningful guidance to the student that aces everything on the first try? How do I get them more involved in finding learning that is meaningful, rather than waiting for me to tell them what to learn?
  • What do the students think? I've collected all sorts of anecdotal evidence that students appreciate the opportunities to reassess, and not just in a superficial way related to their course grade. I've given students an end of year survey to complete, and those results are rolling in slowly as students complete their final exams.

These are the big picture questions that add one more reason to be thankful that the summer is ahead. Getting back to my main point, I am brought back to the idea that quality feedback is the main way we as teachers add value. This, like many things in education, is not easy to scale. This need for improving and scaling the transfer of feedback is really the only basis for innovation in the ed-tech realm that interests me at all these days. So far, despite the best intentions of many that are trying, machine learning is not the answer yet. Make it easy for me to organize and collect student thinking, respond to that thinking, and give helpful nudges to the resources needed to make progress, and then I'll consider your product.

Final exam marking is ahead. Stay tuned.

2012-2013 Year In Review - Standards Based Grading

This is the first in a series of posts about things I did with my classes this year.

When I made the decision last fall to commit to standards based grading, these were the main unknowns that hung at the back of my mind:

  • How would students respond to the change?
  • How would my own use of SBG change over the course of the year?
  • How would using SBG change the way I plan, teach, and assess?

These questions will all be answered as I reflect in this post.

What did I do?

In the beginning of the year, I used a purely binary system of SBG - were students proficient or not? If they were proficient, they had a 5/5. Not yet proficient students received a 0/5 for a given standard. All of these scores included a 5 point base grade to be able to implement this in PowerSchool.

As the semester went on, the possible proficiency levels changed to a 0, 2.5, or 5. This was in response to students making progress in developing their skills (and getting feedback on their progress through Blue Harvest but not seeing visible changes to their course grade. As much as I encouraged students not to worry about the grade, I also wanted to be able to show progress through the breakdown of each unit's skills through PowerSchool. It served as a communication channel to both parents and the students on what they were learning, and I could see students feeling a bit unsatisfied by getting a few questions correct, but not getting marked as proficient yet. I also figured out that I needed to do more work defining what it meant to be proficient before I could really run a binary system.

By the start of the second semester, I used this scheme for the meaning of each proficiency score:

  • 1 - You've demonstrated basic awareness of the vocabulary and definitions of the standard. You aren't able to solve problems from start to finish, even with help, but you can answer yes/no or true or false questions correctly about the ideas for this standard.
  • 2 - You can solve a problem from start to finish with your notes, another student, or your teacher reminding you what you need to do. You are not only able to identify the vocabulary or definitions for a given skill, but can substitute values and write equations that can be solved to find values for definitions. If you are unable to solve an equation related to this standard due to weak algebra skills, you won't be moving on to the next level on this standard.
  • 3 - You can independently solve a question related to the standard without help from notes, other students, or the teacher. This score is what you receive when you do well on a quiz assessing a single standard. This score will also be the maximum you will receive on this standard if you consistently make arithmetic or algebraic errors on problems related to this standard.
  • 4 - You have shown you can apply concepts related to this standard on an in-class exam or in another situation where you must identify which concepts are involved in solving a problem. This compares to success on a quiz on which you know the standard being assessed. You can apply the content of a standard in a new context that you have not seen before. You can clearly explain your reasoning, but have some difficulty using precise mathematical language.
  • 5 - You have met or exceeded the maximum expectations for proficiency on this standard. You have completed a project of your own design, written a program, or made some other creative demonstration of your ability to apply this standard together with other standards of the unit. You are able to clearly explain your reasoning in the context of precise mathematical definitions and language.

All of the standards in a unit were equally weighted. All units had between 5 and 7 standards. In most classes, the standards grade was 90% of the overall course grade, the exception being AP Calculus and AP Physics, where it was 30%. In contrast to first semester, students needed to sign up online for any standards they wanted to retake the following day. The maximum number of standards they could retake in a day was limited to two. I actually held students to this (again, in contrast to first semester), and I am really glad that I did.

Before I start my post, I need to thank Daniel Schneider for his brilliant post on how SBG changes everything here. I agree with the majority of his points, and will try not to repeat them below.

What worked:

  • Students were uniformly positive about being able to focus on specific skills or concepts separate from each other. The clarity of knowing that they needed to know led some students to be more independent in their learning. Some students made the conscious decision to not pursue certain standards that they felt were too difficult for them. The most positive aspect of their response was that students felt the system was, above all else, a fair representation of their understanding of the class.
  • Defining the standards at the beginning of the unit was incredibly useful for setting the course and the context for the lessons that followed. While I have previously spent time sketching a unit plan out of what I wanted students to be able to do at the end, SBG required me not only to define specifically what my students needed to do, but also to communicate that definition clearly to students. That last part is the game changer. It got both me and the students defining and exploring what it means to be proficient in the context of a specific skill. Rather than saying "you got these questions wrong", I was able to say "you were able to answer this when I was there helping you, but not when I left you alone to do it without help. That's a 2."
  • SBG helped all students in the class be more involved and independent in making decisions about their own learning. The strongest students quickly figured out the basics of each standard and worked to apply them to as many different contexts as possible. They worked on communicating their ideas and digging in to solve difficult problems that probed the edges of their understanding. The weaker students could prioritize those standards that seemed easiest to them, and often framed their questions around the basic vocabulary, understanding definitions, and setting up a plan to a problem solution without necessarily knowing how to actually carry out that plan. I also changed my questions to students based on what I knew about their proficiency, and students came to understand that I was asking a level 1 question compared with a level 3 question. I also had some students giving a standards quiz back to me after deciding that they knew they weren't ready to show me what they knew. They asked for retakes later on when they were prepared. That was pretty cool.
  • Every test question was another opportunity to demonstrate proficiency, not lose points. It was remarkably freeing to delete all of the point values from questions that I used from previous exams. Students also responded in a positive way. I found in some cases that because students weren't sure which standard was being assessed, they were more willing to try on problems that they might have otherwise left blank. There's still more work to be done on this, but I looked forward to grading exams to see what students did on the various problems. *Ok, maybe look forward is the wrong term. But it still was really cool to see student anxiety and fear about exams decrease to some extent.

What needs work:

  • Students want more detail in defining what each standard means. The students came up with the perfect way to address this - sample problems or questions that relate to each standard. While the students were pretty good at sorting problems at the end of the unit based on the relevant standards, they were not typically able to do this at the beginning. The earlier they understand what is involved in each standard, the more quickly they can focus their work to achieve proficiency. That's an easy order to fill.
  • I need to do more outreach to parents on what the standards mean. I thought about making a video at the beginning of the year that showed the basics, but I realize now that it took me the entire year to understand exactly what I meant by the different standards grades. Now that I really understand the system better, I'll be able to do an introduction when the new year begins.
  • The system didn't help those students that refuse to do what they know they need to do to improve their learning. This system did help in helping these students know with even more clarity what they need to work on. I was not fully effective in helping all students act on this need in a way that worked for them.
  • Reassessment isn't the ongoing process that it needs to be. I had 80 of the 162 reassessment requests for this semester happen in the last week of the semester. Luckily I made my reassessment system in Python work in time to make this less of a headache than it was at the end of the first semester. I made it a habit to regularly give standards quizzes between 1 or 2 classes after being exposed to the standard for the first time. These quizzes did not assess previous standards, however, so a student's retake opportunities were squarely on his or her own shoulders. I'm not convinced this increased responsibility is a problem, but making it an ongoing part of my class needs to be a priority for planning the new year.

I am really glad to have made the step to SBG this year. It is the biggest structural change I've made to my grading policy ever. It led to some of the most candid and productive conversations with students about the learning learning process that I've ever had. I'm going to stop with the superlatives, even though they are warranted.

Milestones at the start of summer: A tribute

IMG_0720

I used this LEGO car in a five minute demo lesson - my first lesson ever - on Newton's laws of motion. It was a gimmick to get the people in the room thinking about what they knew about forces, and served this purpose perfectly. This was in the beginning stages of my decision during my senior year at Tufts to pursue teaching rather than engineering after graduation.

It sat on the bookshelf next to my desk in both of my New York City apartments. It made its way into a suitcase that a friend took to Zambia. It was one of the items that I took out of the storage last summer with a smile, and was among the knick-knacks that didn't get tossed in the move to the apartment in Hangzhou for next year.

This LEGO car rolled across the floor of the new apartment last week, the final week of my tenth year teaching. It made me think back to the many adventures that have been my life ever since I received my acceptance letter to the New York City Teaching Fellows program in 2003. I worked with an incredible group of teachers in the Bronx for seven years, helped open the KIPP NYC College Prep high school, and then made the move to Hangzhou where I have enjoyed teaching kids and working with some fantastic folks from all over the world.

Even though it is the start of summer vacation, my head is still very much in the teaching game. It's gratifying to know that I can reinvent myself every year after a summer of reflection and meditation on what went well and what did not. I am motivated by my students comments in end-of-year surveys that my enthusiasm for learning and sharing new things gets them excited to be in the classroom with me. The unique experience of working with teenagers compels me to still devote energy and time to making myself better at what I do.

To the students that I have worked with over the past ten years: thank you for giving me the most exhilarating, satisfyingly unpredictable, and meaningful ten years I never knew I wanted in a career. To my colleagues: thank you for teaching me what it means to work hard for the right reasons and toward the right ends. To my family: thank you for supporting me in all that I do.

Have a great summer everyone!