Tag Archives: standards based grading

Standards Based Grading and Leveling Up

I've been really happy since joining the SBG fan club a few years ago.

As I've gained experience, I've been able to hone my definitions of what it means to be a six, eight, or ten. Much of what happens when students sign up to do a reassessment is based on applying my experience to evaluating individual students against these definitions. I give a student a problem or two, ask him or her to talk to me about it, and based on the overall interaction, I decide where students are on that scale.

And yet, with all of that experience, I still sometimes fear that I might not be as consistent as I think I am. I've wondered if my mood, fatigue level, the time of day affect my assessment of that level. From a more cynical perspective, I also really really hope that past experiences with a given student, gender, nationality, and other characteristics don't enter into the process. I don't know how I would measure the effect of all of these to confirm these are not significant effects, if they exist at all. I don't think I fully trust myself to be truly unbiased, as well intentioned and unbiased as I might try to be or think I am.

Before the winter break, I came up with a new way to look at the problem. If I can define what demonstrated characteristics should matter for assessing a student's level, and test myself to decide how I would respond to different arrangements of those characteristics, I might have a way to better define this for myself, and more importantly, communicate those to my students.

I determined the following to be the parameters I use to decide where a student is on my scale based on a given reassessment session:

  1. A student's previously assessed level. This is an indicator of past performance. With measurement error and a whole host of other factors affecting the connection between this level and where a student actually is at any given time, I don't think this is necessarily the most important. It is, in reality, information that I use to decide what type of question to give a student, and as such, is usually my starting point.
  2. The difficulty of the question(s). A student that really struggled on the first assessment is not going to get a high level synthesis question. A student at the upper end of the scale is going to get a question that requires transfer and understanding. I think this is probably the most obvious out of the factors I'm listing here.
  3. Conceptual errors made by the student during the reassessment. In the context of the previous two, this is key in whether a student should (or should not) advance. Is a conceptual error in the context of basic skills the same as one of application of those skills? These apply differently at a level six versus a level eight. I know this effect when I see it and feel pretty confident in my ability to identify one or more of these errors.
  4. Arithmetic/Sign errors and Algebraic errors. I consider these separately when I look at a student's work. Using a calculator appropriately to check arithmetic is something students should be able to do. Deciding to do this when calculations don't make sense is a sign of a more skilled student in comparison to one that does not. Observing these errors is routinely something I identify as a barrier to advancement, but not necessarily in decreasing a student's level.

There are, of course, other factors to consider. I decided to settle on the ones mentioned above for the next steps of my winter break project.

I'll share how I moved forward on this in my next post in the series.

Scaling up SBG for the New Year

In my new school, the mean size of my classes has doubled. The maximum size is now 22 students, a fact about which I am not complaining. I've missed the ease of getting students to interact with simple proximity as the major factor.

I have also been given the freedom to continue with the standards based grading system that I've used over the past four years. The reality of needing to adapt my systems of assessment to these larger sizes has required me to reflect upon which aspects of my system need to be scaled, and what (if anything) needs to change.

The end result of that reflection has identified these three elements that need to remain in my system:

  • Students need to be assessed frequently through quizzes relating to one to two standards maximum.
  • These quizzes need to be graded and returned within the class period to ensure a short feedback cycle.
  • There must still be a tie between work done preparing for a reassessment and signing up for one.

Including the first element requires planning ahead. If quizzes are going to take up fifteen to twenty minutes of a class block, the rest of the block needs to be appropriately planned to ensure a balance between activities that respond to student learning needs, encourage reinforcement of old concepts, and allow interaction with new material. The second element dictates that those activities need to provide me time to grade the quizzes and enter them as standards grades before returning them to students. The third happens a bit later in the cycle as students act on their individualized needs to reassess on individual standards.

The major realization this year has been a refined need for standards that can be assessed within a twenty minute block. In the past, I've believed that a quiz that hits one or two aspects of the topic is good enough, and that an end of unit assessment will allow complete assessment on the whole topic. Now I see that a standard that has needs to have one component assessed on a quiz, and another component assessed on a test, really should be broken up into multiple standards. This has also meant that single standard quizzes are the way to go. I gave one quiz this week that tested a previously assessed standard, and then also assessed two new ones. Given how frantic I was in assessing mastery levels on three standards, I won't be doing that again.

The other part of this first element is the importance of writing efficiently targeted assessment questions. I need students to arrive at a right answer by applying their knowledge, not by accident or application of an algorithm. I need mistakes to be evidence of misunderstanding, not management of computational complexity. In short, I need assessment questions that assess what they are designed to assess. That takes time, but with my simplified schedule this year, I'm finding the time to do this important work.

My last post was about my excitement over using the Numbas web site to create and generate the quizzes. A major bottleneck in grading these quizzes quickly in the past has been not necessarily having answers to the questions I give. Numbas allows me to program and display calculated answers based on the randomized values used to generate the questions.

Numbas has a feature that allows students to take the exam entirely online and enter their answers to be graded automatically. In this situation, I have students pass in their work as well. While I like the speed this offers, that advantage primarily exists in cases where students answer questions correctly. If they make mistakes, I look at the written work and figure out what went wrong, and individual values require that I recalculate along the way. This isn't a huge problem, but it brings into question the need for individualized values which are (as far as I know right now) the only option for the fully online assessment. The option I like more is the printed worksheet theme that allows generation of printable quizzes. I make four versions and pass these out, and then there are only four sets of answers to have to compare student work against.

With the answers, I can grade the quizzes and give feedback where needed on wrong answers in no more than ten or fifteen minutes total. This time is divided into short intervals throughout the class block while students are working individually. The lesson and class activities need to be designed to provide this time so I can focus on grading.

The third element is still under development, but my credit system from previous years is going to make an appearance. Construction is still underway on that one. Please pardon the dust.


P.S:

If you're an ed-tech company that wants to impress me, make it easy for me to (a) generate different versions of good assessment questions with answers, (b) distribute those questions to students, (c) capture the student thinking and writing that goes with that question so that I can adjust my instruction accordingly, and (d) make it super easy to share that thinking in different ways.

That step of capturing student work is the roughest element of the UX experience of the four. At this time, nothing beats looking at a student's paper for evidence of their thinking, and then deciding what comes next based on experience. Snapping a picture with a phone is the best I've got right now. Please don't bring up using tablets and a stylus. We aren't there yet.

Right now there are solutions that hit two or three, but I'm greedy. Let me know if you know about a tool that might be what I'm looking for.

Standards Based Grading & Streamlining Assessments

I give quizzes at the beginning of most of my classes. These quizzes are usually on a single standard for the course, and are predictably on whatever we worked on two classes before. I also give unit exams as ways to assess student mastery of the standards all together. Giving grades after exams usually consists of me looking at a single student's exam, going standard by standard through the entire paper, and then adjusting their standards grades accordingly. There's nothing groundbreaking happening here.

The two downsides to this process are that it is (a) tedious and (b) is subject to my discretion at a given time. I'm not confident that I'm consistent between students. While I do go back and check myself when I'm not sure, I decided to try a better way. If you're a frequent reader of my blog, you know that either a spreadsheet or programming is involved. This time, it's the former.

Screen Shot 2016-02-25 at 9.07.41 AM

One sheet contains what I'm calling a standards map, and you can see this above. This relates a given question to the different standards on an exam. You can see above that question 1 is on only standard 1, while question 4 spans both standards 2 and 3.

The other sheet contains test results, and looks a lot like what I used to do when I was grading on percentages, with one key difference. You can see this below:

Screen Shot 2016-02-25 at 9.10.02 AM

Rather than writing in the number of points for each question, I simply rate a student's performance on that question as a 1, 2, or 3. The columns S1 through S5 then tally up those performance levels according to the standards that are associated with each question, and then scale those values to be a value from zero to one.

 

This information was really useful when going through the last exam with my ninth graders. The spreadsheet does the association between questions and standards through the standards map, so I can focus my time going through each exam and deciding how well a student completed a given question rather than remembering which standard I'm considering. I also found it much easier to make decisions on what to do with a student's standard level. Student 2 is an 8 on standard 1 before the exam, so it was easy to justify raising her to a 10 after the exam. Student 12 was a 7 on standard 4, and I left him right where he was.

 

I realize that there's a subtlety here that needs to be mentioned - some questions that are based on two or three standards might not communicate effectively a student's level with a single 1, 2, or 3. If a question is on solving systems graphically, a student might graph the lines correctly, but completely forget to identify the intersection. This situation is easy to address though - questions like this can be broken down into multiple entries on the standards map. I could give a student a 3 on the entry for this question on the standard for graphing lines, and a 1 for the entry related to solving systems. Not a big deal.

I spend a lot of time thinking about what information I need in order to justify raising a student's mastery level. Having the sort of information that is generated in this spreadsheet makes it much clearer what my next steps might be.

 

You can check out the live spreadsheet here:

Standards Assessment - Unit 5 Exam

2014-2015 Year-In-Review: Standards Based Grading

This was my third year using standards based grading with my classes. I wrote last year and the year before about my implementation.

What did I do differently?

  • I had my WeinbergCloud implementation working from the beginning of the year, so it was part of the expectations I introduced on day one.
  • I also adjusted this system a bit to make it easier to link the reassessments and the content of the standards. There seemed to be too much uncertainty about what each standard represented, which translated into more confusion when signing up for reassessments than I wanted. Creating a list of standards and resources associated with each standard shrank this gap.
  • I did not limit the number of reassessments per day explicitly. I expected that students would not sign up for a ridiculous number given the limitations on their credits, which students earned by doing homework or coming to tutoring.
  • I included time within at least one class a week per student during which students could do reassessments without having to come in outside of class time.
  • Unit exams continued to be assessed purely on course standards, not points. Semester final exams were percentage based.
  • I scaled all of my standards levels from 1 - 5 to be from 6 - 10 to make it easier to communicate the levels to parents and be consistent with our school grading policy of not giving numerical grades below 50%. No student actually received lower grades due to my system of adding a base grade to each standard, but the process of explaining to students and parents that a 1 was really a 60% (5 for the base grade + 1 for the standard level) was clearly more complex than it needed to be.
  • For my combined IB HL/SL class, the HL students had standards that only they were responsible for learning, while also being responsible for the SL standards. More on this later.

What worked:

  • Students seemed to have a better understanding from the beginning of the year of what standards based grading and assessment was all about. I did a bit more deliberate instruction on the ideas behind it at the beginning of the year. I also had smaller classes than before, so I was better able to have individual conversations about signing up for reassessments and talking about the process.
  • A small proportion of students were fully sold on the idea of reassessment as a learning tool. Some students reassessed at least twice a week throughout the semester, and these students had strong performances on the cumulative final exams.
  • By the second unit exam, students were generally not leaving questions blank on assessments. They were trying their best to do some amount of work on each question.
  • As with last year, I gave more challenging questions to assess the range of student ability. Most of these involved either multiple standards combined in one, more open ended responses, or questions requiring explanation. Assessing at the higher levels of mastery became strongly subjective, and students accepted this, though they occasionally advocated for themselves as to why they deserved to be marked higher. They generally felt that it was fair when arithmetic errors kept them in the 8/10 range.
  • Having students report their mastery level when signing up for a reassessment made it much easier for me to know what problem type or category to give them. Furthermore, this made it easier to justify changing the mastery level higher after a successful reassessment, but not making it the highest level on the scale. A student that was a 6 and answered a couple of questions correctly might move to an 8, whereas a student that was previously an 8 would be given more challenging questions and some conversation explaining their understanding in order to move to a 10.
  • It was my priority to get assessments back within the same period, and I estimate that I was able to do this more than 95% of the time. Simple, short, and carefully designed assessments can reveal quite a bit about what students do/don't understand.

What needs work:

  • Similar to previous semesters, I had high participation of a small group of students, with far too many students choosing not to reassess until the very end of each semester. Some students did not initiate their own reassessments at all.
  • Students again hoarded their credits to the end of the semester. I flirted with the idea of adding an expiration date to credits to discourage holding on to credits for long periods of time, but time constraints kept me from implementing this.
  • As a consequence of credit-hoarding, students near the end of the semester signed up for absurd numbers of reassessments in a day - I believe the largest quantity was nine. I shared with students that a good rule of thumb for planning purposes is 10 minutes per reassessment, so doing five reassessments before school isn't practical, but that didn't come across well. Students that couldn't do all of their reassessments in the morning simply pushed them to later in the day. This was a problem for me because I never knew if students were going to show up according to their scheduled time, or just do everything after school. Canceling after no-shows at the end fixed this problem pretty efficiently, however.
  • When a student would answer all questions correctly on an unannounced standards quiz, I generally assigned this a mastery level of 8 on a 6 - 10 scale. Students that had less than an 8 in this case usually had trouble with the same questions on a unit assessment or reassessment on the same standard later on. In other words, the students that had trouble initially learning a concept did not necessarily get the help they needed to make progress before the unit exam. This progress often happened after the exam, but this led to a lot of students falling behind pretty early on. I need to introduce interventions much earlier.

Under consideration for next year:

These are the ideas I am mulling over implementing before school gets started in a month, and I'd love to hear what you think.

  • Make credit expiration happen. This has been an issue for the year and a half of WeinbergCloud's existence. I threatened implementing this in speaking with students, and they were immediately asking me not to because it would prevent them from putting off reassessments as they preferred to do. This includes students that were doing the practice problems between classes anyway, so this wasn't just about losing the credits. Adding a "why not just give a reassessment a try" argument worked in face-to-face conversation with students that were hoarding credits, so forcing the process might be worth the effort. I understand that learning takes time, but many of the students putting off reassessment weren't actively reviewing the standards over time any way. I'd rather force the feedback cycle through more iterations since that is when students seem to learn the most.
  • Introduce submitting work into the process of reassessment. This could be electronic ("To complete your sign up, submit a scan/photo of the work you have done to prepare") or could just be shown before I give them a reassessment. This would reduce some of the sign-ups that happen only based on the mastery score rather than reviewing the concepts that come with it. Students earn credits by doing practice problems or coming to tutoring, and these let them sign up for reassessments - this won't change. To actually go the final step and take the reassessment, I need to see what students have done to prepare. In some cases (students that see me the day before, for example) I may waive this requirement.
  • Require X number of reassessments per two week cycle of the block schedule. This might be in lieu of the previous change, but I'm afraid this might encourage (rather than prevent) a rush of reassessments at the end of a two week period. On the other hand, if the goal is to increase opportunities for feedback, this might be more effective.
  • Make it possible for students to sign-up for an appointment to go over (but not be assessed) material on a given standard. Reassessments are great opportunities for feedback, but sometimes students want to come in to go over material. I get emails from students asking this, but it might be easier to just include this within WeinbergCloud.
  • Introduce skills/definition standards for each unit. This would be a standard for each unit that covers basic recall of information. I'll discuss why I want these (particularly in physics) in more detail within a later post. The short story is that I want to specifically assess certain concepts that are fundamental to all of the standards of a unit with a single binary standard.
  • Classify standards mastery levels in terms of 'likelihood of success'. This is a lower priority, and when I tried to explain this to a colleague, she wasn't convinced it would be worth the effort. If you have a 10, it means you have a 95% or higher likelihood of answering anything I give you correctly. The probabilities might not scale linearly - a 9 might mean between 90-95%, an 8 between 75% and 90, etc. I don't know. The reason I want to do this is to justify giving a 10 to students that have demonstrated solid proficiency without requiring perfection, and have a better reason for only raising a student from a 6 to an 8 after answering a couple questions on a single reassessment.

    Right now the difference between an 8, 9, and 10 are defined (in order) by answering questions correctly on a single standard quiz, a comprehensive unit exam, and correctly answering stretch questions correctly. A student that gets an 8 on a standards quiz before an exam might then answers related questions incorrectly on the multi-standards exam and remains an 8. If this student then takes a quiz on a single standard and answers that question correctly, does it make sense to then raise their mastery level above 8? This is what I often do. I can also control for this by giving a more challenging question, but I'm not sure I need to.

    In short, something is fishy here, and I need to think it out more in order to properly communicate it to students. In my head, I understand what I want to communicate: "yes, you answered these questions correctly, but I'm still not convinced that you understand well enough to apply the concepts correctly next time." This is not the highest priority out of the ones I've mentioned here.

As always, I appreciate your feedback. Thanks for reading!

Coding WeinbergCloud - An Update

Over the past few weeks, I've made some changes to my standards based grading system using the Meteor framework. These changes were made to address issues that students have brought up that they say get in the way of making progress. Whether you view these as excuses or valid points, it makes sense to change some of the features to match the students' needs.

I don't know what standard 6.2 means, Mr. Weinberg.

There are many places students could look to get this information. It does make sense, however, to have this information near where students sign up for reassessments.

When students select a standard, a link pops up (if the standard exists) with a description. This has made a big difference in students knowing whether the standard they sign up for is what they actually intend to assess.

Screen Shot 2015-02-13 at 1.07.27 PM

I also added the entry for the current mastery level, because this is important in selecting appropriate assessments. The extra step looking it up in the online gradebook isn't worth it to me, and asking students to look it up makes it their responsibility. That's probably where it belongs.

Can you post example problems for each standard?

The biggest issue students have in searching for online resources for a specific standard is not knowing the vocabulary that will get the best resources. There's lots of stuff out there, but it isn't all great.

I post links to class handouts and notes on a school blog, so the information is already online. Collecting it in one place, and organizing it according to the standards hasn't been something I've put time into.

Students can now see the standards for a given course, listed in order. If students are interested, they can look at other courses, just to see what they are learning. I have no idea if this has actually happened.

Screen Shot 2015-02-13 at 1.20.19 PM

Selecting a standard brings a student to see the full text and description of the standard. I can post links to the course notes and handout, along with online resources that meet my standards for being appropriately leveled and well written.

Screen Shot 2015-02-13 at 1.20.40 PM

At the moment, I'm the only one that can add resources. I've written much of the structure to ultimately allow students to submit sites, up-vote ones that are useful to them, and give me click data on whether or not students are actually using this, but I'm waiting until I can tweak some UI details to make that work just the way I want it.

Mr. Weinberg, I signed up for an assessment, but it's not showing up.

The already flaky internet in China has really gotten flakier as of late. Students are signing up for reassessments, but because of the way I implemented these requests being inserted into the database, these requests weren't actually making it to the server. I've learned a lot more about Meteor since I wrote this a year ago, so I've been able to make this more robust. The sign-up window doesn't disappear until the server actually responds and says that the insert was successful. Most importantly, students know to look for this helper in the upper left hand side of the screen:

Screen Shot 2015-02-13 at 1.14.13 PM

If it glows red, students know to reload the page and reconnect. Under normal Meteor usage conditions, this isn't a problem because Meteor takes care of the connection process automatically. China issues being what they are, this feature is a necessity.

I've written before about how good it feels to build tools that benefit my students, so I won't lecture you, dear reader, about that again. In the year since first making this site happen though, I've learned a lot more about how to build a tool like this with Meteor. The ease with which I can take an idea from prototype to production is a really great thing.

The next step is taking a concept like this site and abstracting it into a tool that works for anyone that wants to use it. That is a big scale project for another day.

Standards Based Grading(SBG) and The SUMPRODUCT Command

I could be very late to the party finding this out. If so, excuse my excitement.

I gave a multiple choice test for my IB Physics course last week. Since I am using standards based grading (SBG), I wanted a quick way to see how students did on each standard. I made a manually coded spreadsheet eight years or so ago to do this. It involved multiple columns comparing answers, multiple logical expressions, and then a final column that could be tallied for one standard. Multiply that by the total number of standards...you get the drill.

I was about to start piecing together an updated one together using that same exhausting methodology when I asked myself that same question that always gets me going: is there a better way?

Of course there is. There pretty much always is, folks.

For those of you that don't know, the SUMPRODUCT command in Excel does exactly what I was looking for here. It allows you to add together quantities in one range that match a set of criteria in another. Check out the example below:

Screen Shot 2014-10-14 at 3.28.09 PM

The column labeled 'Response Code' contains the formula '=1*(B6=E6)', which tests to see if the answer is correct. I wanted to add together the cells in F6 to F25 that were correct (Response Code = 1) and had the same standard as the cell in H6. The command in cell I6 is '=SUMPRODUCT((F6:F25)*(E6:E25=H6))'. This command is equivalent to the sum F6*(E6=H6) + F7*(E7=H6)+F8*(E8=H6)+...and so on.

If I had known about this before, I would've been doing this in some way for all of my classes in some way since moving to standards based grading. I've evaluated students for SBG after unit exams in the past by looking at a student's paper, and then one-by-one looking at questions related to each standard and evaluating them. The problem has been in communicating my rationale to students.

This doesn't solve the problem for the really great problems that are combinations of different standards, but for students that need a bit more to go on, I think this is a nice tool that (now) doesn't require much clerical work on my part. I gave a print out of this page (with column F hidden) to each student today.

Here is a sample spreadsheet with the formulas all built in so you can see how it works. Let me know what you think.
Exam Results Calculator

Standards Based Grading, Year Two (Year-In-Review)

This was my second year using standards based grading with my classes. I wrote last year about how my first iteration went, and made some adjustments this year.

What did I do?

  • I continued using my 1-5 standard scale and scale rubric that I developed last year. This is also described in the post above.
  • As I wrote about in a previous post, I created an online reassessment organization tool that made it easier to have students sign up and organize their reassessments.
  • The new requirement for students signing up for reassessments involved credits, which students earned through doing homework, seeing me for tutoring
  • I included a number of projects that were assessed as project standards using the same 1-5 scale. the rubric for this scale was given to students along with the project description. Each project, like the regular course learning standards, could be resubmitted and reassessed after getting feedback and revising.

What worked:

  • My rate of reassessment was substantially better in the second semester. I tweeted out this graph of my reassessments over the course of the semester:Reassessment plot EOYBqAGuNKCAAAUZjW.png-large There was a huge rush at the end of the semester to reassess - that was nothing new - but the rate was significantly more consistent throughout. The volume of reassessments was substantially higher. There were also fewer students than in the first semester that did not take advantage of reassessment opportunities. Certain students did make up a large proportion of the total set of reassessments, but this was nowhere near as skewed a distribution as in the first semester.
  • Students took advantage of the project standards to revise and resubmit their work. I gave a living proof project that required students to make a video in which they went through a geometric proof and explained the steps. Many students responded to my feedback about mathematical correctness, quality of their video, and re-recorded their video to receive a higher grade.
  • Student attitude about SBG was positive at the end of the year. Students knew that they could do to improve their grade. While I did have blank questions on some unit assessments, students seemed to be more likely to try and solve questions more frequently than in the past. This is purely a qualitative observation, so take that for what it is.

What needs work:

  • Students hoarded their reassessment credits. This is part of the reason the reassessment rush was so severe at the end of the semester. Students didn't want to use their credits until they were sure they were ready, which meant that a number were unused by the end of the year. Even by the end of the year, more than a quarter of credits that had been earned weren't used for reassessments. <p\> I don't know if this means I need to make them expire, or that I need to be more aggressive in pursuing students to use the credits that they earned. I'm wrestling a lot with this as I reflect this summer.
  • I need to improve the system for assessing during the class period. I had students sign up for reassessments knowing that the last 15 - 20 minutes of the class period would be available for it, but not many took advantage of this. Some preferred to do this before or after school, but some students couldn't reassess then because of transportation issues. I don't want to unfairly advantage those who live near the school by the system.
  • I need to continue to improve my workflow for selecting and assigning reassessments. There is still some inefficiency in the time between seeing what students are assessing on and selecting a set of questions. I think part of this can be improved by asking students to report their current grade for a given standard when signing up. Some students want to demonstrate basic proficiency, while others are shooting for a 4 or 5, requiring questions that are a bit higher level. I also might combine my reassessment sign up web application and the quiz application so that I'm not switching between two browser windows in the process.
  • Students want to be able to sign up to meet with me to review a specific standard, not just be assessed on it. If students know specifically what they want to go over, and want some one-on-one time on it since they know that works well for them, I'm all for making that happen. This is an easy change to my current system.
  • Students should be able to provide feedback to me on how things are going for them. I want to create a simple system that lets students rate their comprehension on a scale of 1 - 5 for each class period. This lets students assess me and my teaching on a similar scale to what I use to assess them, and might yield good information to help me know how to plan for the next class.

I've had some great conversations with colleagues about the ways that standards based grading has changed my teaching for the better. I'm looking forward to continuing to refine my model next year. The hard part is deciding exactly what refinements to make. That's what summer reflection and conversations with other teachers is all about, so let's keep that going, folks.

2012-2013 Year In Review - Standards Based Grading

This is the first in a series of posts about things I did with my classes this year.

When I made the decision last fall to commit to standards based grading, these were the main unknowns that hung at the back of my mind:

  • How would students respond to the change?
  • How would my own use of SBG change over the course of the year?
  • How would using SBG change the way I plan, teach, and assess?

These questions will all be answered as I reflect in this post.

What did I do?

In the beginning of the year, I used a purely binary system of SBG - were students proficient or not? If they were proficient, they had a 5/5. Not yet proficient students received a 0/5 for a given standard. All of these scores included a 5 point base grade to be able to implement this in PowerSchool.

As the semester went on, the possible proficiency levels changed to a 0, 2.5, or 5. This was in response to students making progress in developing their skills (and getting feedback on their progress through Blue Harvest but not seeing visible changes to their course grade. As much as I encouraged students not to worry about the grade, I also wanted to be able to show progress through the breakdown of each unit's skills through PowerSchool. It served as a communication channel to both parents and the students on what they were learning, and I could see students feeling a bit unsatisfied by getting a few questions correct, but not getting marked as proficient yet. I also figured out that I needed to do more work defining what it meant to be proficient before I could really run a binary system.

By the start of the second semester, I used this scheme for the meaning of each proficiency score:

  • 1 - You've demonstrated basic awareness of the vocabulary and definitions of the standard. You aren't able to solve problems from start to finish, even with help, but you can answer yes/no or true or false questions correctly about the ideas for this standard.
  • 2 - You can solve a problem from start to finish with your notes, another student, or your teacher reminding you what you need to do. You are not only able to identify the vocabulary or definitions for a given skill, but can substitute values and write equations that can be solved to find values for definitions. If you are unable to solve an equation related to this standard due to weak algebra skills, you won't be moving on to the next level on this standard.
  • 3 - You can independently solve a question related to the standard without help from notes, other students, or the teacher. This score is what you receive when you do well on a quiz assessing a single standard. This score will also be the maximum you will receive on this standard if you consistently make arithmetic or algebraic errors on problems related to this standard.
  • 4 - You have shown you can apply concepts related to this standard on an in-class exam or in another situation where you must identify which concepts are involved in solving a problem. This compares to success on a quiz on which you know the standard being assessed. You can apply the content of a standard in a new context that you have not seen before. You can clearly explain your reasoning, but have some difficulty using precise mathematical language.
  • 5 - You have met or exceeded the maximum expectations for proficiency on this standard. You have completed a project of your own design, written a program, or made some other creative demonstration of your ability to apply this standard together with other standards of the unit. You are able to clearly explain your reasoning in the context of precise mathematical definitions and language.

All of the standards in a unit were equally weighted. All units had between 5 and 7 standards. In most classes, the standards grade was 90% of the overall course grade, the exception being AP Calculus and AP Physics, where it was 30%. In contrast to first semester, students needed to sign up online for any standards they wanted to retake the following day. The maximum number of standards they could retake in a day was limited to two. I actually held students to this (again, in contrast to first semester), and I am really glad that I did.

Before I start my post, I need to thank Daniel Schneider for his brilliant post on how SBG changes everything here. I agree with the majority of his points, and will try not to repeat them below.

What worked:

  • Students were uniformly positive about being able to focus on specific skills or concepts separate from each other. The clarity of knowing that they needed to know led some students to be more independent in their learning. Some students made the conscious decision to not pursue certain standards that they felt were too difficult for them. The most positive aspect of their response was that students felt the system was, above all else, a fair representation of their understanding of the class.
  • Defining the standards at the beginning of the unit was incredibly useful for setting the course and the context for the lessons that followed. While I have previously spent time sketching a unit plan out of what I wanted students to be able to do at the end, SBG required me not only to define specifically what my students needed to do, but also to communicate that definition clearly to students. That last part is the game changer. It got both me and the students defining and exploring what it means to be proficient in the context of a specific skill. Rather than saying "you got these questions wrong", I was able to say "you were able to answer this when I was there helping you, but not when I left you alone to do it without help. That's a 2."
  • SBG helped all students in the class be more involved and independent in making decisions about their own learning. The strongest students quickly figured out the basics of each standard and worked to apply them to as many different contexts as possible. They worked on communicating their ideas and digging in to solve difficult problems that probed the edges of their understanding. The weaker students could prioritize those standards that seemed easiest to them, and often framed their questions around the basic vocabulary, understanding definitions, and setting up a plan to a problem solution without necessarily knowing how to actually carry out that plan. I also changed my questions to students based on what I knew about their proficiency, and students came to understand that I was asking a level 1 question compared with a level 3 question. I also had some students giving a standards quiz back to me after deciding that they knew they weren't ready to show me what they knew. They asked for retakes later on when they were prepared. That was pretty cool.
  • Every test question was another opportunity to demonstrate proficiency, not lose points. It was remarkably freeing to delete all of the point values from questions that I used from previous exams. Students also responded in a positive way. I found in some cases that because students weren't sure which standard was being assessed, they were more willing to try on problems that they might have otherwise left blank. There's still more work to be done on this, but I looked forward to grading exams to see what students did on the various problems. *Ok, maybe look forward is the wrong term. But it still was really cool to see student anxiety and fear about exams decrease to some extent.

What needs work:

  • Students want more detail in defining what each standard means. The students came up with the perfect way to address this - sample problems or questions that relate to each standard. While the students were pretty good at sorting problems at the end of the unit based on the relevant standards, they were not typically able to do this at the beginning. The earlier they understand what is involved in each standard, the more quickly they can focus their work to achieve proficiency. That's an easy order to fill.
  • I need to do more outreach to parents on what the standards mean. I thought about making a video at the beginning of the year that showed the basics, but I realize now that it took me the entire year to understand exactly what I meant by the different standards grades. Now that I really understand the system better, I'll be able to do an introduction when the new year begins.
  • The system didn't help those students that refuse to do what they know they need to do to improve their learning. This system did help in helping these students know with even more clarity what they need to work on. I was not fully effective in helping all students act on this need in a way that worked for them.
  • Reassessment isn't the ongoing process that it needs to be. I had 80 of the 162 reassessment requests for this semester happen in the last week of the semester. Luckily I made my reassessment system in Python work in time to make this less of a headache than it was at the end of the first semester. I made it a habit to regularly give standards quizzes between 1 or 2 classes after being exposed to the standard for the first time. These quizzes did not assess previous standards, however, so a student's retake opportunities were squarely on his or her own shoulders. I'm not convinced this increased responsibility is a problem, but making it an ongoing part of my class needs to be a priority for planning the new year.

I am really glad to have made the step to SBG this year. It is the biggest structural change I've made to my grading policy ever. It led to some of the most candid and productive conversations with students about the learning learning process that I've ever had. I'm going to stop with the superlatives, even though they are warranted.

A tale of two gradebooks - my SBG journey continues

I realized this morning that I could look back at the assignments from my PowerSchool gradebook from a year ago and see the distribution of assignments I had by the end of the semester:
Screen Shot 2012-12-12 at 8.32.14 AM

My grades were category based - 5% class-work, 10% homework completion, 10% portfolio, 60% unit tests, and 15% quizzes. This comprised 80% of the semester grade, and was the grade that students saw for the majority of the semester. A semester exam at the end made up the remaining 20%.

While I did enter some information about the homework assignments, my grade was just a reflection of how they completed it relative to the effort I expected them to make while working on it. No penalty for being wrong on problems, but a cumulative penalty developed over time for students tending not to turn it in. This, however, was essentially a behavior grade, and not an indication of what they were actually learning. The homework was the most frequent way for students to get feedback, and it did help students improve in what they were learning, but the completion grade was definitely not a measure of what they were learning at all. There were six quizzes that fit into my reassessment system. Not important enough to matter, I realize now with 20/20 hindsight.

The entire Standards-based-grading community shoots me a look saying 'we told you so', but only momentarily and without even a hint of snark. They know I am on their side now.

Here is a screen shot of the assignments in my grade-book as of this morning:
Screen Shot 2012-12-12 at 8.35.40 AM

There is a clear indication of what my students have been working on here. With the exception of the portfolio, a student can look at this (and the descriptions I've included for each standard) and have a pretty good idea of what they did and didn't understand over the course of the semester. They know what they should be working on before the semester exam next week. The parents can get a pretty good idea of what they are looking at as well. I knew making the change to standards based grading (SBG) made sense, but there have been so many additional reasons I am happy to have made the change that I really don't want to go back to the old system.

I'll do more of a post-game analysis of my SBG implementation in PowerSchool soon. I will be making changes and enhancing parts that I like about what I have done so far. I have to first make it through the busy time ahead of marking exams, submitting comments, and getting my life ready for the extended winter break that is peeking its beautiful head over the piles of reassessments on my desk. It is really satisfying to see that my students have weathered the transition to SBG beautifully. Their grades really do emphasize the positive aspects of learning that a pure assignments & points system blurs without thinking twice.

Rubrics & skill standards - a rollercoaster case study.

  • I gave a quiz not long ago with the following question adapted from the homework:

The value of 5 points for the problem came from the following rubric I had in my head while grading it:

  • +1 point for a correct free body diagram
  • +1 for writing the sum of forces in the y-direction and setting it equal to may
  • +2 for recognizing that gravity was the only force acting at the minimum speed
  • +1 for the correct final answer with units

Since learning to grade Regents exams back in New York, I have always needed to have some sort of rubric like this to grade anything. Taking off  random quantities of points without being able to consistently justify a reason for a 1 vs. 2 point deduction just doesn't seem fair or helpful in the long run for students trying to learn how to solve problems.

As I move ever more closely toward implementing a standards based grading system, using a clearly defined rubric in this way makes even more sense since, ideally, questions like this allow me to test student progress relative to standards. Each check-mark on this rubric is really a binary statement about a student relative to the following standards related questions:

  • Does the student know how to properly draw a free body diagram for a given problem?
  • Can a student properly apply Newton's 2nd law algebraically to solve for unknown quantities?
  • Can a student recognize conditions for minimum or maximum speeds for an object traveling in a circle?
  • Does a student provide answers to the question that are numerically consistent with the rest of the problem and including units?

It makes it easy to have the conversation with the student about what he/she does or does not understand about a problem. It becomes less of a conversation about 'not getting the problem' and more about not knowing how to draw a free body diagram in a particular situation.

The other thing I realize about doing things this way is that it changes the actual process of students taking quizzes when they are able to retake. Normally during a quiz, I answer no questions at all - it is supposed to be time for a student to answer a question completely on their own to give them a test-like situation. In the context of a formative assessment situation though, I can see how this philosophy can change. Today I had a student that had done the first two parts correctly but was stuck.


Him: I don't know how to find the normal force. There's not enough information.


Me: All the information you need is on the paper. [Clearly this was before I flip-flopped a bit.]


Him: I can't figure it out.

I decided, with this rubric in my head, that if I was really using this question to assess the student on these five things, that I could give the student what was missing, and still assess on the remaining 3 points. After telling the student about the normal force being zero, the student proceeded to finish the rest of the problem correctly. The student therefore received a score of 3/5 on this question. That seems to be a good representation about what the student knew in this particular case.

Why this seems slippery and slopey:

  • In the long term, he doesn't get this sort of help. On a real test in college, he isn't getting this help. Am I hurting him in the long run by doing this now?
  • Other students don't need this help. To what extent am I lowering my standards by giving him information that others don't need to ask for?
  • I always talk about the real problem of students not truly seeing material on their own until the test. This is why there are so many students that say they get it during homework, but not during the test - in reality, these students usually have friends, the teacher, example problems, recently going over the concept in class on their side in the case of 'getting it' when they worked on homework.

Why this seems warm and fuzzy, and most importantly, a good idea in the battle to helping students learn:

  • Since the quizzes are formative assessments anyway, it's a chance to see where he needs help. This quiz question gave me that information and I know what sort of thing we need to go over. He doesn't need help with FBDs. He needs help knowing what happens in situations where an object is on the verge of leaving uniform circular motion. This is not a summative assessment, and there is still time for him to learn how to do problems like this on his own.
  • This is a perfect example of how a student can learn from his/her mistakes.  It's also a perfect example of how targeted feedback helps a student improve.
  • For a student stressed about assessments anyway (as many tend to be) this is an example of how we might work to change that view. Assessments can be additional sources of feedback if they are carefully and deliberately designed. If we are to ever change attitudes about getting points, showing students how assessments are designed to help them learn instead of being a one-shot deal is a really important part of this process.

To be clear, my students are given one-shot tests at the end of units. It's how I test retention and the ability to apply the individual skills when everything is on the table, which I think is a distinctly different animal than the small scale skills quizzes I give and that students can retake. I think those are important because I want students to be able to both apply the skills I give them and decide which skills are necessary for solving a particular problem.

That said, it seems like a move in the right direction to have tried this today. It is yet one more way to start a conversation with students to help them understand rather than to get them points. The more I think about it, the more I feel that this is how learning feels when you are an adult. You try things, get feedback and refine your understanding of the problem, and then use that information to improve. There's no reason learning has to be different for our students.