Tag Archives: sbar

End of year reflections - SBAR analysis

I wrapped up grading final exams today. Feels great, but it was also difficult seeing students coming in to get their final checklists signed before they either graduate or move on to a different school. Lots of bittersweet moments in the classroom today.

I decided after trying my standards based grading (SBG) experiment that I wanted to compare different students' overall performances among the grading categories to their quiz percentages. In a previous post, I wrote about my experimentation doing my quiz assessments on very specific skill standards that the students were given. As I plan to change my grading to be more SBG based for the fall, I figured it would be good to have some comparison data to be able to argue the many reasons why this is a good idea.

In my geometry and algebra two classes, there are 28 total students. I removed two students from the data set that came in the last month of school, and one outlier in terms of overall performance.

The table below shows the names of each grading category, as well as the overall grade weight for the category used in calculating the grade. The numbers are the correlation coefficient in the data between the variable listed in the row and column. For example, the 0.47 is the correlation between the HW data and the Quizzes/Standards data for each student.


Homework (8%) Quizzes/Standards (12%) Test Average (48%) Semester Exam (20%) Final Grade (100%)
HW 1 0.47 0.41 0.36 0.48
Quiz/Stndrds
1 0.89 0.91 0.95
Tests

1 0.88 0.97
Sem. Exam


1 0.95
Final Grade



1

Some notes before we dive in:

  •  The percentages do not add up to 100% because I am leaving out the portfolio grade (8%) and classwork grades (4%) which are not performance based.
  • Homework is graded on turning it in and showing work for answers. I collect and look at homework as a regular way to assess how well they are learning the material.
  • The empty cells are to save ink; they are just the mirror image of the results in the upper half of the matrix since correlation is not order dependent.
  • I know I only have 25 samples - certainly not ready for a research publication.

So what does this mean? I don't know that this is very surprising.

  1. Students doing HW is not what makes learning happen. I've always said that and this continues to support that hypothesis. It can help, it is evidence that students are doing something with the material between classes, but simply completing it is not enough. I'm fine with this. I get enough information from looking at the homework to create activities that flesh out misunderstanding the next time we meet. The unique thing about homework is that it is often the first time students look at the material on their own rather than with their peers in class.
  2. My tests include some direct assessments of skills, but also include a lot of new applications of concepts and questions requiring students to explain or show things to be true. It's very gratifying to see such a strong connection between the quiz scores and the test scores.
  3. I always wonder about the students that say "I get it in class, but then on the tests I freeze up." If there's any major lesson that SBG has confirmed for me, it's that student self-awareness of proficiency is generally not great without some form of external feedback. If this were the case, there would be more data with high quiz scores and low exam scores. That isn't the case here. My students need real and correct feedback on how they are doing, and the skills quizzes are a formalized way to do this.
  4. I find it really interesting how close the quiz average and the semester exam percentages are. The semester exam was cumulative and covered a lot of ground, but it didn't necessarily hit every single skill that was tested on quizzes. There were also not quizzes for every single skill, though I tried to hit a number of key ones.

This leads me to believe that it is possible to have several key standards to focus on for SBG purposes, and also to dedicate time to work on other concepts during class time through project based learning, explorations, or independent work. It's feasible to assess these other concepts as mathematical process standards that are assessed throughout the semester. It strikes a good balance between developing skills according to curriculum but not making classes a repetitive process of students absorbing procedures for different types of problems. I want to have both. My flipping experiments have worked well to approaching that ideal, but I'm not quite there yet.

I'll have more to say about the details of what I will change as I think about it during the summer. I think a combination of using BlueHarvest for feedback, extending SBG to my Calculus and Physics classes, and less emphasis on grading and collecting homework will be part of it. Stay tuned.

Results of a unit long experiment in SBG and flipping.

I've been a believer in the concept of standards based instruction for a while. The idea made a lot of sense when I first learned about the idea when Grant Wiggins visited my school in the Bronx a few years ago to present on Understanding by Design. Dan Meyer explored the idea quite a bit using his term of the concept checklist. Shawn Cornally talks on his blog about really pushing the idea to give students the freedom to demonstrate their learning in a way they choose, though he ultimately retains judgment power on whether they have or not. Countless others have been really generous in sharing their standards and their ideas for making standards work for their students. Take a look at my blogroll for more people to read about. For those unaware, here's the basic idea: Look at the entire unit and identify the specific skills or you want your students to have. Plan your unit to help them develop those skills. Assess and give students feedback on those skills as often as possible until they get it. In standards based grading (SBG), reporting a grade (as most of us are required to do) as a fraction of standards completed or acquired becomes a direct reflection of how much students have learned. Compare this to the more traditional version of grading that consists of an average of various 'snapshots' on assignments, on which grades might be as much a reflection of effort or completion as of actual learning. If learning is to be the focus of what we do in the classroom, then SBG is a natural way of connecting that learning to the grades and feedback we give to students. My model for several years now has been, well,  SBG lite. Quizzes are 15% of the total grade and test only a couple skills at a time. Students can retake quizzes as many times as they want to show that they have the skills in isolation. On tests, (60% of the total grade) students can show that they can correctly apply the set of all of their acquired skills on exercises (questions they have seen before) as well as problems (new questions that test conceptual understanding). As much as I tell students they can all have a grade of 100% for quizzes and remind those that don't to retake, it doesn't happen. I'll get a retake here or there. I am still reporting quiz grades as an average of a pool of "points" though, and this might leave enough haziness in the meaning of the grade for a student to be OK with a 60%. For this unit in Geometry and Algebra 2, I have specifically made the quiz grade a set of standards to be met. The point total is roughly the same as in previous units. It is a binary system - students either have the standard (3/3) or they don't (0/3), and they need to assess each standard at least twice to convince me they have it. I really like Blue Harvest, but my students didn't respond so well to having twowhole websites to use to check progress. While a truly scientific study would have changed only one variable at a time, I also found that structuring the skill standards this way required me to change the way class itself was structured. This became an experiment not only in reporting grades, but in giving my students the power to work on things in their own way. This also freed me up to spend my time in class assessing, giving feedback, and assessing again. More on this ahead. The details:

Geometry

I started the unit by defining the seven skills I wanted the students to have by the end on this page. The unit was on transformational geometry, so a lot of the skills were pretty straight forward applications of different types of transformations to points, line segments, and polygons. I had digital copies of all of the materials I put together last year for this unit, so I was able to post all of that material on the wiki for students to work through on their own. I adjusted these materials as we moved through the unit and as I saw there were holes in their understanding. I was also able to make some videos using Jing and Geogebra to explain some concepts related to using vocabulary and symmetry, and these seemed to help some students that needed a bit of direct instruction in addition to what I provided to them one on one. I also tried another experiment - programming assignments related to applying transformations to various points. I said completing these assignments and chatting with me about them would qualify them for proficiency on a given standard. Assigning homework was simple: Choose a standard or two, and do some of the suggested problems related to those standards. Be prepared to show me your evidence of study when you come into class. Students that said 'I read my notes' or 'I looked it over' were heckled privately - the emphasis was on actively working to understand concepts. Some students did flail a bit with the new freedom, so I made suggestions for which standards students should spend a particular day working on, and this helped these students to focus. I threw together some concept quizzes for the standards covered by the previous classes, and students could choose to work on those question types they felt they had mastered. Some handed the quiz right back knowing they weren't ready. I was really pleased with the level of awareness they quickly developed around what they did and didn't understand. I quickly ran into the logistical nightmare of managing the paperwork and recording assessment results. Powerschool Blue Harvest, whatever - this was the most challenging aspect of doing things this way. I often found myself bogged down during the class period recording these things, which got in the way of spending quality face time with students around their understanding. Part of this was that I was recording progress for each standard, whether good or bad, in the comment field for each student. "Understands basic idea of translation, but is confusing the image and pre-image" is the sort of comment I started writing in the beginning. While this was nice, and I think could have led to students reading the comments and getting ideas for what they needed to work on, it was a bit redundant since I was having actual conversations with students about these facts. Here is where Blue Harvest shines - I can easily send students a quick message explaining (and showing) what they need to work on. Even more powerful would be recording the conversation when I actually talk to the student, but that would be more practical with an iPad/cell phone app to avoid lugging my computer from desk to desk. Still, I wanted the feedback to be immediate and be recorded, so I knew I had to change my approach. The compromise was to only record positive progress. If a student's quiz showed no progress, it didn't get a comment in Powerschool. If they showed progress, but needed to fix a small detail in their understanding, they might get a comment. If they clearly got it, they got a comment saying that they aced it. Two or more positive comments (and my independent review) led to a 3/3 for each standard. The other promise I made was that if they clearly demonstrated proficiency on the exam (which had non-standard questions and some things they needed to explain) I would give them credit for the standard. The other difficult issue was creating a bank of reassessment questions. My system of making a quiz on the spot and handing it out to individual students was too time consuming. I created an app(using my new Udacity knowledge) to try to do this, the centerpiece being a randomized set of questions that emphasized knowing how to figure out the answers rather than students potentially sharing all the answers. They quickly found all the bugs in my system, and showed that it is far from ready for being an actual useful tool for this purpose. I appreciated their humor and patience in being guinea pigs for an idea. As you might notice from the image above, there is a pretty strong relationship between the standards mastered and the exam scores. Most student exam scores were either the same or better following this system in comparison to previous exams. The most important metric is the fact that most students weren't hurt by going to this more student-centered model. Some student took more notes while working to understand the material than they have all year. Other students spoke more to their classmates and both gave and received more help in comparison to when I was at the front of the room asking questions and doing mini-lessons. While there was a lot of staring at screens during this unit, there was also a lot of really great discussion. I would have focused conversations with every single student three to four times a class, and they were directly connected to the level of understanding they had developed. Some needed direct application questions. Others could handle deeper synthesis and 'why is this true' questions about more abstract concepts. It felt really great doing things this way. I have always insisted on crafting one good solid presentation to give the class - the perfect lesson - with good questions posed to the class and discussions inevitably resulting from them. I have to admit that having several smaller, unplanned, but 'messier' conversations to guide student learning have nurtured this group to be more independent and self driven than I expected before we started.

Algebra 2

The unit focused on the students' first exposure to logarithmic and exponential functions. The situation in Algebra 2 was very similar to Geometry, with one key difference. The main difference of this class compared to Geometry is that almost all of the direct instruction was outsourced to video. I decided to follow the Udacity approach of several small videos (<3 min), because that meant there was opportunity (and the expectation) that only two minutes would go by before students would be expected to do something. I like this much better because it fit my own preferences in learning material with the Udacity courses. I had 2 minutes to watch a video about hash functions in Python while brushing my teeth - my students should have that ability too. I wasn't going for the traditional flipped class model here. My motivation was less about requiring students to watch videos for homework, and more about students choosing how they wanted to go through the material. Some students wanted me to do a standard lesson, so I did a quick demonstration of problems for these students. Others were perfectly content (and successful) watching the video in class and then working on problems. Some really great consequences of doing things this way:

  • Students who said they watched all my videos and 'got it' after three, two minute videos, had plenty of time in the period to prove it to me. Usually they didn't.. This led to some great conversations about active learning. Can you predict the next step in the video when you try solving the problem on your own? What? You didn't try solving it on your own? <SMIRK>  The other nice thing about this is that it's a reinvestment of two minutes suggesting that they try again with the video, rather than a ten or fifteen minute lesson from Khan Academy.
  • I've never heard such spirited conversation between students about logarithms before. The process of learning each skill became a social event - they each watched the video together, rewound or paused as needed, and then got into arguments while trying to solve similar problems from the day's handout. Often this would get in the way during teacher-centered lessons, and might be classified incorrectly as 'disruption' rather than the productive refining and conveyance of ideas that should be expected as part of real learning.
  • Having clear standards for what the students needed to be able to do, and making clear what tools were available to help them learn those specific standards, led to a flurry of students demanding to show me that they were proficient. That was pretty cool, and is what I was trying to do with my quiz system for years, but failed because there was just too much in the way.
  • Class time became split between working on the day's standards, and then stopping at an arbitrary time to then look at other cool math concepts. We played around with some Python simulations in the beginning of the unit, looked at exponential models, and had other time to just play with some cool problems and ideas so that the students might someday see that thinking mathematically is not just followinga list of procedures, it's a way of seeing the world.

I initially did things this way because a student needed to go back to the US to take care of visa issues, and I wanted to make sure the student didn't fall behind. I also hate saying 'work on these sections of the textbook' because textbooks are heavy, and usually blow it pretty big. I'm pretty glad I took this opportunity to give it a try. I haven't finished grading their unit exams (mostly because they took it today) but I will update with how they do if it is surprising.


Warning: some philosophizing ahead. Don't say I didn't warn you. I like experimenting with the way my classroom is structured. I especially like the standards based philosophy because it is the closest I've been able to get to recreating my Montessori classroom growing up in a more traditional school. I was given guidelines for what I was supposed to learn, plenty of materials to use, and a supportive guide on the side to help me when I got stuck. I have seen a lot of this process happening with my own students - getting stuck on concepts, and then getting unstuck through conversation with classmates and with me. The best part for me has been seeing my students realize that they can do this on their own, that they don't always need me to tell them exactly what to do at all times. If they don't understand an idea, they are learning where to look, and it's not always at me. I get to push them to be better at what they already know how to do rather than being the source of what they know. It's the state I've been striving to reach as a teacher all along, and though I am not there yet, I am closer than I've ever been before. It's a cliche in the teaching world that a teacher has done his or her job when the students don't need you to help them learn anymore. This is a start, but it also is a closed-minded view of teaching as mere conveyance of knowledge. I am still just teaching students to learn different procedures and concepts. The next step is to not only show students they can learn mathematical concepts, but that they can also make the big picture connections and observe patterns for themselves. I think both sides are important. If students see my classroom as a lab in which to explore and learn interesting ideas, and my presence and experience as a guide to the tools they need to explore those ideas, then my classroom is working as designed. The first step for me was believing the students ultimately wantneed to know how to learn on their own. Getting frustrated that students won't answer a question posed to the entire class, but then will gladly help each other and have genuine conversations when that question comes naturally from the material. All the content I teach is out there on the internet, ready to be found/read/watched as needed. There's a lot of stuff out there, but students need to learn how to make sense of what they find. This comes from being forced to confront the messiness head on, to admit that there is a non-linear path to knowledge and understanding. School teaches students that there is a prescribed order to this content, and that learning needs to happen within its walls to be 'qualified' learning. The social aspect of learning is the truly unique part of the structure of school as it currently exists. It is the part that we need to really work to maintain as content becomes digital and schools get more wired and connected. We need to give students a chance to learn things on their own in an environment where they feel safe to iterate until they understand. That requires us as teachers to try new things and experiment. It won't go well the first time. I've admitted this to my students repeatedly throughout the past weeks of trying these things with my classes, and they (being teenagers) are generous with honest criticism about whether something is working or not. They get why I made these changes. By showing that iteration, reflection, and hard work are part of our own process of being successful, they just might believe us when we tell them it should be part of theirs.