Category: reflection

Who’s gone overboard modeling w/ Python? Part II – Gravitation

I was working on orbits and gravitation with my AP Physics B students, and as has always been the case (including with me in high school), they were having trouble visualizing exactly what it meant for something to be in orbit. They did well calculating orbital speeds and periods as I asked them to do for solving problems, but they weren’t able to understand exactly what it meant for something to be in orbit. What happens when it speeds up from the speed they calculated? Slowed down? How would it actually get into orbit in the first place?

Last year I made a Geogebra simulation that used Euler’s method  to generate the trajectory of a projectile using Newton’s Law of Gravitation. While they were working on these problems, I was having trouble opening the simulation, and I realized it would be a simple task to write the simulation again using the Python knowledge I had developed since. I also used this to-scale diagram of the Earth-Moon system in Geogebra to help visualize the trajectory.

I quickly showed them what the trajectory looked like close to the surface of the Earth and then increased the launch velocity to show what would happen. I also showed them the line in the program that represented Newton’s 2nd law – no big deal from their reaction, though my use of the directional cosines did take a bit of explanation as to why they needed to be there.

I offered to let students show their proficiency on my orbital characteristics standard by using the program to generate an orbit with a period or altitude of my choice. I insist that they derive the formulae for orbital velocity or period from Newton’s 2nd law every time, but I really like how adding the simulation as an option turns this into an exercise requiring a much higher level of understanding. That said, no students gave it a shot until this afternoon. A student had correctly calculated the orbital speed for a circular orbit, but was having trouble configuring the initial components of velocity and position to make this happen. The student realized that the speed he calculated through Newton’s 2nd had to be vertical if the initial position was to the right of Earth, or horizontal if it was above it. Otherwise, the projectile would go in a straight line, reach a maximum position, and then crash right back into Earth.

The other part of why this numerical model served an interesting purpose in my class was as inspired by Shawn Cornally’s post about misconceptions surrounding gravitational potential and our friend mgh. I had also just watched an NBC Time Capsule episode about the moon landing and was wondering about the specifics of launching a rocket to the moon. I asked students how they thought it was done, and they really had no idea. They were working on another assignment during class, but while floating around looking at their work, I was also adjusting the initial conditions of my program to try to get an object that starts close to Earth to arrive in a lunar orbit.

Thinking about Shawn’s post, I knew that getting an object out of Earth’s orbit would require the object reaching escape velocity, and that this would certainly be too fast to work for a circular orbit around the moon. Getting the students to see this theoretically was not going to happen, particularly since we hadn’t discussed gravitational potential energy among the regular physics students, not to mention they had no intuition about things moving in orbit anyway.

I showed them the closest I could get without crashing:

One student immediately noticed that this did seem to be a case of moving too quickly. So we reduced the initial velocity in the x-direction by a bit. This resulted in this:

We talked about what this showed – the object was now moving too slowly and was falling back to Earth. After getting the object to dance just between the point of making it all the way to the moon (and then falling right past it) and slowing down before it ever got there, a student asked a key question:

Could you get it really close to the moon and then slow it down?

Bingo. I didn’t get to adjust the model during the class period to do this, but by the next class, I had implemented a simple orbital insertion burn opposite to the object’s velocity. You can see and try the code here at Github. The result? My first Earth – lunar orbit design. My mom was so proud.

The real power here is how quickly students developed intuition for some orbital mechanics concepts by seeing me play with this. Even better, they could play with the simulation themselves. They also saw that I was experimenting myself with this model and enjoying what I was figuring out along the way.

I think the idea that a program I design myself could result in surprising or unexpected output is a bit of a foreign concept to those that do not program. I think this helps establish for students that computation is a tool for modeling. It is a means to reaching a better understanding of our observations or ideas. It still requires a great amount of thought to interpret the results and to construct the model, and does not eliminate the need for theoretical work. I could guess and check my way to a circular orbit around Earth. With some insight on how gravity and circular motion function though, I can get the orbit right on the first try. Computation does not take away the opportunity for deep thinking. It is not about doing all the work for you. It instead broadens the possibilities for what we can do and explore in the comfort of our homes and classrooms.

Simulations, Models, and the 2012 US Election

After the elections last night, I found I was looking back at Nate Silver’s blog at the New York Times, Five Thirty Eight.

Here was his predicted electoral college map:

Image

…and here was what ended up happening (from CNN.com):

Image

I’ve spent some time reading through Nate Silver’s methodology throughout the election season. It’s detailed enough to get a good idea of how far he and his team  have gone to construct a good model for simulating the election results. There is plenty of description of how he has used available information to construct the models used to predict election results, and last night was an incredible validation of his model. His popular vote percentage for Romney was predicted to be 48.4%, with the actual at 48.3 %. Considering all of the variables associated with human emotion, the complex factors involved in individuals making their decisions on how to vote, the fact that the Five Thirty Eight model worked so well is a testament to what a really good model can do with large amounts of data.

My fear is that the post-election analysis of such a tool over emphasizes the hand-waving and black box nature of what simulation can do. I see this as a real opportunity for us to pick up real world analyses like these, share them with students, and use it as an opportunity to get them involved in understanding what goes into a good model. How is it constructed? How does it accommodate new information? There is a lot of really smart thinking that went into this, but it isn’t necessarily beyond our students to at a minimum understand aspects of it. At its best, this is a chance to model something that is truly complex and see how good such a model can be.

I see this as another piece of evidence that computational thinking is a necessary skill for students to learn today. Seeing how to create a computational model of something in the real world, or minimally seeing it as an comprehensible process, gives them the power to understand how to ask and answer their own questions about the world. This is really interesting mathematics, and is just about the least contrived real world problem out there. It screams out to us to use it to get our students excited about what is possible with the tools we give them.

First day of Geometry proofs – Refining my process

Last year, I figured out about a week or two after the first introduction to proofs in Geometry last year that I should have started with a more clear connection to the ideas we had been working on in the classes before. We did a progression of logical statements, conditional statements, working on biconditionals as definitions, and then the laws of detachment and syllogism. I realized then that I never made strong references to these concepts and how they all fit together – I just hoped that the students would see how the proofs were built out of these ideas without formally telling them as such.

This year I was much more explicit in how the ideas fit together, particularly by showing a paragraph proof as a series of conditional statements with true hypotheses. I was really happy with the results. I created two videos to use as part of the instruction:
[wpvideo 4LLQ8TEa]

Students watched the video and then worked on identifying the properties of equality and congruence being applied in a few different situations, and completing statements given that a particular property is being applied. This led to some great conversations about subtle differences between the transitive property of equality and the substitution property of equality. (‘If a = 5 and b = 5, then a = b’ is the latter, not the former. This assumes only one property is being applied at a time, of course.)

Once I was satisfied with their progress, I sent them on to watch this video:
[wpvideo EY4lUecB]

Some students immediately took the equation, solved it, and said they were done. This led to more good discussions about the purpose of this lesson. We already knew how to solve equations – what was new this time was justifying each step using a property. It was an opportunity to push these students to then focus on what was happening in each step and not so much on the algebra that they did quickly. Once I was convinced that they did understand what was going on in each step of the video, I had them move to either some problems with the steps written out, missing only the justifications (for weaker students), and for others, full algebraic problems that they do from start to finish.

The thing I did differently here (and which was made easier by the magic of video) is emphasizing that the different steps in a proof are really all either conditional statements, statements of fact (the given information), and possibly steps of arithmetic simplification. Each line should be connected to the previous one in the form of a conditional statement. I have said it in the past, but never explicitly written it out each time so that the students think of it this way.

The whole point of doing things this way is so that students are not introduced to two concepts simultaneously: writing steps in a proof, and proving a statement deductively from scratch. Having a good sense of algebra, this lesson focused on introducing students to the process first. Next time we will move on to actually finding and proving theorems about line segments, with the idea that they already have a basic sense for how different thoughts can be linked together logically in a proof. I am hoping that being this deliberate will pay off – this was definitely the smoothest this lesson has ever gone for me.

Rethinking my linear function approach in Algebra 2

My treatment of linear functions in the past has been pretty traditional. Solve for y, y = mx + b, graphing using slope intercept, then move on to linear inequalities in two variables…it is just dull this way. Most students have seen it before in one form or another, and it wasn’t exciting (or that novel) to them the first time they learned it. It doesn’t have to be this way, and I committed myself this year to doing things differently.

My approach has been centered on two big ideas:

  1. Linear functions have a constant rate of change. All of the other qualities they have are related to this important fact.
  2. There is an amazing connection between graphs, tables of values, and the equations that generate linear functions. These are not three separate skills, they are three views of the same fundamental mathematical object. Corollary: Teaching them on three separate days or sticking to one view at a time creates an unnecessary pigeon-holing effect that sticks with students for as long as conditions in your class permit.

On day one, we did my Robot Tracking activity posted here at GeogebraTube. The video introduction was reviewed in class and students worked on it for much of the period. This emphasized a fundamental concept around linear functions of distance and time that was pretty intuitive to nearly all of the students that did this activity.

Predicting where something is located, assuming it continues moving at a constant rate is one of the most common applications of linearity. We do it all the time. Can we cross the street in front of the bus? Mental calculation. Where should I kick the soccer ball to get it right in front of the forward moving toward the goal? Mental calculation. I don’t mean actually sitting down and calculating where it will be, but that the human brain is pretty good at noticing the velocity of objects, and making a pretty good guess of where it will be. They had a number of methods of coming to an answer that ranged from geometric (simply drawing a line) to counting grid squares, using the trace function, and proportional reasoning.

We ended the period looking at the Python script I posted here and trying to calculate speed from the information generated by the program. Part of the homework assignment for the next class was to try to answer the question posed by another Python program posted here. The table of values is randomly determined each time, and students could (and often did) try it multiple times to get it right.

The next lesson had a single instance of this program as a warm up for the whole class – everyone had to agree on what value of position I needed to enter for the given time value.. They were pretty good at checking each other and having good conversations about how to go about it. They answered correctly, but we had a good conversation about the different ways to get there. They all centered on using the fact that there was equal spacing between all of the points. Most students used some variation of finding the distance moved per second and whether it was positive or negative, and then counted off intervals. In most cases, it was a bit complicated and required a lot of accounting to get to their answer.

We went over the reason we could do this – the constant rate of change – and verified it using a few different pairs of points. I then threw in the idea of using the point (x,y) and using the constant rate of change with that point. We got to $latex frac{y – b}{x – a}=m $ and I asked them to write this using the slope we calculated and any point they liked from the table of data. Students seated next to each other I encouraged to use different points. I then asked them to answer the original question from the Python program using their equation. (Un)surprisingly enough, they all ended up with the correct (and same) answer as before.

Some of them started distributing and writing in slope intercept form. THe thing I was kind of excited about was that they didn’t feel the equation had to be written that way, they just felt like seeing what happened. Many discovered the fact that their answers were the same after doing so, even though they started with different points. We did a couple examples of solving more basic ‘Write an equation for a line that…” questions, but did so without making a huge deal out of slope-intercept form or point-slope form and why one might be better than the other in different situations.

Today was the third day going through this concept – the warm up activity had three levels to it:

The goal here was to constantly push the students to go back and forth between the equation and numerical representations of these functions. There were lots of good things students figured out from these. We then made the jump to looking at how the graph is connected to the table and equation – just one more way of looking at the same mathematical function, and it shares the meaning that comes with the other two representations: a constant rate of change. The new idea introduced as part of this was that of an intercept. What does it mean on the graph? What does it mean for the table? We didn’t talk explicitly about the intercept’s meaning of the equation (again, trying to avoid the “that’s just y = mx + b, I know this already…TUNED OUT”) , but it came out in the process of identifying it algebraically, from tables, and then graphing.

By the end of the period, we were graphing linear functions. Students were asking excellent questions about when the intercepts alone can be used to graph the line, when they can’t ($latex 2x+3y=6$ versus $latex 2x+3y=7$) but they again stuck to the idea of finding a point they know is on the graph, and then using the constant rate of change to find others. Instead of spending a boring lesson explicitly telling them what my expectations are for graphing lines (labeled and scaled axes, line going all the way across the extent of the axes, arrows on axes and lines) I was able to gently nudge students to do this while they worked.

We’ll see how things go as we continue to move forward. The big thing I like about this progression so far is that modeling real phenomena will be a natural extension of what we’ve already done – not a lesson at the end of contrived examples with clean numbers. My goal originally was to get this group comfortable with messy data and being comfortable with using different tools to make sense of it.

I’ve kept my students hermetically sealed from this messiness in the past – integer coefficients, integer values, and explicit step-by-step ways of graphing, generating tables, and writing equations. As I mentioned before, it was, well, boring and predictable, and perpetuated the idea that these skills are all separated from each other. It also continued the pattern that there would be a day in each unit where the numbers are messy, the real world word problems day, but that the pain associated with it would last a day and would be over soon enough.

I’m hoping to reduce this effect by changing my approach. That by seeing the different aspects of linear functions, it will seem natural to use a graph to figure out something that might not make sense algebraically, or use numerical values to solve an algebraic problem. I especially like this because exploring the three views of functions really is, in my opinion, the primary learning goal of the Algebra 2 course. If I can establish this as an expectation early on, I think the latter parts of the course will work much more smoothly.

Why SBG is blowing my mind right now.

I am buzzing right now about my decision to move to Standards Based Grading for this year. The first unit of Calculus was spent doing a quick review of linear functions and characteristics of other functions, and then explored the ideas of limits, instantaneous rate of change, and the area under curves – some of the big ideas in Calculus. One of my standards reads “I can find the limit of a function in indeterminate form at a point using graphical or numerical methods.”

A student had been marked proficient on BlueHarvest on four out of the five, but the limit one held her back. After some conversations in class and a couple assessments on the idea, she still hadn’t really shown that she understood the process of figuring out a limit this way. She had shown that she understood that the function was undefined on the quiz, but wasn’t sure how to go about finding the value.

We have since moved on in class to evaluating limits algebraically using limit rules, and something must have clicked. This is what she sent me this morning:
[wpvideo 5FSp5JDn]

Getting things like this that have a clear explanation of ideas (on top of production value) is amazing – it’s the students choosing a way to demonstrate that they understand something! I love it – I have given students opportunities to show me that they understand things in the past through quiz retakes and one-on-one interviews about concepts, but it never quite took off until this year when their grade is actually assessed through standards, not Quiz 1, Exam 1.

I also asked a student about their proficiency on this standard:

I can determine the perimeter and area of complex figures made up of rectangles/ triangles/ circles/ and sections of circles.

I received this:
…followed by an explanation of how to find the area of the figure. Where did she get this problem? She made it up.

I am in the process right now of grading unit exams that students took earlier in the week, and found that the philosophy of these exams under SBG has changed substantially. I no longer have to worry about putting on a problem that is difficult and penalizing students for not making progress on it – as long as the problem assesses the standards in some way, any other work or insight I get into their understanding in what they try is a bonus. I don’t have to worry about partial credit – I can give students feedback in words and comments, not points.

One last anecdote – a student had pretty much shown me she was proficient on all of the Algebra 2 standards, and we had a pretty extensive conversation through BlueHarvest discussing the details and her demonstrating her algebraic skills. I was waiting until the exam to mark her proficient since I wanted to see how student performance on the exam was different from performance beforehand. I called time on the exam, and she started tearing up.

I told her this exam wasn’t worth the tears – she wanted to do well, and was worried that she hadn’t shown what she was capable of doing. I told her this was just another opportunity to show me that she was proficient – a longer opportunity than others – but another one nonetheless. If she messed up a concept on the test from stress, she could demonstrate it again later. She calmed down and left with a smile on her face.

Oh, and I should add that her test is looking fantastic.

I still have students that are struggling. I still have students that haven’t gone above and beyond to demonstrate proficiency, and that I have to bug in order to figure out what they know. The fact that SBG has allowed some students to really shine and use their talents, relaxed others in the face of assessment anxiety, and has kept other things constant, convinces me that this is a really good thing, well worth the investment of time. I know I’m just preaching to the SBG crowd as I say this, but it feels good to see the payback coming so quickly after the beginning of the year.

Flipping, Week 1: Stop the Blabbing.

One of my major goals this year is to stop talking so much. Even in my tenth year, I still spend far too much time explaining, questioning, and presenting in front of the class.

The nature of this talking has changed a lot though. When I first started teaching, it was almost all explaining. That’s what I thought good teaching was all about – if you could just explain it the right way, then everyone would get the concept you are teaching, right? A perfect lesson consisting of a perfect development, a perfect explanation of all concepts, perfect example problems, and perfect students. This is how I looked at it during my summer training, and before I got into my classroom.

That changed pretty quickly once I actually got started. Explanations were important, but more important was getting students to be somehow involved. My coaching from administration was focused on good questioning over talking and explaining as a way to do this, so I put a lot of energy into this during my first couple years, and it has since stuck.

The problem is that I am often addicted to asking questions when it’s really time for students to get working and thinking on their own. I can ask questions like crazy, which might have really impressed administrators in my room at one time, but it probably infuriated (and still infuriates) my students to no end. As good of a question I could have asked, they were still just sitting there thinking and not doing any active learning on their own. Furthermore, when the one or two students do answer a question I ask, it isn’t necessarily a real indication of what thinking is going on in the heads of the other students in the room. Students who self select to participate make for a bad sample for the level of understanding in the rest of the room.

The technique to address poor participation (as pushed by my administration in my first couple years) was to cold call students. This is a bad sample in the other direction – pushing a student to go from full listening mode to full participation mode with the rest of students is not an effective way to make dialogue an important part of what goes on in your classroom every day. This is especially the case for students that have poor self esteem about math in the first place. Good conversation is rarely one or two to many. When was the last time you saw twenty people actively involved in a discussion? Why would you really try to get that going in a classroom when it doesn’t work for a room full of adults at a faculty meeting?

In reality, real learning doesn’t look like a kid staring into space pondering a good question. It involves experimenting, testing a theory, writing down an idea and trying it out. It involves taking what you have produced out of your own thinking and getting active and reliable feedback.

Back to my main point – I am attempting this year to put any direct instruction for a particular day’s lesson in two minute video chunks, and limiting the number of these to no more than four per day. A few really nice things have happened since doing this:

  • I’m putting a lot of thought into exactly which ideas are best left to video or direct instruction, and which ideas will come out through conversation and the activities. Some things are better taught in a big group, I’m not going to deny this to be the case. Some things are better learned one-on-one, and thinking about the difference has really changed the way I organize the activities for the day.
  • My students are spending a lot less time listening to me, and more time engaged in the videos and what I ask them to do. The videos sometimes include straight example problems, but I try to include a couple things for students to actually do, write down, or talk about with the person next to them as they watch. The conversations that students have during the videos are really rich (and remarkably on-topic), and are so much more useful than having me tell them things while they stare at me.
  • I can do other things while they are working on the activities I give them. I can see how they are watching the video and make suggestions on things they should be writing down. I can test their knowledge by asking one-on-one questions and get a really good sense for the level of understanding for each student. I can look at quizzes I gave at the beginning of the period, make comments on them, and have a conversation with the students about their work before the end of the period! The quality of my interaction with students has been much higher than before, which has resulted in larger amounts of quality feedback. That is really the goal here.
  • My ESOL students are loving it. They can take time with vocabulary, which is the hard part for them, and make note of the mathematics concepts at the same time. Some students are using the videos to create their own glossaries in other languages. I’ve always suggested that students do this, but until now, I haven’t seen them do it so well, let alone of their own volition.
  • The learning in my room is messier than ever before – everyone is at different points and is having different conversations. There are papers all over the place. Students are crowded around tables working and are facing all different directions. Seeing this sort of thing happen my first year would have meant that this was a spent lesson, that I had lost them. Here, it just looks like (and is so far proving to me) good learning experiences for students.

    • This post is partly in response to one of the new blogger prompts about what I want my students to remember ten years down the road. I really don’t care if they remember how to factor quadratics. Moving to a more student-centered learning model though has made the students in charge of making sure they understand what they are learning. I would love if students tell me in ten years that they learned how to learn something new in my class. Real learning is messy, and actually doing math (not watching), making mistakes and growing from feedback is part of the game. There’s not as much room for that in the more traditional math structure of “I-Do,We-do,You-do” model because the last part is where the real learning happens. Maximizing that part (and simultaneously providing ways for that feedback to happen) is the real meat of teaching, and it’s where I am focusing my energy this year.

    Here’s to keeping it going as long as possible!

Standards Based Grading – All in, for the new year

I’ve written previously about wanting to be part of the Standards Based Grading crowd. My quiz policy was based in the idea – my quizzes cover skills only and in isolation, the idea being that if students could show proficiency on the quizzes, then I would know for sure that they had really developed those skills. If they had demonstrated proficiency, but then failed on tests to perform, it was an indication that the problem was seeing all the skills in one place. This is the “I get it in class, but on tests I mess it up” mantra that I’ve heard ever since I first started teaching. My belief has always been that the first clause of that sentence is never as true as the student thinks it is. The quiz grades have typically shown that to be the case.

The thing I haven’t been able to get at is why I can’t get my students to retake quizzes as I thought it compelled them to do. I told them they can get 100%. I reminded them that they just needed to look at each quiz, recognize what they got wrong, and work with me on those specific skills to improve. Then, when they were ready, they could retake and get a better score. Sometimes they do it, but they are always missing either one of those three things. They would retake without looking at the quiz. They would take it knowing what they got wrong, but never asked me to go over the things they didn’t get. There were exceptions, but curiously not enough to impress me.

After really committing to reshaping the quiz grade as a real SBG grade for a unit last year, I saw the differences pretty clearly in how the students went about this aspect of their grade. The standards I expected students to demonstrate were clearly listed in the grade book (fine, Powerschool). The students knew what they needed to work on, and were directly linked to examples and short videos I had created to help them with those specific skills. Class time was spent working around developing those skills, along with some bigger picture ideas to explore separately from the routine skills the standards were centered around for the unit, which was on exponential and logarithmic functions. I was impressed in this short time with how changing this small (15%) portion of the grade changed the overall attitude my students had while they were working with me. It was one step closer to the Montessori style classroom I have always wanted to have while working within the structure of a more traditional program – students walk in knowing what they need to work on, and they get to work. My role becomes more to push them in the way I think they can and need to be pushed. Some need to work on skills, others need to attack context problems and the challenging ‘why is this so’ threads that are usually all teacher driven, but don’t need to be in many cases.

I did some thinking over the last couple of weeks on how I wanted to do things differently, so I wrote up a new grading policy and posted it online. I had renamed my quiz grade to be ‘Learning Standards’, bumped up the percentage by 10% (to 25%), and reduced the homework and classwork components to 5% each, with a portfolio at 10%, and tests to 55%.  In sharing my new grading policy with people through Twitter, there were some key comments that really guided my thinking.

Kelly O’Shea pointed out the fact that even with the change, the standards were not a huge part of the grade. Even by cutting classwork and homework into the standards, it still wasn’t good enough:

A few other people made similar suggestions. John Burk probably put the final nail in the SBG-lite version I thought was safe with this comment:

One problem for getting buy in on SBG is that if it isn’t a big part of the grade, and there are still so many non-sbg things, they might not really understand the rationale for SBG.

If I really believe in the power for Standards Based Grading to transform how learning happens in my classroom, I need to demonstrate its importance and commit to it.

The final result? My grades for Algebra 2/Advanced Algebra, Geometry, Calculus 12, and Physics are going to be 90% Learning Standards, 10% portfolio. I am going to give unit tests, but they are opportunities to demonstrate proficiency on the learning standards. In the case of my AP Calculus students, the grades are still 60% unit tests, 30% standards, and 10% portfolio, primarily because I still will be giving tests that are similar to the AP exam with multiple choice, and free response sections. I also had my first class last year with 100% fives, and am admittedly a bit nervous tweaking what worked last year. That said, I am accepting that this, too, could become a thing of the past.

I am a bit nervous, but that’s mostly because change isn’t always easy. From a teaching perspective, the idea feels right, but it’s not what I’m used to doing. The students sounded pretty cool with it on the first days of class when I introduced the idea though, and that is a major positive. I’ll keep writing as things proceed and my implementation develops – it feels great to know I’m not alone.

I really appreciate all of the kind words and honest feedback from the people that challenged me to think this through and go all in. If I can do nothing else, I’ll pay that advice forward. Cool?

A Response to Slate: How the recent article on technology misses the point.

Ah, summer. A great time to kick back, relax, and have time to write reactions to things that bug me.

I read through the article on Slate titled ‘Why Johnny Can’t Add Without a Calculator’ and found it to be a rehashing of a whole slew of arguments that drive me nuts about technology in education. It also does a pretty good job of glossing over a number of issues relative to learning math.

The problem isn’t that Johnny can’t add without a calculator. It’s that we sometimes focus too much about turning our brain into one.

This was the sub-heading underneath the title of the article:

Technology is doing to math education what industrial agriculture did to food: making it efficient, monotonous, and low-quality.

The author then describes some ancedotes describing technology use and implementation:

  • An experienced teacher forced to give up his preferred blackboard in favor of an interactive whiteboard, or IWB.
  • A teacher unable to demonstrate the merits of an IWB beyond showing a video and completing a demo of an electric circuit.
  • The author trying one piece of software and finding it would not accept an answer without sufficient accuracy.

I agree with the author’s implication that blindly throwing technology into the classroom is a bad idea. I’ve said many times that technology is only really useful for teaching when it is used in ways that enhance the classroom experience. Simply using technology for its own sake is a waste.

These statements are true about many tools though. The mere presence of one tool or another doesn’t make the difference – it is all about how the tool is used. A skilled teacher can make the most of any textbook – whether recently published or decades old – for the purposes of helping a student learn. Conversely, just having an interactive whiteboard in the classroom does not make students learn more. It is all about the teacher and how he or she uses the tools in the room. The author acknowledges this fact briefly at the end in arguing that the “shortfall in math and science education can be solved not by software or gadgets but by better teachers.” He also makes the point that there is no “technological substitute for a teacher who cares.” I don’t disagree with this point at all.

The most damaging statements in the article surround how the author’s misunderstanding of good mathematical education and learning through technology.

Statement 1: “Educational researchers often present a false dichotomy between fluency and conceptual reasoning. But as in basketball, where shooting foul shots helps you learn how to take a fancier shot, computational fluency is the path to conceptual understanding. There is no way around it.”

This statement gets to the heart of what the author views as learning math. I’ve argued in previous posts on how my own view of the relationship between conceptual understanding and learning algorithms has evolved. I won’t delve too much here on this issue since there are bigger fish to fry, but the idea that math is nothing more than learning procedures that will someday be used and understood does the whole subject a disservice. This is a piece of the criticism of Khan Academy, but I’ll leave the bulk of that argument to the experts.

I will say that I’m really tired of the sports skills analogy for arguing why drilling in math is important. I’m not saying drills aren’t useful, just that they are never the point. You go through drills in basketball not just to be able to do a fancier shot (as he says) but to be able to play and succeed in a game. This analogy also falls short in other subjects, a fact not usually brought up by those using this argument. You spend time learning grammar and analysis in English classes (drills), but eventually students are also asked to write essays (the game). Musicians practice scales and fingering (drills), but also get opportunities to play pieces of music and perform in front of audiences (the game).

The general view of learning procedures as the end goal in math class is probably the most destructive reason why people view math as something acceptable not to be good at. Learning math this way can be low-quality because it is “monotonous [and] efficient”, which is not technology’s fault.

One hundred percent of class time can’t be spent on computational fluency with the expectation that one hundred percent of understanding can come later. The two are intimately entwined, particularly in the best math classrooms with the best teachers.

Statement 2: “Despite the lack of empirical evidence, the National Council of Teachers of Mathematics takes the beneficial effects of technology as dogma.”

If you visit the link the author includes in his article, you will see that what NCTM actually says is this:

“Calculators and other technological tools, such as computer algebra systems, interactive geometry software, applets, spreadsheets, and interactive presentation devices, are vital components of a high-quality mathematics education.”

…and then this:

“The use of technology cannot replace conceptual understanding, computational fluency, or problem-solving skills.”

In short, the National Council for Teachers of Mathematics wants both understanding and computational fluency. It really isn’t one or the other, as the author suggests.

The author’s view of what “technology” entails in the classroom seems to be the mere presence of an interactive whiteboard, new textbooks, calculators in the classroom, and software that teaches mathematical procedures. This is not what the NCTM intends the use of technology to be. Instead the use of technology allows exploration of concepts in ways that cannot be done using just a blackboard and chalk, or pencil and paper. The “and other technological tools next to calculators in the quote has become much more significant over the past five years, as Geometers Sketchpad, Geogebra, Wolfram Alpha, and Desmos have become available.

Teachers must know how to use these tools for the nature of math class to change to one that emphasizes mathematical thinking over rote procedure. If they don’t, then math continues as it has been for many years: a set of procedures that students may understand and use some day in the future. This might be just fine for students that are planning to study math, science, or engineering high school. What about the rest of them? (They are the majority, by the way.)

Statement 3: “…the new Common Core standards for math…fall short. They fetishize “data analysis” without giving students a sufficient grounding to meaningfully analyze data. Though not as wishy-washy as they might have been, they are of a piece with the runaway adaption of technology: The new is given preference over the rigorous.”

If “sufficient grounding” here means students doing calculations done by hand, I completely disagree. Ask a student to add 20 numbers by hand to calculate an average, and you’ll know what I mean. If calculation is the point of a lesson, I’ll have students calculate. The point of data analysis is not computation. Just because the tools take the rigor out of calculation does not diminish the mathematical thinking involved.

Statement 4: “Computer technology, while great for many things, is just not much good for teaching, yet. Paradoxically, using technology can inhibit understanding how it works. If you learn how to multiply 37 by 41 using a calculator, you only understand the black box. You’ll never learn how to build a better calculator that way.”

For my high school students, I am not focused on students understanding how to multiply 37 by 41 by hand. I do expect them to be able to do it. Usually when my students do get it wrong, it is because they feel compelled to do it by hand because they are taught (in my view incorrectly) that doing so is somehow better, even when a calculator sits in front of them. As with Statement 3, I am not usually interested in students focusing on the details of computation when we are learning difference quotients and derivatives. This is where technology comes in.

I tweeted a request to the author to check out Conrad Wolfram’s TED Talk on using computers to teach math, and asked for a response. I still haven’t heard back. I think it would be really revealing for him to listen to Wolfram’s points about computation, the traditional arguments against computation, and the reasons why computers offer students new opportunities to explore concepts in ways they could not with mere pencil and paper. His statement that math is much more than computation has really changed the way I think about teaching my students math in my classroom.

Statement 5: “Technology is bad at dealing with poorly structured concepts. One question leads to another leads to another, and the rigid structure of computer software has no way of dealing with this. Software is especially bad for smart kids, who are held back by its inflexibility.”

Looking at computers used purely as rote instruction tools, I completely agree. That is a fairly narrow view of what learning mathematics can be about.

In reality, technology tools are perfectly suited for exploring poorly structured concepts because they let a student explore the patterns of the big picture. The situation in which “one question leads to another” is exactly what we want students to feel comfortable exploring in our classroom! Finally, software that is designed for this type of exploration is good for the smart students (who might quickly make connections between different graphical, algebraic, and numerical representations of functions, for example) and for the weaker students that might need different approaches to a topic to engage with a concept.

The truly inflexible applications of technology are, sadly, the ones that are also associated with easily measured outcomes. If technology is only used to pass lectures and exercises to students so they can perform well on standardized tests, it will be “efficient, monotonous, and low quality” as the author states at the beginning.

The hope that throwing calculators or computers in the classroom will “fix” problems of engagement and achievement without the right people in the room to use those tools is a false one, as the author suggests. The move to portray mathematics as more than a set of repetitive, monotonous processes, however, is a really good thing. We want schools to produce students that can think independently and analytically, and there are many ways that true mathematical thinking contributes to this sort of development. Technology enables students to do mathematical thinking even when their computation skills are not up to par. It offers a different way for students to explore mathematical ideas when these ideas don’t make sense presented on a static blackboard. In the end, this gets more students into the game.

This should be our goal. We shouldn’t going back to the most basic textbooks and rote teaching methods because it has always worked for the strongest math students. There must have been a form of mathematical Darwinism at work there – the students that went on historically were the ones that could manage the methods. This is why we must be wary of the argument often made that since a pedagogical method “worked for one person” that that method should be continued for all students. We should instead be making the most of resources that are available to reach as many students as possible and give them a rich experience that exposes them to the depth and variety associated with true mathematical thinking.

What my dad taught me about learning.

The first time I saw the word ‘Calculus’, I was staring at the spines of several textbooks that sat on the bookshelf at home. I didn’t think much of them; I knew they were my parents’, and that they were from their college days, but had no other awareness of what the topic actually was. I did assume that the reason there were so many of them was because my parents must have liked them so much. After further investigation, I learned that they were mostly my dad’s books. His secret was out: he must have loved Calculus. I believed this for a while.

When my older brother took Calculus, these books came off the shelf occasionally as a resource, though I don’t know if this was his decision or my dad’s. From what I knew, my brother breezed through Calculus. I know he worked hard, but it also seemed to come fairly naturally to him. I remember conversations that my parents had about not knowing where my brother got this talent from. They admitted at this point that it couldn’t have been from either of them. My dad had taken Calculus multiple times and the collection of textbooks was the evidence that hung around for no particularly good reason.

This astounded my young brain for a couple of reasons. It was mind-boggling to me that my parents ever had trouble doing anything. They always seemed to know just what to do in different situations – how could they not do well in a class designed to teach them something? It was also the first time I ever remember learning that my dad was not successful in everything he tried to do. This conflicted deeply with what I understood his capabilities to be.

As I understood it, he just knew everything.

When I was nine and my parents had bought me a keyboard to learn to play piano for the first time, there was no AC adapter in the box I had unwrapped only moments before. My dad scrounged around among his junk boxes and drawers and found one with the correct tip, but the polarity was wrong. I knew I wasn’t going to be able to start jamming that night – it was late and a trip to the store wasn’t an option. He wasn’t going to submit to that as a possibility – he took the adapter downstairs to the basement and had me follow him. There was soldering involved, and electrical tape. I had no idea what he was doing. Moments later, however, he appeared with the same adapter and a white label that said ‘modified’. We plugged it in to the keyboard and it lit up, ready for me to play and drive my parents crazy with my rendition of . I now understand that he switched the wires around to change the polarity – I did it myself with some students recently in robotics. At the time though, it seemed like magic. I just knew I had the smartest dad in the world.

His mantra has always been that if it can be fixed, it should be fixed, no matter the hilarity of the process. I watched him countless times take in the cast-off computers of other people who asked him if he knew how to fix them. Thinking back, I don’t know that he ever specifically answered that question. His usual response was (and still is) “I’ll take a look.” So he would work long hours with a vacuum, various metal tools, and a gray multimeter (that I think he still has) laid out like a surgeon investigating a patient. I rarely had the patience to sit and watch. I would see the results of his work: sheets of yellow legal pad paper filled with notes and diagrams scrawled along the way. In the end, he would inevitably find a solution, though often at this point the person who had asked him to fix the item had gone and bought a new one. I don’t recall ever believing my dad thought it was a waste.

We also worked on things together to try to get closer in my early teens. We both took tests to get amateur radio licenses. I came to really enjoy learning Morse code and got the preparation books to climb the license ladder. He commented repeatedly as I zipped through the books about memorizing the books and not understanding the underlying theory of resonant circuits and antenna diagrams. That was true – at the time I just wanted to pass the tests. I didn’t understand that the process of learning was the valuable part, not the end point. I didn’t see that. I just continued to believe that the tests were a means to an end, just as I viewed through my thirteen year old brain that his herculean efforts to fix things was a means to getting things fixed., and nothing more.

My dad is one of the smartest people I know. As I’ve grown older, however, I have come to understand that it wasn’t that about knowing everything. He instead had been continuously demonstrating what real learning is supposed to be. It was never about knowing the answer; it was about finding it. It wasn’t about fixing a computer, it was about enjoying figuring out how it can be fixed, however much frustration was involved. It wasn’t just about saving money or avoiding a trip to the store to buy an electric adapter. It was about seeing that we can understand the tools we use on a regular basis well enough to make them work for us.

I have seen time and time again how he mentors people to make them better at what they do. I have seen it in the way he mentors FIRST robotics teams as a robot inspector at the Great Lakes regional competition in Cleveland. I have seen it in the way he has spent his time since selling the company he founded with partners years ago. He chooses to do work that matters and makes sure that others are right there to learn beside him. There were times growing up when, admittedly, I just wanted him to fix things that needed to be fixed. To his credit, he insisted on involving me in the process, even when I protested or became impatient.. I didn’t see it when I was younger. Knowing how to go about solving problems is among the most important skills that everyone needs. I was getting free lessons from someone that not only was really good at it, but cared enough about me to want me to learn the joy of figuring things out.

One of my students this year was really into electronic circuits and microcontrollers. He soldered 120 LEDs into a display and wanted to use an Arduino to program it to scroll text across it. The student’s program wasn’t working and he didn’t know why. I had only been tangentially paying attention to the issues he was having, and when he was visibly frustrated, I pulled up a chair and sat next to him, and then said ‘let’s take a look.” We went through lines of code and found some missing semicolons and incorrectly indexed arrays, and I asked him to tell me what each line did. I was only a couple steps ahead of him in identifying the problem, but we laughed and tried making changes while speaking out loud what we thought the results would be. At one point, he said to me “Mr. Weinberg, you’re so smart. You just know what to do to fix the program.”

I immediately corrected him. I didn’t know what was wrong. We were able to make progress by talking to each other and experimenting. It wasn’t about knowing just what to do. It was about figuring out what to try next and having strategies to analyze what was and was not working. I learned this from a master.

On this Father’s day (that also happens to be the day before my dad’s birthday), I celebrate this truth: much of what I do as a teacher comes from trying to channel my dad’s habits while confronting big challenges. I don’t want my students to memorize steps to pass tests; I want them to understand well enough to be able to solve any challenge set before them. I don’t want to fix my students’ problems – I want to help them learn to fix problems themselves. I don’t want my students to be afraid to fail; I want them to understand through example that failure leads to finding a better way.

I am grateful for all that I have learned from him., and I try to teach my students what he has taught me about learning at every opportunity. It would be fine by me if I ever need to do Calculus for him – I’d still be in the red.

Results of a unit long experiment in SBG and flipping.

I’ve been a believer in the concept of standards based instruction for a while. The idea made a lot of sense when I first learned about the idea when Grant Wiggins visited my school in the Bronx a few years ago to present on Understanding by Design. Dan Meyer explored the idea quite a bit using his term of the concept checklist. Shawn Cornally talks on his blog about really pushing the idea to give students the freedom to demonstrate their learning in a way they choose, though he ultimately retains judgment power on whether they have or not. Countless others have been really generous in sharing their standards and their ideas for making standards work for their students. Take a look at my blogroll for more people to read about. For those unaware, here’s the basic idea: Look at the entire unit and identify the specific skills or you want your students to have. Plan your unit to help them develop those skills. Assess and give students feedback on those skills as often as possible until they get it. In standards based grading (SBG), reporting a grade (as most of us are required to do) as a fraction of standards completed or acquired becomes a direct reflection of how much students have learned. Compare this to the more traditional version of grading that consists of an average of various ‘snapshots’ on assignments, on which grades might be as much a reflection of effort or completion as of actual learning. If learning is to be the focus of what we do in the classroom, then SBG is a natural way of connecting that learning to the grades and feedback we give to students. My model for several years now has been, well,  SBG lite. Quizzes are 15% of the total grade and test only a couple skills at a time. Students can retake quizzes as many times as they want to show that they have the skills in isolation. On tests, (60% of the total grade) students can show that they can correctly apply the set of all of their acquired skills on exercises (questions they have seen before) as well as problems (new questions that test conceptual understanding). As much as I tell students they can all have a grade of 100% for quizzes and remind those that don’t to retake, it doesn’t happen. I’ll get a retake here or there. I am still reporting quiz grades as an average of a pool of “points” though, and this might leave enough haziness in the meaning of the grade for a student to be OK with a 60%. For this unit in Geometry and Algebra 2, I have specifically made the quiz grade a set of standards to be met. The point total is roughly the same as in previous units. It is a binary system – students either have the standard (3/3) or they don’t (0/3), and they need to assess each standard at least twice to convince me they have it. I really like Blue Harvest, but my students didn’t respond so well to having twowhole websites to use to check progress. While a truly scientific study would have changed only one variable at a time, I also found that structuring the skill standards this way required me to change the way class itself was structured. This became an experiment not only in reporting grades, but in giving my students the power to work on things in their own way. This also freed me up to spend my time in class assessing, giving feedback, and assessing again. More on this ahead. The details:

Geometry

I started the unit by defining the seven skills I wanted the students to have by the end on this page. The unit was on transformational geometry, so a lot of the skills were pretty straight forward applications of different types of transformations to points, line segments, and polygons. I had digital copies of all of the materials I put together last year for this unit, so I was able to post all of that material on the wiki for students to work through on their own. I adjusted these materials as we moved through the unit and as I saw there were holes in their understanding. I was also able to make some videos using Jing and Geogebra to explain some concepts related to using vocabulary and symmetry, and these seemed to help some students that needed a bit of direct instruction in addition to what I provided to them one on one. I also tried another experiment – programming assignments related to applying transformations to various points. I said completing these assignments and chatting with me about them would qualify them for proficiency on a given standard. Assigning homework was simple: Choose a standard or two, and do some of the suggested problems related to those standards. Be prepared to show me your evidence of study when you come into class. Students that said ‘I read my notes’ or ‘I looked it over’ were heckled privately – the emphasis was on actively working to understand concepts. Some students did flail a bit with the new freedom, so I made suggestions for which standards students should spend a particular day working on, and this helped these students to focus. I threw together some concept quizzes for the standards covered by the previous classes, and students could choose to work on those question types they felt they had mastered. Some handed the quiz right back knowing they weren’t ready. I was really pleased with the level of awareness they quickly developed around what they did and didn’t understand. I quickly ran into the logistical nightmare of managing the paperwork and recording assessment results. Powerschool Blue Harvest, whatever – this was the most challenging aspect of doing things this way. I often found myself bogged down during the class period recording these things, which got in the way of spending quality face time with students around their understanding. Part of this was that I was recording progress for each standard, whether good or bad, in the comment field for each student. “Understands basic idea of translation, but is confusing the image and pre-image” is the sort of comment I started writing in the beginning. While this was nice, and I think could have led to students reading the comments and getting ideas for what they needed to work on, it was a bit redundant since I was having actual conversations with students about these facts. Here is where Blue Harvest shines – I can easily send students a quick message explaining (and showing) what they need to work on. Even more powerful would be recording the conversation when I actually talk to the student, but that would be more practical with an iPad/cell phone app to avoid lugging my computer from desk to desk. Still, I wanted the feedback to be immediate and be recorded, so I knew I had to change my approach. The compromise was to only record positive progress. If a student’s quiz showed no progress, it didn’t get a comment in Powerschool. If they showed progress, but needed to fix a small detail in their understanding, they might get a comment. If they clearly got it, they got a comment saying that they aced it. Two or more positive comments (and my independent review) led to a 3/3 for each standard. The other promise I made was that if they clearly demonstrated proficiency on the exam (which had non-standard questions and some things they needed to explain) I would give them credit for the standard. The other difficult issue was creating a bank of reassessment questions. My system of making a quiz on the spot and handing it out to individual students was too time consuming. I created an app(using my new Udacity knowledge) to try to do this, the centerpiece being a randomized set of questions that emphasized knowing how to figure out the answers rather than students potentially sharing all the answers. They quickly found all the bugs in my system, and showed that it is far from ready for being an actual useful tool for this purpose. I appreciated their humor and patience in being guinea pigs for an idea. As you might notice from the image above, there is a pretty strong relationship between the standards mastered and the exam scores. Most student exam scores were either the same or better following this system in comparison to previous exams. The most important metric is the fact that most students weren’t hurt by going to this more student-centered model. Some student took more notes while working to understand the material than they have all year. Other students spoke more to their classmates and both gave and received more help in comparison to when I was at the front of the room asking questions and doing mini-lessons. While there was a lot of staring at screens during this unit, there was also a lot of really great discussion. I would have focused conversations with every single student three to four times a class, and they were directly connected to the level of understanding they had developed. Some needed direct application questions. Others could handle deeper synthesis and ‘why is this true’ questions about more abstract concepts. It felt really great doing things this way. I have always insisted on crafting one good solid presentation to give the class – the perfect lesson – with good questions posed to the class and discussions inevitably resulting from them. I have to admit that having several smaller, unplanned, but ‘messier’ conversations to guide student learning have nurtured this group to be more independent and self driven than I expected before we started.

Algebra 2

The unit focused on the students’ first exposure to logarithmic and exponential functions. The situation in Algebra 2 was very similar to Geometry, with one key difference. The main difference of this class compared to Geometry is that almost all of the direct instruction was outsourced to video. I decided to follow the Udacity approach of several small videos (<3 min), because that meant there was opportunity (and the expectation) that only two minutes would go by before students would be expected to do something. I like this much better because it fit my own preferences in learning material with the Udacity courses. I had 2 minutes to watch a video about hash functions in Python while brushing my teeth – my students should have that ability too. I wasn’t going for the traditional flipped class model here. My motivation was less about requiring students to watch videos for homework, and more about students choosing how they wanted to go through the material. Some students wanted me to do a standard lesson, so I did a quick demonstration of problems for these students. Others were perfectly content (and successful) watching the video in class and then working on problems. Some really great consequences of doing things this way:

  • Students who said they watched all my videos and ‘got it’ after three, two minute videos, had plenty of time in the period to prove it to me. Usually they didn’t.. This led to some great conversations about active learning. Can you predict the next step in the video when you try solving the problem on your own? What? You didn’t try solving it on your own? <SMIRK>  The other nice thing about this is that it’s a reinvestment of two minutes suggesting that they try again with the video, rather than a ten or fifteen minute lesson from Khan Academy.
  • I’ve never heard such spirited conversation between students about logarithms before. The process of learning each skill became a social event – they each watched the video together, rewound or paused as needed, and then got into arguments while trying to solve similar problems from the day’s handout. Often this would get in the way during teacher-centered lessons, and might be classified incorrectly as ‘disruption’ rather than the productive refining and conveyance of ideas that should be expected as part of real learning.
  • Having clear standards for what the students needed to be able to do, and making clear what tools were available to help them learn those specific standards, led to a flurry of students demanding to show me that they were proficient. That was pretty cool, and is what I was trying to do with my quiz system for years, but failed because there was just too much in the way.
  • Class time became split between working on the day’s standards, and then stopping at an arbitrary time to then look at other cool math concepts. We played around with some Python simulations in the beginning of the unit, looked at exponential models, and had other time to just play with some cool problems and ideas so that the students might someday see that thinking mathematically is not just followinga list of procedures, it’s a way of seeing the world.

I initially did things this way because a student needed to go back to the US to take care of visa issues, and I wanted to make sure the student didn’t fall behind. I also hate saying ‘work on these sections of the textbook’ because textbooks are heavy, and usually blow it pretty big. I’m pretty glad I took this opportunity to give it a try. I haven’t finished grading their unit exams (mostly because they took it today) but I will update with how they do if it is surprising.

Warning: some philosophizing ahead. Don’t say I didn’t warn you. I like experimenting with the way my classroom is structured. I especially like the standards based philosophy because it is the closest I’ve been able to get to recreating my Montessori classroom growing up in a more traditional school. I was given guidelines for what I was supposed to learn, plenty of materials to use, and a supportive guide on the side to help me when I got stuck. I have seen a lot of this process happening with my own students – getting stuck on concepts, and then getting unstuck through conversation with classmates and with me. The best part for me has been seeing my students realize that they can do this on their own, that they don’t always need me to tell them exactly what to do at all times. If they don’t understand an idea, they are learning where to look, and it’s not always at me. I get to push them to be better at what they already know how to do rather than being the source of what they know. It’s the state I’ve been striving to reach as a teacher all along, and though I am not there yet, I am closer than I’ve ever been before. It’s a cliche in the teaching world that a teacher has done his or her job when the students don’t need you to help them learn anymore. This is a start, but it also is a closed-minded view of teaching as mere conveyance of knowledge. I am still just teaching students to learn different procedures and concepts. The next step is to not only show students they can learn mathematical concepts, but that they can also make the big picture connections and observe patterns for themselves. I think both sides are important. If students see my classroom as a lab in which to explore and learn interesting ideas, and my presence and experience as a guide to the tools they need to explore those ideas, then my classroom is working as designed. The first step for me was believing the students ultimately wantneed to know how to learn on their own. Getting frustrated that students won’t answer a question posed to the entire class, but then will gladly help each other and have genuine conversations when that question comes naturally from the material. All the content I teach is out there on the internet, ready to be found/read/watched as needed. There’s a lot of stuff out there, but students need to learn how to make sense of what they find. This comes from being forced to confront the messiness head on, to admit that there is a non-linear path to knowledge and understanding. School teaches students that there is a prescribed order to this content, and that learning needs to happen within its walls to be ‘qualified’ learning. The social aspect of learning is the truly unique part of the structure of school as it currently exists. It is the part that we need to really work to maintain as content becomes digital and schools get more wired and connected. We need to give students a chance to learn things on their own in an environment where they feel safe to iterate until they understand. That requires us as teachers to try new things and experiment. It won’t go well the first time. I’ve admitted this to my students repeatedly throughout the past weeks of trying these things with my classes, and they (being teenagers) are generous with honest criticism about whether something is working or not. They get why I made these changes. By showing that iteration, reflection, and hard work are part of our own process of being successful, they just might believe us when we tell them it should be part of theirs.

1 … 4 5 6 7 8 9