Building (Ev)anAcademy Exercises for Reassessments

I’ve written multiple times previously about WeinbergCloud, the system I created using the Meteor framework that lets students sign up for reassessments. Over the course of the semester, I’ve been developing some aspects of the system that I’m excited about, and I’ll talk about them all eventually. One in particular has held my focus for the past few days, and it’s probably the one feature I’ve been talking about for the longest: building a reassessment engine within the system.

Part of this is out of necessity. The wireless network settings on our school network have changed, so the Python reassessment engine that I’ve used for reassessments over the last two years no longer works when hosted on my personal laptop. I’ve managed reasonably well this year using problems from textbooks and handouts, but it became time to automate this using my new knowledge of Javascript and Meteor.

Whatever your feelings about Khan Academy, the reality is that the organization has put a lot of energy and resources into developing a pretty comprehensive web application built around assessment. Not only are these resources available for free for teachers and students to use, but the source code is as well. The code for anyone to be able to run their own local version of exercises has been around for a while at Github here. The Javascript libraries that go with these exercises are also pretty impressive and capable – generating random numbers with constraints, simplifying fractions and expressions, and numerous other helper functions are already written by people that code much better than I do. They also wrote a math-typesetting library called KaTEX that has some performance advantages over some other libraries…or so I hear. I’m sure much of the ‘why’ here is lost on me.

After two days of tinkering, I’ve adapted some of their code in my app for the purposes of generating questions for reassessments. Writing questions and defining variables is all done in HTML, just as in their own local application, which means it’s possible to add questions without having to load in a database through FTP or some other method. The code rendering the questions onto the webpage I had to write myself, but eventually I determined some ways to make this work for me.

I can put in HTML and Javascript definitions into text areas. Here’s an example of a question asking for a simplified fraction for slope:
Screen Shot 2015-10-02 at 9.14.46 PM

A preview appears below to make sure the question appears the way I want it. In the variable definitions are strings of Javascript code that calculate and define the variables using the Khan Academy utility functions. The question text is then rendered using KaTEX. The random values change on each reload of the page, but these values could potentially be fixed for an individual student’s quiz.

As I create frames for questions, I get a virtually infinite supply of questions I can use for assessing students on learning. Here are a bunch that I put together for testing the interface:

Screen Shot 2015-10-02 at 9.35.43 PM

The next step is to link each question frame to a course standard and build my database of questions. I’m loving the possibilities for building on this further, and will share as they develop. Stay tuned.

Editing Khan

Let’s be clear – I don’t have a problem with most of the content on Khan Academy. Yes, there are mistakes. Yes, there are pedagogical choices that many educators don’t like. I don’t like how it has been sold as the solution to the educational ills of our world, but that isn’t my biggest objection to it.

I sat and watched his series on currency trading not too long ago. Given that his analogies and explanations are correct (which some colleagues have confirmed they are) he does a pretty good job of explaining the concepts in a way that I could understand. I guess that’s the thing that he is known for. I don’t have a problem with this – it’s always good to have good explainers out there.

The biggest issue I have with his videos is that they need an editor.

He repeats himself a lot. He will start explaining something, realize that he needs to back up, and then finishes a sentence that hadn’t really started. He will say something important and then slowly repeat it as he writes each word on the screen.

This is more than just an annoyance. Here’s why:

  • One of the major advantages to using video is that it can be good instruction distilled into great instruction. You can plan ahead with the examples you want to use. You can figure out how to say exactly what you need to say and nothing more, and either practice until you get it right, or just edit out the bad takes.
  • I have written and read definitions word by word on the board during direct instruction in my classes. I have watched my students faces as I do it. It’s clearly excruciating. Seeing that has forced me to resist the urge to speak as I write during class, and instead write the entire thing out before reading it. Even that doesn’t feel right as part of a solid presentation because I hate being read to, and so do my students. This doesn’t need to happen in videos.
  • If the goal of moving direct instruction to videos is to be as efficient as possible and minimize the time students spend sitting and watching rather than interacting with the content, the videos should be as short and efficient as possible. I’m not saying they should be void of personality or emotion. Khan’s conversational style is one of the high points of his material. I’m just saying that the ‘less is more’ principle applies here.

I spent an hour this morning editing one of the videos I watched on currency exchange to show what I mean. The initial length of the video was 12:03, and taking out the parts I mentioned earlier reduced it to 8:15. I think the result respects Khan’s presentation, but makes it a bit tighter and focused on what he is saying. Check it out:

The main reason I haven’t made more videos for my own classes (much to the dismay of my students, who really like them) is my insistence that the videos be efficient and short. I don’t want ten minute videos for my students to watch. I want two minutes of watching, and then two or three minutes of answering questions, discussing with other students, or applying the skills that they learned. My ratio is still about five minutes of editing time for every minute of the final video I make – this is roughly what it took this morning on the Khan Academy video too. This is too long of a process, but it’s a detail on using video that I care too much about to overlook.

What do you think?

Can ideas and a little money be a bad thing?

I was having a conversation with someone recently about technology in education. I brought up Udacity as an interesting model for using video and interactivity for learning. My example was (expectedly) countered with Khan Academy. I shared my opinions about its strengths and weaknesses and we had a really great discussion about what its existence means. I felt good about being able to share some of the ‘other side’ of the argument that Time and CBS haven’t really covered.

One point that was brought up has stuck with me, and I want to explore it a bit. Here’s the basic idea with my own paraphrasing.

Here’s a smart guy with an idea. He sees a problem and wants to help, so he puts his own time and resources into solving that problem. Other people noticed what he was doing and thought it was a good idea, so they put money into his project. What could it hurt?

What could it hurt?

My focus has nothing to do with the fact that many people have benefited from Khan’s resources and his website. Many teachers have used the site as a tool to help their students in skill development. I also don’t want to focus on the fact that many learning professionals have questioned the pedagogy of Khan’s videos given the fact he has no teaching background. Many others have already fleshed out this line of reasoning pretty effectively.

The big problem I see comes from applying how business investment works – a business starting up needs investors so it can start getting what it needs to generate revenue. If the business is actually fulfilling a real need in the market, it will increase in value through the income earned and the equipment and intellectual property generated or acquired by the business. Venture capitalists research companies and their ideas to see which ones have potential to be successful in the market and then select those that, based on their experience in the field, are most likely to succeed. Often these capitalists invest in a number of different ideas to maximize the potential that one will be a real money maker – they understand that not every investment will actually be a winner.

Here’s why I see the hubbub about Khan Academy as an indication of a bigger problem: We get things backward when we see a major investment as a measure of its value, whether in an idea or a business.

So much has been made out of the fact that Bill Gates and Google have invested in the Khan Academy that people might thing it’s a good idea specifically because Gates and Google have done so. Don’t get me wrong – Google has invested in many really great causes (FIRST being one of my favorites) but they don’t always get things right. As I said before, this is the nature of investment though. Not everything works out. I challenge anyone to defend the content of the video below as really, honestly, being truly deserving of a major investment to help it be implemented in schools:


This is the Explicit Direct Instruction initiative that Google recently supported in the Mountain View school district. The manager of community affairs says in the linked article that EDI “…seemed like a really successful program that we want to continue to support.” Google wants to help solve the complex problems of the educational problem, and based on the manager’s assessment, it will continue to do so.

Why might this situation hurt rather than help?

The news story was all about Google’s generous donation in support of the initiative. A person reading this article might make the mental connection that if Google is supporting the idea, then the idea must be one that will effectively address an educational issue. Google’s donation (and subsequent coverage of the donation) have consequently turned it from a company that just wants to help, into a ‘player’ in the educational reform world. What was this originally based on? A manager that saw kids ‘engaged’ because they were compliant, which to her meant that learning was going on in the classroom. What does she know about education?

Many people think they are experts in education because they went to school and they know what worked for them. Salman Khan has been reported to be good at explaining concepts through solving problems – he has said himself that procedures are how he and everyone he knew learned. He makes videos that primarily show procedures, though there are some exceptions. Investors see this and contribute millions. The media picks up on this and says that because of these investments, school has been “rebooted” and education has been revolutionized by his contribution.

Money talks. If the money goes toward Khan, EDI, and other flavors of the week, a few things can happen. The media pounces and says that these ideas that have attracted investors must be what will revolutionize education. Not the genuine ways that many teachers have individually used technology to improve instruction in their classrooms. Not the ways teachers are able to have improved communication with parents and students about their progress. Administrators looking for quick solutions to achievement deficiencies in their districts might sink considerable resources into these ideas without consulting with the teachers responsible for implementing them. Parents can demand that teachers spend less time creating rich explorations and applications of topics in their classrooms in order to focus on this ‘innovative’ idea they learned about on TV or the internet. As I have said before, there is no silver bullet in education, not any one piece of technology, not a single pedagogical technique, nor a single textbook. Solutions to problems in a learning community must be influenced primarily by the parents, teachers, students, and administrators in that community, and not by what the news says is innovative because of a million dollar donation.

It is possible that the involved players can act with a bit more restraint. I know there are many administrators that do. We are so reactionary these days. We want a quick fix. The media’s tendency to hype, the power of the internet to exponentially transmit ideas, and the ability money has to set our priorities: these form a dangerous formula that could lead us to rapidly pursue options that really don’t resolve the issues we face. Hopefully we do not lose sight of what we really value in education. I don’t believe it is too late.

Giving badges that matter.

The social aspect of being in a classroom is what makes it such a unique learning environment. It isn’t just a place where students can practice and develop their skills, because they can do that outside of the classroom using a variety of resources. In the classroom, a student can struggle with a problem and then ask a neighbor. A student can get nudged in the right direction by a peer or an adult that cares about their progress and learning.

If students can learn everything we expect them to learn during class time by staring at a screen, then our expectations probably aren’t what they should be. Our classrooms should be places in which ideas are generated, evaluated, compared, and applied. I’m not saying that this environment shouldn’t be used to develop skills. I just mean that doing so all the time doesn’t make the most of the fact that our students are social most of the time they are not in our classrooms. Denying the power of that tendency is missing an opportunity to engage students where they are.

I am always looking for ways to justify why my class is better than a screen. Based on a lot of discussion out there about the pros and cons of Khan academy, I tried an experiment today with my geometry class to call upon the social nature of my students for the purposes of improving the learning and conversations going on in class. As I have mentioned before, it can be a struggle sometimes to get my geometry students  to interact with each other as a group during class, so I am doing some new things with them and am evaluating what works and what doesn’t.

The concept of badges as a meaningless token is often cited as a criticism of the Khan academy system. It may show progress in reaching a certain skill level, it might be meaningless. How might this concept be used in the context of a classroom filled with living, breathing students? Given that I want to place value on interactions between students that are focused on learning content, how might the concept be applied to a class?

I gave the students an assignment for homework at the end of the last class to choose five problems that tested a range of the ideas that we have explored during the unit. Most students (though not all) came to class with this assignment completed. Here was the idea:

  • Share your five problems with another student. Have that student complete your five problems. If that student completes the problems correctly  and to your satisfaction, give them your personal ‘badge’ on their paper. This badge can be your initials, a symbol, anything that is unique to you.
  • Collect as many people’s badges as you can. Try to have a meaningful conversation with each person whose problems you complete that is focused on the math content.
  • If someone gives a really good explanation for something you previously didn’t understand, you can give them your badge this way too.

It was really interesting to see how they responded. The most obvious change was the sudden increase in conversations in the room. No, they were not all on topic, but most of them were about the math. There were a lot of audible ‘aha’ moments. Some of the more shy students reached out to other students more than they normally do. Some students put themselves in the position of teaching others how to solve problems.

In chatting with a couple of the students after class, they seemed in agreement that it was a good way to spend a review day. It certainly was a lot less work for me than they usually are. Some did admit that there were some instances of just having a conversation and doing problems quickly to get a badge, but again, the vast majority were not this way. At least in the context of trying to increase the social interactions between students, it was a success. For the purpose of helping students learn math from each other, it was at least better than having everyone work in parallel and hope that students would help each other when they needed it.

It is clear that if you want to use social interactions to help drive learning in the classroom, the room, the lesson, and the activities must be deliberately designed to encourage this learning. It can happen by accident, and we can force students to do it, but to truly have it happen organically, the activity must have a social component that is not contrived and makes sense being there.

The Khan academy videos may work for helping students that aren’t learning content skills in the classroom. They may help dabblers that want to pick up a new skill or learn about a topic for the first time. Our students do have social time outside of class, and if learning from a screen is the way that a particular student can focus on learning content they are expected to learn, maybe that makes sense for learning that particular content. In a class of twenty to thirty other people, being social may be a more compelling choice to a student than learning to solve systems of equations is.

If we want to teach students to learn to work together, evaluate opinions and ideas, clearly communicate their thinking, then this needs to be how we spend our time in the classroom. There must be time given for students to apply and develop these skills. Using Khan Academy may raise test scores, but with social interaction not emphasized or integrated into its operation, it ultimately may result in student growth that is as valuable and fleeting as the test scores themselves. I think in the context of those that may call KA a revolution in education, we need to ask ourselves whether that resulting growth is worth the missed opportunity for real, meaningful learning.