You can't really reassess an individual skill

Depending on how you define Standards Based Grading, it gets a lot of things right. You're more likely to get an accurate picture of what someone knows by assessing a skill more than once. What you know now matters more than what you didn't in October. Students need an accurate picture of what they're studying, and "Test #4" doesn't provide that.

Great. But here's something about SBG that's been bugging me for a while.

There's something wrong here, but what is it? 

• The kid showed that she knows all the triangley stuff, but dropped the ball on the square root side of things. She gets a 5/5 on Finding Sides of Right Triangles, but gets a 2/5 on Understanding Square Roots.
• The kid got a question about right triangles wrong, so she gets a 3/5 on Finding Sides of Right Triangles.

Neither of these ideas is quite right. Knowing how to find the square root of 1 is not an all-or-nothing affair. Understanding isn't binary. Rather, understanding comes in degrees, and if a piece of knowledge is weakly understood then it's especially likely to falter when under pressure. 

If you aren't super-comfortable working with right triangles, trying to solve a right triangle problem will be mentally taxing, and when you engage in mentally taxing behavior, you mess things up. But you don't mess up the things that are rock-solid. I doubt that I'll mess up single-digit addition when working on a Calculus problem. Rather, when you're using up mental resources it's the infirm and tentative knowledge that falls apart.

It's the sort of thing that we see all the time on mathmistakes.org. 

This student said something silly, but it's artificial to attribute this to either his understanding of solving quadratic equations or his understanding of what the equation symbol means. It's both.

Would you ask this student to reassess on Doing Arithmetic with Negative Numbers or Finding Equations Given 2 Points? Neither? Both?

There's a larger point here. The idea that you can create a quality assessment that targets an individual skill is a myth. Take the slope question above. You could make the numbers easier so that the arithmetic probably wouldn't be a problem. For instance, you could use (0, 4) and (2, 10). But this is far too easy -- understanding means being able to apply a skill to a difficult context. So you toss in more difficult numbers, but then you're no longer purely assessing a kid's ability to find a line that passes through two points.

I don't know what this means for SBG or reassessing, and I hope that (in addition to challenging the premise of my post) we hash this out in the comments. Maybe this is an argument for fewer, but more substantive standards, like "Doing Stuff With Lines." I'm not sure, though.

Good Writing on Exponents

There's been a lot of good writing about exponents recently, some of it in response to my most recent post. Here's a sampling:

• Christopher Danielson wonders whether the problem with exponents is that we introduce it as repeated multiplication. He's thinking that "number of doublings" might be better, and he tests this theory on his kid. (The comments on the post are great also. Check them out.)
• Chris Robinson modified the exponents survey for his own students, collected a ton of responses and offered his own analysis of what kids are doing with exponents. I still have to dig into his students' survey responses carefully, but I think that my favorite snippet so far is this:

         Exponentiation sometimes defaults to multiplication, and multiplication sometimes defaults to addition.

• ... and some more survey results! Thanks Mrs Reilly! This effect is for realz, guys.
• Andrew Stadel wraps it up with some extremely solid lesson plans that put these mistakes right in front of the kids noses. His theory? Part of the way to change kids intuitions about this stuff is to draw out and directly challenge their previous ideas. I stole his stuff for class, and it went quite well. 

Exponents are cool. 

Why Kids Mess Up Exponents

1. Table of Contents

1. Table of Contents
2. Exponent Mistakes + Teacher Explanations
3. Let's. Try. Evidence.
4. Results
5. What's Going On Here?
6. Sequels

2. Exponent Mistakes + Teacher Explanations

Anyone who teaches exponents is familiar with a series of closely related mistakes:

Students like to treat exponentiation like multiplication. Why?

Thanks to mathmistakes.org, we have a nice collection of attempts to explain this phenomenon.

• "Most kids just try to cram in the fact that negative exponents do ummm, something to the base. Without some conceptual hangar to place this fat they are left wildly guessing. "

• "But really, as much as I try not to write these off to “autopilot”, I can’t think of any other explanation."

• "Often, if you can get a student to slow down and be more present in a problem, they can avoid mistakes they would be prone to make otherwise. We need ways to differentiate true misunderstandings from these sorts of automatic pilot errors."

• "I think that this idea is attractive because many students think that raising a number to a power is the same as multiplying the base by the power as in “4^2 = 8.″

We've got votes for autopilot, wildly guessing, and the idea that students are operating with an incorrect conceptual model for exponentiation.

How can we sort this out?

3.  Let's. Try. Evidence!

This survey was given to students in a first-year Algebra class. They're studying exponents, but have never seen negative powers before, and they've certainly never seen non-integer powers.

What would you expect these kids to answer? 

• If you think that kids are wildly guessing, then they ought to report a relatively low level of confidence in their answers. 
• If you think that kids are mistaken about what exponentiation means, then they shouldn't get the first question right. After all, if they know what exponents mean with one problem, shouldn't they know what they mean just seconds later?

4. Results

This stuff is pretty fascinating. Here's everything, and here are some quick observations:

• Overall, kids answered 50 to the third question, and had a good deal of confidence behind their answer.
• Answers to the second question were more varied, but nobody just multiplied the base and the power together, like they did for 3a (or even 4a!).
• Overall, kids had more confidence with rational exponents than with negative exponents.

(You might be wondering whether these observations are a fluke, which they might be, but they're at least a fluke twice. Here's a repeat of the experiment.)

5. What's Going On Here?

These kids are not guessing. Or they are, and they're lying, because they're telling you that they have confidence in their answers. So you can knock that theory out, it's not what's going on.

These kids are coming into your classroom with ideas about negative and rational exponents. So it's not about rules or memorizing or whatever, these kids have ideas about powers and are pretty confident about them.

These kids do sometimes treat exponentiation as multiplication, even when they're just dealing with plain old positive powers. To me, this supports the idea that when faced with a difficult exponentiation problem, sometimes the mind skips right to multiplication.

These kids are especially confident about rational exponents, where they seemed comfortable answering "50" to the third question. This could be because they've got a fuzzy story they're telling themselves about fractional powers, or it could be because of something more intuitive. Those are your options, and I'm not exactly sure what the difference is between them or how to test for this.

These kids don't just multiply the base and the exponent together when dealing with negative exponents. Instead, they tend to do treat the negative exponent as a positive one and then just tack on the negative to that result. (I suspect that this has something to do with the way we teach kids to do multiplication of negative numbers: do the multiplication, and just tag on the sign at the end.)

Your problem, as an Algebra teacher, is far more serious than autopilot. It's not just thoughtlessness that's responsible for these mistakes. It's a substantive intuition about what the answer to these things should be. It's such a strong intuition that it exists among students who have never even seen these concepts before, in a classroom or (presumably) otherwise.

6. Sequels

The survey file is here, but it's easy enough to make one of your own. I'd love to see what your students respond to math questions that they've never seen before. This all seems like a fruitful way to plumb the images that our kids bring into our classes.

I think the two live explanations for this sort of student work are that (a) kids have explicit, mistaken models about how exponents work and (b) there's a sort of intuition about what exponents should be, and this intuition operates below the level of consciousness.

I don't know how to tease those two possibilities apart, though all my experience in doing and teaching math leads me to think that it's all about intuition, and not really about explicit, mistaken models.

I was surprised that large numbers were sufficient to trigger multiplication of the base and the exponent in a lot of cases. I was also surprised that this didn't happen with negative powers. I'm not exactly sure what to make of that. I wonder what else triggers multiplication.

And does this sort of thing happen with other operations? Is this a general phenomenon or is something special about exponents?

NSF grant proposals in the comments, please. 

Technology lets you say mean things

This whole piece is really good. Here's the goodest bit:

This is why I get so apoplectic when people talk about MOOCs as disruptive technology. There is not a single thing this “New University of California” does that could not have been done technologically in 1898. Has online education suddenly improved to the point where people can gain never-before-seen levels of competency without attending classes? Hardly. Most MOOCs I’ve looked at are poorly designed even by late 90s standards, and besides, education’s killer technology — the book — has made independent learning possible for at least 500 years.

The real question to ask is why policy proposals like this — formerly the domain of fringe elements — are increasingly seen as innovation. What has changed? The answers to that are complex, and have little to do with technology. But understanding the reasons behind *that* is what is crucial to understanding where we are headed and why we are headed there.  I think that “authorized to contract with qualified entities” clause is a piece of it. But the story goes much, much deeper than that….

Here's a story about a teacher. His name is Jerry. Some people don't like the way that Jerry teaches, but they don't want to say so. There's a lot of reasons why they don't want to criticize Jerry. The kids like him, and so do the parents. He's a devoted teacher. He's very not-awful, and there are lots of people that teach like him. And Jerry isn't so into change. He's seen the trends come and go. He's not so into learning the new edu-jargon that is research-based with pie charts and things.

And then some new technology comes out. Jerry's principal gets excited. The people from Teen Einstein (c) have all these awesome ideas about how you can get kids more involved, and they're talking about students taking control and being engaged and personalized whatever, and Jerry's principal is saying Yeah, that's how I'd like Jerry to teach.

And what's Jerry going to say? "No, I don't want to learn how to use that tech." Nah, Jerry's just got to admit that there's something to learn here, that the technology is new to him and so there's something worth looking at. So Jerry's principal is a big fan of technology. He's predisposed to calling it "revolutionary" or "a real game-changer."

Technology solves another problem for Jerry's principal -- how do you tell people that you're improving without admitting that you've got room to improve? You can't just walk behind a podium and tell everyone that you're ending things like hour-long lectures. That's not just change, that's an indictment of your teachers, your district, and everyone else's school experiences.

But technology is (by definition!) new and unanticipated. It's a chance to change without any of the responsibility of inviting change.

---

There are Jerry's in other areas of education, people or institutions that folks are too polite to tear to shreds, and I think that's what's going on with the MOOCs.

Of course (some) people can learn (some) things on their own. Of course, by the time you get to college in a lot of places there isn't a lot of difference between the classroom and learning on your own. But this is not an attractive argument to make because taking this up means indicting the college experiences of everyone, along with the quality of America's college teachers, along with the institutions themselves.

That's not very nice.

But there's this radically new technology that could really change the game. With the internet there has been a revolution in information distribution, and it's changing the way people learn. Something something youtube. Something something social. Something something personalization.

So, don't worry -- nobody's getting hurt. This change isn't about pedagogy, it's not about "You don't teach well" or "What does it mean to learn something anyway?" It's about technology.

Until we figure out a way to convince pretty much everyone that there's good teaching and there's bad teaching and that you can tell the difference between them just by watching carefully, we're going to need -- as a society -- technology to give us a chance to say what we're really thinking.