Sunday, May 18, 2014

Why I Find Standards-Based Grading Fascinating

People ask me all the time, "Hey Michael, why do you spend so much time harping on Standards-Based Grading?"*

* No one asks me anything of the sort, we are all insignificant specks of sand in this grand sandbox that is the internet and even shouting doesn't make us heard as much as fulfill our need to feel as if we're heard and therefore of some consequence to the universe.

The short explanation for why I find SBG interesting is that I have strange and somewhat confusing interests.

The longer version goes like this: I was trained to teach on blogs. I cobbled together enough experience working with kids in camps and summer programs to convince someone to hire me as a math teacher, but I surely had no clue how to teach so I responded by doing the same thing you do when you can't remember how to get to the airport: I googled it.  Quickly I landed on Dan and Sam and Kate's blogs and I clicked and clicked and read as much as I could.

Standard-Based Grading was abuzz, and I brought it into my classroom from Day One. That means that I have a different relationship to it then a lot of other people who transferred onto the SBG-bandwagon. I tend to associate SBG with some of the decisions that I made in my first few years teaching, and I tend to be very critical of my first few years teaching. ("You're in your 4th year, michael, it's still your first few years teaching." OK.)

So much for psychology. But lately I've been reading a bunch about the history of education reform, and I'm fascinated by SBG as an instance of classroom innovation. Why do teachers like SBG? Why do districts and administrators like it? What problems does SBG solve? We are smarter, but not much smarter than teachers in the past. So why is this innovation emerging now?

I have no firm answers. But things I've learned? I have things that I've learned.

First, I know that teachers justify SBG in a variety of ways, some of which are actually incompatible with each other. I did some research and collected a bunch of justifications that bloggers and tweeters use for SBG, all collected in this doc. Here's an especially interesting contrast that I found:
  • SBG Provides Incentives for Students To Learn More, Through Remediation and Reassessment
  • SBG Helps Kids Focus On Learning Instead of on Points
If SBG is helping kids to ignore points, then how is it also using points to motivate learning? Either points are being ignored or points are being leaned on heavily.

The answer, of course, is that different teachers have different reasons for using SBG, to the extent that some people have flatly incompatible motivations for SBG. I recognize this pattern from some of my historical reading. It makes sense, right? An innovation needs a broad base of support to be popular, and if it can draw the alliances of educators with widely varying needs and philosophies then it's more likely to stick. (Of course, it also becomes open to the critique that it stands for very little, but it seems to me that these are competing tendencies in many reform movements.)

By the way I'm obviously not a historian, just some fool who's read a few books. I'm sharing what I'm thinking, not what I know.

Here's the other thing about SBG: it seems to be a classroom innovation very much of our times. We live in the age of Standards-Based Reform efforts, a time when high-stakes testing has made a close correlation to standards a vital concern to all teachers of public schools. I wonder: can this partially explain the broad appeal of SBG? Can it really be a coincidence that SBG emerges in the years of No Child Left Behind?

The story, perhaps, goes like this: in a time when standards-based reforms became a living reality for teachers, a grading system that could more closely align classroom assessments to state assessments starts seeming natural. Because there's widespread dissatisfaction about grading, teachers then use this opportunity to graft on their particular concerns to the innovation in grading. Some teachers wish that there was no grading. Others want grades to be more carefully aligned with incentives. Still others just want to be able to ease communication with parents.

SBG then starts defining itself against "traditional" grading, making it hard to criticize SBG (since everyone has some problem with traditional grading). Since SBG is such a disparate camp, it becomes hard to keep track of what SBG stands for and this aids in its rise, but ends up meaning very different things to very different people, following the same path that "projects" and "problems" and "occupations" and so many other instructional innovations have tread.

So much for amateur history. The last reason why I find SBG so interesting is because I find conflicts between what I'm learning about effective assessment and some especially popular interpretations of SBG. Finding the right tone and angle to criticize a popular ideology is an interesting rhetorical exercise, and one that I think is important because, in my (current) opinion, some of our best have gotten this one wrong in an important way.

Dan is our very best.

Take some of the justifications for SBG that I collected in my doc, put them under the microscope and they end up squirming a bit.
  • SBG Helps Kids Focus On Learning Instead of on Points: How? By assigning them points for everything that they know? 
  • SBG Provides Incentives for Students To Learn More, Through Remediation and Reassessment: One thing that we know is that extrinsic incentives for good behavior often backfire in remarkable ways. (Click that link to read about how financial incentives for parents picking their kids up from day care on time lead to parents showing up later.) Grades should not be used as an incentive. 
  • SBG Assesses A Student On What They Currently Know, As Opposed To What They Knew In The Past: This seems flatly impossible. Grades are static, knowledge is dynamic. In any event, are we really claiming to be able to represent what a student knows in a series of skills and numbers? And what do we care about accurately measuring a student's learning, anyway? Will this help the kid's learning? How?
  • SBG Provides Superior Feedback to Students: Superior feedback comes without a grade, so Standards-Based Grading would seem to be a poor candidate for providing better feedback. Maybe SBG gives better feedback than "traditional" grading, but since we know that kids ignore all feedback that comes in anything as easily fixated-on as a number I can't see much advantage to providing many numbers instead of one number.
  • SBG is an Appropriate Response to Standards-Based Instruction: We also know that a major problem in math instruction is that it comes across as piecemeal and disconnected. Good math instruction is deeply connected, and I can't see how either Standards-Based Instruction or an approach to assessment that atomizes mathematical knowledge can create the deep web of learning whose creation we're trying to foster.
The sheer variety of things that SBG can stand for makes any sort of criticism very very difficult. SBG can become deeply entrenched. But perhaps precisely because it can be safely defended, SBG becomes an excellent forum for hashing out some crucial disagreements concerning assessment, in general. Criticizing any particular plank of SBG is non-threatening, since defenders have many different planks upon which to shift. It's in this context that we can maybe make some progress, getting people with very different uses for SBG to agree that, yes, students learn more from feedback that is completely dissociated from grades. After all, criticism of using SBG for student feedback doesn't mean the end of SBG and another radical change in grading practice. Maybe this is how we make progress through educational reform, even if not through the educational reforms themselves.

It seems to me that SBG is an innovation with a broad base of support, a loose and shifting definition, and a strong connection to tendencies in the wider educational reform movement. There are a network of justifications for it, and this network provides it with a strong defense as well as diffused results when implemented widely. At the same time, I'm betting that the popularity of SBG gives us a safe context in which to debate the essentials of assessment, and perhaps to change a few hearts and minds on some important matters.

Update: Confused by this piece? You weren't the only one. Here's a clearer restatement of the crucial bits.

22 comments:

  1. I'm listening. As you may know, I don't like called what I do SBG, maybe learning-based grading would be better for me. That does nothing to alleviate the concerns you bring up. I don't have answers to them, but I know my students are working harder. Could be because they see my concern better.

    ReplyDelete
    Replies
    1. Interesting! I wonder what aspect of SBG is leading to them working harder.

      Delete
  2. This is an awesome post. You are becoming an educational researcher and you don't even know it. And, you are learning the #1 lesson: things in education are entirely dependent on situation and implementation which makes 'best practice' a sticky notion. Yashar koach.

    ReplyDelete
    Replies
    1. That's definitely something that is coming through in my reading. The sheer number of things that "student interest" or "project" can mean is impressive.

      Anyway, so glad that you liked the post! (Baruch tehiyeh!)

      Delete
  3. I think you may be missing the forest for the trees, or something. For example: on points vs. learning. Yeah, points are there to stay. SBG teachers didn't invent that. Most that I know would like to get rid of grades entirely, but that's not happening any time soon. In the mean time, many of us practicing SBG are listening to the conversations in our classrooms and they are less about grades and more about English, Math, Science, whatever, because the kids figure out that their grade depends on what they learn, so they get down to the business of learning instead of getting down to the business of getting a grade without the effort of having to learn, which many of us saw under traditional grading.
    I don't think many of us who are proponents of SBG are saying, "Great, we get to use standards to give a grade." as much as, "If we have to give a grade, it's good that we get to use standards mastery as the criteria."

    ReplyDelete
    Replies
    1. Thanks for the thoughts, Dave! I hear what you're saying about SBG being (at best) an imperfect solution to an unsolvable problem.

      Maybe SBG is the best of all the lousy solutions of how to handle grading. Still, there are important questions about motivation, feedback and assessments that we can discuss.

      Would you agree that kids learn more from ungraded assessments?

      Do you agree that feedback is better when it comes without a grade, and that we should be trying to give kids lots and lots of opportunities for that sort of feedback?

      Do you think that kids will only do something if there is a grade attached to it?

      Delete
  4. I haven't engaged too much with your interesting series of posts and tweets because ...

    a) ... when I talk about SBG I say it's the worst form of assessment except all the other ones. (Apologies to Churchill.) I am locked in a series of uneasy truces with SBG and I'm not going to go to the mattresses over it.

    b) ... you seem to be overgeneralizing research and setting up some false dichotomies that I don't know if I can easily disentangle. Just two examples:

    (b.1) I believe you're overleveraging Black & Willem for the sake of this "SBG has numbers" vs. "we know numbers are bad for formative assessment" distinction. The numerical scores in SBG are appreciably different from writing "62% D-" at the top of an exam. They could easily be changed to verbal descriptions ("novice" & "master," etc.) for instance, and voila! No more numbers.

    (b.2) "SBG Assesses A Student On What They Currently Know, As Opposed To What They Knew In The Past." Yeah, this is impossible to do perfectly. And different people disagree about what to do when a student's understanding seems to fall from one week to the next. But it seems as though you're making the good the enemy of the perfect here. Traditional unit-based exams do this so much worse.

    ReplyDelete
    Replies
    1. Honestly? I'm surprised if anyone has been able to follow me down this path because I've started with sloppy generalizations and only gotten more on target after taking some bruises and doing some reading.

      Without further ado, let's dig into things.

      The numerical scores in SBG are appreciably different from writing "62% D-" at the top of an exam. They could easily be changed to verbal descriptions ("novice" & "master," etc.) for instance, and voila! No more numbers.

      Yes! This is absolutely true, to the detriment of feedback such as "novice" or "master." There is verbal feedback that is just as unhelpful for learning as are numerical scores.

      We shouldn't be worrying about the difference between numbers and words. This is a superficial distinction, as you argue. So what does matter? Here, a return to the research might be helpful. "Enhancing and Undermining Intrinsic Motivation" (Butler 1988) states plainly that "recent research on intrinsic motivation has consistently found that rewards undermine subsequent interest for initially attractive tasks."

      Both "62% D-" and "novice" are perceived by students as ego-involving judgments on their performance. The real difference comes when we provide feedback that is truly constructive. In Butler's study that involved comments like "You thought of quite a few correct words. Maybe it is possible to think of more short words."

      (god it's annoying to type in this tiny little box)

      This is the sort of thing I want to push people on. Words, numbers, stickers, points, badges, whatever, feedback should be about giving kids a chance to think more and better. Telling kids what you think they're good at isn't an important part of the game here.

      But it seems as though you're making the good the enemy of the perfect here. Traditional unit-based exams do this so much worse.

      Here I certainly need to say more than I've said so far. But, again, I'd go back to feedback and learning. Why is it important for a grade to accurately reflect what a student currently knows? What are we trying to do here? What's the theory of learning? Or is it a theory of fairness? What are we trying to do with this grade, and are we accomplishing it better by going from bad to better?

      Delete
    2. Oh and one more thing.

      I am locked in a series of uneasy truces with SBG and I'm not going to go to the mattresses over it.

      I wrote a whole post about why I keep on talking about SBG and I don't need to drag it out in the comments. My dream isn't that everyone stops SBG, or at least not exactly. I'd love to convince folks that (a) there is feedback that helps kids learn more things and (b) the sort of skill-based rubric-guided assessment that happens under SBG isn't it.

      Basically, when I'm attacking SBG I'm not attacking it as a grading system but as a feedback system. And I think that feedback systems are much more important than grading systems.

      Delete
  5. Michael, I don't think of labels like "mastered" as rewards. I think of them as empirical predictions like, "I think there's a 95% chance that you'd get another question like this one correct." Is that a reward, or an observation?

    I'm trying to implement SBG for Algebra 1 next year, and my biggest obstacle is making equivalent re-takeable assessments that test concepts rather than skills. For example, given scenario X, can you use a proportion to predict a future value, or doesn't proportionality apply here? With skills, it's easy to generate questions of equivalent difficulty (2x+4=10 versus 3x - 6=11), but a different scenario assessing the same concept may prove harder for students just students find that scenario less intuitive.

    Dan, I'm quite interested to read that you have an uneasy relationship to SBG. You're the person who made me look into it!

    ReplyDelete
    Replies
    1. I don't think of labels like "mastered" as rewards. I think of them as empirical predictions like, "I think there's a 95% chance that you'd get another question like this one correct."

      Well let's bounce this right back to old-style grades. I don't think of a D- as a reward, but rather as an empirical prediction: "I think that there's a pretty low chance that if you took this test on Linear Equations again, you'd do better on it."

      Or, if we like making predictions on how well kids things up by standards instead of units, just give a kid a n A on "Solving Equations" and run that "there's a 95% chance..." line back.

      A-F, 0-5, novice-master, what's the difference?

      But I'm not nihilistic here. Not everything is the same. To the extent that A-F, 0-5, novice-master are being used as feedback that helps kids learn, the assumption seems to be something like "a kid's learning is promoted by knowing what they're good and bad at." Based on my read of research and my own classes, this assumption seems fraught. Instead, productive research gives kids chance to reflect and solve-problems that they otherwise wouldn't be able to do.

      Look, maybe SBG is a great grading system, whatever that means. But can we agree that it's a lousy feedback system?

      Delete
    2. Gah. I meant to say "productive feedback gives kids a chance to reflect and solve problems that they otherwise wouldn't be able to do."

      Delete
  6. Michael,

    I've been following your recent journey in this post, on twitter, and in your google doc. Honestly, it's been hard for me to discern what you're trying to write about and what it is you'd like to discuss - SBG as an educational movement? The tension between SBG and grades? Calling on SBG advocates to defend their system so you can understand it/tear it down? This makes it hard for me to figure out how to enter this conversation. It also makes me wonder if this project is an exercise in working out your own thoughts on SBG (I do this too - sometimes I start a blog post just so it forces me to clarify my own thinking on an issue).

    Anyway - in watching from the sidelines, I wanted to point out a few things you've said *outside* of this article that (in my opinon, although you're welcome to ignore it) have helped to clarify some of your ideas and comments:

    "Basically, when I'm attacking SBG I'm not attacking it as a grading system but as a feedback system. And I think that feedback systems are much more important than grading systems." - http://rationalexpressions.blogspot.com/2014/05/why-i-find-standards-based-grading.html?showComment=1400534760301#c3613190178703587519

    "To put it more sharply, I don't think a grading system should ever be used as a feedback system" - https://twitter.com/mpershan/status/468758379612360704

    "Further, some people want to use SBG to motivate kids. But a grading system should never be an incentive system." - https://twitter.com/mpershan/status/468758915279515648

    Anyway - I thought those were an interesting set of ideas that I thought were worth emphasizing and helped me see your ideas a bit more clearly.

    Cheers - Mathy

    ReplyDelete
    Replies
    1. Honestly, it's been hard for me to discern what you're trying to write about and what it is you'd like to discuss.

      This is a very, very fair criticism, as all of this has sprawled out in a series of posts and tweets. Inevitably, some ideas change in content or expression as things go on. You're absolutely right to point to those three quotes as the crucial bits, as they've been the crucial bits all along.

      So, let's tighten up this argument. You'll tell me if this sounds better?

      1. Your grading system should not be your feedback system.
      2. Your grading system should not be used as an incentive system.
      3. Many people who write about SBG want it to be a feedback or an incentive system. This is not a good idea.
      4. The feedback you get from SBG isn't very good.

      That's the crucial bit. Now, because I'm a pain in the ass, there are some related arguments floating around. They are less crucial. Here they are, in short:

      5. We know what good feedback looks like. It should be ungraded, among other things.
      6. We should be a bit more conscious as to the historical context of SBG, and the possibility that the attractiveness of SBG has much to do with the rise of state-wide assessments.

      In general, my writing on SBG has been motivated by the concern that the online community isn't being sufficiently self-critical when it comes to grading, feedback and assessment. My desire is to both change the way people think about feedback as well as the way people think about blogs, twitter and the online community. We've made some big promises about what SBG can do, and I think it's time for us to reckon with our own fallibility.

      Delete
  7. Michael, I think the difference between SBG and your vision is primarily one of time scale. Here's how I see your vision: teachers regularly write comments on ungraded student work, prompting them to reflect on their mistakes and giving them chances to correct their work. That's great for day-to-day work. But what happens if a student's misunderstanding persists past the point of the relevant unit test?

    Students take their cues on what's important from class assignments and assessments. If they've missed 10 concepts over the semester but they think only 4 will show up on future assignments or assessments, they'll skip the other 6, not because they're incapable of being internally motivated, but because our instructional materials signal our priorities.

    So if you want them to keep trying to learn those concepts past the point of the chapter test, you have to make sure lots of assessments/assignments/tasks are cumulative. Once you do that, students are being asked to keep track of their proficiency levels in 40+ skills at all times, and it becomes really helpful to have some kind of chart for them. Just writing thoughtful comments in the margins will help them with the immediate mistake on the paper in front of them, but it won't help them prioritize.

    Referring to your numbered points above:
    1). Okay
    2). Maybe
    3). No, it is necessary feedback if you use cumulative tasks (which you should). It provides students the big picture of what they need to be working on. You're right, though, that written comments or oral feedback on ungraded tasks are also important.
    4). Okay, but it's essential.
    5). Okay.
    6). Yes, but I believe it's our duty as members of the MTBoS to figure out how to serve both masters well: the testing/accountability system, and the conceptual understanding.

    ReplyDelete
    Replies
    1. I agree that it's crucial for assessments to be cumulative. Here's where you lose me:

      "Once you do that, students are being asked to keep track of their proficiency levels in 40+ skills at all times..."

      Wha? How did we get from having cumulative weekly assessments to asking kids to keep track of their proficiency levels in tons of skills?

      Implicit in your comment is the assumption that it's helpful for a kid's learning for the kid to be aware -- or to be able to look up -- their proficiency in 40+ skills. How? What's the learning theory? How does knowing that you're not proficient at a skill lead to a kid knowing that skill?

      Delete
    2. Well, it seems to me that knowing what skills you're missing is a necessary (though admittedly not sufficient) condition for doing something about it. You don't agree?

      That question was partially rhetorical..you must agree (I think). So perhaps the difference in our opinions is this: I want students to be armed to tackle their own learning deficits on their own initiative. If you don't want to keep them informed of their proficiency in all the skills, it must be because you expect the teacher to take all the initiative for reteaching. This might work, but I'm kind of glad when a student approaches me and says she wants to see me after school because she can't figure out factoring when a > 1.

      Delete
    3. I think that this is a crucial area of disagreement. This seems like the core of things. I don't expect students to tackle their own learning deficits on their own initiative.

      This sort of independence is not something that I'm aiming for. It's hard enough for these kids to learn math when I'm there helping them.

      Delete
    4. What ages do you each teach? Perhaps this difference is related to the age of the students. For college students, I'm with Kevin. But for very young students, I'd be with Michael on this.

      Delete
    5. I've taught primarily high school, though lately I also teach elementary school too.

      I think it's worth noting that Kevin, in another comment, shared that his kids take this initiative within the context of a loosely-structured study hall that happens in class, so that this is still happening during class time.

      Delete
  8. I'm definitely struggling with all of this and attempting to rework this summer. This was my first year trying any SBG (2nd year overall) after being so frustrated with the traditional style the previous year. Many of these things you bring up came to mind this year.

    In my Algebra classes, SBG worked as traditional grading under a different name with no real effect on the motivations or learning strategies of the students. While in my Geometry class, it seemed easier (for various reasons) to give constructive feedback that actually affected these motivations and learning strategies.

    I definitely see how SBG in and of itself is not sufficient to change anything, but that it's the feedback strategy, as you said that creates that culture in the room. In the fall, I intend to stop putting scores on assignments after watching the video Ashli posted earlier this spring about the research you mention above.

    Thanks for pushing back.

    ReplyDelete
    Replies
    1. Thanks for the thoughts! You'll let us know how it goes?

      Delete