Saturday, December 20, 2014

In Conclusion (Post 10 of 10)

In conclusion, teaching has a writing problem.

Part of the problem, I've been arguing, is that in education we talk about the wrong things. We talk about concepts that don't get at the heart of classroom teaching -- concepts like feedback -- and so we ask questions that are impossible to answer. As a consequence, these questions -- like how do you give effective feedback? -- are very difficult to respond to. They require the ability to split hairs and to generalize. Academic researchers and consultants are very well-suited to these tasks. Teachers aren't.

And so almost all the writing about teaching comes from researchers or consultants, people who are no longer k-12 classroom teachers. This would be a fine state of affairs if it worked. But writing about teaching rarely does.

My favorite writer about feedback is Dylan Wiliam. His work is as smart as Edutopia's is silly and reductive. Here's a picture that makes him look like a super-villain:


I heartily recommend Embedded Formative Assessment. There's a great passage in there where Wiliam realizes how difficult it is to communicate about feedback.
In 1998, when Paul Black and I published "Inside the Black Box," we recommended that feedback during learning be in the form of comments rather than grades, and many teachers took this to heart. Unfortunately, in many cases, the feedback was not particularly helpful. Typically, the feedback would focus on what was deficient about the work submitted, which the students were not able to resubmit, rather than on what to do to improve their future learning.
Got it! Focus on what the kid can do to improve, right?
I remember talking to a middle school student who was looking at the feedback his teacher had given him on a science assignment. The teacher had written, "You need to be more systematic in planning your scientific inquiries." I asked the student what that meant to him, and he said, "I don't know. If I knew how to be more systematical, I would have been more systematic the first time."
So Wiliam needs a way to say exactly how feedback can tell a kid how to improve. What does he land on?
To be effective, feedback needs to direct attention to what's next rather than focusing on how well or badly the student did on the work.
Feedback should cause thinking. 
If, however, we embrace the idea of feedback as a recipe for future action, then it is easy to see how to make feedback work constructively.
It seems to me that Mr. "Be More Systematic" probably thought that his feedback focused on what's next, caused thinking, and gave a recipe for future action. Wiliam's metaphors and slogans are not enough.

---

It's not Wiliam's fault. He's trying to tackle an incredibly broad subject. (The chapter is titled Feedback That Moves Learning Forward. All of it??)

Here's an embarrassing belief that I have: classroom teachers have a chance to make things better. Nobody in education is better equipped to find the most helpful way to think or write about classroom teaching than classroom teachers. We have the stories. We have the teachers -- novice and experienced -- that we talk to daily. We know what our colleagues find helpful. We know our kids.

When we write about teaching, we have to play to our advantage. We have experience, stories and students to work with. What we lack is precisely what researchers and consultants have -- wide-ranging perspective and time. I think a lot of the silliness we see in education writing comes from writers trying to hit the scope and abstraction of university research. (It's like writing an op-ed in binary.)

There is so much fantastic story-telling from teachers that I read, but just telling stories isn't going to be enough. If we want to contribute to our common knowledge about teaching we need to try to do something with our stories. To analyze them, to collect them, to point out patterns and to generalize from them. That's one of the challenges that teachers writing about teaching face.

Maybe. I think.

Questions

  1. I've argued that revision is a central concept in teaching. No way this covers the entire (but wildly abstract) formative assessment landscape. What other core concepts could we break formative assessment/feedback in to?
  2. It's easy to help other teachers improve their repertoire of activities or teaching moves. (Just tell us, and we add it to the mental list.) But can we also share our decision-making, how we navigate through our repertoire?
  3. Is there an intellectually respectable way for teachers to write about teaching that other teachers will want to read?
  4. Is there an intellectually respectable way for classroom teachers to write about teaching that university researchers will want to read?

Tuesday, December 16, 2014

How To Write About Teaching... (Post 9 of 10)

...question mark? This is really an open question for me. I am far less confident about how to write about teaching than I am about how to teach.

Here's what we know: teachers don't like educational research.



Another thing we know: there is a lot of educational research about feedback.




Look: if you're reading this then you're probably a math teacher. And we've already established that math teachers don't really care for educational research. You might have a faint interest in what the research says, but in all likelihood you don't find that research compelling in any way.

Q: What does compel a teacher, then?

A: Twitter.
Where is the middle ground here? Social media doesn't offer much argument, but academic research is toxic for teachers. Is there any intellectually respectable way to talk about teaching that teachers will care about?

Like I said, to me it's really an open question. I don't know how to do it. I'm struggling here.

My recent writing about feedback is an attempt to find a place to land. Here are the (loose) guidelines I had for this series of posts on feedback:
  • Talk about a kid, not "the kids." Whenever possible, I tried to focus in on a specific student. That helped me give enough context about the kid's specific needs (social, cognitive, motivation, etc.) that you could understand my situation better.
  • Talk about decisions instead of what happened. I tried to avoid the "Here's what I used to do...here's what I did instead...it worked!" cliche of teacher-writing. It's hard to know what's really happening in someone else's classroom. I can't help you see that my feedback is working, and you shouldn't just take my word that it is. Instead, I tried to argue for the way that I was thinking about teaching.
  • "Enough Context" = The math goals + kid's thinking + kid's social status. It was important to me that I give you enough detail so that you could be capable of skepticism. I found that three things often felt important to offer: what I was trying to do, what the kid thinks about the math, and how he/she interacts with math and her peers.
Here's what we know: teachers don't like reading research, but we have no good substitute for it. In a world like ours, the only sensible thing to do seems to be to read research but write from experience. But experience is famously unreliable. We need to find some way to raise the bar on writing from experience. 

But I feel very unsure about this whole thing, so here are some questions.
  1. The nutty thing is that sometimes teachers -- even the very teachers who poopoo research! -- cite or rely heavily on the research world. "Research shows that white boards work," or "Science shows that timed tests are bad for kids." I mean, what's the deal here? (See Schenider, to start?) 
  2. Is blogging the solution?
  3. Forget research for a second. What do you like/hate about non-academic writing about teaching?
  4. The test for teachers often seems to be usability. "It has lots of practical, simple suggestions that I can use in class the next day." Is this what's missing from writing about teaching? Maybe a mix of instantly usable ideas with longer-form arguments could help?

Sunday, December 14, 2014

Does Feedback Need to Be Used In Class? (Post 8 of 10)

I chat with people about feedback and revision online. Here's what I notice.

  • Everybody agrees that feedback is important.
  • Most teachers agree that revision is important. 
  • Very few teachers are giving students a chance to revise their classwork in class.

What are teachers asking students to do with their feedback? They either loosen the "in class" or the "revise their classwork" requirement of that revise their classwork in class formula:
  • Kids are given the option to resubmit their work, but they're expected to do the revision on their own, outside of class.
  • Kids are given feedback and they have future chances to improve their performance on a different, but related problem. (I'm really thinking of the SBG crowd, here.)

This post is here to report that I haven't had much success with either of these common practices. If I want every student to work on something, I find that I need to ask them to work on it in class. And I find that if I just give feedback without giving students a chance to use that feedback (more-or-less) immediately, that feedback tends to be just another thing that I've said instead of something that sticks.

Maybe things are different for your kids? I really have no clue, and would love to know.

In sum, here are some questions that I have.

Questions
  1. Do your kids remember feedback that they don't immediately use in some further classwork?
  2. Are there ways to ensure that your students are thinking about the feedback that you give without asking them to use that feedback on a problem?
  3. What's the theory behind how feedback helps students do better on future tasks, if they aren't using that feedback to practice? 
  4. Do your kids do quality work outside of class? I've never really been able to coax quality out-of-school work from children but kids write essays so it must be possible.

Wednesday, December 3, 2014

Frequently Asked Questions About Feedback (Post 7 of 10)

Q: What does effective feedback look like?

A: It's really best not to worry too much about this. Effective feedback looks wildly different in different situations.

Q: What does effective feedback look like?

A: Seriously, don't sweat this.

Q: What does effective feedback look like?

A: Look I GET IT. You want to know how to use feedback to move your students forward. Me too. But all of the guidelines and slogans you hear flying around about feedback are just plain silly. "The sooner the better"? "Personalized feedback is always better"? These guidelines are wrong, and it's sort of a crazy way to talk about teaching.

If you knew why you were giving feedback, you wouldn't need to ask what effective feedback looks like. We need to shift the conversation in two ways: (1) Away from "what good feedback looks like" to "how do we make good decisions about feedback" and (2) away from "giving feedback" to more specific teaching moves, such as asking for revisions.

We need to focus on decision-making instead of just the product, but feedback isn't specific enough to really gain clarity from thinking through. That's why we should move to thinking about revision.

Q: Why are you yelling, can you please stop yelling?

A: In fact, lots of "bad feedback" is frequently helpful! I mean, if you listen to the way some educators talk it's like there's this golden ideal of narrative, written comments that look a lot like the sorts of notes you're supposed to get on a research paper or a short story. And, sure, that sort of feedback can sometimes be helpful. But when? Why? There's no theory, so no wonder that we end up focusing on what effective feedback looks like.

But "revision" is knowable. We can study it. It encompasses feedback, because feedback is what it's going to take to help students improve their work. At the same time, it's easy enough to pin down "asking for revisions" that we can actually say things about it, like when it is or isn't helpful to give a revision assignment. For example, tasks that are readily comprehensible aren't good candidates for revision.

Q: I just want to know what effective feedback looks like?

A: Here's what I'm proposing:
  • Effective feedback doesn't look any particular way. From a picture, we can't tell whether we're looking at good or bad feedback.
  • Revision of previously attempted work is a powerful tool for certain situations.
  • Good feedback is whatever it takes to help kids improve their work, on their own, in class.
  • We got into this mess, partly, by talking exclusively at a very high level of generality. Feedback is an enormous, fuzzy, abstract concept. We'd do better to bring our discussions down to Earth.
OK, and now I have some questions.

Q: What are some of the decisions that we face when designing a revision activity?
Q: Are there contexts in which different sorts of revision activities tend to be better? 
Q: What sorts of situations tend to need whole-class feedback in order to help kids improve their work?
Q: What other concepts are there out there that are like "asking for revisions" in that they carve out a significant piece of the formative assessment landscape?
Q: What do students whose classwork was at a high level do during a revision activity?
Q: How do you support kids in a revision activity who need help but won't benefit from individual attention because of social reasons?

I have lots of questions, I can keep on going.

Q: What does effective feedback look like?

A: Effective feedback leads to revisions in class.

When Not To Revise (Post 6 of 10)


Elixir or Water?

"Come, gather 'round! Everybody could use a drop of Gray's Fantastick Elixir! Drains the spleen, straightens your spine, furnishes beards and even improves digestion."

"Sir? Will your potion cure me of my ills?"

"Surely, son. The Fantastick Elixir is for all occasions. The more the better, the sooner the better!"

---

Nothing we do -- provided we do it with purpose and intent -- is always helpful. Medicine that heals everything really heals nothing. It's an elixir, a phony, or else it's a universal necessity of life, like water or air.

(Not that there's anything wrong with water, or air. But nobody needs to tell you to breathe.)

So too with learning. If some action always helps a student learn, in any situation, then it can hardly be said to help learning at all. It's a phony. Or it's something so basic to teaching or learning that it barely helps to mention it at all, like "listening." 

I'm not sure that I can tell you when "giving effective feedback" does and does not help learning. I'm sure that it's because giving feedback always works, which means that it's a basic staple of teaching and learning. It's up there on the shelf with "listening well" and "explaining appropriately." 

(Make sure you're drinking! Make sure you're giving feedback! Not everything that's crucial is important.)

On the other hand, I believe that I can say with some specificity when it does and doesn't help students to ask them to revise an assignment. If I can make this case -- and in this post I'll try -- then this should count in favor of "revising" as a concept that is important for teachers. Not because it's fundamental, but because it's not.

The Short, Simplified Answer


Stop here if you'd like. All that follows is evidence and argument.

Would Revision Help Here?


Rachel has been struggling with addition. She's a 4th Grader who -- until a week or two ago -- regularly got things like 6+4 wrong. The rest of the class is far more comfortable with addition/subtraction than she is, and one of the toughest challenges I face is making sure Rachel gets explicit practice with addition while still pushing the rest of my students.

(A recent compromise was to rejigger my units so that we began a unit on addition with larger numbers before diving back into multiplication. I reasoned that Rachel would have a better shot of hitting her trouble spots if I could focus on addition with the whole-class than if we flew on to a unit where addition wasn't at the forefront.)

When it came to adding two- or three-digit numbers, I noticed a few interesting facets of Rachel's work:
  • When given a choice, she always prefers the standard algorithm
  • When Rachel used the standard algorithm, she nearly always messed it up
  • When Rachel was asked to use another strategy -- like breaking numbers apart by place value -- her work was very accurate 

One day, while watching Rachel work, I noticed a mistake with her standard algorithm addition. No shock. So I asked her to use a different strategy, and there she goes, accurately breaking the numbers apart by place and adding. Huzzah. There's a discrepancy between her answers. I ask her which result she believes more. She points to her second, correct sum. Yay.

Rachel: "Yeah, but I really like stacking it."

(Teaching in a nutshell, right?)
So, what do I do? How can I help Rachel improve her adding?

A Cost-Benefit Analysis of Revision

Here's an option: I could pick a problem from Rachel's work, one she used the standard algorithm on. I could give her feedback -- written, oral, whatever makes sense. I can explain where she went wrong. Then, I could ask her to revise it, correct it and improve it.

Would that have been the right call?

I don't think so. Much better, I think, to give Rachel feedback and then a new problem to work on.

Why? There are risks -- in the scheme of things, relatively minor risks -- associated with asking Rachel to continue to work on a problem that she already completed.
  • Engagement Risk - If I ask Rachel to continue working on a question that she already answered -- even one she answered incorrectly -- I run the risk of boring her. After all, she's already seen it before, and it's new things that tend to excite our students. And maybe she won't give the feedback credence at first because she thinks that she already got it right...
  • Social Risk - ...or she'll see that it's wrong and resist engaging because she doesn't really want to feel dumb. Kids like improving, but there's always a risk that feedback will go wrong, or that the personal attention of a one-on-one meeting will be embarrassing.
  • Learning Risk - Another way that this whole thing could go wrong is if Rachel makes the (relatively simple) local correction without actually improving her addition skills. There's not exactly a whole lot to correct or revise in a single addition problem. She's going to need a few examples to improve.
In many situations, these risks are worth taking. Why? Because it would be incredibly costly to assign a new problem to students. We don't want students to spend class time making sense of an entirely new context just so they can tweak one small (but crucial!) detail. And making new problems that target specific areas is time-consuming for teachers. And maybe something about the new task will be more complex than intended, and the conversation won't focus on the area of need.

When revision works, it works because we can get to the point. Yes, we know the context. We understand what the problem's asking, and we even know a lot about how to find the answer. There's just this one area we can improve. This one thing. Let's make this better, I'll help you.

But addition problems? Shoot, I can make 'em up on the spot, and Rachel won't need help understanding what I'm asking her to do. There's little to gain from asking Rachel to revise these sorts of problems.

Revision Is A Sometimes Thing

So revision -- even done well -- isn't an always thing. It's not a staple. It's not a panacea or an elixir either. (And that has got to be at least part of the reason why we don't talk very much about it.) 

Revision is a thing that sometimes helps. We constantly have to weigh the benefit (skipping to the point) with the risks (engagement, social, and learning). It won't always help, and that's a very good thing indeed.

Appendix: What I Ended Up Doing For Rachel


I wrote up this task. I told the class that there was a kid named Joe and that Joe's method for adding was to just add the digits in each place together and then write them down one after the other. (Rachel: "I use Joe's Method a lot.")

You shouldn't take my word for it, but it went well and her work has improved.

Saturday, November 22, 2014

Sooner is Not Better, and There Is No "Best Feedback" (Post 5 of 10)

At the start of this series, I highlighted four claims about feedback that I thought were both widely heralded and wrong.
  1. Oral feedback is always better than written feedback.
  2. When it comes to feedback, the more the better!
  3. When it comes to feedback, the sooner the better!
  4. Feedback is all about helping students understand the mistakes they've made.
Since then, I've shared a case when I gave oral feedback and a case when I gave written feedback. In one case I gave lots of feedback and in the other I gave rather little feedback. 

Sometimes oral feedback is better than written feedback. Sometimes it's the other way around.

The point isn't just that these myths are wrong, though they are. It's that this talk of "best practices" (or "best activities") is such a limited and unhelpful way of talking about teaching. 

In this post I'll argue that sooner isn't always better. Then, it's time to put these sorts of claims to rest. There's a better way to talk about teaching.

Point: When It Comes To Feedback, The Sooner The Better

"In most cases, the sooner I get feedback, the better." (Seven Keys To Effective Feedback)

"Of course, it's not always possible to provide students with feedback right on the spot, but sooner is definitely better than later." (5 Research-Based Tips For Providing Students With Meaningful Feedback)

Counter-Point: Nah

A few weeks ago I gave a quiz to my geometry classes. We were at the end of a unit on quadrilateral properties and proofs, which was the area that my quiz targeted.


When looking at the quiz, I noticed that my three "always/sometimes/never" questions had gotten a huge variety of responses: "always, always, sometimes"; "sometimes, sometimes, it depends"; "sometimes, always, never", etc. 

A closer look confirmed that their thinking was all over the place. A handful of kids indicated that parallelograms have a line of symmetry. Others didn't recall that a kite can be split into two congruent triangles. Others thought that a trapezoid's diagonal divides it into two congruent triangles. Some proofs were nice, others needed lots of improvement. 


I quickly decided to give my class feedback and time to revise: no other activity that I could run in class would be able to address each individual mistake and give each student a chance to think about their specific area of need. 

I decided to wait a week. 

Why wait? It wasn't because I didn't have time. I think that waiting was the right decision. Consider the work of one of my students, Jake:


Jake showed some major limitations in his thinking on this quiz. Jake indicated that only parallelograms can be split along their diagonals to form two congruent triangles and that the consecutive angles of a parallelogram don't sum to 180 degrees. 

Jake was hard-working during class, and he participated actively in many of our conversations in class about parallelograms. In short, he hadn't slacked during the past two weeks. Despite this, he was still having trouble putting the pieces together. Would feedback help him where two weeks of instruction couldn't?

Maybe. But here's another thing I knew: the upcoming week quite possibly would help. Why? The unit we were beginning was studying how one shape (e.g. a parallelogram) can be dissected and rearranged to form a new shape (e.g. a rectangle). The activities that I was planning on running would give us a chance to physically rearrange shapes into other shapes. I thought that these activities would probably give me a number of great opportunities to address some of Jake's misconceptions about congruent triangles in kites and parallelograms. Or, at least, we could draw on these dissection examples when discussing his quiz when I did give Jake feedback.

Another factor: Jake was hard-working, but I saw a drop in engagement after a few days of "always, sometimes, never"-style questions. I wasn't shocked. We had been working on them for a few days, and questions that are posed in a similar fashion can start to bleed together after some time. I thought that Jake (and others) needed a break from these sorts of questions.

So I waited a week, and I gave feedback. And it was fine. 

OK, Great. Every Rule Has Exceptions. Including "The Sooner the Better." Who Cares?

Thanks for the question, sub-heading!

It's true: there are certainly times when the decision to wait a week to give feedback is the wrong one. And is that the majority of situations? The vast majority?

I don't know. I do know that there are lots of situations like my quadrilaterals quiz, and there are a lot of students like Jake. And that "the sooner the better!" is, strictly speaking, false. 

Here's the only question worth asking, then: do we learn much from a slogan like "the sooner the better"?

I don't think so. 

We don't need "5 Research-Proven Teaching Techniques" or "7 Activities For Learning" or "12 Qualities of Excellent Feedback." Ultimately, the work of teaching comes down to the decisions that we make, and writing about teaching should be about improving those decisions. 

What we do need is to talk about the dilemmas that we teachers commonly face, and try to find guidelines that help us make wise decisions in those scenarios.

Talk of "the sooner the better" isn't anywhere close to that. In the rest of this series, I'll try to talk more productively about decisions we face when giving feedback for revision. The stories that I've shared so far are my attempt at a start to that work.

This is the fifth post in a series on feedback. Here's what you've missed so far: 1, 2, 3, 4.

Saturday, November 15, 2014

Against the Feedback Menu (Post 4 of 10)


So you're giving a presentation to teachers. It's called "Giving Effective Feedback to Students." You've got slides. You've got an audience. You've got 60 minutes.

How do you spend it?

The way we teachers talk about feedback, you're likely to present about the menu of options that teachers have for giving feedback. You'll talk about written vs. oral feedback; positive vs. negative feedback; individual vs. whole-class; timely vs. delayed feedback; lengthy vs. brief feedback; feedback for learning vs. feedback for evaluation.

That's certainly what's going on in Types of Feedback and Their Purposes:


Here's an opinion about how we talk about feedback: this is an insane way to talk about feedback.

What makes this crazy is the extraordinarily high level of abstraction. It's so high-level that it's practically philosophical: "What is good?", "How should we write?", "What's best to do?"

In a different planet, we would talk about teaching situations and how to improve them. That seems more sensible to me than talking about things that improve learning and then matching them to various scenarios, post-hoc.

By analogy, imagine that I was presenting on "Giving Effective Drugs to Patients" instead of "Giving Effective Feedback to Students."


My first slide would, of course, cite a relevant research table showing -- conclusively! -- that giving drugs helps patient outcomes.


Of course, first we would have to define "drugs." (The first sign that we're dealing at too high a level of abstraction -- we don't even know what we're talking about!) 


Too many doctors just prescribe Asprin for everything, don't you find? (Compare to: "Feedback vs. Advice".

Now, time to dig into the details! Doctors have to make important decisions about the amount of drugs, as well as the timing.



And don't forget - no two patients are the same!


My point here is not to compare teaching to medicine. They're very different fields! I'm making a more limited analogy between the way we talk about instructional techniques and the way doctors talk about medical strategies.

"What is the most effective way to use drugs?" is an incredibly general question, and not a particularly helpful one. A better question to ask would start with the ailments: "What's the most effective way to treat whooping cough?" or whatever.

So too in math education. The "menu of options" style of presentation is pervasive in talk about teaching, but it doesn't have to be. A more productive route, I think, would be to start thinking about various specific teaching-scenarios that are common throughout the profession. How do we make good teaching decisions in these scenarios?

What sorts of scenarios am I talking about? Here are two that I'm spinning off of earlier posts in this series.
  • Toni's Fixed Mindset: How do you help a talented student who is socially unable to admit to any mathematical limitations to learn a complex skill, like proof? (Post #3)
  • Misinterpreting Bar Graphs: How do you best help a class that has a wide variety of minor, but significant misconceptions that can't be addressed all it once through whole-class instruction? (Post #2)
In the case of Toni, I argued that the best feedback was whole-class, oral instruction that lead to students choosing just one solution to revise. In the case of the bar graphs, I argued for giving individual "highlight"-style feedback and group-time to revise the entire assignment.

Written vs. Oral? Whole-class vs. Individual? Immediate vs. Delayed?

How much good can come of these questions?

---

This is the fourth post in a series on feedback. Here's what you've missed so far: 1, 2, 3.