Monday, August 19, 2013

A Thought Experiment

Imagine your Favorite Author. She writes novels about mothers and sons and the way a city can suffocate its locals and also the Iraq War. Critics adore her work. High Schools can't decide whether to assign or ban her book. Commuters read her on their iThings. She's fantastic.

Your Favorite Author wakes up. She fumbles down the steps and lands in front of a cup of coffee that her husband left her. ("Thanks Dave," she thinks.) She takes a long sip and opens up the latest issue of the Journal for Research in Fiction Writing, which is lying on the table. She flips through it.

A title strikes her: "Characters that Stick: An Exploration of Backstory Tragedy in the Likeability of Supporting Characters." Interesting. She's working on a story now, one with a woman who, as a little girl, watched her parents die. (Spontaneous combustion, a metaphor for the ways in which life and time stories the inability to communicate in the digital era and recession.) She turns to the piece.

She reads the abstract:
This study investigated whether a character produced with a tragic background contributed to 236 randomly selected readers' literary pleasure, that is, the effectiveness of a story at inducing a passionate reaction. An exploratory quasi-experimental study was conducted with a pretest-posttest-control-group design. Men were found to derive more literary value out of a character with tragic backstory than women. Implications of these results for novel-writing are discussed.
Fascinating. Her next story would be better. She takes another sip, takes out her reading glasses. She digs in.


Here's what I'm curious for in the comments:
  • The above scenario strikes me as absurd. Do you share this reaction? 
  • If you think it's ridiculous to imagine novelists improving their craft in this way, then do you also think it's ridiculous to imagine teachers improving their craft in this way?
Here are some points that I think are worth making:
  • Research about how students learn isn't the same thing as research about what makes effective teaching. As an analogy, doctors regularly keep up on research about medications, procedures or treatments. But this is different than research concerning how to make a good diagnosis. It's the difference between research that's relevant to practice, and research on that practice.
  • You might be tempted to say that this comes down to whether teaching is an art or a science, and then (in the spirit of even-handedness) you'll want to say that it's a little bit of both. But you'll have to admit that artists have a wide base of knowledge from which they draw on, and that there is widespread (if far from complete) agreement that at least some art is really good, and some art aint so great. So it's not like facts and knowledge and correct judgement are foreign to art.
  • Besides, what would it mean to say that teaching is a science? Physics is a science; it's the study of the fundamental properties of the natural world. Biology is a science; it's the study of the living world. If teaching is a science, would that mean that it's the study of the something? But teaching, whatever it is, is just not the study of some subject. Maybe instead of "teaching is a science" we mean to say something closer to "there is objective, testable knowledge about what good teaching looks like"?
Looking forward to a spirited, late-August debate here. See you in the comments.


  1. I'm avoiding the art vs science part of the discussion.

    Each teacher's personality affects the way they interact with students (same could be said for the author and her characters). You could tell me tomorrow that the "don't smile til Christmas" rule has been thoroughly researched and it produces greater positive outcomes than smiling and I'm still going to smile.

    I want my students comfortable being themselves, which cannot happen if I'm not also being myself.

    A successful author will look at research and think, good for them but what I'm doing works. And then they will continue doing what has worked for them.

    Teachers interact with students. No amount of research can with any real certainty predict what 30+ students need from their teacher. Sure, suggestions can be made and frequently have value, but not always. I've done lessons which rocked one year (with one group of students) and had that same lesson fall flat the next year. I've done lessons which worked well for me, but the teacher across the hall swears it was a waste of time (and vice versa, to be fair).

    As educators, part of our job is to analyze a group of students and figure out what will work with that group (at that particular time). Show me a well documented study and give me access to all the background on the subjects and I'll point out 50 or more differences between that group and my current group of kids which could potentially skew the results.

    A good educator teaches their students to look at two different concepts, compare them and identify what makes them different and the strengths and weaknesses of the two sides. A great teacher is one who can themselves do that when presented with researched findings.

  2. Teaching is not a science. It's an art. Just as great art often depends on skilled craftsmanship, so good teaching requires skills. And just as the specific craft needed varies by type of artist, so does specific skill needed vary by teacher.

    I am not entirely sure that measuring learning is a good metric for measuring teaching quality.

    We need to measure whether a particular type of teaching is good for a particular level of student. But we won't.

  3. I'm not sure that I agree with the comparison. Writing a novel is (or should be) an act of self-expression; the author is not trying to optimize the readers' literary pleasure.

    Teaching has elements of self-expression, but there are also measurable outcomes. If a scientific study showed that a certain teaching technique was more effective than others, and it did not conflict with my own teaching style, then I would be inclined to adopt it.

    I would like to see math education become more scientific. There are a dozen different ways to teach factorization of quadratic trinomials. It doesn't matter to me what method students use, as long as they can use it correctly with understanding. But I would prefer to choose a method based on empirical evidence instead of gut feeling.

    1. "I express myself with my friends and my family. Novels are not about expressing yourself, they're about something beautiful, funny, clever, and organic." - Zadie Smith, aka like the most critically acclaimed novelist ever.

    2. This doesn't really get at the "measurable outcomes" angle though. Teaching has these measurable outcomes that are different. Like, an athlete has measurable performances and it's not ridiculous to imagine keeping up-to-date on physiological research findings in order to improve as an athlete (indeed, in terms of banned substances, there is a lot of exactly this sort of thing going on).

  4. @Education Realist
    Re: Skilled craftsmanship.
    Do you see a difference between the "delivery" of education and "design" of education?
    There are definitely skills that are required to run a classroom, but there are other skills required to promote solid theoretical foundations.
    I think of knowing viable extensions to problems and designing tasks with multiple avenues as examples of things that are directly informed by research.
    In that way, is there an "art" to teaching, but a "science" of education?

  5. Hi Michael,

    The abstract you published reveals a lot about what your idea of research is: randomized control experiments that reveal generalized truths about the empirical world. As a teacher, you know that the trickiest part of the work comes in sorting out the particulars. So if this is research, I can see why it would not help you in your day to day work.

    There are many kinds of research on teaching. The kind that I do seeks to yield frameworks for navigating the complexity of the work. It is, frustratingly to those who want to reduce teaching to tips and tricks, decidedly non-prescriptive. When people ask me what my research suggests for a given situation, the most honest answer often is, "It depends."

    Effective teaching is often like an optimization problem where you don't have all the parameters available. You become more alert to the clues around you: you start to get better at guessing the probabilities of this information signaling this condition, but the data are necessarily incomplete.

    Why do productive frameworks matter? They help teachers get better at attending to the right things in the blooming buzz of the classroom. The hope is to help the most kids come away with the best understanding of the content, and in ways that help them become fuller human beings. The problem is a tricky one when unpredictability is the base condition.

    See my last storify for my ideas about what gets better as people become more adept at teaching.

    Thanks for this discussion.

    1. Thank you for this, Ilana, and you're right that I ought to be more open to the sort of research that you do. And, I am. Though I'm not familiar with your research in particular, something like the 5 Practices framework came out of research, and I often find it helpful. I should never say that a teacher can't learn much from research, because of course we can.

      But, you describe people who find themselves frustrated with your work. They want tips and tricks, prescriptions, definite knowledge, and truth.

      My argument is that people would have an easier time hearing the truth in your work if they thought of teaching as an art, rather than an act whose success is determined by research knowledge.

      There's a strong set of sensible principles that stand behind theories of improvement in the arts: you need practice and loads of it, you probably need a mentor and an apprenticeship, you need to reflect, you need to find something that works for you, there is such a thing as greatness in art, if people don't get what you're doing then you're doing something wrong, etc.

      I want teaching to have access to this body of wisdom, and this perspective on learning.

  6. "Do you see a difference between the "delivery" of education and "design" of education?"

    Have you read any of Larry Cuban's writing on curricular layers? He posits there are four: designed, delivered, learned, tested.

    Educational policy is concerned with designed and tested layers. That's where we spend the billions, which is too bad, since it's all wasted so far as genuinely influencing or measuring learning.

    Teachers make almost all the calls at the delivered layer. We're the ones in the classroom with kids whose knowledge is far below the expected level, with huge ranges in ability, and so on. But of course, we can't control what the kids take away, because of the hugely different levels of motivation and cognitive ability within the classroom.

    To the extent there is a "science" of education, it involves a level we've barely begun to scratch, in terms of both cognitive ability and motivation. For example, Roland Fryer has done lots of work with paying students to see if that improves test scores, but I don't believe he's controlled for cognitive ability or incentives, at least not that I've seen. There's been some research on teaching math that controls for cognitive ability, and that's encouraging. But not much.

    "I think of knowing viable extensions to problems and designing tasks with multiple avenues as examples of things that are directly informed by research."

    I see that as something I do, and it's not informed by research so much as my own instincts. Art, not science. And the people who design these at the textbook level are not, I think, guided by research, but by what they think would make an interesting exercise.

  7. Maybe teaching is a science in your first sense: teaching is the study of learning.

    1. Success in teaching involves helping people learn. If teaching were the study of learning, then success would involve the teacher understanding more about learning.

      Unless you're willing to give up the success of students as a goal, I don't see how you can conceive of teaching as a study of learning.

  8. This comment has been removed by the author.

  9. Things I'm wondering about: how does the author measure their work? That is an occupation famous for revision - reflection on that level would be amazing for teachers. Where do authors go to get better? They seem to collaborate with other writers in workshop. Teachers need more opportunities to see teaching and to discuss it with the authors.

    1. There we go. This is the line of thinking that I find most helpful, and it's exactly the sort of thinking that you're missing out on if you conceive of teaching as depending on a body of research knowledge.

  10. In the novelist scenario, what are the analogues to issues like:
    - Can my students factor this expression or solve this equation?
    - One of my former students aced the GRE/MCAT and got into a great grad school / med school!
    - etc

    Does the novelist have analogous outcomes that she desires? In what way(s) are they analogous?

    1. There won't be perfect analogies but...

      1. Can I make this passage funnier?
      2. Someone liked my book!

      Does that work?

    2. Those don't really work for me. "Someone liked multiplication!" may be part of a good day for a teacher, but aren't you looking for certain student achievements as well?

    3. (A) An instructor has perfected an awesome activity that really gets her students engaged, achieves content goals (with test scores to prove it), and gets students involved in problem-solving and authentic practices. So of course she makes sure to use this activity every year with her students.

      (B) A novelist writes an awesome novel that is critically hailed, popular, and profitable. So of course she writes the same novel again every year.

      Why is (B) ridiculous but (A) is not?

    4. The product of a novelist is a book. The product of a teacher is student learning.

      What's ridiculous is repeating your product. The novelist in (B) is ridiculous, because he's trying to sell readers the same book twice. The teacher in (A) isn't trying to sell student learning to anyone twice.

      The real analogy to (B) is a teacher who strikes gold with her lesson on Monday and offers it again to students on Tuesday.

  11. "The product of a teacher is student learning."

    Are you sure?

    1. If you say (as I did) that the product of teaching is student learning, then this would make teaching without learning a definitional impossibility, which is problematic.

      So I'd update this to say that the product of a teacher is an experience for students.

      All that I think is crucial to my argument is that the product of a teacher is tied to a definite time, and so repeating a lesson isn't cheap in the way repeating a book would be.

    2. "So I'd update this to say that the product of a teacher is an experience for students."

      Yes! I agree with this.

      Look, I'm not disputing your points about research. I think they are valid. I'm questioning your concept of teaching.

      Because if the product of teaching is an experience for the students, then what research would help? It seems to me that the answer is based on the *type* of student: intellect and motivation first, but other factors as well.

    3. I don't see why research couldn't produce helpful insights or aids for creating experiences for students. I think that the particularity of working with students is a knock on a sort of abstract, generalized, randomly variabled sort of teaching research, but Ilana has already helpfully pointed out that much of teaching research doesn't look like that.

  12. If you're going to analogize with writing, then isn't teaching more analogous to persuasive, non-fiction writing, such as a policy brief, or a health education pamphlet, etc? Something where you're trying to change the thinking and/or behavior of the reader?

    And when put in those terms, it no longer seems ridiculous that the author would consider research findings to improve her writing.

  13. I find this an interesting thought experiment but, after reading this post a few times today and also thinking about the Twitter conversation you've been having (which are harder to follow but still interesting), I think I have a response. It may be too simple/simplistic, but I hope it makes sense.

    I think you are comparing apples (the artist) to oranges (the teacher). As both a dancer (artist) and a teaching artist I find that being creative within my art form is completely different from teaching my art form. Just because you *do* art doesn't mean you can teach it...because there is a whole other set of skills that go into teaching.

    That's it. Well, except to say that as a dancer I learn very little from following the MTBoS, but as a teaching artist I am completely enriched by this kind of conversation. I hope this distinction is useful in some way. :-)

    1. "Just because you *do* art doesn't mean you can teach it...because there is a whole other set of skills that go into teaching."

      Agreed. But teaching is itself a craft or an art form, and that whole other skills constitutes a craft that is not unlike art.

      So: Why do we have such different methods of improving teachers and improving artists?

    2. "that whole other skills constitutes a craft that is not unlike art. "
      [citation needed]
      I think if people are saying they don't buy the analogy then you just restating that A is like B doesn't help.

      "So: Why do we have such different methods of improving teachers and improving artists?"
      Because maybe teaching is different? Maybe the analogy doesn't work?

      It's like you're looking for a simple analogy or a tweetable aphorism that captures whatever it is you mean (and I'm not at all clear on what that is). In one comment you say student learning outcomes are important for teachers and in another comment you say "someone liked my book!" is the analogue for the novelist. People are pushing back that this analogy isn't just challenging (which would be good) but maybe nonsense / apples-to-oranges (which doesn't help).

      Maybe you should just say what it is you mean even if it can't be over-simplified in these ways.

    3. The analogy isn't important. What I'm claiming is that every profession is a craft. Making art is a craft. Doing science is a craft. Teaching is a craft. Doing math is a craft.

      And there are certain traditions of improvement for different crafts. Fiction writers have a tradition of improvement, and it goes something like "Read a ton, write a ton, make sure that you finish stories." Jazz pianists have a tradition of improvement, and it goes something like "Train your ear, play in other people's styles, and then play in your own."

      What's teaching's prescription for improvement? One story that people tell is something like "Stay on top of the current research, learn new techniques, gather good materials."

      I'm trying to expose a gap between the story that we tell about teacher improvement and the story that we tell about the improvement of other crafts. My opinion is that there's an implicit story that people believe about teacher improvement, and that it becomes odder and odder the more you compare it to other crafts.

      The point isn't that teaching is an art in the same sense that fiction writing is. The point is that both are crafts, and that I think that thinking of teaching as a sort of art is helpful because of the tradition of improvement in the arts is sensible and generally applicable.

      The debate that I want to have with people is about the general applicability of these traditions of improvement. Is there a reason why reading journal articles of current research would improve a teacher, but not an author? Is there a reason why a tradition of improvement in the arts (consume a lot of art, sometimes imitate it, produce a ton) is particular to the arts, and not to teaching?

      For example, Dave suggested that writing ought to have a different path of improvement than teaching because writing is about self-expression, and teaching has objective outcomes. This is an objection that is on point. I responded by disagreeing with his premises, but this is the discussion that I want to have.

  14. Isn't the difference in the audience? For example, an artist need merely note the satisfaction of the audience. The audience may be fickle, the audience's opinion may change over time, but if we liken audience approval to engagement, then artists can stop at engagement. Some of them may also want to affect "learning"--in artistic terms, create an enduring work of art, beyond that minute--but an artist has the ability to judge impact solely by engagement.

    Teachers can't. This is *why* you originally say that teacher's product is learning, because you see, correctly, that it can't be merely about engagement.

    And it's more than "if we just want to engage, we can show a movie." No, it's pretty clear that even actual *teaching* that shows engaged students does not necessarily improve learning outcomes.

    So teachers are measured by the experience they provide the students, but that experience isn't simply about engaging their interest or cooperation. At the same time, because student ability and incentive varies so tremendously, we can't measure learning as the outcome.

  15. Thanks for giving useful info...Shop for the best collection of Designer Reading Glasses, Sun Readers, No Line & Bifocal Reading Glasses for men & women.

  16. Thanks for giving useful info...Shop for the best collection of Designer Reading Glasses, Sun Readers, No Line & Bifocal Reading Glasses for men & women.