Tuesday, June 19, 2012

The importance of being really careful in assessing character education

From the NYTimes Q&A with Dominic Randolph:

Q. How do you answer studies that have so far proven that no character education program is effective? Indeed, the implication from the studies is that no program can be effective. That is, October 2010, a federal study, the largest and most thorough ever conducted, found that schoolwide Character Education programs produce exactly zero improvements in student behavior or academic performance.

A. D.R. Exactly the reason that there should be a new effort to create programs that are effective. There is plenty of research to show that “character” as defined by people like Marty Seligman and Chris Peterson is positively correlated with success and high achievement both in individuals and groups. I just think that many supposed character programs have been ill conceived and not well executed. We have not yet worked out an entire program at both our schools, but I think that the work that we are doing is heading in some interesting directions.

Here's the federal study that's referenced above. What drives me crazy is that there's all this money being spent and all this time being invested in programs that don't work. Will my program fail? Yeah, it probably will. That's why assessment is so important.

People resist change. We put students through experiences precisely because we've reasoned that they should impact the students. What that means is that it's really easy to say that kids must have grown because they went through these meaningful experiences. It's so easy to fool yourself into thinking that people have learned something when they haven't learned a thing.

In all likelihood the course that I'm designing will not significantly impact the character of my students. But I think we can try, and if we are careful in the way we measure success we've got a fighting chance of figuring this out eventually.

Here's what I'm proposing as measurements of the success of my program. I'll take as evidence of success any positive, measurable changes in the following areas:
  1. Amount of charity given
  2. Amount of time dedicated towards others
  3. Attitudes towards an individual's obligations to others

Right now I'm looking for surveys that measure any of these. If you've got anything, please drop it into the comments.

11 comments:

  1. Without having read the study, I suspect one reason why these programs have been measured to be ineffective is because in many cases it takes a *really* long time for those lessons to take root. Perhaps the best we can hope for is to "plant the seed" of character development and hope that at some point in the future, it grows.

    ReplyDelete
    Replies
    1. That's dangerous thinking, jason. Does that mean that, in the short-term, effective and ineffective lessons look the same?

      Delete
  2. Please see http://wikipedia.org/wiki/Character_education. I wrote the article.

    The main finding that applies here is that character is situational, and is impacted far more by external factors than something endemic to the individual (original sin, blank slate, etc).

    Humans are social animals and are born naturally cooperative AND competitive, in a dynamic tension. Want a great society? It's as easy as making sure that everyone has equal access to opportunities, provide a solid social safety net, and moderate gaps in wealth.

    Modern socialized societies have been found to be less dysfunctional on nearly every indicator of desirable social outcomes using the Successful Societies Scale. It is ironic that religion is inversely correlated with social success, especially when most character education approaches can be traced to roots in Christian concepts and attitudes.
    http://www.epjournal.net/wp-content/uploads/EP07398441_c.pdf


    Jason, the study also found that the majority of programs studied had no fault in implementation.

    ReplyDelete
    Replies
    1. Hiffi,

      Do you have any recommendations for high quality character education programs that I could study? Thanks in advance.

      Delete
    2. This comment has been removed by the author.

      Delete
    3. In all my research on the subject, I have never turned one up. Going back into antiquity all attempts to improve character at the individual level have failed.

      This is because kids are not born with a character deficit like they are with a math one. I suspect if we ever can measure character, it will be like measuring height - you are born with it. But as things stand, you can't test to find if character lacking, consequently, you can't tell if any program improves it - which goes along way to explain the proliferation of competing and contradictory CE programs!

      If there is anything that works, it's called being born human. We're morally evolved (a tough concept for a religious society). We are experts at cooperating and experts at competing. As a result, overwhelmingly, the most successful interaction strategy is tit-for-tat. No wonder that the most functional societies moderate this dynamic by being appropriately socialized (this goes back to our tribal origins). Behavioral problems are simply anomalies that can be addressed, case-by-case by looking at neurological or environmental causes.

      I believe a lot of the hubbub around CE is due to the mistaken assumption that things are worse than they used to be, and that the young are the problem. It seems to be endemic to the selective memory of old people. Especially odd because, in actuality, things are much, much, better than they used to be.

      From the Wikipedia article:

      Generational character

      Particularly in modern liberal republics, social and economic change is rapid and can result in cognitive stress to older generations when each succeeding generation expands on and exhibits their own modes of expressing the freedoms such societies enjoy. America is a prime example.

      With few traditions, each generation has exhibited attitudes and behaviors that conservative segments of preceding generations uneasily assimilate. Individual incidents can also produce a moral panic.

      Cries about loss of morals in the succeeding generation, overwhelmingly unsubstantiated,[8] and calls for remediation have been constant in America since before its founding. (As an aside, it should be expected that living in a free country - which increasingly supports children's rights - this trend will continue apace.)

      Best

      Delete
    4. You argue that change in X is impossible unless that person has a deficit in X. I disagree. Imagine a basketball player who has bad form, and misses a lot of free throws. He doesn't have a shooting deficit -- he's just bad at it. But through proper coaching and effective practice, he can improve.

      Writing is like this too. Kids who come into 11th grade know how to write. They're just bad at it. They have bad habits and they often don't know how to string sentences together.

      Math is actually like this too. The kids who come into 11th grade math have TONS of ideas and intuitions about numbers, functions and patterns that we need to correct, because they're substantively incorrect.

      And, subjectively, I feel as if my character has changed, and sometimes on purpose. I'm more diligent and less likely to give up than I was two years ago. I'm more patient with loved ones. I'm worse than I was in some ways. I'm less patient with friends. I'm less charitable.

      And I know people whose character gets worse. They become cruel as they age. They lose empathy.

      You might say, "Well, that's not character. That's just changes in the way that you behave or react based on your character." To which I'd respond that it's the same difference to me. If I can change the behavioral or emotional responses to children, so that they can control their determined character that would be great too.

      Delete
    5. Can you teach a kid to reason about math, though? To think creatively? No, that's IQ and can't be improved.

      You give examples of Pavlovian training. It works, sure, but is it character improvement? How do you reward a bully for not bullying (it should be obvious that punishing only teaches bullying). Even then, studies show if you reward kids for desired behavior, they are more likely than those not rewarded to abandon the behavior when the reward is subsequently removed from the equation. Our natural cooperative instinct is STRONGER than a trained one. And training will pervert it!

      Moreover, CE in its current state is an untested experiment on kids (do we really want to be doing that?). Especially, when what we know so far is that there are unintended outcomes. What else can you expect if you don't or can't measure and test things first.

      Character is about making decisions and behavior and the most important one - that shapes everyone else's attitude, if that is what you are after - is what ground-rules are in place for the community. No better example than the Prisoner and Guards experiment to see what you are up against. Cooperation, in tit-for-tat, when the other person is cheating or has all the power makes you a doormat. Only a fool does it, not a smart kid.

      In your personal case, I would argue you changed because you saw that it was to your advantage. So, yes, show kids how a cooperative classroom, school, and society is better for them, collectively and long-term, than a dog-eat-dog, short-term gains one, and who wouldn't adopt it. Want good people, good society? Teach rational politics and economics.

      But if you can't show them that what you prefer is better, or it just isn't, if the rules aren't equitable, why should you expect anything less than the status quo. In fact, the status quo behavior in the face of current conditions, anywhere you look around the world, shows just how good human social instincts are at maximizing any given situation.


      Best

      Delete
  3. I think character should always be taught as part of whatever the regular curriculum using an assessment tool like Benjamin Franklin did with his 13 virtues, see http://en.wikipedia.org/wiki/Benjamin_Franklin#Thirteen_Virtues.

    I wish this message had been given to me in school by the teachers:

    "The most important thing you need to have in order to be a happy and successful person in life is good character. You don't need to know math, or English, or any technical subject. There are plenty of happy, successful people in this world who aren't very good at academics. But no one is truly happy and successful and a blessing to others if he does not possess good character. What is good character? It's a set of good habits that characterize how you live your life. It's habits like:
    - keeping your word,
    - being considerate of other people,
    - always doing your best work,
    - being straight-forward and honest,
    - keeping yourself under control, and
    - being clean and organized.

    No matter what academic subject you are studying, you can at the same time be learning good character. For example, while you are studying math, you can be practicing the habits of always doing your best, keeping your work and workplace clean and organized, finishing what you started even though it's hard, etc.

    And if you do this, if you approach your work here as a character-building exercise, then, it really doesn't matter what you end up doing after you finish school here. You will be a success because you will have good character."

    ReplyDelete
  4. This is called the crumb coat -- and for good reason, as
    much of the crumbs pull away and get mixed up in the
    frosting. But, before any of that takes place I'll have to get through the South’s summer monthsbeing pregnant. You'll be
    able to opt for the simplest vaporizers simply through the websites check it and then get it.


    Also visit my blog post :: volcano vaporizer
    My site > volcano vaporizer

    ReplyDelete
  5. The mist includes the required moisture that basically works in
    relieving child from symptoms by soothing irritated tissues in
    the mouth and the nose, reducing dryness and calming coughs.

    Wow, so many of us today know what a vaporizer is and we all have a handy
    dandy vaporizer with us. Buy one now from the leading vaporizer stores online.



    Also visit my web site; portable vaporizer
    my site: portable vaporizer

    ReplyDelete