Measuring Soft Skills

Op-ed

Measuring Soft Skills

by John Merrow on Jul 27th, 2010

(This post was co-authored with Arnold Packer.)

Reliability and Validity are the Alpha and Omega of testing. A test that is reliable can be counted on each time it’s given, while a valid test measures what it is supposed to. Tests that meet these two criteria are the gold standard of assessment..

Soft Skills For example, making someone swim 100 yards to test whether or not he can swim would be a valid and reliable test. If you sink, you flunk, and that’s true each time the test is given and is independent of who is doing the testing.

However, when teachers are trying to assess ‘soft’ skills, the waters get murky. How can we measure the ability to work with others, process information from disparate sources, communicate persuasively, or work reliably?

Consider the concept of reliability. Is an employee who is late to work one day most weeks reliable? What grade would you assign her, on a scale of 1-5? Suppose you found her explanations plausible (child care problems, for example) and you cut her slack—and gave her a 5– because you’ve been there yourself? However, another employer might give that same employee a grade of 2 or 3, because, after all, late is late.

There’s no set scale for measuring ‘working with others,’ meaning that the rating may vary depending upon who’s doing the rating. And what to one teacher is ‘persuasive communication’ may fall flat with another. There’s just no easy way to measure those all-important ‘soft’ skills.

(more…)

categories: Op-ed

7 comments

Schools and Cyberbullying

by John Merrow on Jul 21st, 2010

In late June Jan Hoffman of the New York Times explored the tough issue of cyberbullying and the schools. She led her provocative piece with an anecdote about parents asking their 6th grade daughter’s principal to intervene in a particularly difficult situation involving abusive and sexually suggestive email from a boy. They didn’t want to involve the police, and they knew the boy’s parents socially. The principal’s response was cut and dried: “This occurred out of school, on a weekend. We can’t discipline him.”

Cyberbulling At first I thought that was a legalistic, hair-splitting response—until I read about a principal who did get involved, was subsequently sued by the angry parent of the offending child, and lost. That’s horrifying, but it’s the reality.

My takeaway, however, is not that schools are right to split hairs and decline to get involved. Instead, I think we need some redefinition, some fresh thinking.

(more…)

categories: Op-ed, oped~below c level

10 comments

Will national standards ever arrive?

by John Merrow on Jul 14th, 2010

As I write this, close to half of the states have signed on to the draft of national standards, officially called the common core. Observers are predicting that well over half will be on board by summer’s end.

There’s a long way to go before we have genuine national standards in core subjects, and there’s no guarantee that they will be challenging enough, given the inevitable pressures to water them down.

And if we do develop worthwhile standards, some form of national testing is likely to follow.

The President of the United States is already on board for that. He said, “I believe we need some national standard education achievement tests—to be used only optionally when states and/or local school systems want them.”

Whoops, that wasn’t Obama; that was Jimmy Carter in 1977.

By the way, the public is on board. 77% of the public favors using national testing programs to measure the academic achievement of students.

Whoops, that was the Gallup Poll back in 1989.

And even teacher unions are supportive, judging from this quote from a powerful union president: “American public schools need a national curriculum to become competitive with school systems in other countries and to reverse plummeting public confidence in education.”

Pretty strong stuff, no? By now you are on to me and have guessed that Albert Shanker said that back in April1990.

I wonder how many of you know just how long the struggle for higher, more comprehensive standards has been going on? (more…)

categories: Op-ed

2 comments

Writing Below C Level

by John Merrow on Jul 7th, 2010

In my work for PBS NewsHour over the past three years, I am most often asked two very specific questions: “Is Jim Lehrer ever going to retire?” And “What is your personal opinion of Michelle Rhee? Do you like her and what she’s doing in Washington?”

To the first question my answer is always the same. ‘I hope not.” Of course I never answer the second question when I am asked, because it’s our job to report what we see happening, not express opinions or pass judgment. I do, however, have some thoughts on the subject, which you will find in Chapter 9 of Below C Level, pages 81-105. Yes, it takes 24 pages.

I spent five and one-half years writing Below C Level. The first drafts of many of the chapters were written on an airplane—I haven’t watched an in-flight movie for years—because my work takes me to distant places, and I have been living on the West Coast for the past eight years.

But, looking back with the first copy of Below C Level on the desk next to me now, I realize that the first five years were a walk in the park, relatively speaking. The last six months were without question the hardest part of the journey. During that time I rewrote every one of the 37 chapters. Once rewritten, it then had to find a place in the structure of the book, or go into the circular file.

Working closely with a brilliant young editorial assistant, Kristen Garabedian, we pretty much scrutinized every sentence. She religiously checked every fact and assertion. More than once she caught me repeating as fact some oft-repeated observation. That only 20% of households have school-age children is a shibboleth I have stated with confidence many times—but that turns out to be difficult to nail down—and therefore suspect.

I wrote Below C Level because I am genuinely worried about our country’s future. (more…)

categories: Op-ed, oped~below c level

2 comments

Summer Reading

by John Merrow on Jul 1st, 2010

Well, it’s finally here, the proof copy of Below C Level. I looked it over and then went on line to give my approval. That one keystroke made the book available immediately.

And if you want to be the first in your neighborhood to have a copy, please visit the Below C Level website or purchase it online directly at www.createspace.com/3422460.

When you do that, you will be getting a pretty good read and also helping Learning Matters–because I am donating most of the royalties to the company.

What’s in the book, you ask? Well, It’s 432 pages before you even get to the index, 37 chapters covering everything from pre-school through higher education.

A wry friend suggested that, if I wanted to sell a lot of books, I should just write about all the people I’ve interviewed over the past 34 years. An option, he said, was to tell the truth about them. I’ve actually followed his advice (including the option); the index is 11 pages of mostly names. (more…)

categories: Op-ed, oped~below c level

0 comments

What you didn’t see on television

by John Merrow on Jun 22nd, 2010

My colleagues and I have spent the past week or more putting the finishing touches on the last installment of our reporting from New Orleans and the Recovery School District there. In all, PBS NewsHour will have aired 12 segments about Paul Vallas and the RSD, and we also produced three other post-Katrina (pre-Vallas) segments. (Watch the full Paul Vallas series here.)

That’s 15 segments, each 8-10 minutes in length, a total of 2 hours of television, roughly. You might be interested to know what went into creating those two hours. Each piece generally entailed three days of shooting, perhaps 6 hours of videotape each day. That 6 (hours) X 3 (days) X 15 (segments) = 270 hours in all.

Our monumental task–15 times over–was to then take that raw material and edit and shape it into a short segment that would tell some part of the story of the effort to transform what was easily one of the worst school districts in the nation.

We produced more than our reports for PBS NewsHour: Each piece was accompanied by as many as four podcasts, usually longer interviews with Vallas, State Superintendent Paul Pastorek, various Teach for America teachers, parents, and so on. (Listen to the podcasts here.)

We’ve been doing the same job in Washington, DC, chronicling the efforts of Michelle Rhee to reform the schools there. We’ve made as many trips, shot as many hours of video, and spent as many weeks editing. We’ll present our final chapter from that city later this summer.

Did we get it right? (more…)

categories: Op-ed

1 comment

Game-Changing Innovation

by John Merrow on Jun 15th, 2010

Those of you who look at these posts with any regularity know that innovation is an interest of mine. I’m a fan of Clay Christensen’s observations about innovation being far more likely to succeed on the margins (where few are paying attention), but these days the margins in education are wide—because only math and English matter to the bean counters!

Innovation is education’s only hope. Even if more stimulus dollars become available, that will only put off the day of reckoning, unless educators wake up and act.

Happily, some are. Here are three examples.

I wrote recently about a charter school in Redwood City, California, that uses the last month of school (after the state tests) as a mini-semester in which kids take one elective class and, at month’s end, present their results to the entire school.

I just came from a Gates Foundation meeting in Seattle, where grantees shared ideas for shaking up and changing post secondary education, particularly the community colleges and their embarrassingly high failure rate. A community college in El Paso, Texas, goes into all the local high schools and gives juniors its placement exam. When the kids see the results, they’re likely to see the light—and work harder so they won’t have to spend their first year of ‘college’ taking remedial (high school) classes. The school districts are so keen on this that they are paying the costs of testing, the community college president told me. And they’re also developing curriculum that addresses the deficiencies revealed by the placement tests. In the end, this ‘experiment’ will reduce both failure and costs. (more…)

categories: Op-ed, oped~Higher Education, oped~charter schools, oped~reform, oped~technology

7 comments

Thinking out loud about remedial education

by John Merrow on Jun 8th, 2010

I’m in Seattle, where about 250 people, mostly grantees of the Gates Foundation, have gathered to talk about increasing the rate of success in higher education, especially among low-income students. My role is to lead a conversation with three individuals who represent three different slices of the industry: public community colleges, for-profit institutions, and on-line universities.

The Foundation was kind enough to suggest some opening questions, such as “How can we serve more low-income students better, faster and cheaper?” and “What do each of your institutions have to offer?” and “What will higher education look like 5-10 years from now?”

Maybe it’s because I am still medicated after my recent knee replacement surgery, but I find my mind wandering to other, perhaps more provocative questions. (more…)

categories: Op-ed, oped~Higher Education

6 comments

Solving a Man-Made Problem

by John Merrow on May 11th, 2010

One of education’s dirty little secrets is that schools give what they call their ‘end of the year’ tests about six weeks before the end of school. The school year is only 5/6th of the way done, but it’s testing time, and everything stops.

Think about that for a moment, maybe put yourself in the shoes of a teacher or a student. If you’re a kid, the message is clear: the year is over! Time to kick back and relax. However, if you happen to be a conscientious teacher, you have to climb a big hill every morning and afternoon for the next five or six weeks, because you have to try to interest your students in what they know doesn’t matter.

Left unexplored is what’s being tested. Do these tests cover everything that the students are supposed to have learned, or merely 5/6ths of the material? If they cover everything, isn’t that unfair to those who are being judged by the results (students and, increasingly, their teachers)? If they cover just 5/6th of the course content, will that mean that many students will never get past, say, World War II in history?

Why this happens is no mystery: it’s done for administrative convenience, to give the test companies time to run the test papers through their machines, process the scores, and get them back to the school districts as early as possible in the summer. With so many tests and so few companies, the bottleneck is frightening, worse than the lines at the toll booths on a summer evening when everyone is driving back from the beach.

What makes this bottleneck worse are the mistakes that occur. Imagine if the toll collectors randomly collected different amounts from some drivers! That’s what happens when the machines begin their mass processing: they make mistakes!

(It’s bad in another way: Districts used to be able to get a cheaper rate if they agreed to give the tests early, a ‘bargain’ that poorer districts took advantage of to save money. This harmed their students, who ended up taking the tests before they had actually covered the material they were being tested on. I recall being in a school in a poor district in Michigan during ‘end of year’ testing—in late February! That may well contribute to education’s infamous ‘achievement gap.’)

This mess is education’s equivalent of an iatrogenic (essentially ‘doctor-caused’) condition in medicine, meaning that we do it to ourselves. Since we created it, we ought to be able to prevent it in the first place. (more…)

categories: Op-ed, oped~charter schools, oped~reform, oped~teaching

10 comments

One Good Turn. . .

by John Merrow on May 5th, 2010

If you’re a reporter or any sort of education wonk, then you’ve been aware of FairTest, the shorthand name for the National Center for Fair and Open Testing. The folks at FairTest have been superb advocates, warning us of the dangers of mindless embraces of bubble tests and cheap standardized, machine-scored exams.

I believe we need FairTest more than ever. No Child Left Behind is still the law of the land and is likely to remain so for many months. As we move toward common standards, we need voices calling for multiple ways of evaluating schools, teachers and students. Those who believe that one score tells it all are sadly wrong, but they are also very powerful.

Sophie Sa, a friend of many years who is now FairTest’s Board Chair, reminded me that FairTest was created in 1985 “to fight against the misuses and abuses of high-stakes standardized norm-referenced tests … while promoting the use of multiple forms of authentic alternate assessments.”

I agree with Sophie that FairTest has been “the leader, the organizational glue, and the legs for individuals and groups concerned with the deleterious effects of standardized testing on students, particularly students in need; on schools and their curriculum; and on the kind of learning that emphasizes critical thinking and deep knowledge.”

Sophie and I go way back, to my own professional crisis back in 1994. (more…)

categories: Op-ed

0 comments

12 3 4 5...7 »

Learning Matters

Op-ed