Some Advice for Common Core Supporters

659315_5ba9794c89Yesterday at TWIE I gave CCSS supporters a hard time for seeming to give up the affirmative case for the new standards:

These days supporters seem to dedicate most of their time to assuring us that the CCSS are not to blame for “fuzzy” math curriculua or “whole language” or questionable history assignments. We are even told that it’s just as well if states opt out of the Common Core altogether because they’re unlikely to gain much from implementation anyway.

On Twitter Morgan Polikoff asked what I would prefer for CCSS supporters to do instead of making these half-hearted defenses. Fair enough, though I don’t really think of myself as part of the team so take all of this with a grain of salt. I’d recommend the following, which are arguably all variations on one or two themes:

1. CCSS supporters need to acknowledge that they overestimated the potential for standards per se to improve curriculum and instruction. Reformers are rarely comfortable wading into the minefields that are debates over C&I, but one of the lessons here should be that you can’t just lean on standards to do that work for you. Frankly, the science of teaching is just not sufficiently advanced and accepted that educators and families will all fall into line if you just give them the right goals to shoot for. Teachers already think their pedagogy is about right for whatever learning objectives you want to establish; if you want them to think differently you need to convince them directly. It is also increasingly apparent that you can’t avoid nasty battles over curriculum by saying “standards are not a curriculum”.  You may as well wage that war directly.

2. CCSS supporters should acknowledge that the new standards are not really as unambiguous as they had thought. By convincing themselves that opponents just don’t understand what the standards “really” say, supporters end up repeatedly missing the point. That tactic may work in a few cases, but whether, for instance, they encourage “fuzzy math” is just not something you’re going to be able to conclusively establish using nothing but the standards themselves. To the extent that teachers think the standards encourage fuzzy math, the standards do encourage fuzzy math. Supporters need to realize that while the CCSS are not “to blame” for peoples’ pre-existing edu-confusions, those confusions are really what we should be dealing with. As with #1, wage those wars directly.

3. CCSS supporters should focus more on Common Core-aligned assessments. What the CCSS “really” mean will be determined in large part by the tests used to hold teachers and schools accountable. So while it’s all well and good to assure us that, e.g., the CCSS “require” a “content-rich curriculum”, that won’t really be true unless the eventual assessments require a content-rich curriculum. I don’t think I’ve seen nearly enough attention paid to this and we’re already pretty far along in the assessment-design process.

4. CCSS supporters should spend more time highlighting “good” Common Core-aligned lessons. Opponents are already more than happy to talk about “bad” lessons;  supporters seem to have decided the best thing for them to do is join in. This is contributing to a growing sense – however unjustified – that the CCSS promote all sorts of educational silliness. That’s a recipe for further political losses and it means that educators aren’t getting the CCSS guidance they deserve.

Personally, I’m agnostic about the Common Core standards. People whose judgment I trust often say very good things about them. At the same time, the new standards are extremely expensive and disruptive and there does not appear to me to be nearly enough in the way of consensus among supporters about what the standards “really” say to make me think they’ll be good for curriculum and instruction on balance. For every supporter who tells me the CCSS “require a content-rich curriculum” there’s another who insists that the new standards “prioritize skills over knowledge”.

So I am not always encouraged. But there’s still plenty of time for “the good guys” to exert influence over CCSS implementation if they’re willing to start playing offense.

Posted in Education Reform | Tagged , | Leave a comment

VAM and Status Quo Bias

4011991883_8cc3a77d69_nYesterday at This Week in Ed I wrote about the American Statistical Association’s report on value-added modeling in education:

[I]f you were to hear about this report only from the staunchest, most ideological opponents of VAM, you would think it says something else entirely. Valerie Strauss, for instance, claims the report “slammed” the use of VAM to evaluate teachers and Diane Ravitch seems to think it is a “damning indictment” of such policies.

The report itself is not nearly so hyperbolic.

For a useful summary check out Stephen Sawchuk, but the report itself is a mere seven accessible pages so I encourage you read it yourself.

The bottom line for the ASA is that they are optimistic about the use of “statistical methodology” to improve and evaluate educational interventions, but current value-added models have many limitations that make them difficult to interpret and apply, especially when evaluating individual teachers.

There’s been considerable confusion about the report because some people seem to be having a hard time getting their heads around the possibility that VAMs have serious limitations but may nevertheless be appropriate for use in education generally or teacher evaluation specifically. 

So, for example, it’s true – as I was told over and over on Twitter – that the report states that VAMs “typically measure correlation, not causation” and “do not directly measure potential teacher contributions toward other student outcomes [besides standardized test scores]” and offer “scores and rankings [that] can change substantially when a different model or test is used”.

Crucially, however, none of this establishes that VAMs shouldn’t be used to evaluate teachers.

I mentioned this in the post, but it bears repeating: all methods of educational evaluation have limitations. That includes the methods we currently use! The trick is to identify and understand those limitations, and then select the best method (or combination of methods) for meeting our objectives.

It just so happens that the methods we use to evaluate teachers now consist mostly of classroom observations by administrators, usually employing a lengthy rubric with varying degrees of clarity and specificity. Those observations are by no means free of limitations.

In fact, it’s hard not to suspect that if, in an alternate universe, reformers were trying to move us away from widespread use of VAMs and toward an observation-based system, many critics would be up in arms about the fact that classroom observations are subject to many of the same limitations as VAMs (plus a few of their own).

Do classroom observations “measure causation” with respect to student outcomes? Do they “directly measure teacher contributions” toward student outcomes at all? Are teacher ratings based on observations stable even when evaluators and rubrics are changed?

As with VAMs, the answer to all of those questions about classroom observations is, of course, “no”. And yet we rely – heavily! – on these observations anyway.

Because classroom observations are a ubiquitous part of the status quo today, we take them for granted and their limitations don’t seem nearly so horrendous. VAMs, on the other hand, represent a departure from the status quo, which means that we’re much more likely to turn a critical eye toward them.

That’s a natural tendency, but it’s also basically irrational. The relevant question is not just, “do VAMs have limitations?”, but “compared to what?”

Posted in Education Reform | Tagged , | 6 Responses

#EduFictionMoviePitches

After watching the trailer for Scarlett Johansson’s new action flick last week, Marc Porter Magee and I agreed that the movie seemed perfectly acceptable until Morgan Freeman intoned that “most human beings only use 10% of their brain’s capacity”. I’m pretty forgiving when it comes to contrived movie gimmicks, but as an explanation for the main character’s superhuman abilities that’s just too goofy to swallow.1

The movie’s use of the “10%” myth got me thinking, however, of other education-related misconceptions that might make for good movie pitches, so I threw a bunch up on Twitter over the weekend. A few that I was especially pleased with:

There were also several good submissions from others:

You can check out all of them here. By all means, let me know if you think up new ones.

  1. I was particularly optimistic about this new movie because director Luc Besson has made some very good action/thrillers, including The ProfessionalThe Fifth Element, and La Femme Nikita. []
Posted in Education, Movies & TV, Pop Culture | Tagged | 1 Response

For Teachers: An Important Paper on Rewards and Motivation

If you ask a teacher about the virtues of giving students rewards for behavior, effort, or accomplishment, there is a very good chance that he will tell you about the dangers of “extrinsic motivators”. Specifically, he me may tell you that giving students rewards for doing something will undermine their “intrinsic” motivation to continue doing that thing in the future, once the reward is no longer offered.

This is an element of folk psychology among educators, but it’s not entirely without justification. Any good (especially progressive) school of education will show its teachers-in-training any number of studies that demonstrate just that danger of rewards. Certainly, my credentialing program did.

As many of my classmates were quick to point out, however, many of those studies seem to have limitations that call their external validity into question. The studies tend to look only at certain kinds of rewards, for example, and tend to involve incentivizing tasks that students are already motivated to perform. Real-world classrooms, we noted, have the potential to include a wide variety of rewards for students and often involve tasks that students do not find very interesting, at least initially.

Still, I had never really managed to find an analysis that called the conventional wisdom about rewards and intrinsic motivation into question. So I was very glad when Harry Webb pointed to just such a paper.

It’s called “Pervasive negative effects of rewards on intrinsic motivation: the myth continues“. In it, the authors meta-analyze 145 studies and summarize their findings this way:

Our results suggest that in general, rewards are not harmful to motivation to perform a task.

There is, however, quite a bit of interesting nuance, as the effects of rewards vary by context and type of reward. So, for instance, it matters a great deal whether the task being performed is already interesting. In fact, if a task is of little interest to begin with rewards seem to have the potential to actually increase intrinsic motivation:

When the tasks used in the studies are of low initial interest, rewards increase free-choice intrinsic motivation and leave task interest unaffected. This finding indicates that rewards can be used to enhance time and performance on tasks that initially hold little enjoyment…Our results suggest that reward procedures are one way to cultivate interest in an activity. In education, a major goal is to instill motivation and enjoyment of academic activities. Many academic activities are not of high initial interest to students. An implication of our finding is that rewards can be used to increase performance on low-interest academic activities.

And even when it comes to tasks that are already very interesting for people, the effect of reward depends on the type of reward and how it is administered. Verbal rewards – like praise – can still increase motivation, and so can some tangible rewards, provided that they are used strategically.

Conveniently, the authors include a chart that summarizes their findings fairly clearly:

click to enlarge

click to enlarge

Of course, as you can see from all of the minus (-) signs in that chart – signifying statistically significant (though often small) negative effects on motivation – teachers should probably still avoid tangible rewards for behaviors students are already motivated to exhibit or tasks students are already motivated to perform.

It is also very probable, however, that many teachers are currently under-utilizing rewards – tangible or otherwise – as tools for encouraging unmotivated students. And some non-negligible number of teachers are likely avoiding these rewards in part because the relevant research literature has been misrepresented to them.

Posted in Teaching & Learning | Tagged , | Leave a comment

For Reformers: An Important Paper on Worker Compensation and Incentives

7214510228_fa7685186d_nI’ve written before that education reformers often have an unfortunate lack of perspective about the way the world works outside of education. This means that reformers often unjustifiably assume – implicitly or explicitly – that their proposed changes would make education more like other sectors. This assumption, in turn, makes reformers’ proposals seem more intuitive and leads reformers to underestimate their potential disadvantages.

As a result, if I had to pick one study that I think all would-be education reformers should read, it would be a paper that I once found via Bryan Caplan. It’s an old paper – from 1988 – and it’s not even about education. Rather, it’s an examination of why most companies don’t use the sorts of compensation and incentive schemes that a simplistic understanding of economics might imply they do or should. Here’s the abstract:

A thorough understanding of internal incentive structures is critical to developing a viable theory of the firm, since these incentives determine to a large extent how individuals inside an organization behave. Many common features of organizational incentive systems are not easily explained by traditional economic theory-including egalitarian pay systems in which compensation is largely independent of performance, the overwhelming use of promotion-based incentive systems, the absence of up-front fees for jobs and effective bonding contracts, and the general reluctance of employers to fire, penalize, or give poor performance evaluations to employees. Typical explanations for these practices offered by behaviorists and practitioners are distinctly uneconomic-focusing on notions such as fairness, equity, morale, trust, social responsibility, and culture. The challenge to economists is to provide viable economic explanations for these practices or to integrate these alternative notions into the traditional economic model.

The authors then proceed to examine all of the ways and reasons that most businesses tend not to implement the sorts of incentive and compensation schemes that reformers typically recommend for schools and teachers.

Are you shocked that teachers are not paid differentially based on their performance? Perhaps you shouldn’t be:

Evidence from research on compensation plans indicates that explicit financial rewards in the form of transitory performance-based bonuses seldom account for an important part of a worker’s compensation.. Medoff and Abraham …examine the pay of managerial and professional employees in two large manufacturing firms and find little differences in earnings resulting from superior performance…

Lawler cites six separate studies of the relationship between pay and performance, and finds that “their evidence indicates that pay is not very closely related to performance in many organizations that claim to have merit increase salary systems. . . . It is particularly surprising that pay does not seem to be related to performance at the managerial level.”

Are you surprised at how few teachers are rated “unsatisfactory” by their supervisors? Again, perhaps you shouldn’t be:

[S]upervisors tend to assign uniform performance ratings and tend not to assign poor performance ratings. Only .2 percent of the 4,788 employees in Company A received the lowest rating; 94.5 percent were rated “Good or “Outstanding”. None of the 2,841 Company B employees received an “Unacceptable” or “Minimum Acceptable” rating, and only 1.2 percent received a rating of “Satisfactory”; 95 percent of the Company B employees are rated “Good” or “Superior”.

Does it seem odd to you that teachers are granted tenure? Perhaps you are beginning to sense a pattern:

Tenure systems appear to prevail in situations where human capital, creativity and an unstructured environment are particularly important in the production process, and where long lags between actions and the observation of outcomes make performance measurement and evaluation difficult…

Tenure, partnership, and up-or-out systems tend to be associated with relatively small organizations with few hierarchical levels.

Over and over again, this paper demonstrates that many of the existing teacher incentive and compensation systems that reformers often object to are in fact not that unusual in other fields. Since reformers seem not to be aware of this – and at times actively assume otherwise – they frequently fail to consider the reasons such systems are popular in other fields.

Fortunately, this paper explores many of those reasons, which goes a long way toward making it essential reading for anybody interested in education policy.

Of course, none of this is to say that any particular aspect of today’s education system couldn’t be usefully improved. Understanding how the world can be improved, however, requires first understanding why it is the way it is.

Posted in Education Reform, Teacher Compensation | Tagged , | 2 Responses

The Limits of Teacher Autonomy

2789011953_eb1bdc27e9_nThis post from Matt Bruenig has just enough educational implications for me to indulge some of my philosophical tendencies. Discussing Justin Green’s complaint about anti-discrimination laws infringing on personal freedom, Matt writes:

Now ask yourself this question: can people in the U.S. refuse to engage in private commerce with anyone for any reason?

The answer is clearly yes. If you do not want to engage in commerce with, say, a black person, you are not forced to. Nobody requires you to operate a hotel, a restaurant, or any other business. If you don’t want to serve a black person at your restaurant, you can refuse to do so by not opening or operating a restaurant. There is no legal penalty for that whatsoever.

We know of course what Green means. He doesn’t mean that people should be able to refuse to engage in private commerce with anyone for any reason (something they already can do). He means that they should have the affirmative ability to engage in private commerce without following the rules we establish for such engagements, in this case non-discrimination rules.

Right. You sometimes see similar arguments from teachers about their own autonomy. Consider this tweet about the Common Core standards:

I don’t actually know how to make sense of the idea that teachers shouldn’t be required to teach standards. What, exactly, are we hiring teachers for if not to teach specific content?

To wax metaphysical, though, note that the Common Core standards – like all previous standards – are in fact completely voluntary for all teachers. After all, any teacher who would prefer not to teach the CCSS is entirely free not to accept a job that requires them to teach the CCSS.

When teachers insist that you should “just let us teach” or complain that we “shouldn’t be forced to do X”, what they sometimes mean is that we should grant them the affirmative right to get hired for a job with a different description. That’s a very different, substantially less-plausible meaning.

Of course, in some cases teacher autonomy is a good thing. Teachers are often best-positioned to make decisions about what their students need or are able to do, for example.

But these are utilitarian considerations; there is no a priori point of principle about whether teachers “should” be “free” to make decisions about what their job entails because everybody is free not to teach in the first place. It’s entirely conceivable, for instance, that our education system would be better on balance if the Common Core standards were repealed or replaced, but the standards are still voluntary for teachers to the extent that nobody is legally required to be a teacher.

This isn’t a major issue in education debates, but every once in a while teachers can’t resist the urge to adopt the appealing rhetoric of “freedom” even though “freedom” is a morally-loaded concept only tangentially related to most educational issues.

Posted in Education | Tagged | 12 Responses

Musical Interlude – Ted Leo and the Pharmacists – Where Have All The Rudeboys Gone?

He’s moved in a somewhat different direction lately, but back in the day Ted Leo was a master of hooks, melody, and falsetto.

Posted in Music | Leave a comment

Educators: Don’t Assume A Can Opener

1282274161_d342cf26f7_nThere is a famous joke about the way economists often undermine the usefulness of their conclusions by making too many simplifying assumptions. Here’s one of the older formulations:

There is a story that has been going around about a physicist, a chemist, and an economist who were stranded on a desert island with no implements and a can of food. The physicist and the chemist each devised an ingenious mechanism for getting the can open; the economist merely said, “Assume we have a can opener”!

It’s probably not fair to pick on economists in this way when then the abuse of simplifying assumptions is at least as widespread in education.

For instance, arguably the trendiest thing going in education today is ‘grit‘: “the tendency to sustain interest in and effort toward very long-term goals”.

We all agree, I suspect, that a tendency to persevere is desirable, and that we should prefer that students have more of that tendency than less of it. So it is perhaps not surprising that since the term was popularized by researcher Angela Duckworth many teachers and schools have begun reorganizing their work to better promote and instill ‘grit’ in their students.

And yet, here’s Duckworth being interviewed by Alexander Russo last month:

Can you talk about how to teach grit in the classroom?
AI don’t know that anybody’s totally figured out how to teach it: What do you do exactly, even when we do have insights from research? How do you get your teachers to speak in ways that support growth mind-set? That’s why, through a nonprofit I helped cofound called the Character Lab, we’re organizing some lectures for teachers about self-control, grit, and related topics. It’s not totally prescriptive, because the science is still developing.

Not to put too fine a point on it, the world’s leading expert on grit is saying that educators who are substantially altering their work to better teach grit are doing so without much in the way of scientific backing or guidance.

In other words, in their excitement over grit many teachers and school leaders have simply assumed – without justification – that it is a trait that can be taught and that they know how to teach it.

This is by no means a problem limited to grit. Before grit it was “21st century skills“, “social-emotional learning”, “critical thinking”, or “scientific thinking”. What unites these fads is that they all, to varying degrees, suffer from a lack of rigorous scientific evidence indicating that they can be taught at all, let alone that we have reliable ways of teaching them in schools. (“Fluid intelligence” may be next.)

Meanwhile, we have good evidence indicating that schools today are reasonably – if imperfectly – effective at teaching kids the less-glamorous knowledge and skills – e.g., in math, science, and history – that we associate with “traditional” education.

So while it’s a good idea for researchers and educators to experiment with methods for teaching other, “higher-order” or “non-cognitive” abilities, it’s also important to remember that it is probably premature to ask schools to move away from their core competencies if we can’t also give them a clear alternate path forward.

Posted in Education | 3 Responses

Musical Interlude – They Might Be Giants – Road Movie To Berlin

De La Soul may have made catalog available to download for free, but TMBG really pioneered distributing music online. This appreciation of “Birdhouse in Your Soul” reminded me that Flood, like many of their albums, has held up much better than other releases of its era.

Posted in Music | Leave a comment

Me on BAM Radio on Teacher Turnover

2341098297_ba2935975dI made a brief appearance on Larry Ferlazzo’s BAM! Radio show to talk more about an upcoming piece about why teachers leave high-poverty urban schools.

Time was short, so I thought I’d collect here, in one place, many of the interesting data points I came across while thinking about the subject.

The best starting place for questions of teacher turnover is probably this report from the National Center for Education Statistics. They surveyed teachers during the 2007-2008 and 2008-2009 school years to look at teacher attrition and mobility during that time period.  They classified teachers as “stayers” if they stayed in the same school, “movers” if they moved to a teaching position in a different school, and “leavers” if they left the profession. Here’s what they found:

Context Stayers Movers Leavers
All public school teachers 84.5% 7.6% 8.0%
Charter school teachers 76.2% 11.4% 12.5%
City schools 84.5% 8.0% 7.5%
≥75% FRPL 84.6% 10.3% 5.1%

Interestingly, neither “city” schools nor lower-income schools have noticeably higher rates of turnover than schools generally. Those classifications don’t perfectly capture what’s going on in “high-poverty urban schools” – “cities” seem to be identified mostly in terms of population and FRPL eligibility is a very imperfect proxy for poverty – but they also don’t indicate that schools in poorer or more urban areas are experiencing noticeably higher rates of turnover than other schools. (I threw the charter school numbers in there mostly out of curiosity.)

It is also worth noting, however, that (via Matt Di Carlo) a more recent data set out of Washington, DC did find noticeably higher rates of turnover in lower-income schools in that district. It’s not clear whether DC is anomalous or if the DC data are in some way better.

The NCES also surveyed “leavers” – from public or private schools this time –  who worked in different contexts about why they left the profession. Excluding vague or more personal reasons (like pregnancy), there were some differences in how often teachers in different contexts cited as very important dissatisfaction with administrators:

Context Dissatisfied with administrators Lack of support from administrators
All leavers 12.8% 12.3%
City schools 17.2% 14.9%
≥75% FRPL 18.8% 18.1%

So teachers who leave teaching after working in city schools and lower-income schools seem significantly more unhappy with their administrators. (Remember, however, that teachers in these schools are nevertheless less likely to leave the profession than teachers in other schools.)

What about teachers who stayed in teaching but moved to a different school? Looking at public and private school movers together, movers cited the following factors as very important:

Context Inconvenient Location Dissatisfied with Admin Student Discipline Lack of Influence
All movers 32.7% 17.3% 11.5% 9.8%
City schools 22.7% 19.6% 14.4% 13.1%
≥75% FRPL 23.4% 21.5% 18.4% 14.8%

So teachers moving from city schools and lower-income schools are more likely to cite dissatisfaction with administrators, behavior issues, and lack of influence, but less likely to move for reasons of geography/convenience.

That NCES report also asked leavers what aspects of their new jobs were better or worse than at their teaching jobs. The five aspects of their new jobs that former teachers (in any context) were most likely to rate as better than teaching were:

  1. Ability to balance personal life and work (56%)
  2. Autonomy (53%)
  3. Recognition from managers (50%)
  4. Salary (47%)
  5. Opportunities for advancement (47%)

This doesn’t speak to high-poverty urban school turnover in particular, but it does speak to what teachers in general might be looking for when they leave the profession.

It also occurred to me that not only should we compare turnover in high-poverty urban schools to turnover in other schools, we should compare turnover in schools to turnover in other sectors.

3716520901_211c447865_nHere I’ve had a little trouble finding exactly the numbers I want. According to the NCES, between the 2007-2008 and 2008-2009 school years 15.5% of public school teachers left their job. Here’s one source estimating total turnover in all sectors in 2008 at 18.7%.

The Bureau of Labor Statistics put the monthly “separation” rate in 2008 at about 3.5% for non-farm jobs. I think that works out to an annual turnover rate of about 42% across all non-farm sectors, but I may be misunderstanding the calculation. If we look just at the “professional and business services” – a generally well-educated sector – the monthly separation rate in 2008 was closer to 5%. Separations are in general higher in the private sector than the public sector.

All else being equal, turnover is generally a bad thing to be avoided when possible. It’s not obvious, however, that high-poverty urban schools or public schools in general have a “turnover problem” compared to other professions. (I’d like to see an annual separation rate for college educated workers to get a clearer picture.)

Posted in Education | 1 Response