How Much Do Reformers Think Job Security Is Worth To Teachers?

7408506410_715acb5f6f_mA couple of weeks back at This Week in Education I tried to explain why the Vergara decision in California doesn’t have easily-predictable major consequences, even if you hand-wave away all of the inevitable legal wrangling and assume tenure and seniority rules  for teachers do end up changing significantly. Partisans really don’t like thinking – or at least talking – about trade-offs1 but they almost always matter in the long-term.

The upshot is that, even if you operate with an extremely naive model in which only student achievement outcomes matter, it’s not obvious that tenure reform2 will have large net benefits:

There will probably be some good effects and some bad effects of tenure reform, and much depends exactly on how the tenure rules are changed and how the state, districts, and schools respond. For example, will districts raise salaries in response to limitations on tenure? Will administrators find work-arounds to reduce energy spent on evaluations?

As a result, it’s hard to know which effects, if any, will dominate in the long-term. They may largely cancel each other out.

One of the central tensions for reformers when it comes to improving teacher quality is that on the one hand they believe teachers are fighting desperately for excessive job security but also, on the other hand, that you can substantially reduce that job security without making teaching significantly less attractive.

In theory this is not impossible. Making it work, however, requires admitting that job security is a benefit for teachers and that taking it away will - all else equal - make being a teacher less appealing.

How much less appealing? I think it’s very hard to say, but via Tyler Cowen, Lee Ohanian makes a (very rough) estimate [emphasis mine]:

I use historical data from the Bureau of Labor Statistics to identify the probability that a worker in the model is involuntarily separated from their job, which is about a 4 percent chance per month, average duration of unemployment, which is about 3.5 months, and the probability of continuing employment, which is about 96 percent.

For the case of the public sector, the probability of involuntary separation is just 1.3 percent, which is one-third as high as the probability in the private sector case. I then calculate the difference in compensation between the public sector (low unemployment case) and the private sector, such that a worker would be indifferent between working in either sector. I find that workers would be willing to work for about 10 percent less compensation in the public sector, given the additional benefit of much higher job security. This estimate is conservative in terms of considering today’s labor market, as average unemployment duration today is much higher than its historical average.

In other words, Ohanian thinks you could use job security as a means of attracting employees into the public sector even if you offered salaries roughly 10% lower than in the private sector because the job security itself has some value.

Now, Ohanian is a conservative economist writing for a conservative think tank, so he unsurprisingly concludes that you can do away with these job security benefits because public sector workers are so wildly overcompensated to begin with that the marginal value of the “excess” compensation is very small.

My sense, however, is that many education reformers – who are often left-leaning – don’t want to say that at all. On the contrary, they will often say that we want the “best and brightest” – i.e., significantly above-average workers - to go into teaching and that teachers should receive more compensation (contingent on performance).

The trouble is that, as Rick Hess puts it, courts are good at “access, not quality”. The Vergara lawsuit is ultimately a very crude way of enacting policy change; the decision does not require, for example, any measures to compensate teachers for reductions in job security.

Reformers may want salary increases but since they weren’t judicially mandated – and do not otherwise appear to be forthcoming - we have to consider a world without them.

I’m not opposed, in principle, to carefully paring back tenure protections for teachers in exchange for higher salaries or other benefits. I could therefore be made more sympathetic to many reformers’ projects if they seemed to be taking these trade-offs more seriously.

  1. At this StudentsFirst forum I submitted to the panelists a question about trade-offs, and was assured all questions would eventually be answered in person or online, but never got a response. []
  2. You can perform a similar exercise for changes to seniority rules. []
Posted in Education Reform, Teacher Compensation | Tagged , | Leave a comment

More Evidence of the Trouble with ‘Student-Centered’ Teaching

248447327_2f8a1c8249_nI’ve long had many related-but-separate complaints about ‘student-centered’ teaching practices. A new study in Educational Evaluation and Policy Analysis lends new evidence to several of them. (You can also check out good write-ups from Sarah Sparks and Bettina Chang.)

The authors used data on a large number of first grade students to see what strategies their teachers used to teach them math. They then looked to see whether teachers tended to use different strategies when students had stronger or weaker math skills, and then grouped these strategies together based on whether they would normally be considered ‘teacher-directed’ or ‘student-centered’.

Their methods – including an elaborate set of statistical controls for variables like student SES and prior achievement – also allowed them to make tentative causal inferences about which teaching strategies seem to be more effective for students who were stronger or weaker in math to begin with.

The results are mostly unflattering to student-centered approaches.

Student-centered Teaching Can be a Pedagogy of Privilege.

A couple of years ago I claimed that teaching methods typically considered ‘student-centered’ together represent a ‘pedagogy of privilege‘; such methods might be good – or at least good enough – for relatively strong students, but they often do not meet the needs of students with weaker skills.

The authors of this new study reach a basically similar conclusion, at least in regards to first grade math instruction:

Controlling for many potential confounds, we also found that only more frequent use of teacher-directed instructional practices was consistently and significantly associated with residualized (value added) gains in the mathematics achievement of first-grade students with prior histories of MD [i.e., mathematics difficulties]. For students without MD, more frequent use of either teacher-directed or student-centered instructional practices was associated with achievement gains. In contrast, more frequent use of manipulatives/calculator or movement/music activities was not associated with significant gains for any of the groups.

An important contribution of our work is that we find that teacher-directed instructional practices are associated with achievement by both students with a prior history of persistent MD, as well as those with a prior history of transitory MD. In contrast, other, more student-centered activities (i.e., manipulatives/calculators, movement/music) were not associated with achievement gains by students with MD.

In other words, the most fortunate students will manage one way or the other but the less fortunate kids are not well-served by student-centered approaches.1

Student-centered Teaching is Attractive in Low-Skill Settings

Despite their inappropriateness for struggling students, I’ve also hypothesized that student-centered approaches may - paradoxically – be more favored when students have fewer or weaker skills.

My guess was that student-centered approaches can obscure skill gaps, which tend to be more salient in low-skill classrooms. When students are mostly proficient or advanced, teachers, administrators, and parents tend to have plenty of independent verification that students are skilled; ambiguous, student-centered activities are not relied on for demonstrations of mastery. With lower-skilled students, adults are more likely to be worried about their students’ skills, because much of the available evidence (e.g., test scores, independent classwork) suggests those skills are absent or weak. When students engage in student-centered activities, they can easily give the illusion of proficiency – talking to one another, handling materials, and so on –  especially if you don’t examine their work too closely or don’t know what you’re looking for.  And it’s easy to interpret ambiguous evidence of learning favorably if you really want to see proficiency (as most educators do).

This new study finds evidence consistent with my theory, at least for some student-centered teaching strategies:

We found no significant relation between the percentage of MD students in the classroom and the frequency of teacher-directed or student-centered instructional activities. However, we did find that…classes of students with higher percentages of MD students were more likely to be taught these skills and with instructional practices emphasizing using manipulatives/calculators and movement/music. As reported below, these instructional activities…were not associated with mathematics achievement gains by students with MD.

Regardless of the reason, however, it seems that teachers are choosing to use less effective methods especially with those students who need the most help.

Student-centered Teaching is not Obviously Research-based

Of course, if you ask adults who favor student-centered methods, they will very often say that those methods are ‘research-based’. There is some sense in which this is true, at least to the extent that you can find seemingly-reputable education studies to support almost any instructional decision.

The trouble is that a great deal of education research is ideologically-motivated, and well-controlled studies of instructional effectiveness are difficult to perform in any case. So how strong, really, is the research base for student-centered teaching?

This new study suggests it is probably not as strong as it is often made out to be:

Some types of instructional practices are commonly considered “evidence-based,” and so presumably their use by teachers should result in increased mathematics achievement. For example, Baker, Gersten, and Lee’s (2002) synthesis of researcher-directed intervention studies yielded a weighted ES of .66 for the use of structured peer tutoring on low-skilled children’s mathematics achievement. Additional syntheses also support peer tutoring as an evidence-based practice (Elbaum, Vaughn, Tejero, & Watson, 2000; Mathes & Fuchs, 1994). Yet our estimate of student-centered instruction, which includes peer tutoring, was statistically non-significant when used with students with prior histories of MD (Guarino et al. [2013] also reported a statistically non-significant finding for peer tutoring).

The authors suggest this might be related to implementation fidelity problems with student-centered approaches, and I suspect that’s a factor.2 It’s also possible, though, that much of the underlying research is just not as strong as we’d like to begin with.

To be clear, nothing here demonstrates that any particular ‘student-centered’ approach doesn’t have its place, even potentially in classrooms with large numbers of struggling students.

This study is, however, more evidence that many traditional, ‘teacher-centered’ approaches are often unfairly-maligned and under-utilized.

  1. In fact, calling those approaches ‘student-centered’ at all seems presumptuous. []
  2. This is not exactly a ringing defense of student-centered approaches; if they are harder to implement, so much the worse for them. []
Posted in Teaching & Learning | Tagged | 6 Responses

Explained Variation Is Not A Measure of Importance

246717376_aa7e238d67Back in early April the American Statistical Association put out a “Statement on Using Value-Added Models for Educational Assessment“.

Last month, Raj Chetty, John Friedman, and Jonah Rockoff issued a response, in part because so many commentators seemed to misunderstand the ASA statement and in part because the ASA seemed not to have incorporated some of Chetty et al.’s most recent research.

Diane Ravitch’s unimpressed follow-up involves a few all-too-common misconceptions:

What do Chetty, Friedman, and Rockoff say about the ASA statement? Do they modify their conclusions? No. Did it weaken their arguments in favor of VAM? Apparently not. They agree with all of the ASA cautions but remain stubbornly attached to their original conclusion that one “high-value added (top 5%) rather than an average teacher for a single grade raises a student’s lifetime earnings by more than $50,000.” How is that teacher identified? By the ability to raise test scores. So, again, we are offered the speculation that one tippy-top fourth-grade teacher boosts a student’s lifetime earnings, even though the ASA says that teachers account for “about 1% to 14% of the variability in test scores…”

The argument is that if teachers account for only a small fraction of the variation in student test scores, teacher quality is probably not a useful lever by which we can improve education outcomes.

This is wrong for at least three reasons.

First, to know whether 1%-14% is a lot of variation to account for we have to compare teachers to something else. It’s not entirely clear from her post, but Ravitch1 seems to want to compare teachers to all other factors put together, but that comparison tells us very little. The 86%-99% of the variation in student test scores not explained by teachers is not explained by a single other factor for us to focus all of our policy energy on; it’s an aggregate of a large number of factors, each likely accounting for a much smaller fraction of the variation.

Second, even if some factors explain more variation in test scores, that doesn’t mean we have to pick just one factor to care about. We may want to prioritize, say, poverty reduction over teacher quality improvements, but that doesn’t mean only the former matters.

Third, and most fundamentally, variation accounted for by a factor is not a measure of that factor’s importance. The ASA statement actually points this out somewhat obscurely:

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores.

The fact that this was included in the ASA statement has not prevented considerable confusion; VAM critics have latched on to the first sentence, but seem not to understand the significance of the second.

Let’s unpack that second sentence.

If a factor doesn’t explain much of the variation in student test scores, that could mean that the factor is relatively unimportant and that even large changes in that factor would not have significant effects on scores.

Another possibility, however, is that that factor doesn’t systematically vary between students.

Consider “access to breathable oxygen”. If you crunched the numbers, you would likely find that access to breathable oxygen accounts for very little – if any – of the variation in students’ tests scores.  This is because all students have roughly similar access to breathable oxygen. If all students have the same access to breathable oxygen, then access to breathable oxygen cannot “explain” or “account for” the differences in their test scores.

Does this mean that access to breathable oxygen is unimportant for test scores? Obviously not. On the contrary: access to breathable oxygen is very important for kids’ test scores, and this is true even though access to breathable oxygen explains ≈0% of their variation.

Now let’s return to the importance of teachers. If teachers account for only a small fraction of variation in student test scores, that may mean that teacher quality is largely unimportant. It may also mean that teacher quality does not vary systematically very much between students.

Another way to think of it is this: if every teacher was exactly as effective as every other teacher, teachers would account for exactly 0% of the variation in student test scores. This would be true regardless of whether these imaginary teachers spent the entire school day reading the newspaper or if they successfully taught advanced calculus to 3rd graders.

In other words, determining statistically how much variation is “explained” by teachers will not, by itself, tell you how important teacher quality is.

This is precisely where research like that of Chetty et al. comes in. It attempts to go beyond simple measures of “explained variation” to quantify teachers’ actual importance and impact.

We can still reasonably disagree about what that research tells us about “the importance of teachers”. What you can’t reasonably do is dismiss that research out of hand using measures of explained variation, as those are not direct measures of importance.

  1. Ravitch is by no means the only one who makes these mistakes, but she’s usefully illustrative here. []
Posted in Education Reform | Tagged | 2 Responses

My Twitter Reactions to the Vergara Ruling

We are already being inundated with analyses of what yesterday’s Vergara ruling “means”, so rather than write up yet another one I just compiled my thoughts from Twitter:

Posted in Education Reform | Tagged , | Leave a comment

There Is Probably No “Crisis” In American Education

Here is a chart of educational attainment in the United States since 1940:

Educational_Attainment_in_the_United_States_2009

When you look at that chart, do you see a crisis?

No? Me neither.

How about in these charts of reading and math achievement on the NAEP for 17-year-olds, broken down by race?reading17

math17

Still hard to see a crisis, at least to my eyes.

Certainly, it’s fair to say that American education has many problems related to effectiveness, efficiency, and equity.

And it’s probably reasonable to say that our country and the world face a number of bona fide crises – war, climate change, poverty, or criminal justice, for example – each  with some relationship to education, however complex or indirect.

But next time you hear someone claim – or are yourself tempted to claim - that American education is in a state of “crisis” or is being “destroyed” by education reform, remember these charts. Then ask yourself whether those terms are being used productively or whether they are being defined down in a way that obscures as much as illuminates.

Posted in Education Reform | Tagged , , , , | 16 Responses

Are the Common Core Standards Voluntary?

50323994Did states adopt the Common Core standards voluntarily, or were they forced to do so by the federal government?

I’m not sure why we would think the answer to that question matters very much. If adopting the CCSS would be good for a state, then their adoption by that state would be good. If adopting the new standards would be bad for a state, then the state shouldn’t adopt them. Information about how the standards are (or are not) adopted seems neither necessary nor sufficient to determine whether adoption should proceed.1

Still, suppose we’re really interested in whether or not the standards were “voluntary” for states. And let’s suppose, for the sake of argument, that it makes sense to talk about states doing things “voluntarily”, a substantial assumption given that “freedom” is a difficult concept to apply even to individual people.

So, were the standards voluntarily adopted by states? There is a prima facie case that they were, even if the Obama administration ended up supporting them. As Michael Petrilli says:

These standards started out as a state effort, with support from private entities like the Gates Foundation. It was the governors and state superintendents who came together, voluntarily, to draft higher common standards, because they acknowledged that their own state standards were set too low. There was already momentum behind the standards when the Obama administration intervened.

Along with the fact that not every state has adopted the CCSS, this is basically dispositive. If states mostly supported the standards prior to the Obama administration’s endorsement, it hardly makes sense to talk about the federal government “coercing” the states into adoption. If Arne Duncan hadn’t gotten involved it’s possible that some states may ultimately have decided not to adopt the standards they had helped to create, but “a few states were marginally more likely to adopt” hardly seems sufficiently coercive to justify the resulting libertarian outrage.2

Libertarians, however, have been outraged. Why? Here’s Neal McCluskey:

From the outset of the Obama administration, officials talked about a need for national standards, and under the mammoth 2009 “stimulus” they got a lever by which to push that: the $4.35 billion Race to the Top program. To fully compete for Race to the Top money states had to adopt standards common to multiple states, and only one set of standards fully met the definition: the Common Core.

This argument will baffle most people. How is being offered money to do something “coercive”? Presumably McCluskey is offered monetary compensation of some kind for his work with the Cato Institute; has he therefore been “coerced” into working there?

Most people, however, are not libertarians. McCluskey sneaks his premises in a little later in the argument:

Adopting the Common Core was, in principle, no more voluntary than having a mugger take your money — [state] taxpayer money — then let you “voluntarily” hand him the keys to your car to get the dough back.

Wait, what? McCluskey thinks paying your taxes to the federal government is morally equivalent to being mugged? You could be forgiven for thinking you misunderstood, but here he is replying to me last week on Twitter:

This is the classic anarcho-libertarian line on government: that its activity necessarily involves forcing people to do things they would prefer not to do and is ipso facto illegitimately coercive.3 Because this is such a fringe view many CCSS supporters have had a hard time understanding why anybody would consider, e.g., Race to the Top to be “coercive”. The answer is that some people think taxation is essentially theft.

Without getting too far into the weeds here, it’s worth mentioning why most people think being anti-coercion doesn’t mean that taxation is theft. First, “taxation is theft” strikes most people as a reductio ad absurdum of naive definitions of “coercion”. If you are using “coercion” in such a way that you have to conclude that taxation is the moral equivalent of armed robbery, something is probably wrong with your use of the word.

Second, most people believe that taxation is justified in one way or another. This may be because they view taxes as “the price we pay” for the various (typically larger) benefits of membership in society. Or it may be because they realize that your possessions and what they are worth are determined to a large extent by our social context, including our governmental institutions. If, e.g., the nominal and real value of your wages is a function of our social (and governmental) institutions, it becomes very difficult to discern how much of them (if any) is “yours” in a morally fundamental sense.4

So it is safe to say that the federal government did not coerce states into adopting the Common Core standards by offering  money to do so. But if carrots aren’t coercive, what about sticks? It is probably fair to say that sticks were in play:  

We’ve dispensed already with the “Fed $ from taxpayers under force of law” issue, but NCLB waivers appear to be somewhat different.

No Child Left Behind, you will recall, requires that states set learning standards for basic skills – reading and math, mostly – and get ever-increasing proportions of students over related proficiency thresholds. Over the years the requirements of NCLB have become more difficult for states to meet as proficiency targets have risen and Congress has failed to modify the legislation. As a result, the Obama administration offered states “waivers” to avoid many of NCLB’s accountability provisions in exchange for adopting other various reforms including, in many cases, the CCSS.

A somewhat more plausible case for federal coercion is this: States that did not adopt the CCSS would have a much harder time getting a waiver from NCLB, and without a waiver they would be subject to increasingly harsh penalties for failing to satisfy the law’s requirements.

Here, too, the libertarian arguments mostly don’t work.

First, it was possible to get a waiver without adopting the CCSS (as Virginia did) or just skip the standards altogether (as several states have).

Second, the provisions of No Child Left Behind are substantially voluntary. This has been largely forgotten, but states have several options for avoiding NCLB’s accountability penalties. NCLB allows states to set their own standards and proficiency levels, for example, and there are various ways of making sure your schools are not actually held to the highest proficiency standards. NCLB, moreover, is just another carrot: required only for states that wish to receive federal education funding.

In other words, when confronted with the possibility of adopting the Common Core standards, states actually had a substantial amount of flexibility.

There is arguably a naive sense in which states were “coerced” into adopting the CCSS in that some states may not have been completely satisfied with any of the options available to them. But by this standard virtually everything is coercive to one degree or another and the term is rendered meaningless for normative purposes. That conventional, lay-person use of “coercion” may be practically useful on a day-to-day basis, but it has no philosophical precision so it can’t do any philosophical work.

And this, I think, is why the debate over whether the CCSS are “voluntary” has been so intractable. The question is a philosophical (and possibly incoherent) one, but the parties involved - especially libertarian opponents – insist on using notions of coercion that are so imprecise (or loaded) that they can’t possibly settle it one way or the other.

  1. Of course, we should prefer to have a set of background institutions that effectively distributes power between different levels of government. Those concerns, however, are mostly neither here nor there when evaluating the merits of CCSS adoption since the standards have only minor – if any - implications for our background institutions. []
  2. Note also that the Obama administration’s belated involvement in the adoption process has been so polarizing that it’s not obvious the net effect was pro-CCSS in each individual state. There was a point at which CCSS supporters started asking them to back off the advocacy precisely because it seemed to be backfiring. []
  3. McCluskey claims not to endorse the anarcho-libertarian view, but it’s hard to know how else to interpret an analogy between taxation and armed robbery. []
  4. We have socially and economically useful private property institutions, but they depend heavily on coercion, especially government coercion, themselves. []
Posted in Education | Tagged , | Leave a comment

Everything is Untested Until You Test It

9486459224_e1517c1dab_nTeacher tenure is a hot topic in California these days thanks to the Vergara trial, so it was newsworthy when the San Jose chapter of the California Teachers Association asked the State Board of Education for a waiver from state law extending the probationary period for some teachers using a system of peer review. They didn’t get it:

The State Board of Education on Thursday denied the San Jose Unified School District and its teachers union their request for the authority to require some probationary teachers in the district to work an additional third year before becoming eligible for tenure.

Granting a one-year waiver from state law, Jennifer Thomas, president of the San Jose Teachers Association, had argued, would send a message that “a union can be an incubator of innovation in pursuit of educational excellence.” Board member Carl Cohn, a retired superintendent of Long Beach Unified, agreed, urging the board to encourage the cooperation between a teachers union and a district reflected in the request. “Sometimes it is easier to teach kids to read than to get this kind of trust,” Cohn said.

But with its 7-2 vote, the majority sided with the California Teachers Association and the state Department of Education’s position that the district and the union should ask the Legislature to change the tenure law or grant it an exception. Waivers, said board member Sue Burr, should be narrow and limited to circumstances involving individual teachers, such as a teacher who went on maternity leave while on probation. “For better or worse, we’re being asked to waive a fundamental personnel protection,” she said.

One of the big disappointments here is that San Jose Unified and its teachers union were cooperating(!) to try something new and different and it’s state law – not mutual mistrust – that’s preventing them from moving forward. The proposed experiment wasn’t a big one in absolute terms – it’s unlikely that many teachers would have their probationary periods extended in any case – but it involved grappling with hot-button issues of teacher evaluation and tenure that rarely receive even that level of experimentation.

One of the lines often leveled against education reforms is that they are “untested” and therefore shouldn’t be imposed on schools because we don’t have a good sense of what the effects are likely to be.

In many cases it’s true that a proposed reform is untested, but note that this can often be avoided by letting schools and districts test them.

Of course, it may not always make sense to give local education agencies flexibility on every issue. But the disappointment you see expressed over the San Jose case is due, in part, to the sense that it should be possible for the district, especially in collaboration with the union, to perform modest experiments with tenure and peer review.

Much has been made of the fact that the CTA lobbied against the local chapter’s request, but it’s not hard to understand why the Board would be reluctant to waive away the legislatively-established two-year probationary requirement. Arne Duncan’s NCLB waivers have been harshly criticized in some circles as bureaucratic overreach, and that’s a situation in which virtually everyone agrees that the law being circumvented is long overdue for revision. There is less consensus about California’s tenure laws, so it’s natural that the Board may not want to pick that fight with the legislature.

So while the outcome of this decision is frustrating, the underlying issue in this case – and likely many others – is that the laws themselves may need to be loosened to some degree to allow reasonable experimentation at the local level.

And if we can’t test new ideas at all, it’s not clear how we make progress.

Posted in Education Reform | Tagged , , , | Leave a comment

Why Education Reform is Probably Not The Best Way to Fight Poverty

12383520_78bc1448e2_nDoug Lemov is skeptical that I’m right about education being a (relatively) ineffective way of fighting poverty. His response is thoughtful and deserves a response of its own.

First, it’s worth being clear that the only claim I’m prepared to advance and defend is that education reform is not the best way to fight poverty in the United States today. This is entirely compatible with the ideas that (1) education might be modestly effective at fighting poverty, even if only indirectly, and (2) improving education is worthwhile for reasons entirely unrelated to poverty. In fact, I believe both of those things.

The problem I have is with the contention – made by many prominent reformers – that education is the best way to fight poverty. And yes, this is often what they say.

Consider, for example Michelle Rhee, who claims that when it comes to poverty, “teachers can be the answer for so many kids”; otherwise you are going to “write a generation of children off”. Arne Duncan apparently thinks that “the only way to end poverty is through education.” Mike Petrilli claims that “schools are the indispensable anti-poverty program”. He also, curiously, thinks that poverty is “(mostly) not about money”, which is both (mostly) false by definition and an implicit dismissal of money-based methods of fighting poverty.

It’s that denigration (implicit or otherwise) of other poverty-fighting programs that’s the problem. If education is the best way to fight poverty – or the only way – then that means we should worry relatively less about other methods. Here is Rhee again explicitly disparaging other ways of fighting poverty, saying “the best tool that we have to fight intergenerational poverty and break that cycle of poverty is education. It’s not a particular social program or this or that.”

So I object to the claim that education is the best way to fight poverty (in the United States) because (1) we should not in general claim things that are false and (2) we should not in any way discourage people from supporting other, more effective methods of fighting poverty.

With all of that being said, let’s look at Doug’s defense of education as a poverty-fighting measure and why I find it unpersuasive, at least on this narrow question of relative effectiveness.

My Original Post

First, Doug is absolutely right that my international comparison in my original post does not in any way prove that education is unrelated to poverty. All I did was show that countries with higher PISA scores today also tend to have higher pre-tax/transfer poverty rates. There would be lots of ways of running that analysis more persuasively, but my goal was really just to try to prod people into making the affirmative case for education as a poverty-fighting tool.

Part of the issue, here, is that I view the claim that education is the best way to fight poverty as radical, and therefore requiring substantial evidence. I would like to see reformers make the case that all of the other developed countries that have dramatically reduced their poverty rates have done so through education, but I don’t think I’ve ever even seen that attempted.

If we want to ignore my graph, that’s fine, but let’s also be clear that strong claims about education reform’s poverty-fighting power have not (yet) been supported with much international evidence.

On the other hand, I did also point out that domestically it is noteworthy that decades of improving educational outcomes have not done much to reduce poverty rates. I think that’s a more substantive point that I’d like to see addressed. There’s a lot more to say about it – for both sides – but it’s a good starting place (and one I’ll briefly return to below).

Doug’s Affirmative Case

Doug’s affirmative case for education as a poverty-fighting measure rests primarily on three arguments. First, that educational outcomes strongly predict economic growth; second, that the individual economic benefits of more or better education are substantial; and third, that we should worry less about relative measures of poverty and more about absolute measures of poverty.

Education and Economic Growth

Doug bases his case for the importance of education to economic growth on Eric Hanushek’s analysis finding that

If one country’s test-score performance was 0.5 standard deviations higher than another country during the 1960s—a little less than the current difference in the scores between such top-performing countries as Finland and Hong Kong and the United States—the first country’s growth rate was, on average, one full percentage point higher annually over the following 40-year period than the second country’s growth rate.

Doug continues:

That may not seem like much, but, as Hanushek notes, world economic growth averages about 2 to 3 percent of GDP annually, so a difference of a full percentage point in growth is massive.

I, for one, have little trouble believing that education is correlated with economic growth. While his analysis is in many respects interesting, however, there are lots of reasons to doubt that Hanushek has found useful causal phenomena of the magnitude he suggests.

Hanushek admits that “[o]ther economic research has identified two additional factors that affect a country’s economic growth rate: the security of its property rights and its openness to international trade.” This highlights the fact even though economic growth is likely a complicated process dependent on many factors, we understand very few (only two?) of those factors. This makes it unlikely that it is currently possible to adequately control for all of the relevant factors to identify the “true” explanatory power of education.

Additionally, explanatory power in a correlation is not the same as magnitude of causality. Correlation is not causation, etc. I suspect that the correlation Hanushek identifies reflects some causality in the direction he and Doug expect, but our certainties about how much causation runs in each direction should be quite low.

More importantly, there are at least two reasons that Hanushek’s analysis does not speak directly to the question Doug and I are discussing (viz., whether education is the best way to fight poverty in the U.S.).

First, when we talk about “the importance of education” in the United States, we are really talking about the importance of education reform. We already have an extensive education system which means we are already reaping a substantial fraction of the potential benefits of education. The question is about the extent to which we can additionally improve our education system, and we can’t just assume we’re going to be able to get anywhere near the sorts of gains that Hanushek uses to illustrate “the importance of education” for economic growth. Those are hypothetical numbers, not options for us to choose off of a public policy menu.

Second, and as I mentioned to Doug on Twitter, economic growth is not the same thing as poverty reduction. Economic growth means increases in productivity and wealth, but poverty reduction is about the extent to which those economic gains flow to, and improve the material well-being of, the poorest people. To see why it’s a mistake to conflate economic growth and poverty reduction, these graphs of inflation-adjusted median household incomes are a good place to start:

click to enlarge

click to enlarge

Median household data can be misleading – e.g., family sizes have shrunk, inflation measures are imperfect, and people can move between income quintiles  - but this is not a graph of an economy where the benefits of economic growth are flowing proportionally to the poorest households. During the time period examined in that graph, real GDP roughly tripled and per capita GDP roughly doubled, but our economic “floor” budged fairly little.

And lest you think that some sort of Simpson’s paradox is obscuring substantial gains by subgroups, consider how the household income numbers break down by race:

click to enlarge

click to enlarge

Again, remember that this covers a time period of well-documented academic gains for most subgroups of students and for Black and Hispanic students in particular, but their incomes have not obviously risen accordingly. There are ways to slice the data that make the economic gains look somewhat better, but you have to slice them pretty fine to make it look like educational improvements are contributing to substantially reduced rates of poverty as a result of economic growth per se.

The Individual Returns to Education

Here’s Doug:

In the US, I reminded myself, not finishing high school or finishing high school with skills that don’t allow you to go on to college is one of the fastest way to increase your likelihood of ending up in poverty. (Would anyone dispute that? The trend in fact has steepened recently as middle class industrial jobs erode and “knowledge work” is the overwhelming way to the middle class and above).

Make no mistake: education seems to have positive, causal effects on earnings. Frankly, I find that literature daunting and my understanding of it is limited.

For present purposes it will suffice to say that however substantial the individual returns to education may be, they do not appear to be putting a major dent in our poverty rates. I would guess various factors are at play. For one thing, the returns to education are heterogeneous. Consider, for example, that a substantial fraction of recent college graduates earn less than the average high school-only graduate.

So while college is lucrative for the average student – and very lucrative for some fraction of graduates – a great many college graduates appear to be reaping much smaller economic gains from their degrees. If this is particularly true of marginal college students – those who would not have been ready for college at all if not for reform and who are perhaps less likely to enter the most lucrative fields – the poverty-fighting power of additional education may be limited.

Second, large shifts in overall levels of education may have effects on individuals’ returns to education. If, for example, the supply of high school graduates increases, that may put downward pressure on wages for high school graduates or drop-outs. This sort of dynamic effect would (I think) further reduce the impact of education on poverty at the national level.

Finally, it is worth remembering, again, that we are considering the effects of education reform, not of education as such. The standard line among education reformers today is that the best route to improving educational outcomes is through improvements in teacher quality, but you apparently have to improve a teacher’s quality by a full standard deviation to get a student even a 1% income boost at age 28. That’s not nothing, but it’s also a purely theoretical hypothetical; it’s not the kind of return that gives us much reason to think education reform is going to be the best way to fight poverty.

Measures of Poverty

Here’s Doug again:

My understanding is that Paul may have used GINI as a proxy for poverty but according to the World Bank, GINI is a measure of income distribution rather than of poverty. And this could be a major confounding factor. For example, roughly speaking, being in the bottom quintile of incomes in the United States makes you “poor”…relatively speaking. I do not intend to mitigate the suffering or difficulties that come with that poverty. (In fact, I work primarily with high poverty schools because i care so much about it) but being in the bottom quintile in the US would not make you poor in India or El Salvador.  Your income and quality of life would be far above the mean.  So your opportunity and quality of life even at the bottom of the economic hierarchy in the US is far better than that of people far higher on a relative ranking of poverty within other countries. So if i improved my nation’s schools and GDP growth went up steeply, everyone would become more prosperous and better off and poverty would be mitigated but my country’s GINI scores might not change. In fact they’d be just as likely to become more pronounced–poverty and income distribution are different things.

To clarify, the measure of poverty I used in my original post was the “poverty rate before taxes and transfers” with the “poverty” line set at 50% of each country’s median income. This is not technically the same as using Gini though most of the differences are unimportant here. Doug is correct that the measure of poverty I used is defined in some sense relative to each country’s overall wealth and that in absolute terms the poorest Americans are often better off than the median residents of much poorer countries.

Still, I think my chosen measure is preferable.

The fact that my chosen poverty measure is in some sense relative is a feature, not a bug. As far as I am aware all conversations about education reform in the United States are about improving welfare in the United States. Whether the poorest Americans are richer than the average residents of poorer countries is therefore immaterial. In a hypothetical conversation in which I have to choose between improving the absolute well-being of the poorest people in the world and improving the absolute well-being of the poorest quintile in the United States, I will likely prioritize the former. In our actual conversation about fighting domestic poverty, however, it seems not just useful but essential to factor in our overall level of wealth when thinking about what is possible or acceptable.

Also, even if we prefer to ignore our considerable national wealth when evaluating how effectively we are ameliorating suffering for our poorest citizens we still come out poorly in international comparisons. Here, for example, is a comparison of international child poverty rates measured both in relative (median income) and absolute (PPP) terms:

click to enlarge

click to enlarge

Note that even using this more “comparable” measure the child poverty rate in the U.S. is still double or triple that in many other developed countries.

Compared to What?

The effectiveness of an intervention should always be understood in comparative terms. If my claim is that education is probably not the best way to fight poverty, that must be because I think there are better ways and I have been remiss in not stating them explicitly.

We know that the United States has already significantly reduced its poverty rate by collecting taxes and then transferring the money to less-fortunate families through tax credits and benefits:

dajoi-fig-1

We also know that it is possible to have substantially lower rates of poverty than we have in the United States because other countries have successfully reduced their poverty rates well below ours. (Remember, this is true regardless of whether we are measuring poverty in relative or absolute terms.)

How have these countries achieved these reductions in their poverty rates? For the most part, they do it the way we’ve done it: via government redistribution (i.e., taxes and transfers). Here is a comparison of OECD countries’ pre- and post-transfer poverty rates:

click to enlarge

click to enlarge

This is what education reform is competing against for the title of “best way to fight poverty”.

It will not do here to say that tax-and-transfer programs are less effective by hand-waving away their political viability. Their benefits are substantial in magnitude, we’ve implemented major programs in the past, and many programs today could still enjoy bipartisan support (especially if we avoid the ‘education-is-the-best-or-only-way’ narrative). We may not be close to implementing other, more effective poverty-fighting schemes, but we’re not obviously “close” to achieving any particular educational reform panacea, either.

Conclusion

Education is good and we should improve education when reasonably possible. Education may even have some utility when it comes to reducing poverty. Nevertheless, the claim that education is the best way to fight poverty seems both empirically false and politically destructive for other, more direct and effective poverty reduction measures.

Posted in Education Reform | Tagged | 10 Responses

Reformers and International Comparisons

Olympic_rings_with_white_rims_svgOn Tuesday at This Week in Ed I asked what the evidence is that education reform is the best way to fight poverty. The domestic evidence doesn’t strike me as obvious, but I was especially curious about the international evidence.

After writing that post I started nagging people on Twitter to tell me which countries have  - at least according to reformers – education systems we should be aspiring to.

My main reason for doing this was that if reformers really think education is the best way to fight poverty, then presumably countries with the best education systems should have substantially lower poverty rates as a result. My guess is that there is little evidence for this; if a developed country is successful at reducing poverty (and many are), that is usually because of government taxes and transfers, not education. Still, since reformers seem largely unwilling to identify the countries they believe have the most effective education systems, their claims about the benefits of reform (e.g., poverty reduction) are difficult to test.

Unsurprisingly, I didn’t have many takers on Twitter. A few reform-sympathizers chimed in that while we might want to learn from aspects of different countries’ education systems, reformers aren’t really committed to the idea that copying other countries would be useful in general. We have a different context, after all!

I agree whole-heartedly that you can’t judge the effectiveness of an education system just by looking at its outcomes. Context matters a great deal. But, of course, a fair amount of the education reform movement is premised on the idea that we can tell discern the mediocrity of our own education system by noting the mediocrity of our outcomes.  If the inferiority of our education system is demonstrated by our test scores, it’s hard to see why the superiority of other systems wouldn’t be. Raw international test scores either tell us about educational quality or they don’t.

Jason Becker stepped up with a more radical defense of reformers, arguing that they are not, in fact, making claims about educational effectiveness at all.

As far as I can tell, this is false as an empirical matter of fact. For example, here’s StudentsFirst with a series of ads explicitly claiming that “our education system” is a joke.

Hypothetically, though, could education reformers take the position that the our ranking on international test scores is interesting just because our outcomes as such have implications for our “competitiveness”?

I’m skeptical for two reasons.

First, the extent to which our relative educational outcomes matter for our international competitiveness is an empirical question that does not seem to me to have an obvious answer. America has not “lead the world” in educational outcomes for decades, if ever, and yet we seem to be reasonably successful and “competitive” (whatever that means). If reformers want to go this route, they’d have an awful lot of work to do in terms of demonstrating this relationship. (Note that they rarely bother to pursue this, which suggests that this is not really their main concern anyway.)

Second, and more fundamentally, would it even make sense for education reformers – as education reformers - to make a big fuss about our outcomes and competitiveness without making any claims (implicit or otherwise) about our educational effectiveness? Probably not: why bring up unequal education outcomes in a conversations about education reform unless you think the problem could be addressed by improving our educational effectiveness?

In other words, it wouldn’t make sense for education reformers just to point out that our outcomes are too low unless they are also – at least implicitly – trying to argue that those outcomes could be improved through changes to our education system.

Admittedly, reformers are often not explicit about what they think international comparisons demonstrate, probably because when you get right down to it they demonstrate very little. Nevertheless, those international comparisons either tell us something about education reform or they are non sequiturs in a conversation about education reform.

If we suppose reformers do not intend for international comparisons to be non sequiturs, there is still no plausible argumentative path forward for them.

1. Reformers may intend the international comparison to demonstrate that our education system is unusually or unnecessarily ineffective. This would be a bad use of raw test score data, which measure student achievement, not school effectiveness. Furthermore, as noted above, this would imply that reformers should be able to point to “more effective” education systems in other countries. They seem loathe to do this.

2. Reformers may intend the international comparison to demonstrate that, regardless of how effective our education system is now, it needs to be more effective in order to “catch up” our students. However, the plausibility of this argument rests in no small part on what we believe about our system’s current educational effectiveness. The more effective we believe our system already is, the less plausible we will find it that our international standings can be meaningfully altered by trying to make it even more effective. It will seem plausible that education reform can substantially close the gaps between the U.S. and the highest-achieving countries only if we think that our current education system is quite ineffective in relative terms. Which is to say, here reformers have subtly begged the question at hand, namely: whether our mediocre educational outcomes are best understood as resulting from educational ineffectiveness.

The most frustrating aspects of all of this are the obvious double standards.

In other contexts reformers have acknowledged, however grudgingly, that raw test scores don’t tell us much about effectiveness and what really matters is “value added”. When comparing scores internationally, however, suddenly all we need to know is that our test scores are lower than the scores of people in (some) other countries and this is supposed to make us very embarrassed about our education system.

Moreover, while these test scores tell us a great deal about how poor our own schools are, they are apparently of no use in evaluating other countries’ schools.

All in all, then, the reformers’ position often turns out to be simply that ranking countries by educational outcomes is useful for denigrating American schools…and nothing else. It is profoundly unlikely that that is the correct way to use international test scores.

Posted in Education, Education Reform | Tagged , | 6 Responses

Some Advice for Common Core Supporters

659315_5ba9794c89Yesterday at TWIE I gave CCSS supporters a hard time for seeming to give up the affirmative case for the new standards:

These days supporters seem to dedicate most of their time to assuring us that the CCSS are not to blame for “fuzzy” math curriculua or “whole language” or questionable history assignments. We are even told that it’s just as well if states opt out of the Common Core altogether because they’re unlikely to gain much from implementation anyway.

On Twitter Morgan Polikoff asked what I would prefer for CCSS supporters to do instead of making these half-hearted defenses. Fair enough, though I don’t really think of myself as part of the team so take all of this with a grain of salt. I’d recommend the following, which are arguably all variations on one or two themes:

1. CCSS supporters need to acknowledge that they overestimated the potential for standards per se to improve curriculum and instruction. Reformers are rarely comfortable wading into the minefields that are debates over C&I, but one of the lessons here should be that you can’t just lean on standards to do that work for you. Frankly, the science of teaching is just not sufficiently advanced and accepted that educators and families will all fall into line if you just give them the right goals to shoot for. Teachers already think their pedagogy is about right for whatever learning objectives you want to establish; if you want them to think differently you need to convince them directly. It is also increasingly apparent that you can’t avoid nasty battles over curriculum by saying “standards are not a curriculum”.  You may as well wage that war directly.

2. CCSS supporters should acknowledge that the new standards are not really as unambiguous as they had thought. By convincing themselves that opponents just don’t understand what the standards “really” say, supporters end up repeatedly missing the point. That tactic may work in a few cases, but whether, for instance, they encourage “fuzzy math” is just not something you’re going to be able to conclusively establish using nothing but the standards themselves. To the extent that teachers think the standards encourage fuzzy math, the standards do encourage fuzzy math. Supporters need to realize that while the CCSS are not “to blame” for peoples’ pre-existing edu-confusions, those confusions are really what we should be dealing with. As with #1, wage those wars directly.

3. CCSS supporters should focus more on Common Core-aligned assessments. What the CCSS “really” mean will be determined in large part by the tests used to hold teachers and schools accountable. So while it’s all well and good to assure us that, e.g., the CCSS “require” a “content-rich curriculum”, that won’t really be true unless the eventual assessments require a content-rich curriculum. I don’t think I’ve seen nearly enough attention paid to this and we’re already pretty far along in the assessment-design process.

4. CCSS supporters should spend more time highlighting “good” Common Core-aligned lessons. Opponents are already more than happy to talk about “bad” lessons;  supporters seem to have decided the best thing for them to do is join in. This is contributing to a growing sense – however unjustified – that the CCSS promote all sorts of educational silliness. That’s a recipe for further political losses and it means that educators aren’t getting the CCSS guidance they deserve.

Personally, I’m agnostic about the Common Core standards. People whose judgment I trust often say very good things about them. At the same time, the new standards are extremely expensive and disruptive and there does not appear to me to be nearly enough in the way of consensus among supporters about what the standards “really” say to make me think they’ll be good for curriculum and instruction on balance. For every supporter who tells me the CCSS “require a content-rich curriculum” there’s another who insists that the new standards “prioritize skills over knowledge”.

So I am not always encouraged. But there’s still plenty of time for “the good guys” to exert influence over CCSS implementation if they’re willing to start playing offense.

Posted in Education Reform | Tagged , | Leave a comment