Politics and polls: a few observations on Nate Silver’s 2008 results.

OK, so the “occasionally other stuff” really means politics, and that’s been a major distraction for the past few weeks. The differences between electoral vote tallies from state-by-state polling, which gave Obama persistent (though at times small) EV lead, and the national polling which gave Romney a persistent (though at times small) lead are an obvious conundrum, and I had begun to poke around in data that’s fairly easily available to see what biases there might be in state polling and models such as Nate Silver’s, even before attacking Silver’s model because the new favorite pundit sport.

My first question was: How good were state polls in past elections?  I didn’t find an easy way to pull together information on that, so I settled for the closely related “How good was Nate Silver’s model in 2008?”  The short-hand view — it missed only one state (Indiana) and one Nebraska CD — isn’t actually very helpful since this year’s election could easily come down to one or two states.

To look a little more deeply, I compared 538′s final prediction to the actual state-by-state results. A few things I found:

1) Comparing Obama’s actual percentage of the vote vs. the 538 model for the 56 electoral vote entities (50 states, plus DC, plus 2 Maine CDs and 3 Nebraska CDs), on average 538 underestimated Obama’s vote share by 0.9%, quite comparable to the 1.1% underestimate in the 538 popular vote estimate.

2) The bias was noticeably smaller (-0.07% vs. 1.32%) in the states/entities where the race was close (an Obama-McCain difference of 10% or less) than those where the difference was large.

3) There were a number of significant (>2%) “misses” in states where the race was predicted to be close.  However, of these, only three — Colorado, where the Obama vote was underestimated by 2.4%, Indiana where the Obama vote was underestimated by 2.5%, and Nevada where the Obama vote was underestimated by 8% — were in states that had been heavily polled prior to the election (see Silver’s discussion of the accuracy of state polling)

4) For the states predicted to be close, where there was substantial polling, on average, the 538 model was slightly closer to the final result than the polling alone, though in the one case (Indiana) that the polling and model predicted different winners, the polling was correct.

5) Finally, a subtle but interesting trend in the comparison: the model had a fairly strong tendency to underestimate the margin of victory of the winner of each state — i.e., underestimating Obama’s margin of victory in states that Obama won, and McCain’s margin of victory in states that McCain won.  The relationship was approximately linear — underestimates increased with the candidates winning percentage.  Could this be the result of bandwagon effects, or possibly unmodeled political polarization?

Details and graphs to follow..

Pathways to the Middle Class — first thoughts

I’ve been reading Pathways to the Middle Class from Brookings’ Center on Children & Families.  So far the most interesting point is that, as they put it, “there are second acts,” and in particular that kids who seem “off track” as teens manage to be “on track” at the end of their 20′s.  This seems particularly true for boys.

It seems possible that this is an artifact of  the method they use to piece their data set together (maybe?).  The individuals in the “younger” life stage cohorts were born more recently than those in the “older” life stage groups, and there is a thread of conventional wisdom that says that males, particularly males without a high school or college diploma, are doing less well than in the past.

But it also may reflect the disparate impact that family responsibilities have on women in their twenties and thirties, or a greater willingness of employers to forgive a disorganized youth in men than in women  (stereotype-based gender bias in hiring seems to persist in other areas).

Taking the data at face value, adolescence appears to be the riskiest stage — a relatively large probability of moving from “on track” to “off track” and a relatively low probability of the reverse.  But it also seems to suggest that for many students simply successfully navigating the pitfalls of the teen years is as crucial for future success as adequate preparation for college/career.

Today’s reading

Pathways to the Middle Class from Brookings’ Center on Children & Families.

Lots to digest. More later.

Quantitative Reasoning/Literacy

A good overview of the concept of quantitative literacy (as opposed to just passing math classes) by Deborah Hughes-Hallet (who I had the good fortune to have as one of my first supervisors when I was a college senior teacher calculus sections).

The Role of Mathematics Courses in the Development of Quantitative Literacy

So far, most of the interest in quantitative literacy is at liberal arts colleges — Carleton’s QUIRK Center is one of the notable efforts — but much of the discussion seems relevant to K-12 education, both in terms of the purposes of mathematics education, and the pitfalls of the current approaches.

A-G graduation requirements

One of the issues I’ve found myself thinking about a lot is the trend toward stricter (more “rigorous”) high school graduation requirements, in particular the trend in California towards “A-G” graduation requirements which would make passing the courses needed for admission to UC/CSU a requirement for a high school diploma.

I think perhaps the best arguments for doing this, or something like it, are that it will help address equity issues.  Currently Hispanic and African-American students are less likely to complete A-G requirements in high school therefore under-represented among UC and CSU applicants.  Requiring all students to complete A-G classes helps level the playing field.

But it seems to me, like many reform trends in education, to be a simplistic response to a complicated problem.  The most obvious UC and CSU are mandated to serve (respectively) the top 12% and to 33% of high school graduates; preparing more students to meet the entrance requirements is not going to significantly increase the number of students that UC/CSU serve, leading to an obvious question:  Is an A-G curriculum the best preparation for the 2/3 of CA high school graduates who will not attend UC or CSU?  It’s not obvious that the answer is “no,” but it’s also not obvious that the answer is “yes.”

Hello world!

I’m a physicist with a long standing interest in K-12 education (including nearly eight years as a member of a local school board) recently transplanted from California to Oregon, and on leave from a college teaching position in CA.

My goal for this year is to gain a deeper understanding of science education and the education policy questions relating science education and the connections between K-12 and higher education.  My hope is that blogging will help me keep my efforts organized — as well as providing an opportunity to mull over more general questions.  Hope you’ll join me!

– Rachel