US: Will The Polls Be Quite As Bad As 2020?

With Bonus Echelon Slander

I have a stupidly good memory for certain, stupid things, and one of those things is remembering where I was when I heard something – be it a song, or a podcast, or a conversation, I can put myself right back to where I was when I first heard it.

My ability to remember this sort of things is only marginally helpful – it has allowed me to remember a lot of podcast moments for hosts of shows I listen to who need it for Best Of content, I think most notably – but it also is of occasional use here, because I remember listening to a Times live podcast recording right before the 2017 UK Election. I was walking to my local corner store to buy a Slushie, and as I was walking back, one of the guests said (I’m paraphrasing) “May’s run a shit campaign, but the polls were wrong last time so the Tories will be fine”.

That statement was true – in 2015, the polls had expected a deadlock in the popular vote and the Tories won it by 6%, leading to Cameron’s shock majority government. In the interim, Labour had elected a leader widely viewed as unelectable, and the widespread assumption was that the 2015 poll bias would repeat itself.

And then Jeremy Corbyn came six seats short of depriving Theresa May of a functional government with the DUP.

…

If the midterms were taking place in any other country, I’d be screaming from the rooftops that the existence of the 2020 polling miss was, in fact, evidence that the GOP won’t beat their polls in the same way again – an argument that has worked in the last decade in Canada, Australia, and the UK. Obviously we’ve already covered the UK, but in 2011, the Tories were undersold by the final polls in Canada and then four years later Justin Trudeau’s Liberals romped to a majority government that few – amongst those who did includes myself – saw coming. In Australia in 2019, the polls blew it, predicting a Labor lead of about 3% on TPP (51.5/48.5), which was actually a Coalition 3% lead on the night. (If you’re an Aussie reading this and mad at me calling that a 3% lead – I know that’s not how you call it, but I’m standardizing here, please go with me.) In 2022, the polls were basically bang on, in aggregate.

If you want to go deeper, one of the plausible reasons for the 2019 Australian miss was the 2018 Victorian State election, where the polls also missed – but massively understated the size of Labor’s win. Counterfactuals are hard, but it’s also hard for me to think that if the Victorian polls had been right, or had underestimated the Coalition, that we would have seen the 2019 Federal miss.

In 2019 in the UK, the 2017 miss was again being used to justify unskewing the polls, this time by deluded Labour supporters who had their heads up their asses about the state of play going into that night. Shockingly, on the night, the Tories won by basically what their polls had said they would, and the “unskew the polls by the amount they were wrong” people had been wrong two elections in a row. In Canada, after two elections where the polls were merely mediocre, the last two federal ones have been very good in terms of final accuracy.

Yes, there are some consistent sources of error – the Conservatives (both federally and provincially) tend to be underestimated in Alberta, and that is going to matter when I focus in on Alberta next year – but in aggregate, poll errors don’t tend to repeat too often, for a simple reason. Pollsters want to be right, because when they’re wrong, they get fired. That said, this isn’t any other country.

I made this argument in 2020, actually, quite a bit – the polls in 2016 had been wrong in large part because they didn’t weight for education, so therefore once they weighted for education in 2020, they’d be better. In theory, that made a lot of sense, and the polls in 2020 got a lot of the 2016 errors fixed. Their samples, however, were just much much worse, and so the crisis of polling continues. Whether that was pandemic effects or not, it’s hard to say – but I think just assuming the 2020 polling miss again is wrong.

Let’s be clear – I think assuming that the polls will be wrong in the same way directionally makes some sense – Republicans will beat their polls in Ohio, like, sure – but I think a lot of people are assuming they’ll do so to the same magnitude, and that’s where the problems lay.

Let’s use Wisconsin as an example, where the polls were wrong by 8% last time. Applying that same penalty takes the GOP from underdogs to favourites in the state, but I don’t think that blindly doing that makes a lot of sense. The reason for the 2020 error being so big was mostly because of two polls – a Biden +11 Times/Siena poll, and the famous ABC/WaPo Biden +17. Neither of those pollsters have released any state level polls this cycle, and the two pollsters who have – Marquette and Fox – both had Biden +5. Not exactly amazing work, but certainly not 8% off, either.

Go wider, and the worst 2020 pollsters have either changed their methods, or are mostly not polling this cycle so far. NYT/Siena has released one national poll, ABC/WaPo hasn’t dropped any polling since Dobbs, Quinnipiac used to be one of Biden’s worst pollsters in the post-Afghanistan, pre-Dobbs interregnum, and Fox had fucking Youngkin +8, so it’s not like they’ve been reliable Dem hacks.

Throw in the massive decrease in Dem-commissioned state polls, and the fact that Data For Progress have shown their whole ass with their horrible New York 19 poll, and you see a situation where the averages are less likely to be so biased. We know YouGov – who amounts for a decent amount of the state polling, because of their partnership with a lot of universities in the US – is willing to radically overhaul methods, from their UK and Australian work (YouGov runs Newspoll in Australia), and Emerson really feels like they’re overcorrecting for the last polling error.

The fact that the right are flooding the zone with shit like Trafalgar’s increasingly fake seeming polls and whatever the fuck passes for “polling” from InsiderAdvantage also helps make us confident that the averages won’t be so wrong, because between the lack of left wing partisan polls and the increase in weight prominent averages (*cough*FiveThirtyEight*cough*) has given to places like Trafalgar because of 2020, there’s less room for pure annihilation.

None of this is to say the polls will be right, but it is to say that assuming that a 2020 miss is somehow the median expectation is probably wrong. Is it possible? Of course it is, because assumptions that the 2016 polling miss represented the highest end of the possible 2020 miss turned out to be shit. I think assuming the Midwest polls will be correct is also probably stupid, but the thing is, there’s a lot of room between “the 2020 miss will repeat” and “the polls are correct”, especially in the Midwest.

In the Sunbelt, assuming the polling is correct is probably fine – the misses weren’t big in 2020, and in Nevada specifically, assuming the 2020 miss will repeat would assume that the one time the GOP have beat their polls becomes a pattern, which seems wrong. In Georgia, the 2020 polls were mostly fine in aggregate, and in Arizona Blake Masters is so fucked that it doesn’t really matter what you assume, he’s still gonna be down and down a lot.

I get the instinct to unskew the polls, because there’s no intellectual penalty for being low on Democrats this year, just as there was no penalty for being low on Aussie Labor in May. Had the Aussie right won again, there would have been hell to pay for commentators and pundits who had believed the polls, but the poll-skeptics on the right faced no such sanction for being low on Labor. In the same way, Crystal Ball and Cook will probably go into November with Democrats as extremely narrow Senate favourites (assuming conditions close to the current ones) and very sizable House underdogs, and then basically have a free bat. If Democrats win the House, or even beat their expectations, nobody will give them shit for totally downplaying the chances for months on end even once the tide had turned. Being low on Democrats is a free swing, and one being taken advantage of by people whose reputations are underwater with a segment of the right.

I know the impulse to left punch as much as any – I always try to make sure when I’m picking column topics for my betting columns over at TheLines to find some that allow me to tip Republicans, so that I’m not accused of being a solely Democratic-biased fuckwad. I know that every time I go out of my way to shit on Democratic chances in Florida or Ohio that I am doing it, in part, to salvage some reputation for impartiality or ability to be unbiased. Does this mean I actually secretly believe we’re getting Senator Ryan or Governor Crist? Lord no, but it does mean that I let a lot of other shit go by without comment, and I never let an opportunity to left punch go.

Hell, today we’ve gotten four high quality Senate polls – two Pennsylvanias (Fetterman +5 and +10), a Wisconsin (Johnson +1) and a Georgia (Warnock +6) – and it’s telling the response to the Wisconsin one. Like, if we ignore everything we know about Wisconsin polling, this poll still shows a tossup race, but it’s a very bad sign for Democrats – and it was probably fair to question moving the state. That said, the Pennsylvania polls are both disasters for the GOP, because in both Oz’s approvals are hellacious and in the CBS/YouGov +5, that’s an LV poll weighted by 2020 recalled vote, so we know it’s likely not some wildly off sample.

(Also, this is a wildly tangential point, but fuck the “Toss it in the average” people when we get shit like Trafalgar’s Vermont poll or whatever the fuck that was from Echelon yesterday. Obviously, Democrats are not winning by nearly that much in any of those races, and probably not even winning Kansas. Don’t be dumb. And no, we do not have to pretend that bullshit actually makes sense! We don’t! In fact, doing so makes us stupider, because we then have to contort ourselves into thinking that very obviously stupid things are within the bounds of possibility. We have brains, and expertise, and sometimes calling very obviously bullshit things bullshit is what has to happen. You want to know why the polling averages missed so much in 2020? In part because it treated very obviously bullshit things like Biden +17 in Wisconsin as literally possible. Fuck this bullshit.)

At the end of the day, taking polls at face value is absolutely a scary proposition, but it’s also fair to say there are reasons to think they’ll be, on average, less wrong in 2022. Is that much comfort? No, not really, but when a party’s lost all sense of momentum or message because of a policy disaster that they can’t move past – be it the Dementia Tax in 2017 for the Tories or abortion now – the “but the polls will be wrong again just like last time” might be true.

But just know it isn’t always true.