Polling Myths, Debunked

On Everyone’s Favourite Polling Conspiracies

One of the worst parts of election campaigns is the ramping up of the polling and modelling conspiracies, so let’s have some fun here and debunk these one by one, shall we? (I’m so annoyed already I don’t even have the energy for a pithy intro, be warned.)

X & Y Pollster Is A Partisan Hack, Their Data Is Biased

Greg Lyles was Gordon Campbell’s campaign manager in the 90s, Quito Maggi is an Ontario Liberal, Darrell Bricker is certainly known for having Conservative tendencies, and Frank Graves hasn’t made his contempt for Poilievre a secret. Yes, partisans sometimes get into public polling. No, it’s not a conspiracy.

EKOS had a Poilievre lead of 25 when Chrystia Freeland resigned, which is statistically indistinguishable from David Coletto’s 26 point lead. Ipsos - again, led by the guy who wrote The Big Shift, one of the most ludicrous books written on the flimsiest of pro-Conservative premises - was the first pollster to show a Liberal lead in like 2 years (albeit by less than an hour). David Coletto - two time Scrimshaw Show guest, two timer giver of proprietary Abacus data to me for free to help me write columns, and friend of this show - has what I believe to be the only poll fully taken with Carney as PM right now showing Poilievre with a popular vote lead. Quito’s polling had the worst Liberal results pre-Freeland resignation.

There is this fantasy that you can map vote intention or general sentiment of the proprietors of these firms and cleanly discern bias. You can’t. It’s a fiction perpetuated by dumb people who want to believe they’re smarter than the rest of us.

Pollsters Only Release Data When It’s Good For Their Side

Most pollsters in Canada have a media deal - Abacus with the Star, Leger with Postmedia and/or the Canadian Press, Ipsos with Global - or they monetize their data in some way (Mainstreet and Nanos with crosstabs behind paywalls). In most cases, their schedules are dictated by the terms, explicit or implicit, of their media deals or their subscription packages. Nanos is a weekly tracker out of writ and a daily tracker in writ, Mainstreet does a Federal poll monthly(-ish) out of writ and a daily tracker in it, unless there’s a provincial election in which case the monthly may blow out to 5 or 6 weeks for very understandable reasons.

Leger, Abacus, and Ipsos are at the whims of their contracts and their media sponsors’ budgets. There is no conspiracy. I wish others would self-choose to go on more of a regular schedule to avoid the appearance of impropriety, but like, everybody basically agrees the Liberals are ahead now. This isn’t a conspiracy.

Tracking Polls Are Gaming The Models

This one is always a fun conspiracy - the idea that because pollsters release lots of polls, they’re somehow artificially gaming the models by flooding the zone, gaining an airquotes “artificial” share of the weight in polling averages, and then messing with the models. I get why people think that, but the problem with it is it holds the modellers in quite low intellectual capacity to think we just get gamed like this.

I can’t speak for Fournier or Grenier or Bryan or whoever in specific, but I know for me, once a new Nanos or whoever is running a tracking poll comes in, the old numbers go out. I don’t double count the data, I don’t boost it for recency, I don’t do anything to the data except take out the previous day’s number and put the new day’s number in. It’s also the case that you’d see such manipulation in the models, and there’s never been a situation where it’s happened.

I’m not exactly sure how others handle it, but plainly you’d have to be an idiot to let your model get gamed by a tracking poll. Lord knows I have often had my issues with some of my fellow modelers, but they’re not idiots. (Most of the time, anyways - we all have our moments.)

Margins Of Error Make Polls Worthless

Canadian polls are almost stunningly accurate. One of the things that my American friends and readers always marvel at is how good everybody manages to do at Canadian projection. It is harder to model Canada than the US in some ways - the combination of third parties, vote splits, tactical voting, and regionalism makes projecting this right really hard. But, as a collective we consistently do it very well for a reason - our polls are very good.

Margins of errors are real, but irrelevant to most discussion. Polls are estimates, and it’s reasonable to point out that there is some error to be expected. There is no way to perfectly know beyond a shadow of a doubt what will happen on election day before the votes are cast and counted. This doesn’t mean it’s not scientific - it is - but it does mean that focusing on MOE is a waste of time. The numbers the pollsters produce are their results. Pretending that actually if you tilt your head and hop on a trampoline 16 times and sneeze in a West South West direction there’s a different result is cope.

At this point I’m out of myths, but let’s hit one other topic.

Incumbency

I think I’m different in how I handle incumbency than others - I don’t give every MP who is running again a boost. Let’s take Leah Gazan in Winnipeg Centre as an example - she won in 2019 for the first time, served as a MP for two years, and got a big swing to her in 2021, a combo of a slight NDP vote increase and incumbency. That incumbency is now built into her margin. If I keep giving her an incumbency boost, on top of the baseline result that already includes her personal vote, you’re basically modelling that it’s impossible for incumbents to lose the longer they’re in office.

The way I do it is changes in incumbency merit change in status. Take a seat like Anita Anand’s Oakville seat - when she announced wasn’t running again, the Liberals faced a penalty. When she announced she was coming back, the penalty was removed. As a general rule I do 4% on margin for seats where the retiring incumbent flipped a seat (2% of vote share gained by the new incumbent, 2% of vote share lost by the party they beat) that inverts when they retire.

The exceptions to this are few but worth noting. Leaders contesting seats for the first time as leader get a bigger benefit (2x the usual), so Carney will get a hefty boost once I add the Leader’s Benefit to Nepean, but Singh, Blanchet, May, and Poilievre get nothing because their Leader’s Bonus is already built in. I’ve also imposed a 10% penalty on the Liberal margin in 3 Jewish-heavy seats (York Centre, Eglinton-Lawrence, and Mount Royal) because I am aware of the fact that the Jewish community is not exactly thrilled with the handling of certain …foreign policy questions, let’s say.

On the question of tactical voting, the NDP federally are much more exposed to losses than the Ontario NDP for a few reasons. In many seats, especially in BC, the Liberals project ahead of the NDP. It’s a lot harder to get Liberals to back the NDP incumbent when the Liberal projects ahead than when they’re at 16% and the NDP are down 4. Also, the NDP have a lot of exposure to the Liberals, and in those seats there’s no incentive for Liberal supporters to back Gazan or Niki Ashton or Peter Julien or whoever.

I do think that in the few competitive NDP-CPC battles you’ll see some tactical voting - London Fanshawe, Elmwood Transcona, a couple of the Vancouver Island seats - but the ONDP’s success is not instantly transferable across the country.