Coletto, Methods, And The Return Of The Pollster Wars

On The Latest Rendition

“Either way, Pierre Poilievre won’t be Prime Minister, and everyone who thinks he can be will eventually blame their belief he can on a rush of blood to the head.”

I wrote these words in February of 2022, when Pierre Poilievre first signalled he was going to run to replace Erin O’Toole as Conservative leader. Pierre Poilievre would go on to lose the next election after I wrote it, and it was an important column in the history of this site. It was what I believe to be my first piece to ever break 10k views, and it certainly helped drive traffic and attention. And yet, it’s probably one of the worst columns I’ve ever written.

David Coletto has decided to (re-)ignite the Pollster Wars, one of my favourite conversations, and it’s in that context that I’m thinking of that Poilievre piece. As someone who has spent the last decade thinking about polling and modelling, whether in public or private, I have a lot of thoughts about the topic. I’m quite sure that these thoughts will make precisely nobody happy, because it’s not as simple as believing Coletto unconditionally nor condemning him as a closet conservative hack.

But as this conversation happens, I’d keep one fact in mind - being proven right in the end doesn’t mean the work was good.

..

The reason I’ve never taken a victory lap on the column that leads this piece is that column was fucking crap. It was nonsense, idiocy on stilts, the kind of arrogance and certainty that is unbecoming a student pontificating in class, let alone someone who wants to be taken seriously. I was an arrogant ass off the high of being right about 2021, and I let that arrogance lead me down a path of bad work. Just because it ended up (for now) turning true doesn’t make it good work.

The reason I’m thinking about this is simple - there’s another round of the pollster wars, and I think that everybody in this industry would do well to learn from my mistake in 2022. The problem that every pollster faces is that it’s impossible to know who was right outside of an election, because elections are discrete, one off events. It’s impossible to know what the true state of the country is at any given time, outside of election day. In late February 2025, EKOS and Abacus both released polls, whose final day in the field were one day apart. EKOS had a narrow Liberal lead; Abacus had a 12% Conservative one. There isn’t any empirical way of knowing who was right at the time.

One of the many problems with this conversation is that we’re fighting about accuracy over a long time with metrics that only judge the final poll. If you want to criticize EKOS, the argument would be the Liberals never had a double digit lead in the writ period, and if you want to criticize Abacus, you’d say that they took far too long to show a competitive race. Subjectively, I agree with both these statements. But I don’t have any proof for them beyond what the other pollsters showed, which is less proof and more choosing to drive down the world’s wonkiest cul-de-sac.

There’s a similar inherent uncertainty in seat projections, and who was the most right. My 2025 performance was done - I overestimated the Liberals to some degree, but in terms of raw seats I got wrong, I did quite well. But given that my misses were mostly predicting Liberals who ended up losing to Conservatives, did I do better than others whose topline seat totals were much closer, but worse at predicting 343 winners? I don’t know, which is why I didn’t spend any time on the debate. Despite the veneer of math and stats, this game on both the polling side and the seat side is about narrative, and what’s convenient to claim.

If my methodology was showing Conservative friendly results compared to consensus, and I believe my methods, I’d want to defend them. There’s nothing wrong with that, and if anything more people should be willing to defend their work and ensure it stands up to scrutiny. Accusations of bad faith get thrown around with a frightening frequency, and as someone who’s been accused of being a sexist for wanting Anita Anand to be made Finance Minister (I wish I was joking), I’m not going to start throwing accusations around for fun. And I’m certainly not going to indulge that about someone who has never been anything but good to me and this site.

But it’s equally worth noting that there is a price when pollsters start talking about bias without a robust data set to prove it. Three byelections this week saw the Conservative vote down double digits, and a third pollster has finally seen Doug Ford’s vote share in Ontario starting with a 3; if only barely. For all the talk of anti-Conservative biases in the industry, the fact that Liaison was the first to see the current discontent with Ford that two other firms have since shown makes me dubious that their federal polling is wrong. Throw in the byelections and the case for anything other than the pollsters showing a double digit lead nationally being right looks shakier than Marilyn Gladu’s principles.

In years past, these minor differences in the polls and the fighting in the industry would have been my everything. But with the greatest respect to everybody involved, this latest round of the Pollster Wars is just vaguely pointless. The evidence for some form of Conservative optimism is flimsy at best, and resembles less a rational argument on the merits of the evidence the more the byelections, and the leaks about the mood of caucus, settle in. That means that any election now would end up in a Liberal landslide, and there’s no real reason to doubt that