Nate Silver's Model An Incoherent Mess
What A Nightmare
“What in the ever loving fuck did you build, Nate?”
On Thursday, the 538 Midterm models came out, and this morning I just looked at the overall, aggregate probabilities, and basically nodded – it’s an ever so slight R edge in the Senate and the GOP 87/13 in the House, two outcomes that in aggregate look fine. I don’t really know how Nate is getting a R+6 national environment, but assuming it’s some historical trends-based thing, whatever. The problem with the models are they’re dogshit in specific, but a lot of the dogshit cancels out to a reasonable national outcome.
The House model basically fails on the same basis everywhere – they overstate Democrats in working class, right trending seats like Dan Kildee’s Biden +2 Michigan 8th and understate Democrats in left-trending suburban seats like David Trone’s Biden +10 Maryland 6th. The reason is their partisanship metrics use some 2016 data, and some state data, and therefore has a very big lag. Using that as their metric of partisanship is basically an admission they have no idea what the fuck they’re doing, because this exact model misses all of the Global Fucking Realignment stuff and implicitly assumes that the trends will reverse without Trump on the ballot (which we know won’t happen, because even in Virginia and New Jersey, the swings were much more concentrated with white non degree voters than white degree holders.
Knowing this, their House model is bad, but bad in a way that’s mostly irrelevant – the GOP are sizable favourites, and as much as they’re low on a lot of the left trending suburban seats, they have Jared Golden as a fucking tossup and Henry Cuellar as a fucking 59/41 favourite. In aggregate, it’ll cancel out, but this model is just asking “what if everything we’ve learned in the last decade just didn’t happen?” and it’s dogshit for that reason.
I have no idea how they came up with their Senate ratings, admittedly – Lean D Arizona, Tossup (but D favoured) Pennsylvania, Tossup (but R favoured) Nevada and Georgia – so I’ll mostly skip those, because the true buffet of bullshit is in the Governor’s Model, where Ron DeSantis, Governor of the state famous for always being close regardless of the national environment, is favoured by 12%, where Gretchen Whitmer in Michigan is also up by the same amount, where Tony Evers in Biden +1 Wisconsin has a better chance than Democrats do in Biden +16 Oregon, where Laura Kelly is in a tossup race in Kansas, and where Greg Abbott is gonna beat Beto O’Rourke by more than he beat … I genuinely don’t even know who he ran against in 2018? Like, I’m no Beto fan, but if you’re making big assumptions around candidate quality and past races – which is the only way they get Mark Kelly Lean D and Laura Kelly in a tossup – then how is Beto losing by 14%? (To be clear: Beto is losing. He’s just not losing by 14%.)
You can make a case for some of these in isolation – Oregon Governors races have been historically closer than federal partisanship and there’s that independent, Whitmer is getting to run against joke candidates, Evers has a good approval according to Marquette – but in the same forecast they’re absolutely not happening at the same time. If Kansas is a tossup, then Beto is losing by 6. If Beto is losing by 14, then maybe DeSantis is winning by 12, but those two things aren’t happening on the same day Democrats win Wisconsin comfortably and Michigan in a rout. Yes, federal partisanship means less, but this isn’t 2006, when Democrats won fucking Tennessee and Wyoming.
“Ah, but Rick Scott only won by 1 on the same night the GOP won Illinois in 2014,” I hear someone say, as if that’s proof that these kinds of wildly divergent results are possible and not that Florida is always gonna be close. Also, yes, there will be some localized oddness, but the reason it is odd, and not the conventional take, is because when it happens, it’s a fringe case, not the modal expectation. The notion that this is the likeliest outcome beggars belief, and brings me to the thing that wounds me so. Nate, buddy, what the fuck did you build?
I know what he would say if he read this and replied (which, he shouldn’t do, he’s a busy guy and I’m just some dude), which is that he sets the outputs based on a criteria of X and it spits out Y and he doesn’t build models to conform to expectations and I get all of it. The problem is, if your model spits out R+12 Florida Gov, you gotta fucking get your head out of your ass and realize that there is no chance in hell that actually will happen, because it’s fucking Florida.
The advantage of smart data-based analysts – a rank I am decidedly not including myself in, I’m just an asshole – is that they have a literacy in backing up their arguments. Where this detaches itself from reality is in its purpose – Nate is a smart guy who knows a lot about politics. We know this because he can talk at length about it, he pays a lot of attention to it, and he isn’t just the stats nerd with a spreadsheet. At its best, data journalism is supposed to be about accentuating what we see ourselves – whether it’s Zach Lowe in the NBA space or any of the brilliant people running Football Outsiders or ESPN’s NFL advanced stats department, we would hate them if they didn’t also have a smarter understanding of the game based on ability to watch tape than us.
Why is Mina Kimes the best football thinker in the world right now? Because she can walk you through a highlight package, talk about scheme and fit and personnel choices, and she knows how to accentuate that brilliance with the analytical background and the stats to prove it. When Dan Orlovsky talks about how part of why Russell Wilson takes so many sacks is that he holds the ball, she has the numbers to prove it, but also, it’s not like she’s never seen Russ take a sack he absolutely didn’t need to because he wouldn’t get it out fast enough.
The problem with Nate is that his forecast is the product of someone who has no innate feel for the thing he’s built, and it’s robotic, and frankly dogshit. It’s a joke unbecoming of the backing of a major news organization, because it’s bad at two levels. It’s wrong in many places in a “this race won’t go to X party” sense, but it’s also incoherent, because the things they have happening at the same time will never happen together. For whatever you think of the 413ers, of which I was proudly one then and am unproud to have been one, 413 was a much more internally consistent forecast than this, because it properly understood that in a wave, that would have been the map. The 538 model has a built in wave assumption, but somehow has Democrats making a net gain of 1 Governorship. It’s incoherent, internally inconsistent, and straight up bad.
Ryan Jakubowski and I did a draft of the Cook rated tossups for my podcast last night, and they suffer from the same problem – there is no universe in which Ohio 9 and Rhode Island 2 are tossups at the same time. It’s incoherence, and the reason I wanted to do that with Ryan is to show up the ludicrous nature of these forecasts. (Also, Ryan is one of the smartest people thinking about US elections out there, so getting to talk through the House map with him is always a pleasure.) Like, this isn’t about whether or not it’ll be a R+6 or a R+2 national environment – that’s not the point. What is the point is that regardless of the environment, this is idiotic and incoherent.
I got into election projection and electoral math as a devotee of Nate Silver’s. The Nate I loved would never have bought this bullshit and would be relentlessly mocking this absolutely dogshit model. And now he has become what he got famous for hating, a shill for an output that doesn’t pass a basic smell test. It’s sad to see someone I respected so fall so low, but this is where he is.