So did the model predict the election?

A month has passed since the general election, and this weekend we celebrate the centenary of the birth of our democracy.

I’ve been pouring over the many data points from the election and have put together the Irish Election Stats forecasts alongside other the others. I note only one pundit who published their own individual forecasts – Ivan Yates. His forecasts are included. If more had made public forecasts I would have included them.

 

Which forecasts are being compared?

The website Election Hub Ireland compiled election forecasts throughout the campaign. You can see each constituency on their site. They noted the forecasts made by:

  • RTÉ
  • Newstalk
  • Irish Times
  • Irish Independent

As far as I understand, these forecasts were made at earlier stages of the election, and weren’t responding to poll changes.

Ivan Yates made his forecasts public the day before the election.

The Irish Election Stats forecast used here was the final one of the campaign – RTÉ’s exit poll on the morning of the count.

 

How do you compare accuracy of results?

The nature of our system doesn’t lend itself to easy forecast comparisons. This isn’t like a US presidential election where the forecast is one name – it’s either right or it’s wrong. I’ve had to create three methods of comparison:

  • Party total (i.e forecasting that Fine Gael won 50 seats)
  • Constituency forecasts (i.e that Dublin Central was 1 FG, 1 IND and 1 SF)
  • Name forecasts (i.e naming 150 of the 158 TDs)

 

Party totals

For this measure I take the “error” of each forecast for each party. The sum of these errors across the nine party groups gives “total error” which is then compared across the various forecasters.

party fore table

party fore chart

In this case Ivan Yates comes out on top with 16. This is followed by IES, and the media outlets. It’s notable that the Fine Gael/ Labour result was substantially less than the forecasts made by media organisations.  The coalition ultimately returned 57 TDs, in contrast to forecasts of 68 by Newstalk and RTÉ,  70 by the Irish Independent, and 71 by the Irish Times!

The Sinn Féin forecasts are close to the final result, with Fianna Fáil captured poorly by everyone. Even exit polls failed to capture its strong performance, thus worsening the accuracy of the IES model.

Not a single forecast foresaw the demise of Renua. It won zero seats, with the Irish Independent initially forecasting four.

 

Constituency forecasts

The first thing you will note about the table below is the fractions. This is because not all media organisations gave “deterministic” forecasts. Some avoided predictions for the final seat in constituencies and instead listed those they thought were “in contention”. If two were listed for one seat, only 1/2 could be counted where one of the two went on to win the seat.

In the below charts, only parties are counted towards an accurate forecast. If the correct party won a seat (but a different candidate) it is still counted.

con fore chart

con fore

 

On this measure Yates is ahead again, just two ahead of IES. I note again the worse accuracy by major news organisations, although their forecasts were made much earlier in the campaign.

 

Name forecasts

This is by far the most difficult way to make forecasts for Irish elections. It’s one thing to forecast a party to take a certain number of seats, but the candidates to win those seats is often anyone’s guess.

The media organisations avoided name forecasts at this election, leaving just IES and Ivan Yates to compare results.

“1” signifies that the TD was correctly forecast. I note where a TD wasn’t forecast by anyone. 20 TDs fall under this category, showing how such candidates came from the outside to win, but also highlighting how challenging it is to forecast Irish elections.

name fore

 

Once again, the IES model is just behind Yates by two.

 

In conclusion

For a statistical model created by a 23 year old – compared to the experience of a former minister, bookie and now broadcaster – I’m pleased with the results overall.

If there’s another election later in the year, I’m sure we’ll come back to these numbers again.

Leave a Reply