With the regular season over, it’s time for our second annual review of the preseason predictions and projections that we pitted against each other back in April. Last season, the Effectively Wild podcast preview guests came away with a surprising victory; could they retain their crown?

Let’s take a look at the final standings, for which I’ve calculated the mean absolute error (MAE) – the average difference between the predicted total and the actual – and the root mean squared error (RMSE) – the square root of the average of the squares of all the differences. If neither of those made much sense to you, the easiest way to think about it is that RMSE gives greater weight to large errors because they are squared, so a projection set that missed a win total by 20 wins will suffer more in RMSE than MAE. The plus & minus numbers indicate the amount of wins the prediction/projection was out by in either direction: for example, PECOTA’s -6 for the Royals means PECOTA projected them to win 6 fewer games – 75 – than the 81 they actually did.

The entrants consisted of BP’s PECOTA system, the FanGraphs (ZiPS+Steamer) system, Clay Davenport’s system, our own season preview writers, and Effectively Wild guests/BP Annual essay writers. A composite of these five is also included. APEC is Adjusted PECOTA – based on fan responses to our PECOTA wins over-under poll.

So, did EW guests retain their title?

 PECFGDavBttPEssayCompAPEC
MAE6.2005.8006.9675.9676.3675.7335.800
MAE Rank5274612
RMSE7.9507.3488.8117.2968.5267.4217.465
RMSE Rank5271634
DivTeamActualPECFGDavBttPEssayCompAPEC
ALCCLE94-2-8-12-4-3-7-10
ALCKCR81-6-4-1196-23
ALCDET86-7-5-5-1-6-6-3
ALCMIN5919192117241918
ALCCHW7862107552
ALETOR89-3-5-1-1-2-31
ALEBAL89-15-10-12-8-8-12-11
ALETBR6819131415171514
ALEBOS93-6-5-7-4-9-7-7
ALENYY840-2-310-20
ALWLAA7427109355
ALWHOU843378645
ALWOAK6971115619115
ALWSEA86-2-41-60-3-8
ALWTEX95-16-15-15-8-11-14-11
DivTeamActualPECFGDavBttPEssayCompAPEC
NLCMIL735-3-501-1-4
NLCSTL86-5-1-39302
NLCCHC103-9-8-3-7-1-7-8
NLCPIT7845758512
NLCCIN6864-17-32-1
NLEATL6811-2130-2
NLEFLA79-41-1-6-4-4-4
NLENYM873324420
NLEWSH95-8-7-7-7-7-8-6
NLEPHI71-5-7-7-14-4-2
NLWARI699121514141215
NLWLAD91313432-2
NLWSFG87-2112301
NLWSDP6886071164
NLWCOL75-1-1-8-1-3-4-8

Not even close; in fact, only the Davenport projections – a new addition for this year – saved the EW guests from going from first to last. As for the winners, that’s up for debate. The composite projection actually finished first in MAE, while it was our very own BttP writers who took the RMSE crown, turning around last year’s basement finish in style. Meanwhile, FanGraphs were second in both, tied with the Adjusted PECOTA projections in MAE. The fact is that these four sets were all incredibly close and it’s probably fairest to call it a draw, as a couple of wins in any set’s favour could have changed the final results and these differences of a tenth of a win or two are insignificant.

BttP writers only missed on three teams by double-digits and their worst pick was 17 wins out, helping them edge the RMSE victory; by contrast, every other set had at least one miss of 18 or more, and Davenport had a whopping 10 misses by 10 wins or more. The Twins clearly confounded everyone by being much more terrible than we could have imagined, while the Rays, Orioles, Rangers and Diamondbacks also caused a lot of trouble. The Giants – again – proved rather easy to predict, as did their division rivals in Los Angeles, and the majority of the NL East. Congratulations to BttP’s Ross Bukouricz for getting the Brewers’ total exactly right, EW guest Kenny Ducey, who was spot on for the Yankees, and Baseball Prospectus’ Meg Rowley, who nailed the Mariners’ total.

It should also be noted that this season was much easier to predict than 2015: the EW essay writers were in first place in 2015 with a MAE of 7.3 wins, worse than even 2016’s last-place Davenport projections (7.0). The Adjusted PECOTA experiment was again fairly successful, as 7 out of PECOTA’s 8 most inaccurate predictions were improved with the input from the Effectively Wild listeners, particularly the Rays and Orioles predictions, which both moved 5 wins in the right direction.

So several sets did a pretty good job of predicting wins this season; what about the final team ranks? Below are the same tables, but this time with each team’s actual final MLB rank on the season, and how each set compared to that.

 PEC DFG DDav DBttP DEssay DComp DAPEC D
MAE5.4675.0676.8335.5336.0335.3335.233
MAE Rank4175632
RMSE7.5196.5018.2046.5527.4186.7826.676
RMSE Rank6172543
DivTeamRankPEC DFG DDav DBttP DEssay DComp DAPEC D
ALCCLE41-4-12-21-1-7
ALCKCR16-8-8-8106-45
ALCDET11-5-3-7-2-11-7-5
ALCMIN301271081288
ALCCHW1870135070
ALETOR7-2-32-3-3-15
ALEBAL7-19-15-16-13-14-16-13
ALETBR262112101012159
ALEBOS501-5-3-10-3-5
ALENYY1432-41-1-13
ALWLAA210795-312
ALWHOU1497111091010
ALWOAK24361211660
ALWSEA110-12-10-10-9
ALWTEX2-14-16-18-10-13-14-9
DivTeamRankPEC DFG DDav DBttP DEssay DComp DAPEC D
NLCMIL224-6-3-5-5-4-4
NLCSTL11-42-49434
NLCCHI10000000
NLCPIT1847726716
NLCCIN260-1-13-4-2-2
NLEATL26-3-3-3-4-3-3-4
NLEFLA17-7-1-5-10-8-7-6
NLENYM95654660
NLEWSH2-3-2-3-8-6-3-2
NLEPHI23-7-7-7-6-2-7-3
NLWARI246101286813
NLWLAD65444442
NLWSFG9-1541442
NLWSDP265113311
NLWCOL20-6-5-7-6-8-7-8

A double win for FanGraphs this time, although again the differences are negligible in some cases, particularly the one-twentieth of a rank that separated FG from the BttP writers. The Cubs were the reverse of last year’s Phillies – everyone predicted them to finish first and everyone was right. Once again, the NL was particularly predictable, with only the Diamondbacks really drawing fairly big misses across the board.

So that’s prediction season over for another year. If you have any suggestions for prediction or projection experiments you’d like to see next season, or further features we could add, I’d love to hear about them so leave a comment and I’ll see what I can do. Enjoy the playoffs, and I’ll see you next year!

Next post:
Previous post:

Trackbacks/Pingbacks

  1.  2017 Milwaukee Brewers Preview | Banished to the Pen
  2.  2017 Season Preview Series Index | Banished to the Pen

Leave a Reply

Your email address will not be published. Required fields are marked *