For the third year running, our review of preseason predictions and projections is back, crushing all the hopes of those baseball writers who never wanted to make predictions in the first place and would rather they were forgotten forever.

For each of the sets of predictions and projections that featured in our preseason analysis, the mean absolute error (MAE) and root mean squared error (RMSE) has been calculated. MAE is the average difference between the predicted total and the actual, while RMSE is the square root of the average of the squares of all the differences. RMSE gives greater weight to large errors because they are squared.

Before we get to the results, a quick recap of the predictions and projections featured and their abbreviations here:

PECOTA (PEC): Baseball Prospectus‘ projected win totals based on their in-house projection system.

FanGraphs (FG): The FanGraphs Depth Charts projected totals, which are a combination of the Steamer and ZiPS projection systems, with an additional playing time adjustment applied by FanGraphs staff.

Davenport (Dav): Totals based on Clay Davenport’s projection system, with Clay’s own playing time estimates.

Banished to the Pen writers (BttP): Predictions from each of our writers from our season preview series.

Effectively Wild guests (EW): Predictions from each of Effectively Wild‘s team preview podcast guests.

Composite (Comp): The average of the five projection/prediction sets above.

Adjusted PECOTA (APEC): A modified PECOTA prediction set, based on EW listener responses to a preseason over/under poll using the first PECOTA release, as discussed in the preseason edition of this series. Each PECOTA total is modified by up to ten wins, based on the net total percentage of listeners taking the over/under; for example, if 70% of listeners took the over, 20% under, and 10% the push, the PECOTA total would be moved up by 5 wins.

First, the win total predictions themselves. Effectively Wild season preview guests were attempting to recover from their sixth-place finish last season, while BttP writers had a first place in RMSE to defend.

It didn’t go well for either the EW or BttP contingent, but it’s a real improvement from PECOTA, which was a mediocre fifth in both MAE and RMSE last year. The first place in RMSE was helped by both PECOTA’s insistence that the Dodgers were an exceptional baseball team, and the fact that the system saw more in two of the season’s biggest surprises, the Brewers and Twins, than anyone else did. Once again, the FanGraphs system also performed extremely well, after finishing second in both measures last year. With PECOTA performing so well, the Adjusted PECOTA set didn’t repeat the success of 2016, meaning that all three sets which had the more significant human touch lagged behind the projection systems.

Unsurprisingly, the Giants came out as the biggest miss across the board thanks to their disaster season: the seven sets missed the San Francisco win total by an average of 24.3 wins. In a not particularly close second were the Tigers, at 18.7, closely followed by the Mets at 17.4. At the other end of the spectrum, the Angels, Braves and Marlins totals were out by less than three wins on average. Overall, this season was much harder to predict accurately from a wins perspective than 2016: both the MAE and RMSE first-placed scores would have been comfortably last in 2016. 2015 was much closer to 2017 in terms of accuracy, although no set was as far out this year as BttP writers were in 2015 (so we’re improving, right?).

Special mention for accurately predicting the exact win total goes to Alex Speier of the Boston Globe, who nailed the Red Sox total of 93. PECOTA was spot on an impressive three times, for the A’s, Marlins, and that much-derided Cubs prediction of 92 wins. The Adjusted PECOTA set added two more correct totals, for the Braves and PECOTA’s long-time nemesis, the Royals. If only I could find a way to combine the two.

If the win totals were harder to predict, which is unsurprising in a league that saw some greater extremes at the ends of the spectrum than recent years, what about the actual team ranks?

A victory for the composite projection, which could be interpreted as some success for everyone, if you’re feeling really generous. The composite by its nature is never going to be the furthest out on a team, so those huge misses by certain sets are always mitigated. After finishing first last year, FanGraphs was very close to doing so again, and after coming second in both the win total and rank measurements in 2015, it’s fair to say that the FanGraphs projections are as close as we’ve come in three years of these reviews to any kind of consistent strong performance. This measurement is kinder to the BttP writers, and another potential trend to watch for next year is Davenport’s tendency to miss big on the rankings side: it was also last in RMSE for ranks in 2016, and the system’s strong projected finish for the Tigers and Mariners really hurt it this year.

That’s all for this year. Congrats to FanGraphs for another impressive performance, PECOTA for rebounding from a poor 2016, and to the BttP writers for beating the EW guests for the second year running. Join us again at the end of the 2018 preseason as we track another futile battle to predict baseball.