When I finished the 2015 season preview prediction analysis in April, I promised to come back at the end of the season and find out just how wrong everyone was. For those of you who were really, really wrong, time to look away. Everyone else can laugh at how bad we still are at trying to predict baseball.

In order to evaluate how successful our previewers and projection systems were, each team’s final win total was subtracted from the projected total from each set. The average difference of all 30 predictions were then calculated, as well as the correlation of each set of predictions to the actual win totals. The results can be seen below.

In a stunning turn of events, the BP Annual essay writers/Effectively Wild podcast preview guests crushed not only the other human predictions (we’ll gloss over where those came from), but both the PECOTA and Fangraphs predictions, finishing with the lowest average difference of 7.3 wins. The EW set was also the only set to finish with a correlation stronger than 0.5. PECOTA wasn’t all that far ahead of the less successful human predictors, while Fangraphs came in a comfortable second. The composite projections I calculated in the original post would have been the second-best system by this methodology, and successfully predicted the Rays’ total after some significant disagreement between the four sets.

Also included are the averages per team, to give an indication of just how predictable each team was. Although this author was quite pleased with only missing the Giants’ total of 84 by two wins, the truth is that the Giants were clearly the most predictable team this season. Both PECOTA and EW guest Grant Brisbee nailed the win total precisely, and Fangraphs’ 3-win miss looked positively huge by comparison. No other prediction from any of the sets was spot on, but several only missed by one, including BttP alum and current BP star Matt Trueblood’s Phillies total of 64.

At the other end of the spectrum was, of course, the Kansas City Royals, and no system hates the Royals more than PECOTA, which whiffed on the AL Central champs’ total by an incredible 22 wins. If PECOTA hadn’t disliked KC and Pittsburgh so much, it certainly would have made a contest of this. Let that be a lesson to you, PECOTA. All four sets were also much too optimistic about the Athletics and Reds, with double-digit misses all around.

Readers of the original prediction post or EW Facebook group members may also remember that listeners were pitted against PECOTA in an over/under poll for each of the original 30 projected win totals from early February. Initially I just calculated whether the highest percentage of voters was correct on the over/under in each case, but that didn’t strike me as an entirely satisfactory way to determine a winner, especially considering the score came out at 15-15. The over/unders were therefore also evaluated using the following entirely arbitrary scoring system:

-PECOTA was automatically awarded a win for correctly predicting a win total.

-If PECOTA was within six wins (just under one standard deviation) of a team’s actual total and the majority of listeners failed to correctly pick the over/under, PECOTA took the win.

-If PECOTA was within six wins of a team’s actual total and the majority of listeners did correctly pick the over/under, it was scored as a tie

-If PECOTA missed by more than six wins and the majority of listeners correctly picked the over/under, the listeners won.

-If PECOTA missed by more than six wins and the majority of listeners was also wrong with the over/under, neither side received a point.

The results were as follows:

Of course, after my efforts to decide a winner, the final score works out at 8-8. with two ties and ten totals both sides were pretty wrong about. Frankly, PECOTA got away with a couple, because had just a few more listeners come to their senses and realised that the Rockies would be terrible, it would have lost a win. It was also six wins away from both the Rangers and Braves totals, which I generously awarded as wins to make PECOTA feel slightly less bad after receiving such a drubbing from the essay writers. I’m therefore calling this for the listeners; make that 2-0 to the humans.

Predicting win totals isn’t the easiest job in the world. Were our sets any better at predicting the overall standings? Below is the same table as our win total comparison, but this time comparing the predicted rank with the final rank of all 30 teams.

The overall ranking of the four sets doesn’t change, but BttP’s predictions nearly overtake PECOTA. Baseball Prospectus Annual essay writers can congratulate themselves on comfortably defeating two projection systems, and now must carry the incredible weight of the baseball community’s expectation going into 2016.

This second table perhaps makes it even more obvious how impossible it was to predict the AL this year. The Nationals and Padres aside, the NL predictions had an average difference well into the single digits; our projections collectively missed by double-digits for more than half of the AL teams. There was not a single division winner correctly predicted by any set in the AL, and all four also missed on the Nationals. At least both humans and projection systems alike got the Cardinals and Dodgers right.

Finally, congratulations must go to the Phillies, who successfully proved all four projections right by finishing dead last. Sometimes, you can predict baseball.