Heaps and Nelson by the Numbers

Editor’s Note: The conventional stats favor Riley Nelson’s season over Jake Heaps’ season. But Heaps supporters point to the the tougher schedule Jake faced vs. what Riley faced. We asked loyal podcast listener and reader Nick to crunch some numbers and answer the question of 2011 Heaps vs. 2011 Nelson through more advanced statistics. His thoughts are below. No nerds were hurt in the crunching of these numbers.

In the spring of 2010 Jake Heaps came to BYU as one of the most-heralded recruits in the school’s history, the second coming of Ty Detmer, and the next great BYU quarterback. All of this before he ever played a down. Riley Nelson was a Utah State transfer who looked like a career back-up that might bring a nice change-of-pace option to the offense. Two seasons later Nelson is the undisputed coaches’ favorite and Heaps is transferring after two lackluster seasons. The big question is did Bronco make the right call? Far be it from me to question St. Mendenhall, but let’s see if we can shed some light on the situation using statistics.

Before I launch into this, let me explain the statistics I’ll be using. Aside from standard metrics like passer rating, total offense (YPG), and offensive points per game (PPG), I’ll also be using Football Outsider’s S&P+ defensive rankings for BYU’s opponents. For a full description of S&P+ see the FO website, but basically S&P+ measures a team’s defensive effectiveness while adjusting for the quality and tempo of the opponent.

Using a team’s S&P+ rating allows us to compute expected statistics – what an “average QB” with an “average offense” should do against the defenses BYU faced this year. You can think of this as similar to the “wins above replacement” statistics used in baseball, where a player’s performance is compared with what a statistically average player would have done. For a larger sample size I’ve included the Utah State and Idaho games where Nelson and Heaps split time and I’ve given each credit for the fraction of the games they played.

The first question is just how much harder was the schedule Heaps faced than the one Nelson faced. The answer is not as much as you might think. While Heaps faced three of the top four defenses BYU played this season in Texas, Utah, and UCF, he also faced the three worst in New Mexico State, Mississippi, and Idaho. Nelson, on the other hand, faced one good defense (TCU), one porous defense (Idaho), and four average defenses. In quantitative terms, Heaps faced an average defensive S&P+ rating of 104.46 (roughly equal to the defense of West Virginia, Boston College, or Nevada) and Nelson faced an average of 98.32 (roughly equal to the defense of Houston, Syracuse, or Northern Illinois).

Now let’s take a look at Heaps. He had an average passer rating of 111.8 on the season, while an average QB against the teams he faced would have had a rating of 127.6. Heaps came in at least 10 points below average in every game except against Idaho and New Mexico State. While Heaps was in at QB, BYU’s offense averaged 317 YPG compared to an expected output of 376 YPG. The Heaps-led offense came in at least 20 yards below average against every team it faced except Utah. Finally, with Heaps at QB BYU scored less than expected in all games except against Idaho and New Mexico State. Overall, the offense scored 20.1 PPG compared to an expected 25.6 PPG against those defenses. In all three metrics Heaps preformed significantly below average with particularly poor games against Mississippi, Central Florida, and Utah State compared to expectations.

Now let’s look at Nelson. He had an average passer rating of 157.3, 25.8 points better than an average QB’s expected rating of 131.5. Nelson’s QB rating exceeded the expected value by at least 24 points in every game except against TCU. With Nelson leading the offense, the team produced an average of 502 yards and 32.7 points per game, compared to the expected production of 391 yards and 27.0 points per game.

The Nelson-led offense rated above average in YPG in every game he played. He was within 5 points of expectations against San Jose State, TCU, and Hawaii and at least 10 points better than expectations against Utah State, Oregon State, and Idaho. Nelson had a significantly better-than-average season in all the metrics, buoyed by strong games statistically against Utah State and Hawaii.

Clearly the numbers side with Nelson. I have assumed, however, that the rest of the offense was static throughout the season, which of course was not the case. The evolution of Brandon Doman’s play-calling, the emergence of Mike Alisa as a power running option, and the momentum that comes from winning games all played a very real factor in the offense’s improvement as the season progressed. No statistical measure can really ever explain everything that goes into a win or loss, but the bottom line is that using the best statistics available, the numbers show that Nelson was a better-than-average QB this season and Heaps was not.