Beyond the Box Score: The '+' Concept
When I started entering all this play-by-play data, one of my main goals was simply to apply some of the basic sabermetric ideas to football. I mean, if they make sense in one sport, they should make sense in another, no? The idea behind my 'EqPts' (and therefore PPP) measure came from two baseball measures: 'EqR', the Equivalent Runs concept that take a series of offensive stats and determines how many runs those stats should have produced on average, and Expected Runs, the matrix that shows you, on average, how many runs you can expect out of specific "__ runners on, __ outs" situations. And of course the S&P (Success Rate + PPP) measure was an obvious rip-off of OPS.
Well, the next one I'm going to rip off (actually, I prefer co-opt) is the '+' concept. The idea of an Adjusted ERA or Adjusted OPS figure (also known as ERA+ or OPS+) starts with saying, basically, that not every 3.68 ERA (or 0.890 OPS) is created equal. Was it during the deadball era? Was it in a hitter's park or the Polo Grounds? You try to put everybody on as even a playing field as possible to evaluate their stats. That idea should work for football too, right?
Last year Colt Brennan threw for 416 yards and 6 TDs against Northern Colorado on September 1, while Tim Tebow threw for 304 yards and 2 TDs (and 15.8 EqPts) against a decent South Carolina defense on November 10. By all basic statistical accounts, Brennan's stats were insanely good and easily better than Tebow's performance against SC. However...could Brennan have put up Tebow's numbers against SC? What would Tebow have done against Northern Colorado? In a nutshell, the goal of the '+' concept is, for me, to adjust for what's expected against different opponents.
For every major measure I use, both the ones I created and the ones I, uhh, co-opted--Success Rate, PPP, S&P, Line Yards/Sack Rates, etc.--you could create '+' measures that compare an offense's or defense's performance to what their opponents typically averaged. And here's how we're going to do it, using a blurb from my Buffalo BTBS piece as an illustration:
Let's take [Buffalo's] October 4 matchup against Ohio, a game they won 31-10. For that game (without taking turnovers into account) they put up a 0.780 S&P and scored 30.3 EqPts, while Ohio garnered a .576 S&P and 12.0 EqPts. How did that compare to what an average opponent did against Ohio? Ohio gave up 22.0 EqPts per game and a 0.704 S&P while gaining 18.3 EqPts and a 0.665 S&P. So Buffalo gained 1.66 times more than the average Ohio opponent gained, 166% of normal. Ever heard of the OPS+ measure? Basically it compares people to averages, with a score of 100 more-or-less meaning that the person gained exactly 100% of what was expected. So if we use this concept, we can say that Buffalo's offense put up an EqPts+ measure of 166 against Ohio, meaning they gained 166% of what Ohio normally gave up. Get it? They also put up a 110.9 S&P+.
Meanwhile, if you flip the equation, you can come up with a defensive score as well. (You have to flip the equation so a good defensive performance also results in a score above 100.) Buffalo's defensive scores against Ohio were a 109.6 EqPts+ and a 115.6 S&P+.
So to summarize that in pretty bullet points...
- For every game they play, a team's output (offensive and defensive) is compared to the expected output considering the team they're playing.
- 100 = dead average. Over 100 = good, under 100 = bad.
- The purpose of this is to give (or take away) credit for teams' statistics based on the quality of their opponents. Technically you could do this same thing with rushing yards, points (the real kind), or anything else, but since I've been doing all this measuring of EqPts, success rates, etc., and since I'm very much sold on the quality of these measurements, by god we're going to use them.
- The trick here is that, for each game played, there are two sets of offensive ratings and two sets of defensive ratings. Why two? Well, taking into consideration the Buffalo-Ohio game above, Buffalo got an offensive score for their performance against Ohio's averages and a defensive score for their performance against Ohio's averages. However, Ohio also got offensive/defensive scores compared to Buffalo's averages (a 47.3 EqPts+ on offense and a 51.8 EqPts+ score on defense, if you're scoring at home...or if you're alone). The key is that Ohio's score isn't simply an inverse of Buffalo's score. If that's still not clear, I'll illustrate with more examples, but for now we'll move on.
So what's the point of doing all of this? Quite simply, we can more accurately measure how good teams really were. The best way to illustrate that is to show you some rankings. I have lots of '+' measures to choose from, and I haven't yet figured out the most accurate one to use, but let's just run through some for now.
EqPts+ (Offense)
1. Florida (172.45 avg)
2. Oregon (159.16)
3. Louisville (156.05)
4. West Virginia (155.88)
5. Tulsa (154.83)
6. Kentucky (151.31)
7. Missouri (151.02)
8. Texas Tech (150.68)
9. LSU (149.22)
10. Navy (148.74)
Now, none of the names on that list are particularly surprising, but how do these rankings compare to pure scoring and yardage rankings?
Florida: #3 scoring offense, #14 total offense
Oregon: #12 scoring offense, #10 total offense
Louisville: #18 scoring offense, #6 total offense
West Virginia: #9 scoring offense, #15 total offense
Tulsa: #6 scoring offense, #1 total offense
Kentucky: #15 scoring offense, #24 total offense
Missouri: #8 scoring offense, #5 total offense
Texas Tech: #7 scoring offense, #2 total offense
LSU: #11 scoring offense, #26 total offense
Navy: #10 scoring offense, #22 total offense
And what about some of the teams who ranked high in the 'regular' rankings but didn't appear in the top 10 above?
Hawaii: #1 scoring offense, #3 total offense...#12 in EqPts+
Kansas: #2 scoring offense, #19 in EqPts+
Boise State: #4 scoring offense, #18 in EqPts+
Houston: #4 total offense, #37 in EqPts+
Oklahoma: #5 scoring offense, #14 in EqPts+
As you would expect, teams with tougher slates--i.e. a lot of SEC teams--were held in higher regard using the '+' concept.
So what about the S&P+ measure? That takes efficiency and explosiveness into account instead of simply explosiveness.

S&P+ (Offense)
1. Florida (again!) (156.69)
2. West Virginia (134.71)
3. Navy (131.61)
4. Texas Tech (130.81)
5. Louisville (129.48)
6. Hawaii (129.07)
7. Oregon (128.11)
8. Missouri (127.30)
9. LSU (126.82)
10. California (125.07)
So first of all...kudos to Florida and to Heisman voters. I admit it--they nailed it with Tim Tebow. I'm a Mizzou (and therefore Chase Daniel) fan, so naturally I'm supposed to think that Tebow is overrated and Daniel should have won the Heisman. I was rather unimpressed with the '20 TDs passing, 20 rushing' thing (if Mizzou had only called QB keepers inside the 5, Daniel would have that many rushing TDs too), but...no matter what I thought of that particular distinction, Tebow QB'd what was simply the best offense in the country according to these numbers, and since he basically was the rushing game...yeah, Tebow gets some dap.
And just for fun...
Rushing EqPts+ (Offense)
1. Navy
2. West Virginia
3. Oregon
4. LSU
5. Florida
Some dap for Navy there as well...of course they put up big-time rushing numbers running Paul Johnson's option system, but they apparently did it against a series of respectable rushing defenses. This should make Georgia Tech fans happy.
Passing EqPts+ (Offense)
1. Louisville
2. Texas Tech
3. Hawaii
4. Tulsa
5. Florida
Again, no surprising names here...though it's doubly impressive that Florida was in the Top 5 in both.
Quick sidebar: a few interesting conclusions were reached about the 'national averages' post I wrote last week, including this one from SMQ:
I welcome this, personally, as an empirical base that bolsters my usual emphasis on keeping the entire playbook open: outside of talent, predictability is the number one killer of offenses, and defenses that stop the run and make offenses one-dimensional are, well, see above.
The 'above' being a youtube video of Auburn sacking Alabama roughly 176 times. Well, an EDSBS commenter made an interesting point--Florida was one of the most predictable teams in the country, and yet they were by far the most successful. I figure there are two main explanations for that. 1) Execution matters. Call it the Remember the Titans Postulate. Even if they know what's coming, they still have to stop you from doing it. 2) Down yardage matters. Florida was able to succeed on third downs because they kept things manageable on third downs. Not even Superman Tebow would succeed very often if most of his third downs were 3rd-and-9's instead of 3rd-and-3's. It does go to show how interesting football is, though. There are a million things that have to happen for you to succeed or fail. Though I guess you don't need a bunch of new statistics to reach that conclusion...so forget I said anything.
Anyway...on to defense. I should mention that, while I have no problem sharing the rankings, there's a caveat: I still have some tinkering to do with these numbers. For one thing, it's very much possible for an offense to put up like 0.32 EqPts in a given game. Since you're flipping the equation now, the opposing team's offensive average is in the numerator, and the 0.32 would be in the denominator. If you take that team's average (say, 15.0) and divide it by their 0.32 output for that game, you're going to get an insanely high defensive EqPts+ score (4687.5, to be exact), and obviously that can skew averages.
The first thing I did was put a cap on scores. For all '+' numbers so far, no particular unit's game score can be higher than 300. I have to do some further tinkering, as that still leads to a lot of 300's (and therefore higher averages than on the offensive side of the equation), but here's what we've got so far.
EqPts+ (Defense)
1. Ohio State (216.02)
2. USC (200.94)
3. Kansas (193.43)
4. LSU (187.53)
5. Hawaii (180.24)
6. Boise State (179.52)
7. Oklahoma (171.53)
8. Texas (171.08)
9. Texas Tech (170.65)
10. Arizona State (168.49)
Beyond the first 4 teams and OU, that's not exactly what you would have considered a murderer's row of defenses there.
S&P+ (Defense)
1. Ohio State (182.64)
2. USC (161.45)
3. LSU (161.18)
4. Virginia Tech (156.96)
5. Rutgers (156.62)
6. Oregon State (156.51)
7. Oklahoma (150.14)
8. Penn State (149.37)
9. Boise State (148.44)
10. Arizona State (146.95)
These numbers as a whole run a little lower, and that makes me a bit more comfortable about them, but...those are still a lot of the same teams there. With either measure, tOSU is by far #1, which suggests that they at least somewhat earned their good fortune last year despite a horrendously weak schedule. They will probably remain #1 no matter what kind of tweaking I do, so...good for them, I guess.
Rushing EqPts+ (Defense)
1. West Virginia (#18 in plain old rushing yards allowed per game)
2. Texas (#6)
3. Florida (#10)
4. Air Force (#45)
5. Navy (#81)
Passing EqPts+ (Defense)
1. Texas Tech (#12 in passing yards allowed per game)
2. Hawaii (#37)
3. Ohio State (#1)
4. Rutgers (#5)
5. Kansas (#49)
Now...this is pretty damn interesting, really. Teams with good rushing games (WV, AFA, Navy) came out of nowhere to place at the top of the rushing defense list, while pass happy teams (Tech, Hawaii) were at the top of the passing defense list. Of course, this is measuring EqPts Allowed in those areas...teams playing Tech and Hawaii were likely to run the ball a lot (and therefore avoid passing) to keep the ball out of Tech/Hawaii's hands. So let's check out S&P+...since it looks a per-play average instead of a per-game average.
Rushing S&P+ (Defense)
1. Ohio State (#3 in rushing yards allowed)
2. Oregon State (#1)
3. UCLA (#14)
4. Penn State (#7)
5. Wyoming??? (#27)
Passing S&P+ (Defense)
1. Ohio State (#1 in passing yards allowed)
2. Rutgers (#5)
3. Utah (#11)
4. Arkansas (#23)
5. Virginia Tech (#31)
Okay, this is better. I obviously don't need these numbers to precisely resemble the yards per game stats--why the hell would I be doing all this if that were the case?--but it is certainly strange that these numbers could be so drastically different. At least the offensive numbers were in the same ballpark. Any ideas as to what I should maybe do different are welcome.
CONCLUSION
So what I've just done with some basic S&P and EqPts numbers, I could do this with every stat in the catalog...S&P+ by down, quarter, field position, etc. Line Yards/Sack Rates, etc. And to some degree, I'm going to do just that (though I don't yet know what I'll find that will be interesting enough to share!). There are four purposes to all of this.
1) This could obviously be interesting form an evaluative perspective. It's always fun trying to come up with more and more precise ways of evaluating and ranking teams.
2) It could be even more interesting from a predictive perspective. The only thing more fun than ranking teams is making accurate predictions, am I right? Of course, the main problem is, with only one year of data it's somewhat impossible to actually know what's predictive and what just seems like it should be predictive. We'll all be discovering together which tools are and are not good forecasting tools.
3) It will make for a more informed experience while watching football...which is just awesome. It was fun watching Mizzou games last year, knowing that an opponent's best quarter is Q2, or little things like that. It will be even more fun this year, knowing even more.
4) It's becoming clear to me that there are little pockets of stat nerds out there--stat nerds who enjoy the practicality of stats, which is even better--and this is an excellent opportunity to build something of a communicative community based around college football stats. I'm just going to keep on writing these posts and asking for opinions and new ideas and seeing where this goes.
All comments on 'Sunday Morning Quarterback' are the views of the individual commenter and do not necessarily reflect the genius of SMQ, Sports Blog Nation, etc.
0 recs |
13
comments
Read Related
Comments
I have a couple of questions/ideas...
1. In the EqPts section Navy and WV placed best against the run while Hawaii and Texas Tech placed best against the pass. Wouldn’t this logically happen because these teams spend most of their time practicing against their respective offenses who dedicate themselves to one way of moving the ball? I mean, Navy should be really damn good at stopping the run because they face it all of the time.
2. While thinking about Navy I harkened back to their game against Notre Dame which went into overtime. Do you have any idea on how to normalize the affects that OT and, for that matter, situational football has on the stats? Sometimes all you need is a field goal to win at the end of the game and you could conceivably put the ball in the end zone but you elect to kick. It seems to me that situational football could throw off the numbers a decent amount.
3. Would another logical step to take in testing the EqPts stat be comparing the difference in average points scored in a game/points scored in a specific game against the difference in average EqPts/ EqPts scored in a specific game?
Sorry if that last part was hard to understand.
by gahnki on Jun 21, 2008 9:45 PM EDT 0 recs
Responses...
1. I thought about that after I submitted the post…that would, to some extent, explain why they’d be good at containing similar offenses…it’s something to look into, though I’m certainly wondering why those teams would be at the top of my list here, and not the typical ‘rushing/passing defense’ lists…
2. I have all sorts of situational stats that I haven’t fully unleashed yet—all of these stats I’ve measured (S&P, PPP, success rate, etc.), I’ve measured by down, quarter, score margin, etc. It will be pretty fun (to me, anyway) to look into who did better in certain situations, and how that may have affected their overall numbers…and I plan to do just that.
3. Are you talking about something like the % difference between EqPts and actual points? If so, here’s what I’ve gathered so far. By taking into account this EqPts figure, along with a ‘turnover costliness’ figure I’ll discuss in detail later, my ‘equivalent’ scores are about 8 points off of what the actual scores are. Which suggests that special teams and luck account for about 8 points a game (at least that’s what it suggests in my head). I’m tossing around different ways of measuring special teams, and we’ll see where that goes.
And with all of this, I’m extremely open to suggestion, so whatever ideas you have, feel free to pass them along…
http://www.rockmnation.com
Thrust nunchuk upward!
by The Boy on
Jun 21, 2008 10:47 PM EDT
up
0 recs
Well, I’m just interested in further testing EqPts because it’s still a new statistic that could have some holes in it. Success Rate is a fantastic stat that makes a lot of sense so I’m just focusing on testing some of the other stats validity to make sure the numbers are solid.
Let me try and clarify what I originally meant. Let’s say FSU scores 24 point against Florida’s defense while the national average against Florida’s D is 21. That would be a +3 for Florida State’s offense. Now we compare that +3 to the EqPts difference between what FSU scored on offense against what teams normally score on offense against Florida. Would that be a way to compare just how close the EqPts stat matches up with the points scored in a game?
by gahnki on
Jun 23, 2008 2:21 PM EDT
up
0 recs
gotcha...
...that’s definitely something in the works. I want to get special teams figured out first (and write a corresponding post about that), but comparing points (and Pts+) to EqPts (and EqPts+) is certainly an end goal…
http://www.rockmnation.com
Thrust nunchuk upward!
by The Boy on
Jun 23, 2008 2:47 PM EDT
up
0 recs
I don't think this is true
“Navy and WV placed best against the run while Hawaii and Texas Tech placed best against the pass. Wouldn’t this logically happen because these teams spend most of their time practicing against their respective offenses who dedicate themselves to one way of moving the ball? I mean, Navy should be really damn good at stopping the run because they face it all of the time.”
People say this all the time, but it doesn’t make sense. Teams don’t practice against their own scheme. They watch film and practice against their opponents’ schemes. The only time Navy should have seen a similar offense in practice was the week it played Air Force.
by SMQ on Jun 23, 2008 8:21 AM EDT 0 recs
Spring Practice
What you said isnt completly true. Georgia Tech’s defense, for example, spent most of the spring practicing against an option offense. While come fall, they wont, they still have more familiarity with it than other teams.
by gtne91 on
Jun 23, 2008 11:06 AM EDT
up
0 recs
Spring practice and fall practice both have these teams facing the specific offense that their team runs. They will face the offense that their opponent runs the week of the game. Sometimes a team will practice against a specific opponents offense in fall practice but they are still facing the offense their team runs a majority of the time.
by gahnki on
Jun 23, 2008 2:07 PM EDT
up
0 recs
A-ha!
I figured out why the defensive rankings didn’t look quite right—it was a damn Excel VLOOKUP error on my part. It affected the EqPts+ rankings, but not the S&P+ ones. Here are the real rankings:
EqPts+ (Defense)
1. Ohio State (140.98)
2. USC (129.18)
3. LSU (128.24)
4. Virginia Tech (125.37)
5. Oklahoma (125.16)
6. TCU (123.28)
7. Rutgers (123.26)
8. South Florida (123.09)
9. Penn State (122.97)
10. Auburn (122.64)
Rushing EqPts+ (Defense)
1. Oklahoma (140.24)
2. Ohio State (128.09)
3. Wyoming! (126.49)
4. UCLA (126.14)
5. Penn State (125.09)
Passing EqPts+ (Defense)
1. Ohio State (186.76)
2. Rutgers (175.72)
3. Arkansas (153.56)
4. South Carolina (152.62)
5. Fresno State (144.81)
http://www.rockmnation.com
Thrust nunchuk upward!
by The Boy on Jun 23, 2008 4:13 PM EDT 0 recs
I applaud your enthusiasm and hard work – I’ve done a bunch of statistical work on OOC schedules for all the BCS teams, myself. The issue I’ve run into, though, is that you have no assurance that your samples are either random or independent. There are factors beyond our control here.
In your case, while the numbers are very useful, I hesitate to put too much weight on them. The reason goes back to why sabremetrics is such a successful set of analytical tools: in baseball, an out is an out. There is no clock and all at bats are created equally. While you’ve done a good job of taking a lot of the factors out (i.e. 3rd downs aren’t created the same as 2nd downs), there is still a problem of 1) how much time is on the clock and 2) is the team trailing and if so by how much. I’m honestly not sure how much you can do to neutralize these extraneous variables.
In any event, it’s interesting to see your analysis. Perhaps I’ll post my OOC schedule analysis at some point.
by The Iron Colonel on Jun 25, 2008 11:28 AM EDT 0 recs
Yeah...
...I tried keeping track of the clock for a while, but within the box score, the only guaranteed times mentioned are at the start and end of possessions…and that obviously makes it extremely hard to document each play with any consistency or accuracy. Football Outsiders have done a lovely job with NFL games, but that’s because they’re able to watch every game and document the clock, who’s on the field, etc. There are too many college games, and far too many untelevised college games, to do this, so the stats will always be somewhat limited in that regard. I’d love to know who was on the field for each play, as you can come up with great ”% of plays made” stats for defense, among other things, and you can document specifically who gave up a sack…things like that. But that’s simply not going to be possible for college football as a whole…
http://www.rockmnation.com
Thrust nunchuk upward!
by The Boy on
Jun 25, 2008 11:38 AM EDT
up
0 recs
Don’t take that the wrong way, I think your analysis is a big step up from what was previously available. I just recognize that the number of extraneous variables present in football is much larger than those present in baseball.
by The Iron Colonel on
Jun 25, 2008 2:34 PM EDT
up
0 recs
no offense taken by any means...
...I knew when I was jumping in that I wouldn’t be able to get into as much detail as other sports. However, there’s still a lot of potential here, and being able to measure things like the expected points from certain situations (and tons of other tendencies measured over 142,000 plays) is quite lovely…
http://www.rockmnation.com
Thrust nunchuk upward!
by The Boy on
Jun 25, 2008 2:52 PM EDT
up
0 recs
Out of curiosity, how did you transcribe the play by play data? I’ve done it manually (ugh) for several games to see how the mean YPC and compares to the median YPC and measure how much long runs skew the reported numbers, and that was painful. I assume you wrote some kind of script to handle that?
by The Iron Colonel on Jun 25, 2008 3:35 PM EDT 0 recs





