Sunday Morning Quarterback: An SB Nation Community

Navigation: Jump to content areas:


Pro Quality. Fan Perspective.
Login-facebook
New Blog: Voodoo Five for South Florida Bulls Fans!

On the Trail: Minimally Creepy Economist Edition

Normally, SMQ would be suspicious and probably immediately dismissive of an e-mail from the PR division of SAS, "The billion-dollar leader in business intelligence and analytics," i.e. prediction models based on hordes of data mining. After calling Friday to allay suspicions he was just one target of a media daisy cutter, though, SMQ is sufficiently interested in therecruiting model the company's beginning to hawk to run with the concept: based on the eventual decisions of 3,395 prospects from 2002-2004, three economists from Elon and Mercer universities and ERS Group Inc. in Tallahassee developed a model to a) determine the factors that do and do not matter in most recruits' decisions to attend a certain school and b) use those factors to forecast where each member of the Top 100 will sign this season.

This is pretty close to what SMQ is attempting with his stats relevance watch series in terms of applying data to success and determining a correlation that differentiates what generally works from what doesn't. The difference in the more sophisticated recruiting model is the inclusion of a predictive element that makes an educated guess . Which, if it works - and it has in this class, for 69 of the 87 Rivals Top 100 that had committed as of Wednesday, about 79.3 percent of the time - is a fairly lucrative proposition.

A couple decisions lowered that rate over the last two days: one was the surprise defection of L.A. cornerback Brandon Warren, not a commit but a longtime lock to sign with hometown USC, to Michigan, making jubilant the Big Blue hordes that freaked over the apparent loss of local star Ronald Johnson at the same position after some bizarre flap with UM coaches Wednesday (composing himself Friday, Brian thinks Johnson is still a possibility or the Wolverines - more on RoJo momentarily). The other is much further down the list, to linebacker Allen Bailey, who shocked the world (er, Jamie Newberg) by crossing his projected destination, Georgia, off his short list. Bailey hasn't committed yet to any other school, but the model predicted he'd wind up at UGA, and it won't be right unless he changes its mind again (more on the Bailey prediction later, too).

So, presuming Johnson is still in play at Michigan and Bailey is not going to Georgia, that leaves the model's success at 69 of 89 Top 100 commits, 77.5 percent with a dozen players still out there. That sounds like a moneymaking percentage.

Factors that matter, according to the model, only a couple of which are slightly novel:

• Whether the athlete made an "official visit" to a specific college

 • Whether the school is in a BCS conference

 • The distance from the high school athlete's hometown to a specific school

 • Whether the recruit is in the same state as a specific school

 • The final AP Ranking of a specific school in the previous year of competition

 • The number of conference titles a school has recorded in recent years

 • Whether the school is currently under a "bowl ban" for violating NCAA rules

 • The current number of scholarship reductions a school faces for violating NCAA rules

 • The size of the team's stadium (measured in terms of seating capacity)

 • Whether the school has an on-campus stadium

 • The current age of the team's stadium

So when the developers of the model write "high school athletes prefer winning programs that are close to home, are in possession of good physical facilities, and are in good graces with the NCAA," college football cognoscenti reply, "duh." It's no surprise academics and graduation rates have no place on the list, either, though BCS bowl appearances, NFL draft picks and past national championships also had no discernible influence. And none of those factors explain the success of Illinois, Ole Miss and North Carolina this season. The one counterintuitive element of the work is that reduced scholarships (due to availability, not NCAA sanctions) actually increases the likelihood of signing a top recruit, the explanatory guess there being fewer scholarships means less competition for playing time and exposure.

Remember, though, that the model predicts with better than three-fourths accuracy without talking to players, coaches, friends, parents or any expert  or knowing anything about a prospect, specifically, other than his ranking in the Rivals Top 100 and the location of his hometown. In other words, the eventual decisions of a certain set of 18-year-olds is extremely predictable on the whole, regardless the hormonal eccentricities of the individual. However obvious, at least this is an attempt to quantify the speculative vaguery of the seedy, seedy business, and a pretty effective one to date.  Compared with the predicted destination of Scout.com, here's the SAS model on the players still on the market as of Wednesday, one week before Signing Day:


RB Joe McKnight
River Ridge, La.

DT Marvin Austin
Washington, D.C.

CB Ronald Johnson
Muskegon, Mich.

WR Terrance Tolliver
Hempstead, Tex.
SAS Prediction SAS Prediction SAS Prediction SAS Prediction
LSU (22%) USC (19.6%) Michigan (20.3%) LSU (21.1%)
USC (19.8%) No. Carolina (17.3%) Mich. State (19.2%) Oklahoma (20.7%)
Ole Miss (18.7%) Tennessee (16.9%) Ohio State (18.5%) Florida (20.1%)
Fla. State (10.7%) Fla. State (16.2%) USC (16.9%) USC (19.9%)
Alabama (10.1%) Illinois (16%) Tennessee (18.2%)
Scout Prediction Scout Prediction Scout Prediction Scout Prediction
USC Fla. State Florida Florida

S Chad Jones
Baton Rouge

RB Noel Devine
Ft. Myers, Fla.

WR Deonte Thompson
Belle Glade, Fla.

RB Lennon Creer
Tatum, Texas
SAS Prediction SAS Prediction SAS Prediction SAS Prediction
LSU (18.5%) West Va. (19.3%) Florida (14.7%) Oklahoma (19.6%)
Florida (16.3%) Alabama (18.7%) Miami (14.4%) Tennessee (18.1%)
USC (15.8%) Florida (17.6%) USC (14.4%) Texas (15.5%)
Fla. State (15.4%) Fla. State (14.8%) Ohio State (13.2%)
Miami (13.3 %) USC (14%) LSU (13.1%)
Scout Prediction Scout Prediction Scout Prediction Scout Prediction
LSU ? (WVU/FSU) Florida Tennessee

DE Sidell Corley
Mobile, Ala.

DB Stefoin Francois
Reserve, La.

LB Allen Bailey
Darlen, Ga.

WR Brandon Gibson
Mobile, Ala.
SAS Prediction SAS Prediction SAS Prediction SAS Prediction
Alabama (19.9%) LSU (17.7%) Georgia (20.2%) Auburn (20.6%)
Florida (19.3%) Fla. State (16%) Florida (19.9%) Alabama (20.2%)
Oklahoma (18.2%) Arkansas (15.5%) Alabama (18%) So. Car. (17.9%)
Tennessee (18.1%) Tennessee (15.4%) Miami (17.6%)
Scout Prediction Scout Prediction Scout Prediction Scout Prediction
Alabama LSU Miami Alabama

Opinions diverge on seven of the twelve, including, most interestingly, McKnight, the top-ranked undecided across the board, and Johnson, whose midweek backtrack leaves the door open for Florida. SMQ mentioned Allen Bailey above, the Georgia linebacker who apparently was considered a UGA lock before dropping the Bulldogs last week. But where "the recruiting world" was "shocked" by that move, the SAS model only had Georgia a couple tenths of percent ahead of Florida, one of the new favorites for Bailey along with Miami and Alabama.

Two commits, Noel Devine (grades) and Chad Jones (baseball) aren't very likely to sign anything Wednesday. Otherwise, SAS only needs one commit from the other ten to go its way to hit 70 percent for the second straight season. We'll return later in the week to check on its success.

0 recs  |  Comment 8 comments

Story-email Email Printer Print

Comments

Display:

That's really interesting...
There's a lot of different ways coaches can use the data. More than the 70% correct I think there's more to learn from the 30% that don't follow the model.

It would be interesting to interview the recruits that didn't follow the model and start making a list of other criteria that influenced them to make their choice.  It's in this 30% I suppose that the salesmanship of the recruiter comes into play (as well as other factors).

If a coach knows a certain recruit is very likely to commit to his program he can perhaps spend less time, energy and resources on that recruit and dedicate more time to filling another need with a recruit that is less likely to come to the school.  In other words getting a recruit to shun the "natural" or "predictable" choice.

by Henry Gomez on Feb 4, 2007 11:33 AM EST reply actions   0 recs

Am I reading this correctly?
[quote]Within the sample used to build the model, we accurately predicted approximately 68 percent of the choices of high school athletes.[/quote]

Are they serious?  You can't test a prediction model on the data you used to build it!  From this one sentence alone I'm beginning to be skeptical of everything on that webpage. Unless I'm reading something wrong.  

by samr on Feb 4, 2007 1:59 PM EST reply actions   0 recs

Yea,
You read that right. Poor form, but I'm guessing this season's predictions were from out-of-sample data. 77.5 percent is pretty good. I'm interested to see the article (read: dissect it without apology) when it comes out.

I learned a new word. co·gno·scen·ti: persons with superior, usually specialized knowledge of a field or highly refined taste.

by EconoMz on Feb 4, 2007 2:40 PM EST up reply actions   0 recs

Don't think so
"The model was built on a database capturing characteristics and decisions of 3,395 recruited athletes for the three 'recruiting seasons' between 2002 and 2004."

Any application of the model for any year 2002-2004 would obviously be retroactive. Although it should have come out better than 68 percent, I would think. I don't think there's anything wrong with necessarily running the numbers gathered from 2002-04 against any of those single seasons as an exercise, but I agree it's not an example I'd use to sell the system - it's not a valid test of the model's accuracy. It doesn't change the model, though, and has no effect on its rate of success in the future. I suspect it will (or should) be tweaked every year based on new information.

by SMQ on Feb 5, 2007 2:12 AM EST up reply actions   0 recs

Ronald Johnson
committed to USC.

by Henry Gomez on Feb 5, 2007 1:13 AM EST reply actions   0 recs

Depth Chart
One glaring thing they seemingly left off the list is the depth chart. Kids are more likely to go someplace where they have a good chance to play right away. The style of their play also makes a difference. A school with a conservative style of offense is going to have a tough time landing superstar QB's and WR's.

by Mike on Feb 5, 2007 3:54 PM EST reply actions   0 recs

Actually...
They specifically addressed playing time in the form of available scholarships, and so did I: "...there being fewer scholarships means less competition for playing time and exposure."

It would require a statistical analysis to develop an empirical "style of play," I guess. It's not able to take a coach's personality into effect, either, that  presumably being a factor behind the success of Butch Davis and Ron Zook right now at unsuccessful programs.

by SMQ on Feb 5, 2007 6:50 PM EST up reply actions   0 recs

Comments For This Post Are Closed


User Tools

College Football Coverage
Start posting on Sunday Morning Quarterback »

Join SB Nation and dive into communities focused on all your favorite teams.

Connect_with_facebook

FanPosts

Community blog posts and discussion.

Recent FanPosts

+ New FanPost All FanPosts >

SBNation.com Recent Stories

Tennessee running back Bryce Brown (11) runs past Ohio's Melvin Payne (55), Dak Notestine (51) and Lee Renfro (32) during the first half of an NCAA college football game Saturday, Sept. 26, 2009 in Knoxville, Tenn. Tennessee won 34-23. (AP Photo/Wade Payne) +1 updates

Bryce Brown Leaving Tennessee? Former 5-Star RB 'Not A Part Of The Team'

Ohio State head coach Jim Tressel, left, jokes with offensive lineman Bryant Browning (70) before taking the team picture during an NCAA college football media day, Thursday, Aug,13, 2009, in Columbus, Ohio. (AP Photo/Terry Gilliam)

Jim Tressel Receives Two-Year Contract Extension, No Raise

The offensive line is reflected in the visor of Wisconsin running back John Clay during the second half of an NCAA football game against Michigan, Saturday, Nov. 14, 2009, in Madison, Wis. (AP Photo/Morry Gash) link

Wisconsin RB John Clay Will Miss Entire Spring Season

More from SBNation.com >


Official Partner of CBS Sports