NCAA – CoachRank Quick FAQ (AKA Read Before Killing The Messenger)

What Is CoachRank?

CoachRank is a data-driven, (hopefully) objective system of measuring coach performance over a long-term (three-year and above) period of time.  CoachRank includes multiple factors to formulate a score on a 0-100 scale that seeks to enable valid and accurate comparisons between DI coaches regardless of program or conference size.  “Season” scores are taken for every season a club plays under a current manager (currently dating back to 2009), with the average of those season scores being the base CoachRank score.

What Factors Are Used To Calculate CoachRank?

The exact factors and formula used to calculate the CoachRank score are proprietary information at this time, but the six factors used to calculate the base score are a combination of a club’s league performance, conference tournament performance (if applicable), and NCAA Tournament performance.  The base score is used mostly to compare coaches at the same club while limiting the effects of shifts in conference strength in measurements.

The adjusted score, is used for just about everything else with CoachRank, including the big list.  Adjusted scores are calculated by multiplying the base season score by an adjustment percentage calculated by measuring the school’s conference’s performance in that season’s NCAA Tournament. The adjustment percentage can give a maximum of fifty percentage points to multiply the base score with, but the highest adjustment percentage at current is 24% for the ACC in both 2012 and 2013.  As with the base score, adjusted season scores are averaged to get a total adjusted score for a certain time period.

Adjusted scores allow CoachRank to accurately reflect the difficulties of playing in and succeeding in the best conferences, while also not arbitrarily giving an inaccurate boost to clubs from big conferences who don’t perform well.

What About The Conferences That Don’t Play Conference Tournaments?

Measures have been taken to compensate for the lack of conference tournaments for the Ivy League, the Pac-12, and the WCC.  While not perfect, we feel that these measures successfully mitigate the lack of a conference tournament to be measured for these leagues.

Why Isn’t <INSERT CRITERIA> Included?

-Anything Financial – There isn’t data available for all 300+ clubs.  If there is, be my guest and forward it here.

-Non-Conference Results – The vast differences in quality in non-conference scheduling make this an absolute impossibility to calculate between clubs.  I mean, are we seriously supposed to regard the non-conference schedules of Iowa and Penn State similarly?

-Anything Earlier Than 2009 – The eventual goal is to get every season into the database, but for practical purposes, we’ve chosen to only examine the period from 2009 onward.

Why Aren’t These Results Weighted By Year?

We actually experimented with weighted results by season, giving progressive bonuses with the most current season getting the biggest bonus.  However, the differences between these weighted results and unweighted results were marginal.  Additionally, concerns about a disproportionate bonus being applied to one good season in a string of four other bad ones and vice-versa led us to drop plans for weighting results by season at this time.

Hey Wait, Where Is…?

-Anyone Who Took Control Of A Team After the 2011 Season – This data set only includes coaches with at least three seasons at their current school.  We deemed this the bare minimum amount of data that would provide a suitably accurate picture of performance.

-Anyone With Less Than Three Seasons of League Performance Data – A big part of the CoachRank formula is league performance.  Obviously, if there isn’t league performance data, it’s pretty hard to get an accurate season score.  Thus, if a club played as an independent in a season, that season is excluded from the data, meaning that some schools/coaches with less than three seasons in which they played in a league are not included in these rankings.

What About Schools Transitioning to Division I From Lower Divisions?

Provided these schools have at least three seasons of league data, they’re included in the rankings, with Division II data calculated into the CoachRank measures.  Steps have been taken to compensate for clubs not eligible for postseason play as they transition to Division I.

This Is Biased Against Large Conference Schools!

Note: There are plenty of opportunities for bigger conference schools to rack up points despite playing in tougher leagues than some smaller clubs who can feast on their woeful brethren.  Bigger clubs usually fare better in NCAA Tournaments, which is a big component in the CoachRank system.  Bigger clubs also usually play in conferences which get the adjustment bonus thanks to their NCAA Tournament performance.  So really, bigger conference schools shouldn’t complain too much.

This Is Biased Against Small Conference Schools!

Look, the NCAA Tournament isn’t the be all, end all if you want a decent rating in CoachRank.  Win and win often in your league and rake in a couple of conference tournament titles and you’ll be fine.  Seven of the top twenty-five are from non-Big Eight conference clubs.  Will a small conference club ever get to the top of the mountain?  Probably not.  But there’s no reason why they can’t get within a hop and a step away from the top.

What’s In Store For The Future?

The long-term aim is to input every season, from every DI club into the database.  Eventually.  But if it gets done, we’ll be able to compare coaches from different eras and get a broader and complete picture of coaches throughout their whole careers spanning multiple programs (or a long-term stint at one club).  We’ll also be able to compare programs as a whole regardless of how many managers they have had.  It’s going to be a lot of work, but it’ll be a great project once complete.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>