Are the committee’s selections slowly getting better or just easier to predict?


The 65-team field was statistically puzzling in 2007. Things have been more predictable lately, though the courts are less distinctive visually.

One way to think of the NCAA tournament is as the most popular nightclub in town. The men’s basketball committee is the bouncer, of course, stationed outside the club and working the rope line. There are 351 teams queued up hoping to get in, and the bouncer gives familiar and knowing nods to the first 20 or so teams as they breeze in. It’s pretty much the same group every year.

Next, after the usual teams have been waved through, there is always the same number of unfamiliar and oddly attired out-of-towners who show up. Once they explain that they’ve won the conference tournaments in their one-bid leagues, however, these teams are also let in.

Lastly, there are the toughest decisions of all. These are the final at-larges, and up until a few years ago the best way to get in was to engage in some form of the following conversation. For example this particular discussion took place in 2012:

SOUTHERN MISS: Uh, hi, I’m Southern Miss. I think I’m on your list for 2012.

BOUNCER (suspicious, checks clipboard). Nope, sorry, pal. Don’t see you. Wait behind the rope, please.

SOUTHERN MISS: I think there must be some mistake. I’m pretty sure I’m supposed to be on the list. My RPI is 21.

BOUNCER: Right this way.

In 2016 the RPI still functions as the bone and sinew of the team sheets that the committee uses to evaluate the entire field, and very possibly this is a lamentable state of analytic affairs. For its part the NCAA has maintained until it’s blue in the face that the committee really, truly and absolutely does look at more than just one aging metric that has no basketball-specific content and thus can be (and is) applied to any number of sports.

Well, maybe the NCAA’s right. In recent years something rather interesting has occurred on the rope line. The RPI still offers one method of entry, but it’s not the only way into the club. Teams that look good on basketball-specific metrics like KenPom, the BPI, Massey Ratings,, KPI Rankings, et al., are now getting into the tournament at a higher rate than we saw in previous years.

I looked at the average pre-tournament ratings of the five best eligible teams according to KenPom that have been left out of the tournament each year since 2002. It turns out the 2015 selections comprised the most KenPom-congruent field we’ve seen over the course of those 14 tournaments. (Congruent in terms of population only, of course. Seeding is another matter.) Each field is its own snowflake, naturally, but the overall trend across the past 14 tournaments has been toward greater agreement with the basketball-specific metrics. And, as first pointed out by Michael James of The 14 Game Tournament site, this trend has actually accelerated in more recent years.

The 2007 field teaches a lesson, but maybe not the one we think it does
Conversely, the 2007 field marked the nadir for the “We look good at KenPom” crowd. Syracuse, Clemson, Mississippi State, Florida State and Air Force were all ranked at No. 35 or higher on Selection Sunday by Ken’s laptop, but none of them received a bid. Are you not outraged?

Hold that thought. Maybe 2007 can tell us something important about the inherently unjust task of selecting which teams get to play for a national championship. Simply put, I doubt very much that a committee populated by members serving multi-year terms was somehow uniquely incompetent in one particular March.

Sure, you can make a case that the committee did whiff on the Orange that year, but the other four decisions were at least more understandable. The three major-conference snubs had a combined league record of 22-26, and in the Mountain West that season Air Force finished behind two teams that got in (BYU and UNLV) and tied with one that did not (San Diego State).

These were legitimately good teams (as a group they fairly dominated that year’s NIT), but in the games played before Selection Sunday they just didn’t give the committee enough reasons to bump other good teams. There will always be outliers, and the number of information sources that the committee now has at its fingertips should, one hopes, decrease the chances that any one of those sources — the RPI, a basketball-specific rating system, a number for “good wins,” or anything else — will be the tail that wags the dog.

Everyfan his or her own bracketologist…no, seriously
The good news for amateur bracketologists is that the process that has arisen by default — doddering old feeb RPI is still in the driver’s seat, but at least now there are KenPom guardrails — makes the field easier to predict than ever. And by “easier” I am referring not only to accuracy rate but also and especially to time expended.

Four years ago Drew Cannon introduced the Easy Bubble Solver, which is simply the sum of a team’s rankings in the RPI and at KenPom. At the time Drew rolled it out, he received his fair share of rather smug feedback (I know because I was his editor) about how the EBS really wasn’t “all that great” at predicting past selections.

Then a funny thing happened. After it was introduced, the EBS did in fact become better at predicting tournament selections. Maybe this is a neat inversion of the phenomenon described by Jeff Sullivan, wherein umpires in baseball stopped being influenced by catchers who were good at pitch framing because the umpires had become aware of metrics that measured how much they were being influenced by catchers who were good at pitch framing.

Maybe now that the committee knows that things like basketball-specific metrics do exist, they can’t help blending those insights in with what the RPI-structured team sheets are decreeing. In any event, Drew’s EBS offers today’s busy fan a shot at predicting the field about as well as actual bracketologists, but with far less effort.

The “about as well” claim, granted, comes with a huge asterisk. In recent years the committee has taken to lobbing at least one analytic grenade per year, almost as a token yet cathartic gesture that screams “I got your fancy numbers right here, pal!” Two years ago that grenade assumed the form of NC State, and in 2015 it was UCLA.

No doubt you’ll recall that the Bruins then went on to make the Sweet 16, a development which was held by some observers to be sufficient justification for what on Selection Sunday had been a mystifying decision. Of course if an even better team had been selected instead of UCLA and had also been given the benefit of a questionable goal-tending call in the round of 64, they too might have made the second weekend.

Anyway, good luck with your projected bracket, and do keep the committee’s annual grenade in mind. Personally I hypothesize that as the committee becomes progressively more responsible with the other 67 selections it will wax even bolder and Trumpier with the 68th team. My hypothesis gives my alma mater a glimmer of hope.