Announcement

**OnMAA** · 03-06-2011, 11:18 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by dave1381 View Post

No posts supporting an anti-WCHA conspiracy allowed in here. This thread instead seeks to spread the truth that the NCAA selection committee followed a NCAA handbook procedure to the letter, but it happened to be produce a terribly unfair outcome for WCHA schools this season.

I'll paste the selection and pairing portions of the NCAA handbook below. How would you improve on this? What would the budget impact be of your suggested improvements? If you propose criteria that's a radical departure from selection criteria in other sports, how do we ever convince the NCAA that it makes sense?

If you think the problem is simply money, where does the money come from? Is it fair to give women's hockey that money when most sports have 25% of their brackets seeded, and women's hockey gets 50% of its bracket seeded? Why would women's hockey deserve any more? (playing devil's advocate)

I want to start an actual informed discussion about this, while understanding the institutional constraints involved.

http://fs.ncaa.org/Docs/champ_handbo..._icehockey.pdf

Simple suggestion one: pass a by-law or whatever it takes to allow the National Collegiate Championship to be officially referred to as the D-I championship, as in this thread title. Everyone does it anyway. Who really cares that there are a few Division II teams who have never once played enough D-I opponents to be eligible? Why do they count in the RPI, and show up in the RPI and just confuse anyone who takes the time to look at it? Teams that don't play 20 D-I opponents shouldn't count in the RPI. Go away. Stay out of the D-I RPI.

While I'm not up on all the nitty gritty rules, two rules should be foremost in my mind:
1 - Number one and two seeds must be in opposite brackets
2 - Repeating matchup of teams within same conference must be avoided in round 1, specially if they just met in a league final.

Long story short, if you are not one of the top two seeds, you gotta beat one of them at some point to get the big prize, unless there are multiple upsets along the way.

**LakersFan** · 03-07-2011, 07:26 AM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

The one thing that troubles me about this years final eight teams is that there were only four non-conference games contested during the regular season featuring teams in the final eight: Merychurst at Cornell, Cornell at Mercyhurst, Mercyhurst at Wisconsin, and Dartmouth at Boston College. While this makes it easier to rank teams within each conference, it makes it more difficult to rank the top teams when it comes to seeding to top eight.

Part of this is "bad luck," as St. Lawrence and Harvard are two teams usually in the top 8 that experienced down years. Had those two teams made the top 8, the number of non-conference games between top 8 teams would have doubled.

So if the NCAA is not going to ditch RPI for a better method of numerically ranking the teams (please note that I am the Rutter in Rutter rankings, so I am biased in this respect), how about they start rewarding teams that play non-conference games against top 8 opponents, especially on the road, when it comes to seeding and hosting. This has been done in the past for hockey and basketball on the men's side, although I am not sure why it was dropped. The problem is usually trying to seed the teams in spots 3-6, so a reward based on schedule would encourage more games against the top teams in the country.

What bothers me is that there could be a trend developing. The WCHA, due to the heavy conference schedule demands and they fact that all of their teams are in the “west”, play very few games against the top teams in the east. Each year, usually two WCHA teams host quarterfinal games. This is deserved, however, as the WCHA has won every Frozen Four.

This year, however, the top two teams in Hockey East played schedules that ended up having only one non-conference game against a top 8 team (I am not counting the BU/BC Beanpot game, even though it is labeled non-conference). This was not intentional, as games were scheduled against ECAC teams that have made the top 8 in the past but did not this year. But will the success in terms of hosting two NCAA quarterfinal games create a change in how BU and BC schedule their non-conference games? I hope not.

Part of the problem is that schedules are created before the games are played, sometimes in the previous year. For example, Minnesota hosted Harvard this year after traveling to Boston last year. Now, if Minnesota knew that Harvard would be down and could have replaced Harvard with any of the other five non-WCHA top 8 teams and won both games, they would be hosting a quarterfinal game. I would love to see a system that rewards teams for scheduling two year, home-and-home series with other top teams and not breaking the agreement even though it may hurt their chances in terms of seeding.

One of my goals this summer is to finally do the research for the paper that looks at the effect of conference scheduling on the Pairwise Rankings. Does the WCHA conference schedules help or hinder the big 3? What is the impact of the limited Ivy league schedule? How do I do all of this in an unbiased manor? I think this really needs to be looked at.

**ARM** · 03-07-2011, 08:08 AM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by dave1381 View Post

If you think the problem is simply money, where does the money come from?

As much as we don't like the math behind RPI and PWR, where the system really broke down is where the committee was forced to ignore the math and prioritize travel. Is it possible to replace the "flawed" math by a better system and see improvement in the bracket if a financial override is still present?

Originally posted by LakersFan View Post

One of my goals this summer is to finally do the research for the paper that looks at the effect of conference scheduling on the Pairwise Rankings. Do the WCHA conference schedules help or hinder the big 3?

The CHA and WCHA are the only conferences where each conference rival plays a 4-game series each year. As non-conference play ends and the requisite number of losses must be spread around a conference, I think any league that is more balanced will tend to see its teams drift toward the middle. Because of the large number of intra-conference games, this effect is most pronounced in the WCHA. The team most adversely impacted this year was North Dakota, who suffered hugely that the WCHA's #5 was a quality team in Bemidji St. Of course, the same could be said of Princeton, but the Tigers weren't in the at-large discussion and they didn't face Quinnipiac as many times during the season. For WCHA teams, I think we've seen all three of the Big 3 move up in the PWR in years where they've enjoyed some separation from the league pack (2005, 2009), but do worse as a group mathematically in years where the conference has been more than 3-deep (2007, 2011).

**UCONN FAN** · 03-07-2011, 08:57 AM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Is it safe to assume that ND is the team that the WCHA fans feels should have been in? I guess maybe, just maybe, scheduling Vermont was not in their best interest. Vermont has been struggling for years so scheduling those games, especially when their conference schedule is so demanding, was not a good idea.

**hockeyfan111** · 03-07-2011, 09:07 AM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Agree with ARM. Although some extremists might argue that their particular team should have been in the Top 8, there is overwhelming consensus that at this point in time, the selection committee picked the right 8 teams. Now, did the selection committee rank the teams in the appropriate order? Not alot of consensus in the varying polls to indicate the selection committee's rankings are appropriate. But given a little latitude to reward teams that have won their conference championship (minus Mercyhurst), the biggest issue is surrounding the travel restrictions. In my opinion, this one item accounts for the majority of dilution associated with this year's selection and match up process. Going out of your way to address travel restrictions and having Dartmouth vs Cornell and Wisco vs UMD as NCAA tournament opponents in the quarter finals is one of the most disrespectful acts that could be directed towards Division I Women's Ice Hockey. Might as well say, be happy with your scholarships and the fact we let you play 30 games during the season and don't bother us again until next year. Total ineptness in leadership across the NCAA.

Originally posted by ARM View Post

As much as we don't like the math behind RPI and PWR, where the system really broke down is where the committee was forced to ignore the math and prioritize travel. Is it possible to replace the "flawed" math by a better system and see improvement in the bracket if a financial override is still present?

The CHA and WCHA are the only conferences where each conference rival plays a 4-game series each year. As non-conference play ends and the requisite number of losses must be spread around a conference, I think any league that is more balanced will tend to see its teams drift toward the middle. Because of the large number of intra-conference games, this effect is most pronounced in the WCHA. The team most adversely impacted this year was North Dakota, who suffered hugely that the WCHA's #5 was a quality team in Bemidji St. Of course, the same could be said of Princeton, but the Tigers weren't in the at-large discussion and they didn't face Quinnipiac as many times during the season. For WCHA teams, I think we've seen all three of the Big 3 move up in the PWR in years where they've enjoyed some separation from the league pack (2005, 2009), but do worse as a group mathematically in years where the conference has been more than 3-deep (2007, 2011).

**ARM** · 03-07-2011, 09:17 AM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by UCONN FAN View Post

Is it safe to assume that ND is the team that the WCHA fans feels should have been in?

I'm not saying that UND should have been in. My point is that for any 4th place team in the WCHA, the margin for error on making the tournament becomes small because they have a dozen regular-season games against the 3 teams above them in the standings, then presumably more in the tourney. North Dakota did well against Minnesota and UMD, but still wound up with 8 losses vs the top 3 teams. Then what has absolutely killed the 4th place teams over the years is that the separation isn't great between the teams below them, so they lost additional games to Bemidji State, and worst of all for the Sioux, Minnesota State. Are they as good as Dartmouth? My guess is that they are roughly equivalent. Does UND deserve to be in the field over the Big Green? No, not under the criteria that have been in place all season.

**Watson Rink** · 03-07-2011, 11:56 AM

Increasing Interconference Play

A Modest Constructive Proposal

There would be more interconference games during the regular season each year if academic year schedules and finances permitted visiting teams to play in four-team tournaments rather than in away-away doubleheaders. Imagine if Minny and ND had come to play travel buddies Harvard and Dartmouth instead of Minny and H playing back-to-back games against each other. BU and BC would make an attractive travel buddy pairing as well against WCHA teams.

(Parenthetically, the same holds true within conferences. It would have been a lot more interesting if H and D could have played each other at different junctures during the season rather than a home-and-away series a day apart. Or perhaps there are insurmountable scheduling difficulties in that one case due to the rest of the universe not having adopted Dartmouth's academic calendar? There didn't use to be back-to-back games when H and Bruno were travel buddies.)

On the other hand, even though the number of interconference games would be increased if H scheduled two more Hockey East teams rather than scheduling BU and BC during the regular season, old pre-split ECAC loyalties/rivalries are too strong to entertain that idea for a moment because the Beanpot format is elimination, not round robin, and there would therefore have to be some years in which there would be no H-BC or H-BU game at all. Unthinkable! Totally unthinkable!

**brookyone** · 03-07-2011, 12:02 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by hockeyfan111 View Post

Going out of your way to address travel restrictions and having Dartmouth vs Cornell and Wisco vs UMD as NCAA tournament opponents in the quarter finals is one of the most disrespectful acts that could be directed towards Division I Women's Ice Hockey. Might as well say, be happy with your scholarships and the fact we let you play 30 games during the season and don't bother us again until next year. Total ineptness in leadership across the NCAA.

Bravo!

**granddaddyscout** · 03-07-2011, 12:14 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Personally I think UND is quite a bit stronger than Dartmouth, but they failed to demonstrate that several times during the year against Vermont, MSU, and others. So they basically screwed themselves. That said I think where the system suffers the most is in not truly factoring in a meaningful strength of schedule factor, which I know is very hard to do when teams don't play against enough common opponents. I think where thing broke down this year the most is in Minny dropping out of the top 4 based on an OT loss to UW and then BC getting bumped up based on their conference championship. Using the same theory for bracketing having Minny at 4 would have sent UMD to Minneapolis and then they would have been forced to send Dartmouth to Madison. It would have then sent BC to BU, and MC to Cornell.

**Watson Rink** · 03-07-2011, 12:15 PM

An Immodest Fiscally Unsound Proposal

It would be the most fun if finances permitted using geographic diversity as the first criterion for seeding the quarterfinals.

Column A would have all the WCHA teams, Column B would have all the HE and ECAC teams, and the CHA teams would fill in columns A and B as necessary. If there were five WCHA teams, the easternmost would go to Column B, and if there were five WCHA teams, the westernmost would go to Column A. Columns A and B would each be seeded and A-1 would play B-4, A-2 would play B-3 &c.

Sure, bracket integrity would be violated to some extent, but it would be due to a consistent element of the structure, just as in MLB in any particular year the least successful division winner may or may not be a better team than the wild card.

Unfortunately, I haven't thought of a way to finance all the airfares beyond parents holding bake sales.

**papulaisle** · 03-07-2011, 01:34 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by ARM View Post

I'm not saying that UND should have been in. My point is that for any 4th place team in the WCHA, the margin for error on making the tournament becomes small because they have a dozen regular-season games against the 3 teams above them in the standings, then presumably more in the tourney. North Dakota did well against Minnesota and UMD, but still wound up with 8 losses vs the top 3 teams. Then what has absolutely killed the 4th place teams over the years is that the separation isn't great between the teams below them, so they lost additional games to Bemidji State, and worst of all for the Sioux, Minnesota State. Are they as good as Dartmouth? My guess is that they are roughly equivalent. Does UND deserve to be in the field over the Big Green? No, not under the criteria that have been in place all season.

A level playing field. That is the goal.
In attempting to create such, the "requirements", I offer some thoughts:

1. The 'criteria', or requirements which need to be fulfilled in order to make the tournament, get a bid'-as above mentioned by ARM, must be known before competition commences ("...be in place all season" as he states.)

2. Criteria/requirements need to be easily understandable by all. Even so, perhaps, the least fan knows what is at stake.

3. Strict adherence to the criteria must be rigidly followed without deviation.

Pretty easy. But apparently in coming up with workable requirements and/or criteria, things became complicated, some. Dave and Arm, others, have good understanding of what is needed, what they ARE, but maybe EVEN MORE simplicity is needed.

Or more discussion of the requirements is needed. Or maybe starting right from the 'get go' the RPI, etc. , other stats used, could be broadcast weekly, daily, so all folks would become FULLY CONVERSANT with who stood where and what would be needed by whom to get included in the tourney?

Deviation has seemed to occur say the fans from the west.

But maybe the 'deviation' isn't within (from) the rules or committee but within the overall background within which the teams compete-the US economy. Strict adherence to saving money which IS clearly stated in the requirements as to travel has been this year forced on the committee, member teams, by the difficult economic times.

People-above mentioned-who understand the requirements agree that the right eight teams made the tourney. And deviation from the usual travel arrangements isn't a deviation from the rules-merely a change brought about due to harsh economic times.

It would seem to be incumbent upon those who feel otherwise (as to whether criteria are faulty), to identify where the present criteria/requirements broke down and chose the less worthy team.They need to present changes that remedy the 'failure' that they see occurred (seems-how Dartmouth got in over UND...principally.)

But at the same time they also need to admit that the criteria-as constituted-DID choose the right eight this year based upon their implementation as written.

System worked. Economy caused usage of heretofore little used provisions (as to travel.) Criteria, perhaps, as to admitting lesser able team, need altering.

**granddaddyscout** · 03-07-2011, 01:51 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by papulaisle View Post

A level playing field. That is the goal.
In attempting to create such, the "requirements", I offer some thoughts:

1. The 'criteria', or requirements which need to be fulfilled in order to make the tournament, get a bid'-as above mentioned by ARM, must be known before competition commences ("...be in place all season" as he states.)

2. Criteria/requirements need to be easily understandable by all. Even so, perhaps, the least fan knows what is at stake.

3. Strict adherence to the criteria must be rigidly followed without deviation.

Pretty easy. But apparently in coming up with workable requirements and/or criteria, things became complicated, some. Dave and Arm, others, have good understanding of what is needed, what they ARE, but maybe EVEN MORE simplicity is needed.

Or more discussion of the requirements is needed. Or maybe starting right from the 'get go' the RPI, etc. , other stats used, could be broadcast weekly, daily, so all folks would become FULLY CONVERSANT with who stood where and what would be needed by whom to get included in the tourney?

Deviation has seemed to occur say the fans from the west.

But maybe the 'deviation' isn't within (from) the rules or committee but within the overall background within which the teams compete-the US economy. Strict adherence to saving money which IS clearly stated in the requirements as to travel has been this year forced on the committee, member teams, by the difficult economic times.

People-above mentioned-who understand the requirements agree that the right eight teams made the tourney. And deviation from the usual travel arrangements isn't a deviation from the rules-merely a change brought about due to harsh economic times.

It would seem to be incumbent upon those who feel otherwise (as to whether criteria are faulty), to identify where the present criteria/requirements broke down and chose the less worthy team.They need to present changes that remedy the 'failure' that they see occurred (seems-how Dartmouth got in over UND...principally.)

But at the same time they also need to admit that the criteria-as constituted-DID choose the right eight this year based upon their implementation as written.

System worked. Economy caused usage of heretofore little used provisions (as to travel.) Criteria, perhaps, as to admitting lesser able team, need altering.

I disagree that it worked to the extent that the current system selected BU and BC as the 3rd and 4th seeds especially given their respective challenges with mediocre teams down the stretch. I can understand the NCAA's sensitivity to cost, however had we had a system that properly rewarded Minnesota as the 3rd or 4th seed much of the injustice would have been avoided. Minnesota as the highest ranked of the unseeded teams is rewarded with the longest travel distance. The right 8 teams were selected based on the current system, but they were not seeded correctly. I also would still argue that a better means of addressing competition quality would have likely placed UND in the 8th spot.

**ARM** · 03-07-2011, 02:29 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Originally posted by granddaddyscout View Post

I can understand the NCAA's sensitivity to cost, however had we had a system that properly rewarded Minnesota as the 3rd or 4th seed much of the injustice would have been avoided. Minnesota as the highest ranked of the unseeded teams is rewarded with the longest travel distance.

Minnesota had a lousy October and a bad first game in Bemidji. Still, they probably could have hosted had they a) held onto a third-period lead against UMD in January; b) held onto a late second-period lead in Grand Forks; or c) held onto a late-two goal lead in the WCHA championship. They had their chances, but were unable to do any of those. I think the Gopher staff and team understands that their problems are of their own making, and now they have the chance to prove they are better than BC, if they indeed feel that is the case.

**dave1381** · 03-07-2011, 06:04 PM

Re: The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Thanks all for the great comments. Don't have time to respond to everything -- hopefully more while watching the games Saturday.

I'll start here, since this is what I understand the best.

Originally posted by papulaisle View Post

It would seem to be incumbent upon those who feel otherwise (as to whether criteria are faulty), to identify where the present criteria/requirements broke down and chose the less worthy team.They need to present changes that remedy the 'failure' that they see occurred (seems-how Dartmouth got in over UND...principally.)
But at the same time they also need to admit that the criteria-as constituted-DID choose the right eight this year based upon their implementation as written.

For starters, to echo, what ARM said, Dartmouth clearly deserved to be selected based on the criteria in place. I will, however, describe to some degree why I dislike the criteria. But I can't rule out the possibility that if different criteria were in place, maybe teams would've performed differently. And yes, if my preferred criteria were in place, North Dakota would be selected over Dartmouth.

We had some talk over the transparency of the criteria. I don't think it's terribly complicated. There's one element that's an index taking into account the whole season and adjusting for strength of schedule, and there are other elements that give more weight to common opponents, records vs top teams, and head-to-head play. Fundamentally I don't think there's anything wrong with these elements of the selection criteria. Now it certainly can get complicated how you put these criteria into practice, but the ideas of what matter are fairly simple, and I don't necessarily have a problem with them.

As Prof. Rutter and others have rightly pointed out in the past, the RPI is a lousy system that fails to adequately capture conference strength. To give a simple example of why, consider two conferences where the teams in conference A are all better than the teams of conference B. Now suppose they only play one nonconference game each, and many conference games. As the number of conference games gets larger, the nonconference game becomes insignificant, and for the most part, the RPI is reflecting conference standings. In fact, many of the selection criteria reflect this fallacy.

To a large degree, the reason why Hockey East got two teams to host is not because Hockey East actually had the strongest second-place team, but rather because they were the league with the biggest split for 1st place. If you rank teams simply by their conference win pcts, you'll get pretty close to the selection criteria (an exception is Mercyhurst because they actually do play a relatively strong nonconference schedule).

The advantage of a system like the KRACH or the Rutter rankings with firm statistical foundations, and it doesn't suffer from the kind of flaw I described above. But there's a lot of institutional inertia that'll prevent it being adopted. Clearly a better system will expose some conferences being weaker, and the weaker conferences won't like that. The RPI has a long history in the NCAA across all sports, and there'll be an uphill climb to get any one sport to do anything innovative. A final problem is that RPI formula is very transparent and easy to understand (though it's weights are completely arbitrary and the women's hockey RPI uses a few tweaks that probably most coaches don't follow understand), while I don't think you'll have as much success getting coaches to buy into maximum likelihood estimation or Bayesian statistics.

And then to the extent that the RPI is flawed, it spills over to the record vs. RPI top 12 criteria as well. Again, the top 12 weren't necessarily the actual top 12, but the conference with the most top 12 teams was the one with the biggest split in the teams ranked 2-4 -- the ECAC. By any reasonable method Bemidji was on par with Harvard and better than Quinnipiac, but you wouldn't know it from the NCAA criteria.

So when you look at teams results against the top teams, you don't give enough credit to the teams who beat truly great opponents. BC got a lot of credit for its sparkling 8-3-1 record vs. top 12 teams, while Minnesota was about .500, and Mercyhurst 1-2. Not only that BC went 2-2 against the No. 3 team, while Minnesota was 1-3-1 against the No. 1 team and 1-3 against the No. 9 team. Clearly BC looks great there.

But if you break this down by a better ranking,
Minnesota went 1-3-1 against the No. 1 team, 2-2-1 against the No. 5 team, 1-3 against the No. 7 team, 3-1 against the No. 11 team and 2-0 against the No. 12 team. BC went 2-2 against the No. 6 team, 1-0 against No. 9, 3-1 against No. 10, and 2-0 against No. 12. It's much less clear BC's results against the better teams are better now.

So hopefully that gives some insight into the faults in the criteria that came into play this year, and how a better criteria would correct some of these faults, and would have allowed Minnesota to host and North Dakota into the tournament if this better criteria were in place and results had remained the same.

Announcement

The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

The Thread for Constructive Ideas for Improving the NCAA D-I Selection Process

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment