Announcement

Collapse
No announcement yet.

2010-2011 Rutter DI Computer Rankings Thread

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • 2010-2011 Rutter DI Computer Rankings Thread

    We are back for another year of algorithm based fun. For the complete rankings and information on how they are calculated, please visit the website: http://math.bd.psu.edu/faculty/rutte...sRankings.html

    For games played through 10/10/10

    Code:
       	Team  		Rating  
    1 	Mercyhurst 	1.7743 	
    2 	Minnesota 	1.4368 
    3 	Wisconsin 	1.2998 
    4 	Quinnipiac 	1.1193 	
    5 	Ohio State 	0.9929 
    6 	Boston College 	0.9259 	
    7 	Harvard 	0.6314 	
    8 	UMD	 	0.5116 	
    9 	Cornell 	0.5085 	
    10 	Providence 	0.4357
    Yes, Quinnipiac is ranked above OSU as the started the season ranked a little higher.

    I have added another page to the website this year describing the uncertainty in the rankings: http://math.bd.psu.edu/faculty/rutter/CurrentD1Unc.html

    Since I use a statistical model to estimate a teams quality rating, there is some degree of uncertainty in the estimates. Very rarely are these estimates of uncertainty reported, but they do contain some interesting information. Here is the Top 10 from that page:

    Code:
       	Team  		25th 	75th   	Percent Top 8
    1 	Mercyhurst 	1 	6 	86.8
    2 	Minnesota 	2 	7 	82.5
    3 	UMD	 	2 	10 	70.4
    4 	Wisconsin 	3 	10 	69.0
    5 	Quinnipiac 	3 	10 	67.0
    6 	Boston College 	4 	12 	57.8
    7 	Ohio State 	4 	12 	57.4
    8 	Harvard 	5 	18 	43.7
    9 	Cornell 	6 	19 	39.4
    10 	Princeton 	10 	25 	21.1
    The table is sorted by Percent Top 8. This is determined by looking at simulations of all 33 teams' quality ratings and ranking them over 10,000 times. Percent Top 8 is the percentage of times these simulated rankings have that team in the top 8, the same number of teams that make the NCAA tournament. The 25th and 75th percentile represent the innerquartile range of all the simulated rankings, that is the middle 50% of the rankings. So, for at least half the time, Merychurst was ranked between first and sixth.

    Can the Percent Top 8 column be used to predict the probability that a team will be in the top 8? Yes, IF the team played an infinite number of games against all of the other teams and the true quality rating did not change over time. That is a big "if." In reality, the teams play a finite number of games against a subset of all the teams, and the Percent Top 8 calculation does not take into account future opponents.

    You will note that Ivy league teams have the widest spread of rankings, as they have not played any games yet and their ratings are based on last season's results. These results will improve over time as more games are played. For example, in 2008-2009, Wisconsin finished ranked number 1 with an interquartile range of [1,2] and a 100% of being in the top 8.

  • #2
    Re: 2010-2011 Rutter DI Computer Rankings Thread

    Originally posted by LakersFan View Post
    For games played through 10/10/10

    Code:
       	Team  		Rating  
    1 	Mercyhurst 	1.7743 	
    2 	Minnesota 	1.4368 
    3 	Wisconsin 	1.2998 
    4 	Quinnipiac 	1.1193 	
    5 	Ohio State 	0.9929 
    6 	Boston College 	0.9259 	
    7 	Harvard 	0.6314 	
    8 	UMD	 	0.5116 	
    9 	Cornell 	0.5085 	
    10 	Providence 	0.4357 
    I can't complain because Providence is ranked , however this must make OnMaa very happy as he has been lobbying for the Q!

    Originally posted by LakersFan View Post
    Yes, Quinnipiac is ranked above OSU as the started the season ranked a little higher.

    Comment


    • #3
      Re: 2010-2011 Rutter DI Computer Rankings Thread

      Originally posted by scrambledlegs View Post
      I can't complain because Providence is ranked , however this must make OnMaa very happy as he has been lobbying for the Q!
      You got my allegiances way off, but I did put Quinnipiac in the top ten (deservedly so) based on a 4-0 record. However, they should IMHO not be ranked ahead of UMD or BU, as Quinnipiac's strength of schedule does not match that of BU and UMD. The Rutter ranking also shows Clarkson ranked 30th. Clearly a reflection of their poor start and not of their real strength as perceived by many including yours truly.

      The statistical rutter method done by Lakersfan will become more accurate once more games are recorded in the books. Watched these rankings in years past and found them to be A: good and B: impartial. Keep up the good work Lakerfan.
      Last edited by OnMAA; 10-12-2010, 10:41 AM. Reason: sp

      Comment


      • #4
        Re: 2010-2011 Rutter DI Computer Rankings Thread

        Hi Lakersfan,

        Thanks for the nice addition to the rankings.

        One question -- your method isn't actually taking into account the uncertainty in the ranking parameter estimates for each team, right? It sounds like you're taking the parameter estimates as given, and then calculating the expectation of each team finishing in the top 8 given the parameters. Just wanted to be clear.

        Comment


        • #5
          Re: 2010-2011 Rutter DI Computer Rankings Thread

          Originally posted by dave1381 View Post
          One question -- your method isn't actually taking into account the uncertainty in the ranking parameter estimates for each team, right? It sounds like you're taking the parameter estimates as given, and then calculating the expectation of each team finishing in the top 8 given the parameters. Just wanted to be clear.
          No, it is a true measure of the uncertainty. I apologise in advance for all the statistical terminology I am about to post.

          Since I use a Bayesian approach to estimate a team's rating, it is very easy to take a random walk through the joint distribution of all the teams' ratings. I do this as part of the estimation process, and usually report the median of each team's rating as a point estimate. I could easily report a 95% credibility interval (Bayesian form of confidence interval) for each team's rating. For example, the 95% CI for Wisconsin's rating (after their great win over Ohio State, wait, wrong sport) on Sunday morning is (-0.06, 2.87). By itself, that interval is scary not only because it falls below 0, which would indicate an average team, but it is also difficult to compare to the other teams' ratings, since they would be effected/correlated with Wisconsin's. Looking at 33 simultaneous CI hurts my head.

          Instead, for all 30,000 visits to the joint distribution, I rank the ratings, therefore looking at how Wisconsin compares to all the other teams for that 1 simulation from the joint distribution. That seems like a lot of work, but computers are fast and enjoy endless, repetitive tasks. Therefore, I have 30,000 samples from the distribution that describes the rank of Wisconsin with respect to the other 32 teams. I could plot a histogram of those 30,000 values, but I decided to summarise the distribution by looking at the middle 50% and the probability (calculated via simulation) of being in the top 8.

          If you look at today's report, you see that none of the teams are ranked number one more than 25% of the time. To me, this indicates that the top 7 teams are indistinguishable from each other at this point in the season. Given the limited number of games that have been played and the fact that many of the top teams have lost a game or two, this is not a surprise.

          There will be a quiz on Wednesday

          Comment


          • #6
            Re: 2010-2011 Rutter DI Computer Rankings Thread

            Originally posted by LakersFan View Post
            There will be a quiz on Wednesday
            I plan to just audit this course. Nice work.
            "... And lose, and start again at your beginnings
            And never breathe a word about your loss;" -- Rudyard Kipling

            Comment


            • #7
              Re: 2010-2011 Rutter DI Computer Rankings Thread

              For games played through October 17, 2010

              Code:
                 	Team            Rating  	
              1 	Wisconsin 	1.3296 	
              2 	UMD	 	1.2343 	
              3 	Boston College 	1.1127 	
              4 	Ohio State 	1.1007 	
              5 	North Dakota 	1.0811 	
              6 	Mercyhurst 	1.0510 
              7 	Boston Univ.	1.0278 	
              8 	Providence 	0.7194 	
              9 	Minnesota 	0.6559 	
              10 	Harvard 	0.6314
              Where is Cornell? Well, they have a rating of .51 based on last year's regular season. Their rating hasn't changed, the other teams have just jumped over them. Once they start playing games, they will settle in where they belong. Still early in the season and the ratings are very sensitive.

              Comment


              • #8
                Re: 2010-2011 Rutter DI Computer Rankings Thread

                For games played through October 24, 2010

                Code:
                  	Team 		Rating 	
                1 	Wisconsin 	1.8320 	
                2 	UMD	 	1.3278 	
                3 	Boston Univ. 	1.1463 	
                4 	Mercyhurst 	1.1330 	
                5 	Boston College 	1.0245 	
                6 	Cornell 	0.9519 
                7 	North Dakota 	0.9248 
                8 	Quinnipiac 	0.7220 	
                9 	Minnesota 	0.7055 
                10 	Harvard 	0.6283

                Comment


                • #9
                  Re: 2010-2011 Rutter DI Computer Rankings Thread

                  Lakerfan -- thanks for the explanation. I didn't get a chance to read the response until now. Now I understand.

                  One more quick question: in the past when I've used your rankings to calculate playoff probabilities, I've calculated the inverse normal cdf of the difference in team rankings. Is that still correct -- i.e. the prior/posterior distributions are all multivariate normal? Or is it just a decent approximation? I guess I'm also ignoring the issue that you have ties in the regular season and not the postseason.

                  Comment


                  • #10
                    Re: 2010-2011 Rutter DI Computer Rankings Thread

                    Originally posted by dave1381 View Post
                    Lakerfan -- thanks for the explanation. I didn't get a chance to read the response until now. Now I understand.

                    One more quick question: in the past when I've used your rankings to calculate playoff probabilities, I've calculated the inverse normal cdf of the difference in team rankings. Is that still correct -- i.e. the prior/posterior distributions are all multivariate normal? Or is it just a decent approximation? I guess I'm also ignoring the issue that you have ties in the regular season and not the postseason.
                    Yes, that is the easy way to calculate the probability of one team defeating another. The model for predicting a single game's outcome is univariate, while the distribution that describes the teams' ratings is multivariate. Now, to be 100% correct, you would calculate the probability of team A beating team B for every 30,000 combinations of the two ratings, and then find the average the results. And you would have an answer very similar to the one calculated from the point estimates.

                    Comment


                    • #11
                      Re: 2010-2011 Rutter DI Computer Rankings Thread

                      For games played through 10/31/10

                      Code:
                        	Team 	Rating 	
                      1 	Wisconsin 	2.0486 
                      2 	UMD 	        1.5473 	
                      3 	Cornell 	1.3705 
                      4 	Mercyhurst 	1.1801 	
                      5 	Ohio State 	0.9873 	
                      6 	Boston Univ. 	0.9834 	
                      7 	North Dakota 	0.8624 	
                      8 	Boston College 	0.5805 	
                      9 	Minnesota 	0.5137 	
                      10 	Dartmouth 	0.4874
                      I will update the ratings after Tuesday's games.

                      Comment


                      • #12
                        Re: 2010-2011 Rutter DI Computer Rankings Thread

                        Ok, at this point it looks to me like Cornell, Mercyhurst, BU/BC are all a class above in their conferences right now, and most of the regular season excitement in terms of making the top 8 is going be jostling among WCHA teams. The WCHA will get 4 teams, barring an upset of Cornell in the ECAC tournament or BU/BC in Hockey East. Of course other teams are capable of improving in ECAC/Hockey East, but no one else in those leagues will improve enough fast enough to earn an NCAA at-large bid.

                        Comment


                        • #13
                          Re: 2010-2011 Rutter DI Computer Rankings Thread

                          Originally posted by dave1381 View Post
                          Ok, at this point it looks to me like Cornell, Mercyhurst, BU/BC are all a class above in their conferences right now, and most of the regular season excitement in terms of making the top 8 is going be jostling among WCHA teams. The WCHA will get 4 teams, barring an upset of Cornell in the ECAC tournament or BU/BC in Hockey East. Of course other teams are capable of improving in ECAC/Hockey East, but no one else in those leagues will improve enough fast enough to earn an NCAA at-large bid.
                          With St. Lawrence and Harvard appearing to be below average teams this year (at least so far), the quality of the non-conference games have taken a hit. Below is the list of non-conference games between top 10 (according to my rankings on 11/6) that have or will occur. I may have missed a couple, and it will of course change when the top 10 changes.

                          Wisconsin-Mercyhurst
                          Bemidji State-Mercyhurst x2
                          Cornell-Mercyhurst x2
                          BU-North Dakota x2

                          Four of those games (BS-MC, BU-ND) would not have been thought of as top 10 non-conference games before the season, I think. This is going to make it difficult for any ranking system to rank the WCHA vs. the east, as there are so few games against the top teams in each league. Also makes Merychurst's schedule look a little stronger.

                          Please, for the sake of computer models everywhere, let OSU join the CHA when Penn State goes DI so each WCHA team can gain four (hopefully meaningful) non-conference games.

                          Comment


                          • #14
                            Re: 2010-2011 Rutter DI Computer Rankings Thread

                            Regarding disappointing teams for the sake of nonconference matchups, topping the list has to be UConn (UMD's only nonconference opponent) and Clarkson (a Minnesota opponent) have to be tops on the list. They were both top 10 teams last year and are struggling mightily this year.

                            SLU was average last year so they don't really belong in this conversation. I agree Harvard has disappointed so far, but in terms of the rankings they've lost to the No. 4 team and beaten the worst team --- really their ranking (aside from the prior) is mainly determined by one game, the tie against Yale. I expect they'll be more of a 9-15 team rather than a below average team.

                            Mercyhurst looks stronger, not just because of the direct head-to-head matchups with Bemidji, but also Wayne State's wins over Bemidji and North Dakota are impressive.

                            I'd never have imagined the WCHA would have 6 of the top 9 teams, that's for sure.

                            Comment


                            • #15
                              Re: 2010-2011 Rutter DI Computer Rankings Thread

                              For games played through 11/7/10

                              Code:
                                	Team 	Rating 	
                              1 	Wisconsin 	1.6556 	
                              2 	UMD		1.4347 	
                              3 	Boston Univ.	1.3097 
                              4 	Cornell 	1.1924 	
                              5 	North Dakota 	1.0688 	
                              6 	Mercyhurst 	0.8699 	
                              7 	Minnesota 	0.8207 	
                              8 	Ohio State 	0.7148 	
                              9 	Bemidji State 	0.5747 	
                              10 	Boston College 	0.5441

                              Comment

                              Working...
                              X