PDA

View Full Version : Octonion Power Rankings



Pages : [1] 2

octonion
03-13-2013, 06:59 AM
Poisson model, pooled NCAA divisions, home/away/neutral factors.

div = NCAA division
str = team strength
ofs = offensive strength
dfs = defensive strength
sos = strength of schedule



rk | school | div | str | ofs | dfs | sos
-----+---------------------------------+-----+-------+-------+-------+-------
1 | minnesota | 1 | 4.230 | 1.916 | 0.453 | 1.489
2 | quinnipiac | 1 | 3.858 | 1.536 | 0.398 | 1.460
3 | miami | 1 | 3.469 | 1.402 | 0.404 | 1.458
4 | minnesota-state | 1 | 3.443 | 1.753 | 0.509 | 1.502
5 | north-dakota | 1 | 3.321 | 1.727 | 0.520 | 1.526
6 | denver | 1 | 3.259 | 1.837 | 0.564 | 1.548
7 | st-cloud-state | 1 | 3.216 | 1.720 | 0.535 | 1.499
8 | new-hampshire | 1 | 3.173 | 1.687 | 0.532 | 1.470
9 | umass-lowell | 1 | 3.113 | 1.565 | 0.503 | 1.428
10 | notre-dame | 1 | 3.084 | 1.605 | 0.521 | 1.482
11 | union | 1 | 3.012 | 1.570 | 0.521 | 1.448
12 | rensselaer | 1 | 2.931 | 1.551 | 0.529 | 1.496
13 | wisconsin | 1 | 2.868 | 1.369 | 0.477 | 1.496
14 | providence | 1 | 2.863 | 1.544 | 0.539 | 1.482
15 | boston-college | 1 | 2.808 | 1.761 | 0.627 | 1.437
16 | western-michigan | 1 | 2.786 | 1.295 | 0.465 | 1.459
17 | nebraska-omaha | 1 | 2.679 | 1.760 | 0.657 | 1.495
18 | air-force | 1 | 2.673 | 1.534 | 0.574 | 1.349
19 | ferris-state | 1 | 2.625 | 1.463 | 0.558 | 1.475
20 | dartmouth | 1 | 2.618 | 1.526 | 0.583 | 1.462
21 | cornell | 1 | 2.541 | 1.377 | 0.542 | 1.502
22 | yale | 1 | 2.527 | 1.576 | 0.624 | 1.486
23 | niagara | 1 | 2.508 | 1.494 | 0.596 | 1.354
24 | colorado-college | 1 | 2.507 | 1.798 | 0.717 | 1.567
25 | michigan-tech | 1 | 2.493 | 1.611 | 0.646 | 1.516

FlagDUDE08
03-13-2013, 08:30 AM
Poisson model, pooled NCAA divisions, home/away/neutral factors.

div = NCAA division
str = team strength
ofs = offensive strength
dfs = defensive strength
sos = strength of schedule



rk | school | div | str | ofs | dfs | sos
-----+---------------------------------+-----+-------+-------+-------+-------
1 | minnesota | 1 | 4.230 | 1.916 | 0.453 | 1.489
2 | quinnipiac | 1 | 3.858 | 1.536 | 0.398 | 1.460
3 | miami | 1 | 3.469 | 1.402 | 0.404 | 1.458
4 | minnesota-state | 1 | 3.443 | 1.753 | 0.509 | 1.502
5 | north-dakota | 1 | 3.321 | 1.727 | 0.520 | 1.526
6 | denver | 1 | 3.259 | 1.837 | 0.564 | 1.548
7 | st-cloud-state | 1 | 3.216 | 1.720 | 0.535 | 1.499
8 | new-hampshire | 1 | 3.173 | 1.687 | 0.532 | 1.470
9 | umass-lowell | 1 | 3.113 | 1.565 | 0.503 | 1.428
10 | notre-dame | 1 | 3.084 | 1.605 | 0.521 | 1.482
11 | union | 1 | 3.012 | 1.570 | 0.521 | 1.448
12 | rensselaer | 1 | 2.931 | 1.551 | 0.529 | 1.496
13 | wisconsin | 1 | 2.868 | 1.369 | 0.477 | 1.496
14 | providence | 1 | 2.863 | 1.544 | 0.539 | 1.482
15 | boston-college | 1 | 2.808 | 1.761 | 0.627 | 1.437
16 | western-michigan | 1 | 2.786 | 1.295 | 0.465 | 1.459
17 | nebraska-omaha | 1 | 2.679 | 1.760 | 0.657 | 1.495
18 | air-force | 1 | 2.673 | 1.534 | 0.574 | 1.349
19 | ferris-state | 1 | 2.625 | 1.463 | 0.558 | 1.475
20 | dartmouth | 1 | 2.618 | 1.526 | 0.583 | 1.462
21 | cornell | 1 | 2.541 | 1.377 | 0.542 | 1.502
22 | yale | 1 | 2.527 | 1.576 | 0.624 | 1.486
23 | niagara | 1 | 2.508 | 1.494 | 0.596 | 1.354
24 | colorado-college | 1 | 2.507 | 1.798 | 0.717 | 1.567
25 | michigan-tech | 1 | 2.493 | 1.611 | 0.646 | 1.516


Any description as to how you are coming by these numbers?

goblue78
03-13-2013, 09:09 AM
For a fairly simple exposition of how to do Poisson modeling, see https://dl.dropbox.com/u/5755704/ranking.doc (I wrote it a couple of years ago, so it refers to the 2011 season.) The only difference between this and what Octonion has done is normalization to create the rankings, I think. But he or she will, I'm sure, correct me if I'm wrong.

octonion
03-13-2013, 09:22 AM
Sure, it's a mixed-effect Poisson regression model. I'm assuming that the goals scored by a team is modeled as a Poisson distribution depending on the team's offensive strength (random effect), opponent's defensive strength (random effect), home/away/neutral factor (fixed effect) and year (fixed effect). Offense and defense are nested within NCAA divisions, and these are pooled over the last 14 years.

As an example, I get than teams score about 7% more goals and allow about 7% fewer goals at home. This implies by the Pythagorean expectation (with exponent about 2.2) that the home team's winning percentage should be about 57%, and this is exactly what we see.

-Chris

goblue78
03-13-2013, 09:28 AM
Interesting, Octonion. What do you get out of the 14 year pooling (given year fixed effects) other than more accurate (presuming invariance) estimates of the home team effect? If I understand you correctly, are you assuming that, in essence, a team's offensive and defensive prowess rises and falls by a fixed (estimated) amount, or are the defense and offense allowed to move independently from year to year?

William Blake
03-13-2013, 10:14 AM
what? Only 9 WCHA teams in the top 25? Eastern bias.

FlagDUDE08
03-13-2013, 10:19 AM
what? Only 9 WCHA teams in the top 25? Eastern bias.

Goofers are #1? Western bias.

HoosierBBall_GopherHockey
03-13-2013, 10:26 AM
Union not #26? Michigan Tech in the top half of anything?


https://www.youtube.com/watch?v=zSTEqHxh3fI

gopheritall
03-13-2013, 10:28 AM
So, was the model applied to previous years? If so, how did it compare to the NCAA tournament results (especially in regards to insulated teams that do not play each other such as MN and QU this year)?

I assume the reason to rank the teams is to give some insight into how the teams will perform. If the purpose is for something else, what is it?

Jon
03-13-2013, 01:30 PM
So, was the model applied to previous years? If so, how did it compare to the NCAA tournament results (especially in regards to insulated teams that do not play each other such as MN and QU this year)?

I assume the reason to rank the teams is to give some insight into how the teams will perform. If the purpose is for something else, what is it?

To start a discussion on the internet. Duh. :D

octonion
03-13-2013, 07:30 PM
It's a first model - with sports you do have to be really careful regarding any rule changes. In particular, can the rule changes be absorbed into the fixed yearly effect?

The strength of a team's offense and defense are separately estimated for each season, but the overall pool strength of D1, D2 and D3 is assumed to be consistent. The home field advantage is assumed to be consistent, too, but that could easily be allowed to vary by team. That makes a difference for some (but not most) teams.

octonion
03-13-2013, 09:54 PM
It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?

Numbers
03-13-2013, 10:27 PM
It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?

In last year's tournament, I believe that all 4 top seeds qualified for the FF, and in the FF, everything went according to PWR seeding. That means that, at worst, the PWR predicted the same 11/15 (and I am fairly sure that UND was a #2, so that makes 12 right atleast), and it's not designed as a predictive model. Draw your own conclusion.

octonion
03-13-2013, 11:48 PM
It looks like the tournament seeding had two #1 seeds in the final 4, whereas I had three. What's the overall accuracy of the tournament seeding over the last 10 years or so?

http://en.wikipedia.org/wiki/2012_NCAA_Division_I_Men's_Ice_Hockey_Tournament

Numbers
03-14-2013, 12:06 AM
It looks like the tournament seeding had two #1 seeds in the final 4, whereas I had three. What's the overall accuracy of the tournament seeding over the last 10 years or so?

http://en.wikipedia.org/wiki/2012_NCAA_Division_I_Men's_Ice_Hockey_Tournament

I am sorry. I had thought that Minny and Ferris were #1s. Interesting that they each defeated #1 seeds from their own conference...

Patman
03-14-2013, 12:11 AM
It predicted 11/15 (73.3%) of the NCAA D1 tournament outcomes correctly last year, fitting on only non-tournament games. Is that good or bad?

Its a small sample size. I've thought about yearly correlation but then you get into the trickery of time-series behavior. I'm a little leery of a mixed-effect model for this... and with the constraint my thoughts would be that it would be hard to calculate... while one can always recode the predictors with respect to constraint, that a "2" may appear in the data is a little leery.

Also, if you are using 58 separate random effects, I would be shocked if the computation wasn't difficult in some measure.... oh, wait... random effect within year and then moving value for each. I suppose but I'm not sure what's gained or if model strength actually borrows over. "Nested within NCAA divisions" so every team by year comes from the same pool for offense and then again for defense?

As the complexity increases the ease of modeling by standard software also increases. GLMM models can be nasty if you throw enough into the works.

Hammer
03-14-2013, 07:39 AM
I am sorry. I had thought that Minny and Ferris were #1s. Interesting that they each defeated #1 seeds from their own conference...

No. Ferris beat #3 seed Denver, then #4 Cornell to get to Tampa, then #1 Union to get to the final.

octonion
03-15-2013, 04:33 AM
Sure, one tournament is a small sample. What would be your metric for measuring relative predictive ability? Octonion is likely better than KRACH even now, as KRACH doesn't account for winning margins. That's why KRACH has Quinnipiac ranked over Minnesota.

goblue78
03-15-2013, 09:12 AM
Relative predictability for any set of games is measured as sumof(f(prob(win)-win)), where the sum is taken over all out-of-sample games and f() is a penalty function which is always positive and, ideally punishes longshots, like f(x)=x^2. Thus, a "pure" PWR based metric that predicts the higher seed always wins (or a pure metric based on any ranking system that predicts the higher seed will always win, generates either f(1) or f(0) for every game. Your system, by contrast, will generate f(1-pr(win)) when you get it right and f(pr(win)) when you get it wrong. Thus, predicting a game as 50-50 will generate f(.5) no matter what the outcome. Nicely, the binomial nature of the probabilities also will tell you whether or not your predictive score is within the tolerance of the probabilities: for any game with true probability p, the standard error is sqrt(p(1-p)). Since games are independent, you can then form a compounded expected predicted accuracy over N games which rises as the square root of N. This will at least tell you something about whether or not you've overfitted the model.

One more thing: it's true that your model uses score and KRACH doesn't, but KRACH uses who won the game and your method doesn't. Unless the Poisson model is exactly right (and it isn't), it's up in the air which is more accurate for predicting the outcome of actual games. I tested this by resimulating the season using only the Poisson model and looking at team records: try it, you will find some surprises.

Patman
03-15-2013, 09:27 AM
Sure, one tournament is a small sample. What would be your metric for measuring relative predictive ability? Octonion is likely better than KRACH even now, as KRACH doesn't account for winning margins. That's why KRACH has Quinnipiac ranked over Minnesota.

Thank you for not responding to substantive issues, as a journal referee I would not be pleased. Remember that in the future should you proceed.