Thursday, October 23, 2008
A Minor Point on Conference Strength
I was thinking about the BAW conference rankings and realized something needs to be highlighted. The BAW rankings point out how strong a conference is. The rankings do not point out how tough it is for an individual team in that conference. Other than the Pac10, BigEast, and Mountain West, teams do not play a full round-robin. Even with a full round-robin, teams do not play themselves. For example, think of USC. In the last post, it was noted how easy it is for USC to trollop through the Pac10 this year since they do not have to play themselves. The rest of the conference is a considerably weaker slew of teams. I think it is important to note that the BAW conference ratings only reflect how strong the conferences are. The BAW conference rankings should not be used to compare individual teams. Stick to the individual teams BAW rankings to compare teams.
Tuesday, October 21, 2008
First 2008 Conference Rankings
Which conference is the toughest? Well, that question is hard to answer. Following is the BAW analysis of playing every team in a conference on a neutral field:
If I had to rate them based on BAW, I'd go
1. BigXII
2. SEC
3. Big Ten
4. ACC
5. Pac10
6. Big East
7. Mountain West
The debate between the BigXII and SEC is really close. It would be tougher for a top echelon team to play every BigXII team, but it would be easier for a middle of the pack team. The BigXII has a very weak base compared to the SEC but has the edge at the top. That is somewhat surprising and likely due to the collapse of Auburn, while Oklahoma State has surprised positively so well.
It's somewhat surprising that Big Ten came in 3rd in that there is a clear gap between the Big Ten and the ACC, Pac10, Big East grouping. The bottom of the Big Ten has shrunk with Indiana and Purdue being the only truly dreadful teams. In the last few years, one could point out 4 dreadful Big Ten teams in a season. Still, that rating systems such as sagarin would put the Big Ten number 2 right now highlights the problems with some of the widely used metrics for conference strength. It's clear that if a team had to play every team in a conference, that team would be insane to choose to play the SEC or BigXII ahead of the Big Ten.
Another interesting thing that I've seen is that it is nearly as tough for a top team to play every Mountain West team as it is the Pac10. Furthermore, the Pac10 is heavily reliant on USC to make it tough for a top team to play that conference. Without USC, it would be pretty easy for a top team to stroll through the Pac10 (0.881 expected win pct).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| BAW5 | BAW25 | BAW50 |
| ||||||
|
|
| W | L | pct | W | L | pct | W | L | pct |
|
|
| SEC | 9.16 | 2.84 | 0.764 | 6.40 | 5.60 | 0.533 | 4.78 | 7.22 | 0.399 |
|
|
| BigTen | 8.71 | 2.29 | 0.792 | 6.45 | 4.55 | 0.587 | 4.93 | 6.07 | 0.448 |
|
|
| BigXII | 8.87 | 3.13 | 0.739 | 6.47 | 5.53 | 0.539 | 5.01 | 6.99 | 0.417 |
|
|
| ACC | 9.95 | 2.05 | 0.829 | 7.37 | 4.63 | 0.614 | 5.62 | 6.38 | 0.468 |
|
|
| Pac10 | 8.43 | 1.57 | 0.843 | 6.56 | 3.44 | 0.656 | 5.41 | 4.59 | 0.541 |
|
|
| BigEast | 6.82 | 1.18 | 0.852 | 5.37 | 2.63 | 0.671 | 4.2 | 3.8 | 0.526 |
|
|
| MountW | 7.74 | 1.26 | 0.860 | 6.27 | 2.73 | 0.697 | 5.12 | 3.88 | 0.569 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If I had to rate them based on BAW, I'd go
1. BigXII
2. SEC
3. Big Ten
4. ACC
5. Pac10
6. Big East
7. Mountain West
The debate between the BigXII and SEC is really close. It would be tougher for a top echelon team to play every BigXII team, but it would be easier for a middle of the pack team. The BigXII has a very weak base compared to the SEC but has the edge at the top. That is somewhat surprising and likely due to the collapse of Auburn, while Oklahoma State has surprised positively so well.
It's somewhat surprising that Big Ten came in 3rd in that there is a clear gap between the Big Ten and the ACC, Pac10, Big East grouping. The bottom of the Big Ten has shrunk with Indiana and Purdue being the only truly dreadful teams. In the last few years, one could point out 4 dreadful Big Ten teams in a season. Still, that rating systems such as sagarin would put the Big Ten number 2 right now highlights the problems with some of the widely used metrics for conference strength. It's clear that if a team had to play every team in a conference, that team would be insane to choose to play the SEC or BigXII ahead of the Big Ten.
Another interesting thing that I've seen is that it is nearly as tough for a top team to play every Mountain West team as it is the Pac10. Furthermore, the Pac10 is heavily reliant on USC to make it tough for a top team to play that conference. Without USC, it would be pretty easy for a top team to stroll through the Pac10 (0.881 expected win pct).
First 2008 Rankings Are Here!
Methodology used
Well, the BCS announced it's rankings this week, so I thought I might as well get cracking and post the BAW5 rankings for title contenders. With apologies to Georgia Tech, Pitt, Tulsa, and Ball State, I only considered Texas, Alabama, Penn State, Florida, Georgia, LSU, USC, Ohio State, Oklahoma State, Texas Tech, Oklahoma, and Utah in this poll. I just figured those were the 12 teams with the best case for national title consideration by the end of the year. I'll likely put up Tulsa and Ball State if they keep winning as other major contenders fall away. Keep in mind that this is an early poll subject to all the issues of early polls. Without further verbage, here are the rankings of the 12 contenders:
1. Texas
2. Alabama
3. Oklahoma State
4. Penn State
5. Ohio State
6. Utah
7. Texas Tech
8. Oklahoma
9. Georgia
10. LSU
11. USC
12. Florida
Here is how the number 5 team in the country should have done verse each team's respective schedule and the BAW5 ranking for how each team exceeded that number 5 team's win total.
It's no surprise that Texas and Alabama lead here just as they do in the BCS polls. What is remarkable is that Oklahoma State is ahead of Penn State already.
Who of the 12 has played the easiest schedule so far? No surprise that it is Utah coming out of the weaker Mountain West.
The Epct shows the expected win percentage for a number 5 team playing the respective schedules. What many might find shocking is that Ohio State has played the most difficult schedule thus far. This is due to the somewhat surprising strength of the bottom and middle of the Big Ten with the addition of a road game against USC. Another interesting result is that Penn State's schedule has been nearly as weak as Utah and Texas Tech.
Well, the BCS announced it's rankings this week, so I thought I might as well get cracking and post the BAW5 rankings for title contenders. With apologies to Georgia Tech, Pitt, Tulsa, and Ball State, I only considered Texas, Alabama, Penn State, Florida, Georgia, LSU, USC, Ohio State, Oklahoma State, Texas Tech, Oklahoma, and Utah in this poll. I just figured those were the 12 teams with the best case for national title consideration by the end of the year. I'll likely put up Tulsa and Ball State if they keep winning as other major contenders fall away. Keep in mind that this is an early poll subject to all the issues of early polls. Without further verbage, here are the rankings of the 12 contenders:
1. Texas
2. Alabama
3. Oklahoma State
4. Penn State
5. Ohio State
6. Utah
7. Texas Tech
8. Oklahoma
9. Georgia
10. LSU
11. USC
12. Florida
Here is how the number 5 team in the country should have done verse each team's respective schedule and the BAW5 ranking for how each team exceeded that number 5 team's win total.
|
| EW | EL | BAW5 |
| Texas | 5.881 | 1.119 | 1.119 |
| Bama | 6.100 | 0.900 | 0.900 |
| Ok St | 6.341 | 0.659 | 0.659 |
| Penn St | 7.369 | 0.631 | 0.631 |
| Ohio St | 6.505 | 1.495 | 0.495 |
| Utah | 7.535 | 0.465 | 0.465 |
| T Tech | 6.544 | 0.456 | 0.456 |
| Oklahoma | 5.719 | 1.281 | 0.281 |
| Georgia | 5.931 | 1.069 | 0.069 |
| LSU | 5.932 | 1.068 | 0.068 |
| USC | 4.990 | 1.010 | 0.010 |
| Florida | 5.353 | 0.647 | -0.353 |
It's no surprise that Texas and Alabama lead here just as they do in the BCS polls. What is remarkable is that Oklahoma State is ahead of Penn State already.
Who of the 12 has played the easiest schedule so far? No surprise that it is Utah coming out of the weaker Mountain West.
| Team | Epct |
| Utah | 0.942 |
| T Tech | 0.935 |
| Penn St | 0.921 |
| Ok St | 0.906 |
| Florida | 0.892 |
| Bama | 0.871 |
| LSU | 0.847 |
| Georgia | 0.847 |
| Texas | 0.840 |
| USC | 0.832 |
| Oklahoma | 0.817 |
| Ohio St | 0.813 |
Monday, October 20, 2008
2008 Changes: Team Specific Home-Field Advantage
The second major change to the BAW methodology is that home-field advantage will no longer be a blanket +/- 3 points. The flaw in that system should be obvious. Playing a night road game in Baton Rouge verse LSU is much different than playing an 11 AM game in Evanston against Northwestern. It defies common sense to put the disadvantage to the road teams as equal in terms of points for these games.
Instead, for 2008, it's time HFA became team specific for BAW rankings. There were a few possible ways to do this. One is to go through and backtest based on past scores how much HFA a team has had. This would be the most rigorous number method. However, this system is fatally flawed in the sample size. In addition, there are numerous minor flaws with internal and external consistency. Basically, the numbers are not stable year-to-year and even within years. Plus, the numbers are not consistent with general observations-ie, some places rated quite difficult by every human show up as no HFA in the numbers! Even those numbers do not account for bye weeks, time of game, etc.
So, what to do? The alternative I chose was to follow Phil Steele's HFA rankings. These are widely available in Steele's yearly guide. Phil Steele is well respected. I think he is THE expert when it comes to college football. His opinions generally align with all of the talking heads. When he disagrees, it's most often that Phil has the extra insight to the situation. While I hate being beholden to one specific guy (you know, why I changed from using sagarin predictor so much), this seems like the best available and easiest way to use HFA.
Keep in mind that there are still some flaws. HFA matters more in expected close games and games were the crowd is 'up,' such as nationally televised and rivalry games. However, the difference from the norm is highly exaggerated. I don't think it will change the BAW rankings to not account for those factors.
Still, I see using the Steele HFA as a major improvement verse straight up +/- 3 points.
Instead, for 2008, it's time HFA became team specific for BAW rankings. There were a few possible ways to do this. One is to go through and backtest based on past scores how much HFA a team has had. This would be the most rigorous number method. However, this system is fatally flawed in the sample size. In addition, there are numerous minor flaws with internal and external consistency. Basically, the numbers are not stable year-to-year and even within years. Plus, the numbers are not consistent with general observations-ie, some places rated quite difficult by every human show up as no HFA in the numbers! Even those numbers do not account for bye weeks, time of game, etc.
So, what to do? The alternative I chose was to follow Phil Steele's HFA rankings. These are widely available in Steele's yearly guide. Phil Steele is well respected. I think he is THE expert when it comes to college football. His opinions generally align with all of the talking heads. When he disagrees, it's most often that Phil has the extra insight to the situation. While I hate being beholden to one specific guy (you know, why I changed from using sagarin predictor so much), this seems like the best available and easiest way to use HFA.
Keep in mind that there are still some flaws. HFA matters more in expected close games and games were the crowd is 'up,' such as nationally televised and rivalry games. However, the difference from the norm is highly exaggerated. I don't think it will change the BAW rankings to not account for those factors.
Still, I see using the Steele HFA as a major improvement verse straight up +/- 3 points.
2008 Changes: Hello, Massey Combined!
In a quest to make the BAW rankings better, there are two major changes to the methodology this year. The first one is less reliance upon the sagarin rankings by using the massey combined rankings. The reason for this change is two-fold.
First, using sagarin exclusively often meant that the BAW rankings reflected pretty much the sagarin rankings. While there was some new information added, it was not nearly enough IMO. Most people could just learn everything they needed from sagarin. The BAW was not adding enough new information to make it worth it for enough people to check it out.
Second, and most importantly, the massey combined is a much more robust ranking measure. Massey combined relies on the logic of The Wisdom of Crowds. While sagarin could very well be the most expert of all systems, its system will still be inferior to a weighted combination of all systems out there in theory backed up scientific studies. Massey combined takes all the numerous ways that people have devised for ranking college football teams and combines them into one consensus rating. The details are on Massey's site. They make sense to me. This should really help things.
This will be the new way of ranking teams 1 through 120. The Massey Combined will give the ordinal rankings used in the methodology. However, this says nothing about the cardinal value of those rankings. We need to have a good estimate at the point value of being, say, #17 v #77. For that power index point ranking, we will again be using the 120 sagarin predictor scores for FBS. I have contemplated doing a weighted average of sagarin predictor, teamrankings, and some proprietary indexes I have developed on my own. For right now, I'm not doing that, but I reserve the right to do it if sagarin predictor starts looking quite different in its distribution. So, sagarin predictor is still a big factor in the cardinal, power index values for team. Now, sagarin predictor will be a very minor factor with massey combined being the major factor in the ordinal ranking of FBS teams 1 to 120.
First, using sagarin exclusively often meant that the BAW rankings reflected pretty much the sagarin rankings. While there was some new information added, it was not nearly enough IMO. Most people could just learn everything they needed from sagarin. The BAW was not adding enough new information to make it worth it for enough people to check it out.
Second, and most importantly, the massey combined is a much more robust ranking measure. Massey combined relies on the logic of The Wisdom of Crowds. While sagarin could very well be the most expert of all systems, its system will still be inferior to a weighted combination of all systems out there in theory backed up scientific studies. Massey combined takes all the numerous ways that people have devised for ranking college football teams and combines them into one consensus rating. The details are on Massey's site. They make sense to me. This should really help things.
This will be the new way of ranking teams 1 through 120. The Massey Combined will give the ordinal rankings used in the methodology. However, this says nothing about the cardinal value of those rankings. We need to have a good estimate at the point value of being, say, #17 v #77. For that power index point ranking, we will again be using the 120 sagarin predictor scores for FBS. I have contemplated doing a weighted average of sagarin predictor, teamrankings, and some proprietary indexes I have developed on my own. For right now, I'm not doing that, but I reserve the right to do it if sagarin predictor starts looking quite different in its distribution. So, sagarin predictor is still a big factor in the cardinal, power index values for team. Now, sagarin predictor will be a very minor factor with massey combined being the major factor in the ordinal ranking of FBS teams 1 to 120.
2008 Changes: New Name---BAW!
Well, it's getting time to start releasing new rankings. With that comes some changes. As mentioned in the previous post, the first thing was to come up with an easier name. 2007 featured OTFRR (On-The-Field Results Ranking). I never really liked it. The new name is BAW.
Baseline
Adjusted
Wins
From thefreedictionary.com
base line or base·line (b
s
l
n
)n.
Let's show how it will work in action. For looking at title contenders, I often compared how teams did versus a hypothetical number 5 team in the country playing the same schedule. That will now be known as the BAW5 team ranking. The baseline is how the hypothetical #5 team should do verse the schedule. That would be +/- 0.00 wins. BAW5 reflects how well a team has done in the wins/losses to that level. We'll likely see some BAW25 (how a team does compared to the hypothetical #25 team) and BAW50 usage as well.
So, you are now in the domain of the BAW rankings.
Baseline
Adjusted
Wins
From thefreedictionary.com
base line or base·line (b
sln)n.1.
a. A line serving as a basis, as for measurement, calculation, or location.
b. A measurement, calculation, or location used as a basis for comparison.
ad·just·ed (
-j
st
d)
ad·just·ed (
-jstd)adj.
1. Arranged or changed so as to match, conform, or function, especially in a specified way: an improperly adjusted scale.
2. Having achieved psychological balance, especially regarding others or the demands of everyday life: a poorly adjusted teenager.
Let's show how it will work in action. For looking at title contenders, I often compared how teams did versus a hypothetical number 5 team in the country playing the same schedule. That will now be known as the BAW5 team ranking. The baseline is how the hypothetical #5 team should do verse the schedule. That would be +/- 0.00 wins. BAW5 reflects how well a team has done in the wins/losses to that level. We'll likely see some BAW25 (how a team does compared to the hypothetical #25 team) and BAW50 usage as well.
So, you are now in the domain of the BAW rankings.
Friday, October 10, 2008
Name, Name, Name, Name
I realize that the name of this ranking system sucks. Right now, I have used On-The-Field-Results-Rankings (OTFRR). That is no good. I want the name to reflect what the measurement is. The ranking reflects who has earned the best wins/losses record verse the competition they played. Any Ideas?
Right now, I have W-Field, or WAP (wins above projected), or BAW (baseline adjusted wins).
Suggestions are welcome!
Right now, I have W-Field, or WAP (wins above projected), or BAW (baseline adjusted wins).
Suggestions are welcome!
Why No Rankings?
Basically, it is too early for them to mean anything.
You can read an expounded thought on this here:
http://cfbbreakdown.blogspot.com/2008/10/polls-right-now-garbage-in-garbage-out.html
The earliest this poll will show up is post week 8, but my guess is it will be after week 9.
You can read an expounded thought on this here:
http://cfbbreakdown.blogspot.com/2008/10/polls-right-now-garbage-in-garbage-out.html
The earliest this poll will show up is post week 8, but my guess is it will be after week 9.
Subscribe to:
Posts (Atom)