How the Other Rating Systems Screw Up Strength of Schedule

Well, austigers on twoplustwo forums asked
why the strength of schedules I give differ
from sagarin and other methods when I use
sagarin heavily for inputs....

My response must've been good, because
NebraskaSucks from phog.net hunted it down
on a poker forum to post it to the phog.net.

Hence, I'll repost it here
----------------------------------------------------
Sagarin and most other rating systems screw
up strentgh of schedule

they'll often just do an arithmetic or
geometric mean of the rankings of all
opponents played

that doesn't work when trying to evaluate
how tough it is to win a game

for instance, a home favortie of 21.5 has
won 94.7% of their games from 1993 to
2006 while a home favorite of 27.5 has
won 97.5%. That isn't much difference
for 6 points. However, a home favorite of
3 has won 54.5% of their games while a
home favorite of 9 has won 74.1% of the
time. That difference is huge. Here,
almost all rating systems just look at the
opponent being 6 points different. A
team that played the 21.5 and 9 point
favorite would be getting just as much
credit as the team playing as a 27.5 and 3
point favorite. However, it was much,
much, much more difficult for the latter
team to win both games.

In summary, all the strength of schedules
I've seen publicized fail because they do
not account for the non-linearity of win/
loss pcts in relation to the ranking of the
opponent.

I use the same inputs as sagarin, but feel
it's a much better methodology for getting
the output.

The other methods fail Kansas due to
Kansas playing some really bad squads...
so bad that I gave Kansas an outright 100%
chance to win some games instead of
capping at 99%. However, there shouldn't
be a huge difference due to Kansas playing
a 100% gimme like SE Louisiana and
Arizona St playing SDST at home (a game
they over 97% of the time). Yet, ASU is
getting credit for playing the #100 overall
(#73 predictor) team while Kansas gets
credit for #203 overall (#186 predictor).
When the schedules are weighted, that will
plummet Kansas even though the onfield
win/loss difference is minimal.

Kansas has had the toughest schedule of
the undefeated so far because they have
played @ K St, @ Colorado, and @ Texas
A&M...all of which are very losable games
even for a #5 team.


This idea had buzzed around my head for
a long time, but I finally broke through
conceptualizing it when I read the
coltonindex criticism of the rpi in college
basketball. The rpi is a sham that has teams
gaming the system to secure automatic
wins from teams rated 130 instead of 270.
The same phenomena is showing up in
college football only the football teams
haven't really thought of 'gaming' the
system yet.

Methodology

Why?
and
How?

That's the two main questions that are
brought up over and over as I discuss my
new rating system and its results. Why do
we need another ranking system? How
does the ranking system really work?

First, the why.
The problem with college football is that
there are only a limited number of games
and teams vary greatly in strength of the
competition they play. You'll see many
try to get around this by opining on what
teams they think are the 'best,' pointing
out records v top25/top30/top10 or some
other level according to some poll, looking
at conference and/or road records, or
quoting some esoteric strength of schedule
that is often just a linear weighting of
opponents' computer ranking or based on
opponents' win percentage. This always
struck me as a very poor way of doing
things. I came across the colton index for
NCAA bball and its objections to the rpi's
use for that game. Many things in that
complaint struck a chord in me. What really
matters is how tough it is to win or lose
given a schedule. A linear rating is very poor
for that. For instance, a home favorite of 21.5
wins about 94.7% of the time while a home
favorite of 27.5 wins 97.5% of the time. That
isn't much of a difference for playing a team
that is 6 points better than another. Compare
that instead to a home favorite of 3 points
winning 54.5% of the time verse a 9 point
home favorite winning 74%. The same 6 point
difference is HUGE as far as the difference
between winning and losing. The system
advocated in this blog tries to play upon that
difference to point out whom are the best
teams in wins and losses. While arguing who
is/are the 'best' teams is a fun topic and leads
to interesting debate, the champion in nearly
every sport is given to the team with the best
win/loss records. Champion trophies are not
given to the team that 'experts' think is the
'best.' This rating system tries to reward the
teams that earn it on the field!

Now, for the how:

1) (edited in italics 10/20/08) Each team
is ordinally ranked 1 to 120 following the
massey combined rankings. Massey
combined is a system that uses weighting
of every available poll to create a consensus
poll. For every ranking, I gather that ranking's
sagarin overall score. Sagarin overall is the best
known computer evaluation system and the
most reliable from everything I've seen and used.

2) I then pull out the rating for the a base
level team according to sagarin predictor.
When trying to pick out the top teams, I've
compared to a #5 team. If I was looking
at the differences between teams about
number 20, I'd use the #20 team's score
as a baseline.

3) (edited in italics 10/20/08)
Next, I construct a ex post facto 'spread'
for each game by taking the difference
between the baseline ranking and each
opponent's sagarin predictor and add or
subtract a xxblanket 3xx point home-field
advantage. Beginning 2008, the home-field
advantage is no longer a blanket 3 points.
Instead, the HFA listed by Phil Steele for
each individual team will be used. These
are available in Phil Steele's yearly guides.
Phil Steele is a well known and respected
college football guru.

4) From there, I convert the 'spread' into
a win pct by comparing the 'spread' to a
database that has the actual win pcts for
each game played at that spread in college
football from 1993 to 2006 (ty to goldsheet
for that info).

5) After obtaining win pcts for a baseline
team against each team on the schedule, I
just use simple math to project the average
amount of wins the baseline team should
have against that schedule.

6) I then subtract the projected baseline
team's win total from the actual win total for
the specific team under consideration to
obtain a result for how many wins a team
has outperformed a generic baseline team.

7) For giggles, I go further and work through
every possible combination to see what would
a baseline teams' win total look like against
that schedule for all possible outcomes.

It's time for a ranking tool that isn't clouded
by 'brand name' schools or preseason polls.
I'm not arguing this ranking identifies the
'best' teams. I am arguing that this ranking
does a great job and should be a tool in
identifying the teams that have proven it on-
the-field with win/loss results.

Kansas Remains #1! Post Week 9

Methodology for ranking system. Rankings
are on-the-field win-loss results based.

So, another Saturday is in the books and it's
time to update the ratings. What follows is
the OTFRR#5 through Saturday, October
27th. The top 4 did not shift at all after all of
them surprisingly won their games, most in
dominating #1 fashion. The big news is that
Oregon has caught upto LSU among one loss
teams. Look for a post on the one-loss teams
coming up in the next few days.

Here are the rankings compared to a #5 team
Kansas +1.005W
Ohio State +0.887W
Arizona State +0.844W
Boston College +0.774W
Oregon +0.210W
LSU +0.204W

Kansas is over a full game ahead of what a
typical #5 team would be playing their
schedule. That's right-Kansas could've lost
a game this season and still had a top5 resume.

Oregon has played the toughest schedule so far
for the top 6 teams, but is essentially tied with
LSU for that honor. BC and Ohio State have
played the weakest schedules of the six teams
profiled here.

Expected #5 team performance for team schedules:
Oregon 6.790W 1.210L 0.8488winp
LSU 6.796W 1.204L 0.8495winp
Kansas 6.995W 1.005L 0.874winp
Arizona State 7.156W 0.844L 0.895winp
Ohio State 8.113W 0.887L 0.901winp
Boston College 7.226W 0.774L 0.903winp

Despite what the talking heads on tv say,
there are teams earning it on the field. The
tv pundits prefer to ignore it because it is
not the 'best' teams that are showing it on
the field. Sorry, it's not the 'best' team that
wins, it's the team with the best results.

Right now, I'd guess that Kansas and
Arizona State are the teams in control of
their destiny according to this ranking
system with BC a close third and Ohio
State a distant fourth.


Here are the details for each team:
Ohio State
Youngstown St 99.00%
Akron 99.00%
Washington 82.83%
N'Western 97.51%
Minnesota 93.33%
Purdue 73.76%
Kent St 99.00%
Mich St 93.15%
Penn St 73.76%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.3707
8 0.4150
7 0.1753
6 0.0352
5 0.0036
4 0.0002
3 0.0000
2 0.0000
1 0.0000
0 0.0000

Boston College
Wake Forest 89.33%
NC St 96.31%
G Tech 73.99%
Army 99.00%
Umass 99.00%
B Green 99.00%
N Dame 91.96%
V Tech 73.99%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.4203
7 0.4113
6 0.1443
5 0.0225
4 0.0017
3 0.0001
2 0.0000
1 0.0000
0 0.0000

Arizona State
SJ ST 99.00%
Colorado 87.40%
SD ST 97.51%
Org St 80.02%
Stanford 86.57%
Wazzu 94.15%
Washington 94.74%
Cal 76.24%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.3975
7 0.4030
6 0.1622
5 0.0334
4 0.0038
3 0.0002
2 0.0000
1 0.0000
0 0.0000

Kansas
C Mich 99.00%
SE La 100.00%
Toledo 99.00%
FIU 100.00%
K St 50.83%
Baylor 99.00%
Colorado 73.43%
Texas A&M 78.27%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.2835
7 0.4641
6 0.2177
5 0.0339
4 0.0009
3 0.0000
2 0.0000
1 0.0000
0 0.0000

LSU
Miss St 94.15%
V Tech 88.44%
MTSU 99.00%
S Car 88.94%
Tulane (N) 99.00%
Florida 68.67%
Kentucky 69.15%
Auburn 72.23%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.2490
7 0.4044
6 0.2547
5 0.0785
4 0.0124
3 0.0010
2 0.0000
1 0.0000
0 0.0000

Oregon
Houston 94.74%
Michigan 69.01%
Fresno St 95.05%
Stanford 86.57%
Cal 76.24%
Wazzu 97.51%
Washington 83.68%
USC 76.24%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.2552
7 0.3969
6 0.2495
5 0.0817
4 0.0150
3 0.0016
2 0.0001
1 0.0000
0 0.0000

One Week Makes a Big Difference

Heading into this week's games, I want to use
the 4 undefeateds to highlight how big real
conference contests are at demonstrating who
are the best teams in the country.

Going into this week, we have four undefeateds
in the BCS conferences.
According to the OTFRR#5, here are the wins
ahead of a #5 team playing their schedule and
the chance a #5 team would be undefeated
right now having played those respective
schedules.
Kansas +0.703 0.4219
Ohio State +0.659 0.4872
Arizona State +0.619 0.5185
Boston College +0.461 0.6047

Now, assuming those teams all win and
using the post week 8 sagarin predictors,
we'd have the following:
Ohio State +0.951W 0.3452
Kansas +0.894W 0.3413
Arizona State +0.821W 0.4139
Boston College +0.663W 0.4828

There are quite a few things I want to
highlight from this exercise. One, notice
how each team's chances of being
undefeated are now below to way below
50% even assuming they were a #5 team.
Along with this, notice how the top teams
are nearly a full win above what one
would expect a top 5 team to be. That
means that even with one loss, they'd
almost be right on the pace of a top 5
team playing their schedule.

Secondly, the potential week 9 
numbers show Ohio St jumping Kansas
even though Kansas still has the least
chance of being undefeated. In this case,
it shows how the depth of opponents
and more games can overweigh verse a
schedule that has a few more really
tough games.

Finally, the chart lends to the argument
that perhaps the voters have gotten it
right all this time in putting undefeated
teams into the title game ahead of very
talented one loss teams. It appears that
even the softest of BCS conference
schedules separates the wheat from the
chaff.

Notre Dame (Charlie) v Washington (Ty) and some Island Love

I've recently come across many debates on the
whole Charlie Weis v Ty Willingham, ND v
Washington mumbo jumbo. While I hope to
keep my blog Irish free, since it's about
winning teams, I thought I'd add my two cents
based on the on-the-field Results Ranking for
a #50 teams (OTFRR50). OTFRR50 looks how
a generic #50 team in the country would do
verse a schedule and compare that to how the 
actual team did based on the methodology here. 
While, I'm at it, I'll throw in a comparison 
between the 2 schools and Hawai'i so that the 
island folk don't get on me too much for the 
last post.

The results first
OTFRR50
Hawai'i
+0.933W expected 6.067W 0.933L 0.867winp
Washington
-0.255W expected 2.255W 4.746L 0.322winp
Notre Dame
-1.893W expected 2.893W 5.108L 0.362winp

While Ty's team has underperformed compared
to a #50 team, Notre Dame is a complete joke to 
even be mentioned as a # 50 team. Also, the 
Huskies have played a tougher schedule for a #50 
team to compete against, while Notre Dame 
still gets lots of credit for a schedule that isn't 
that close to Washington's in difficulty. Finally, 
we see that Hawai'i does stand out well above a 
typical #50 team even with a light schedule.

What follows are the details:

Notre Dame if #50
G Tech 51.61%
Penn St 26.88%
Michigan 26.88%
Mich St 51.61%
Purdue 32.56%
UCLA 16.74%
B C 40.60%
USC 42.37%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.0002
7 0.0031
6 0.0222
5 0.0865
4 0.2023
3 0.2907
2 0.2513
1 0.1196
0 0.0241

Washington if #50
Syracuse 82.50%
Boise ST 48.84%
Ohio St 18.83%
UCLA 16.74%
USC 42.37%
Arizona St 5.43%
Oregon 10.74%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.0000
7 0.0000
6 0.0012
5 0.0157
4 0.0949
3 0.2782
2 0.3741
1 0.2065
0 0.0294

Hawai'i if #50
N Colorado 100.00%
La Tech 70.85%
UNLV 70.85%
Charleston So 100.00%
Idaho 91.96%
Utah St 92.39%
SJST 80.66%

WINS P
12 0.0000
11 0.0000
10 0.0000
9 0.0000
8 0.0000
7 0.3440
6 0.4240
5 0.1907
4 0.0380
3 0.0033
2 0.0001
1 0.0000
0 0.0000

Sorry Domers, we deal in cold hard facts around here.