Post by kevin on Feb 26, 2019 16:32:04 GMT -6
As you may know, I've been critical of the power rating system used for selection and seeding of the playoff teams. While its intent is admirable--select and seed the teams in a fair and impartial manner--it has clear flaws.
It is true that the PR system gets most things right. But that's not a particularly high standard to meet. I looked over the results of the DI girls' playoffs. The better seed, according to the LHSAA system, won 19 of 23 games. That's an impressive-sounding record. But what if you had simply picked the team with the better W-L-T record? You'd also have picked 19 winners correctly.
Another problem is that too much of the system is based on your record and your opponents' records, without any regard for what those records actually mean. Not every 15-win team is the same. Some teams have 15 wins against a very tough 20-game schedule. Other teams have 15 wins against a weaker 25-game schedule. The current system does not control for the number of games played, nor does it look at how good your opponents' opponents really are.
Before I go any farther, I'd like to thank 3balz, who was generous enough to send me the spreadsheet he had compiled in order to update the power ratings every day. His data had every single game played all year, with easy access to any team's schedule, results, etc. Without that there is no way I'd have been able to put this together.
I first tried some simple tweaks to the current system. I tried to account for the different numbers of games played by different teams by calculating each team's win percentage and using that number against a hypothetical 20-game schedule. It didn't make much difference. I tried using an RPI-style system (25% your record, 50% your opponents' records, 25% your opponents' opponents' records). That also didn't make much difference.
It became clear that the schedules that teams play are simply too different for these methods to work well. What I wanted to do was create a method that would be much more similar to what a human would do in comparing teams, while still being completely objective and suitable for a computer to perform. I thought back to what I used to do during the days of coaches' seeding. At the end of the season, I would take the playoff teams and create a brief resume for each one, looking at record, key wins and losses, etc. When I then set out to compare teams, the first thing I looked for, of course, was head-to-head. If Team A beats Team B, Team A is probably better. I know that upsets happen, and that sometimes teams play more than once, but that's my first starting point.
Unfortunately, not all teams play head-to-head. So what's my next step? I look at common opponents. If team A beats teams C, D, and E, but team B loses to C, D, and E, we can probably say that team A is better than team B. Obviously, it doesn't always work out quite that cleanly, but the basic idea should still work. If two teams play the same five opponents and one team goes 3W-1L-1T and the other goes 1W-2L-2T, the team that won three games is probably better.
What I decided to do was to compare every single team in a division against everyone else and award a team a point for each team it defeated, either in a head-to-head matchup (if applicable) or by comparing winning percentage vs. common opponents. If two teams had no head-to-head and no common opponents (or if the records vs. common opponents were identical), I called it a tie and did not award a point to either team.
I tested this method with the DI girls results. With 41 teams in the division, the most points a team could've earned would be 40. One drawback to this method is that it's quite easy for teams to end up tied. If two or more teams were tied, I looked at how they fared in their comparisons to each other. Here are the results of the method:
1. Byrd, C.E. 39
2. Mandeville 38
3. St. Scholastica 37
4. Dominican 36
5. Northshore 33
6. Mt. Carmel 32
7. Acadiana 29
8. Baton Rouge 28
9. St. Joseph's Academy 27
10. Barbe 24
11. Fontainebleau 22
12. Dutchtown 22
13T. West Monroe 21
13T. Lafayette 21
15. St. Amant 19
16T. Sulphur 15
16T. Walker 15
18. East Ascension 14
19. Thibodaux 14
20. Hahnville 13
21. Slidell 13
22. Captain Shreve 11
23. Comeaux 10
24. Denham Springs 10
25. Zachary 10
26T. Alexandria 8
26T. West Jefferson 8
28. Ehret, John 6
29. Chalmette 5
30. Bourgeois, H.L. 4
31. Bonnabel 4
32. Airline 4
33. Higgins, L.W. 4
34. Ponchatoula 3
35. New Iberia 2
36T. King, Grace 1
36T. Covington 1
36T. Pineville 1
39T. Southwood 0
39T. Hammond 0
39T. East St. John 0
You'll notice that this method correctly predicts the winners of 21 of the 23 playoff games, an improvement over the current PR system. In the next post I'll be addressing some of the questions I think people may have, and outline some areas for potential improvement.
It is true that the PR system gets most things right. But that's not a particularly high standard to meet. I looked over the results of the DI girls' playoffs. The better seed, according to the LHSAA system, won 19 of 23 games. That's an impressive-sounding record. But what if you had simply picked the team with the better W-L-T record? You'd also have picked 19 winners correctly.
Another problem is that too much of the system is based on your record and your opponents' records, without any regard for what those records actually mean. Not every 15-win team is the same. Some teams have 15 wins against a very tough 20-game schedule. Other teams have 15 wins against a weaker 25-game schedule. The current system does not control for the number of games played, nor does it look at how good your opponents' opponents really are.
Before I go any farther, I'd like to thank 3balz, who was generous enough to send me the spreadsheet he had compiled in order to update the power ratings every day. His data had every single game played all year, with easy access to any team's schedule, results, etc. Without that there is no way I'd have been able to put this together.
I first tried some simple tweaks to the current system. I tried to account for the different numbers of games played by different teams by calculating each team's win percentage and using that number against a hypothetical 20-game schedule. It didn't make much difference. I tried using an RPI-style system (25% your record, 50% your opponents' records, 25% your opponents' opponents' records). That also didn't make much difference.
It became clear that the schedules that teams play are simply too different for these methods to work well. What I wanted to do was create a method that would be much more similar to what a human would do in comparing teams, while still being completely objective and suitable for a computer to perform. I thought back to what I used to do during the days of coaches' seeding. At the end of the season, I would take the playoff teams and create a brief resume for each one, looking at record, key wins and losses, etc. When I then set out to compare teams, the first thing I looked for, of course, was head-to-head. If Team A beats Team B, Team A is probably better. I know that upsets happen, and that sometimes teams play more than once, but that's my first starting point.
Unfortunately, not all teams play head-to-head. So what's my next step? I look at common opponents. If team A beats teams C, D, and E, but team B loses to C, D, and E, we can probably say that team A is better than team B. Obviously, it doesn't always work out quite that cleanly, but the basic idea should still work. If two teams play the same five opponents and one team goes 3W-1L-1T and the other goes 1W-2L-2T, the team that won three games is probably better.
What I decided to do was to compare every single team in a division against everyone else and award a team a point for each team it defeated, either in a head-to-head matchup (if applicable) or by comparing winning percentage vs. common opponents. If two teams had no head-to-head and no common opponents (or if the records vs. common opponents were identical), I called it a tie and did not award a point to either team.
I tested this method with the DI girls results. With 41 teams in the division, the most points a team could've earned would be 40. One drawback to this method is that it's quite easy for teams to end up tied. If two or more teams were tied, I looked at how they fared in their comparisons to each other. Here are the results of the method:
1. Byrd, C.E. 39
2. Mandeville 38
3. St. Scholastica 37
4. Dominican 36
5. Northshore 33
6. Mt. Carmel 32
7. Acadiana 29
8. Baton Rouge 28
9. St. Joseph's Academy 27
10. Barbe 24
11. Fontainebleau 22
12. Dutchtown 22
13T. West Monroe 21
13T. Lafayette 21
15. St. Amant 19
16T. Sulphur 15
16T. Walker 15
18. East Ascension 14
19. Thibodaux 14
20. Hahnville 13
21. Slidell 13
22. Captain Shreve 11
23. Comeaux 10
24. Denham Springs 10
25. Zachary 10
26T. Alexandria 8
26T. West Jefferson 8
28. Ehret, John 6
29. Chalmette 5
30. Bourgeois, H.L. 4
31. Bonnabel 4
32. Airline 4
33. Higgins, L.W. 4
34. Ponchatoula 3
35. New Iberia 2
36T. King, Grace 1
36T. Covington 1
36T. Pineville 1
39T. Southwood 0
39T. Hammond 0
39T. East St. John 0
You'll notice that this method correctly predicts the winners of 21 of the 23 playoff games, an improvement over the current PR system. In the next post I'll be addressing some of the questions I think people may have, and outline some areas for potential improvement.
