Human Influence in The Toy

Mike

ModeratorMike

Six-man master
Administrator
In another topic the idea that the ratings could be flawed because of arbitrary numbers that are sometimes assigned to teams. Since someone (now, me) does have to set up the pre-season rankings, which @granger agrees are "just a guess... nothing more. something to get the system started", I wondered how much affect that might have on the outcome of the rankings, especially this late in the season. It's worth noting that the pre-season rankings are the only time the ratings are altered by a human. After that, it's all left up to The Toy.

Anyways, since I'm new to the internal workings of the rankings, and really wanted to answer this question for myself just as much as for anyone else, I decided to run a few tests.

Because the schools being discussed in the topic that sparked this idea were the top 3 schools in division 2, the schools being altered for the tests are Motley County, Strawn, and Richland Springs. No other school's rating, or rank, was changed from what it was to start this season.

The results show the school's rating in comparison to what their rating is at the time of this post. In other words, the results show what the outcome would be at this exact point in the season had I interfered more with the pre-season data, as compared to the way they are currently on the site.

Test 1

For the first test, I took 200 points off of each of the top 3 D2 schools' (Motley, Strawn, Richland) pre-season rating. Keep in mind this is the only time that the rankings are altered by human at any point during the season, and 200 points would be a very dramatic decrease that simply wouldn't happen. I wouldn't lower a school by 200 points even if I knew they were losing all 6 starters. Anyways, I then re-ran the rankings as I normally would.

Motley County's rating dropped from a 391.9 to a 378.83. Oddly enough, though, this would have them sitting at 2nd overall instead of 3rd.

Strawn dropped from 368.78 to 358.08. Their overall dropped from #7 to #10.

Richland Springs went from 351.82 to a 347.99 and their remained the same at #18.

Test 2

Next I removed 100 points from each of the schools.

This time Motley County dropped from 391.9 to 383.76. Their overall ranking remained unchanged at #3.

Strawn went from 368.78 to 362.29 and their ranking fell from #7 to #9.

Richland Springs went from 351.82 to 350.21 and their ranking rose from #18 to #17.

Test 3

In the next test I changed Motley County's rating to a 59, which would put them in last place to start the season.

In this test Motley County's rating ended up being a 381.25 (-10.65), and they (again) went from #3 to #2 overall.

Because of the affect that Motley's lower ranking had on other teams, Strawn's rating also went down to 361.19 (-7.59) and they fell from #7 to #8 overall.

Richland Springs also took a slight hit to their rating, lowering them to 350.34 (-1.48), and bringing them up (again) to #17.

Test 4

In the final test I kept Motley County at a 59 rating (last place overall). Then I bumped Strawn up 100 points, putting them well into 1st place overall.

This time Motley County landed at a 383.67 (-8.23) and #2 overall.

Strawn ended up with a 377.25 (+8.47) and was sitting at #4 overall.

Richland Springs got a 355.56 (+3.74) and was sitting at #17 overall.

Summary

So, to summarize, I'd say human interaction has very little affect on the rankings, especially as the season progresses. The fact that there was less than a 4% change in all ratings, even with very drastic changes to the pre-season ratings should put to bed the idea that The Toy can be affected by human influence. Regardless of what I did, these three teams still ended up in the exact same order in their division and pretty close to where they were in the overall rankings too.

I think it's fair to say that what matters to The Toy most is the outcome of the actual games, which is the only way to know exactly what's going to happen anyways.
 
Last edited:
What formula is used to get that initial preseason rating? Do you start at the previous seasons final rating and then make adjustments based on starters lost and returning lettermen? We are about to lose 10 seniors and this is a very interesting topic for me.
 
phimutau said:
What formula is used to get that initial preseason rating? Do you start at the previous seasons final rating and then make adjustments based on starters lost and returning lettermen? We are about to lose 10 seniors and this is a very interesting topic for me.
Click to expand...

There isn't any specific formula for pre-season rankings and, honestly, I'm not even sure what I'll do yet. It was so late when I picked things up this year that I decided to just use last year's final rankings as this years pre-season rankings. I adjusted the ratings a bit bit let the order the same. That will probably be the starting point for next year but, yes, if I know a team is losing a lot of their starting team I'll probably drop that team's rating by a bit.

I wouldn't expect next year's pre-season rankings to vary all that much from the final rankings this season though. I certainly won't have information on all the teams.
 
I have followed these rankings since Granger was handing out the rankings on paper. People have always insisted he adjusted the rankings manually at times. This is poppycock of course that arises because very few people have a basic understanding of advanced statistical analysis. I mean who wants learn that stuff other than a few random nerds like me? Your tests are interesting fun, thank you for sharing.

I don’t remember when “The Toy“ became common to the Texas Sixman culture but it has been many years now. It will be an uphill battle and a bit sad for people like me to see the end of that expression.
 
GetARealProblem said:
I have followed these rankings since Granger was handing out the rankings on paper. People have always insisted he adjusted the rankings manually at times. This is poppycock of course that arises because very few people have a basic understanding of advanced statistical analysis. I mean who wants learn that stuff other than a few random nerds like me? Your tests are interesting fun, thank you for sharing.

I don’t remember when “The Toy“ became common to the Texas Sixman culture but it has been many years now. It will be an uphill battle and a bit sad for people like me to see the end of that expression.
Click to expand...
Goob? Popycock, catfish and everkleer?
 
GetARealProblem said:
I don’t remember when “The Toy“ became common to the Texas Sixman culture but it has been many years now. It will be an uphill battle and a bit sad for people like me to see the end of that expression.
Click to expand...

Turns out, you're right about it being many years. I found a thread from 2011 where Granger referred to it as The Toy. So, let's just stick with that.
 
Mike said:
So, instead of calling it the "toy", since I'm really not even sure how that got started, I'm just going to start calling it "The Machine" (you can thank Bert Kreischer).

Ok, so The Toy wasn't something that was just recently introduced. Evidently this name has been around for quite some time. So, we'll just stick with that.

Anyways, in another topic the idea that the ratings could be flawed because of arbitrary numbers that are sometimes assigned to teams. Since someone (now, me) does have to set up the pre-season rankings, which @granger agrees are "just a guess... nothing more. something to get the system started", I wondered how much affect that might have on the outcome of the rankings, especially this late in the season. It's worth noting that the pre-season rankings are the only time the ratings are altered by a human. After that, it's all left up to The Machine Toy.

Anyways, since I'm new to the internal workings of the rankings, and really wanted to answer this question for myself just as much as for anyone else, I decided to run a few tests.

Because the schools being discussed in the topic that sparked this idea were the top 3 schools in division 2, the schools being altered for the tests are Motley County, Strawn, and Richland Springs. No other school's rating, or rank, was changed from what it was to start this season.

The results show the school's rating in comparison to what their rating is at the time of this post. In other words, the results show what the outcome would be at this exact point in the season had I interfered more with the pre-season data, as compared to the way they are currently on the site.

Test 1

For the first test, I took 200 points off of each of the top 3 D2 schools' (Motley, Strawn, Richland) pre-season rating. Keep in mind this is the only time that the rankings are altered by human at any point during the season, and 200 points would be a very dramatic decrease that simply wouldn't happen. I wouldn't lower a school by 200 points even if I knew they were losing all 6 starters. Anyways, I then re-ran the rankings as I normally would.

Motley County's rating dropped from a 391.9 to a 378.83. Oddly enough, though, this would have them sitting at 2nd overall instead of 3rd.

Strawn dropped from 368.78 to 358.08. Their overall dropped from #7 to #10.

Richland Springs went from 351.82 to a 347.99 and their remained the same at #18.

Test 2

Next I removed 100 points from each of the schools.

This time Motley County dropped from 391.9 to 383.76. Their overall ranking remained unchanged at #3.

Strawn went from 368.78 to 362.29 and their ranking fell from #7 to #9.

Richland Springs went from 351.82 to 350.21 and their ranking rose from #18 to #17.

Test 3

In the next test I changed Motley County's rating to a 59, which would put them in last place to start the season.

In this test Motley County's rating ended up being a 381.25 (-10.65), and they (again) went from #3 to #2 overall.

Because of the affect that Motley's lower ranking had on other teams, Strawn's rating also went down to 361.19 (-7.59) and they fell from #7 to #8 overall.

Richland Springs also took a slight hit to their rating, lowering them to 350.34 (-1.48), and bringing them up (again) to #17.

Test 4

In the final test I kept Motley County at a 59 rating (last place overall). Then I bumped Strawn up 100 points, putting them well into 1st place overall.

This time Motley County landed at a 383.67 (-8.23) and #2 overall.

Strawn ended up with a 377.25 (+8.47) and was sitting at #4 overall.

Richland Springs got a 355.56 (+3.74) and was sitting at #17 overall.

Summary

So, to summarize, I'd say human interaction has very little affect on the rankings, especially as the season progresses. The fact that there was less than a 4% change in all ratings, even with very drastic changes to the pre-season ratings should put to bed the idea that The Machine Toy can be affected by human influence. Regardless of what I did, these three teams still ended up in the exact same order in their division and pretty close to where they were in the overall rankings too.

I think it's fair to say that what matters to The Machine Toy most is the outcome of the actual games, which is the only way to know exactly what's going to happen anyways.
Click to expand...
turned out well
 
Just for the sake of transparency, I wanted to circle back to this again. I apologize if this turns into a long-winded post and I realize there may be a lot of this that some of y'all couldn't care less about, but I wanted to post it for those who do. Also, full disclaimer, that I still haven't completely wrapped my head around this algorithm. I'm not a mathematician so this is certainly not my area of expertise like it was for Granger. I'm just a dude who knows how to write code and build websites. With that said, I want you all to know that I am 100% dedicated to ensuring that this site continues to roll on as best I can, as accurately as I can, for as long as I can.

As some of y'all know, as soon as I took over this site my first priority, after getting things working again, was to re-write The Toy in a coding language that I know how to use. This took me a few years because it was originally written in a coding language I'd never even seen before, so I had to learn that before I could convert it. Once that was done, I made sure to build in a lot of statistical reporting so that I can keep up with how well The Toy is doing from week to week, and also because I wanted to be able to run tests in the off-season to find out if there are small tweaks I could make to the algorithm to make it more accurate from year to year.

Just this year, thanks to the pick 'em, I noticed that the statistical reporting was, at least sometimes, incorrect. I don't remember which week it was, but during one week early in the year, after a Thursday night, the pick 'em showed The Toy as having more wrong games than my stats did for the entire night. The way the pick 'em picks are entered is pretty much the same as anyone else so, as long as I don't screw up when I prepare it each week, the picks cannot be wrong. It was only at that point that I knew there was something bad wrong with my stat reporting, so I went to digging around in the code, trying to figure out what the heck I'd done wrong. It took me a few weeks but I was able to figure out what happened. I thought I was recording the stats in the same way (the same place) that Granger had been, but the differences in the way the arrays are stored made it incorrect, and I never noticed it until early this year. Come to find out, where I was recording the stats for each game was, sometimes, after one (or both) of the teams involved had already had some processing done on their rating. The way this worked was to always make The Toy's stats look better than they actually were.

The reason for this post is for me to be able to say that all of the off-season testing and tweaking I've done, over the last several years, has been for almost nothing and it's quite possible that The Toy has not been as accurate as it could have been the last several seasons, including this one. Now, I won't know that to be the case until I get to start testing again this coming off-season with actual accurate stat reporting, but I'm going to assume that there are things that could be done to make it a percentage point, or two, better. It could be why, no matter what I did in my tests, the results were largely the same.

All that said, I do believe I've got the stat reporting fixed now and I've been (and will keep on) monitoring the results each week, looking for inconsistencies. Assuming it is fixed, I am happy to say that, since week 4, The Toy is still hitting over 80% accuracy every week, and topping out at 86.96% a few weeks ago. Pretty darn good if you ask me, for a computer with absolutely no human influence other than the pre-season ratings, and especially if there's something I could've done to make it better. I'll update this thread again this off-season if I'm able to find out that things could have been better this season had my statistical reporting been right.
 
Thanks for all you are doing Mike! You've taken a fun website and continued its positive evolution. There are a large number of us who are so very pleased with the transition. Truly, thanks!

Ps. I know my Underwood's 6-Man Football is in high demand, and I also have a number of old, paper Huntress Reports. #windfall
 
Dale Miller said:
Thanks for all you are doing Mike! You've taken a fun website and continued its positive evolution. There are a large number of us who are so very pleased with the transition. Truly, thanks!
Click to expand...

I appreciate that, but there are always folks on both sides of the fence where change is involved. This post wasn't meant to be about that, but just about the fact that the rankings, which I didn't intend to change, may not be what they used to be because of an error on my part. We soon shall see!
 
Mike said:
I appreciate that, but there are always folks on both sides of the fence where change is involved. This post wasn't meant to be about that, but just about the fact that the rankings, which I didn't intend to change, may not be what they used to be because of an error on my part. We soon shall see!
Click to expand...
My favorite rankings are produced on the field, November into December. But I absolutely follow The Toy; it should be fun hearing updates during the off season.
 
Back
Top