GUESS by the numbers Topic

One of the "criticisms" of the GUESS reports is that it undervalues QB's.  And I can't disagree with that - certain positions consistently score higher in the ratings than others.

I took a look at approximately 80 seasons of recruiting classes, across all 11 worlds, and pulled some raw stats and re-ran GUESS ratings, along with the "base" ratings from the GD 3.0 engine currently in Beta.  The results below are for D1A; all other divisions showed similar results, albeit with QB coming in slightly lower.  I tossed out some players with abnormally low scores - they were obviously AC signings and were skewing the result because the AC doesn't sign players in the same kinds of distribution as human coaches.



Most of the columns should be self explanatory - the " % " column shows how the average rating compares to the highest average rating across positions.  For example, the QB average rating is 84% of the Running Back average rating.  For D1AA, D2, and D3, that number runs closer to 80%.

However, as you can see, it's not so much a result in deficiencies of the formulas as it is deficiencies of the STATS.  The CORES column, which is the result of just adding up the three main "core" attributes at each position, shows core stat scores for QB's are substantially lower than they are for other positions.  Totals are included just for comparison sake.

One of the things we've talked about in the past is "normalizing" the ratings between positions, so that the average rating across all positions are near (and hopefully the MAX ratings follow suit).  With a sample set this large, I think I can comfortably adjust the formulas to do so, not just for QB's but for all positions.  The two big questions are -- 1) Should I do so, or should I let the raw stats dictate ratings regardless of position, and 2) Implement now, or wait until the big revision I'm working on in conjunction with the beta?

For those who are interested, current formulas can be found here.  Note that for DL / LB / DB, and the purposes of this exercise, I averaged the weights between the two sub-positions (i.e. average of S/CB weights to get DB formula.)

Other thoughts and constructive criticism welcome.

 

4/29/2013 3:44 PM
Thanks bhazlewood. Thanks for all of the hard work that you have done for us.
4/29/2013 3:53 PM
I say go ahead and do it.  There will always be issues, like the fact that having the best kicker or TE isn't as important as having the best QB, but it will be a step in the right direction.
4/29/2013 5:16 PM
I hate the word "normalization", so my vote is no...
4/29/2013 7:11 PM
I'm all for it if you think it will make the Guess Ratings better! I really enjoy them now but it does suck getting knocked down for having a QB in your class.
4/29/2013 7:25 PM
I like your idea and thank you for your efforts.
4/29/2013 8:35 PM
I also resist the efforts of normalization - but that is just for my personal lifestyle - for the GUESS ratings GO FOR IT! - Probably wait for the beta though if it makes more work for you.
4/29/2013 11:47 PM
I say, do whatever makes sense to you. You are the developer, so work with your end in mind, and make it as easy on yourself. What you are doing is hard enough as it is.
4/30/2013 12:06 AM
Posted by aegsm76 on 4/29/2013 7:11:00 PM (view original):
I hate the word "normalization", so my vote is no...
+1
4/30/2013 2:18 AM
How would you do it? Would the best player at a position rate a 1 and the worst rate a 0 in terms of the overall guess score or would you normalize each stat individually?
4/30/2013 11:37 AM
My vote would be to wait and include these findings and adjustments in the revision that's coming with the beta going live.  No need to add the extra work of two updates to the already fantastic job that you do.
4/30/2013 12:29 PM
Bob, can you show us the same data again, but this time with the STDDEV added? If it's not to much work, that is...
4/30/2013 1:11 PM

4/30/2013 5:00 PM (edited)
Yeah, that's what I thought. So my normalization attempts were first to shift the means up to the highest level. So I'd bump every QB by some factor that made the QB mean equal to the RB mean. This didn't really get me a good mix of positions/players in my Top 100, because of those STDDEV differences. You're still overloaded with RBs, TEs and WRs.

So then I tried to normalize through using # of STDDEV +/- (from the Mean). This gives you a better representation of how good a given player is, relative to his peers (ie, other players at that same position). But then what you end up with is QBs, Ks and Ps looking like your best overall players. That's because the "Max" players at those positions are more STDDEV above their positional means than the others.

So neither of these results made me feel like I'd arrived at a satisfactory result. I also played - shortly - with my own hard-coded normalization factor that I arrived at by simply playing around with the numbers and seeing my mix of players. But ultimately, it felt too forced, so I abandoned it.

I needed/need a better statistics background to know how to work around these issues.

4/30/2013 8:00 PM
i've gotten used to the guess player ratings, and i'm OK with my qb being inherently less "valued" than my rb or wr.  it's just that the recruiting and overall team ratings aren't normalized.  that's something that could easily be adjusted, right?
4/30/2013 9:51 PM
1234 Next ▸
GUESS by the numbers Topic

Search Criteria

Terms of Use Customer Support Privacy Statement

© 1999-2024 WhatIfSports.com, Inc. All rights reserved. WhatIfSports is a trademark of WhatIfSports.com, Inc. SimLeague, SimMatchup and iSimNow are trademarks or registered trademarks of Electronic Arts, Inc. Used under license. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.