Joined on 8/29/12
I totally approve this XD
It's actually pretty rare to have the winner change in TWO pairings after normalizing. IIRC I've only seen one pairing flip in my years of doing the stats.
Thanks for doing the hard (and hopefully fun!) work of putting this together!
Haha I've learned a lot more about statistics in two years so now I actually know what I'm talking about! XD
Yeah I was surprised to see two winners—but then again, not *that* surprised considering the margins between the two pairs originally were 0.002 and 0.004 points (this is not a typo).
Oh it's totally fun! And thank you for paving the way for this post to exist! XD
Neeeeeeeeeeeeerrrrrrrd.
Just kidding, this is awesome. I love statistics.
my laveneeeeeeeeeeeerrrrrrrrrrrrrrd air balloon will come lift you off your feet
there's only hentai in that .zip; where's the spreadsheet
ohhh wait the OTHER "NGADM '17 R1 Stats' zip oops
Neat. Would be interesting to see full stats on the pairings "predicted", but overall, good number fun.
Ahhh I would've included them in the spreadsheet except I did them by hand! :P (Mostly because I couldn't think of an Excel-friendly way to calculate them haha)
IIRC though, the most controversial pairing was SoundChris vs EvilRaccoon, where 4/6 judges slightly preferred SoundChris' piece but the other two loved the raccoon and hated the Chris so much that they tipped the scale in EvilRaccoon's favor! Weeeird.
Really cool work here, LunacyEcho! While I'm not convinced that normalization is a completely fair way of calculating scores, it's totally worth consideration as a fun statistical analysis. :) Hope to see more of these in future rounds!
Thanks, TL! :D Tbh I would still support normalization for two main reasons—accuracy and fairness.
Accuracy | One judge's 8.0 may not be another's 8.0. What each judge is doing is comparing every song against each other, and the difference between judges' likings of different songs is not equal. Without a unified scale, the processing of averaging un-normalized scores is ultimately an inaccurate representation of what they actually think.
Fairness | Each competitor should be given an equal shot, and similarly, each judge should be given the same weight. However, without normalization, the judges can actually have different weights on the scores, making some judges more *valuable* (for lack of a better word) than others in terms of deciding the overall scores. This upsets the inherent idea that of the six judges, each should have a fair and equal say in the scoring process.
However, you're totally right that normalization does have its drawbacks! imo one of the biggest reasons NOT to use normalization is that there's actually a score cap of 10. Because judges are only human, and because there will always be a better song (no matter *how* good a song is), the scores get more iffy the closer they get to a perfect 10. This means that the judges' scoring scales may not be linear, which is an assumption necessary for normalization to be accurate.
...wow, I ended up typing a lot more here than I thought I would. In the end, it's really up to the organizers. But no matter what the scoring process is, that doesn't change the fact that there is so muCH GOOD MUSIC PRODUCED in this competition and in the end, that's what really matters! :D :D
You, sir, have done your homework! And that's putting it lightly...
I quite enjoy delving into all these statistics, so thank YOU for taking so much time to compile all of this data! I'm just stopping by for a moment before continuing on my merry track-scoring way, but I appreciate the heck out of all the effort you put into this.
I hereby dub thee Protégé of Gadogry!
Haha thanks! Good luck scoring, and it's an honor to be dubbed thusly! :)
Hmm...while I agree that normalization maximizes "fairness," I think that fitting the judges' scores into a fixed distribution is problematic because it distorts the extent to which they believe the participants are similar in skill level. Based on the non-normalized figures, SkyeWint may have more "weight" than Samulis, but that weight is a product of the judges' assessment of the similarity of everyone's ability, reflected in the standard deviation. Never mind the fact that each judge uses different scoring standards that are not easily comparable.
Not that our current system is perfect, but I question the value of "equal weight" in judging.
Sorry if I'm being too argumentative, though. I do appreciate all the work you've put into this! :)
McGorilla42
Wow. That was really interesting! Thanks for all the work you put into it! Will you do it again for the rest of the rounds?
LunacyEcho
Thanks, McGorilla! I'm planning on it, as long as I can find the time. :P (Although once it gets down to the semifinals and finals, there are so few data points that I may have to get creative...heh)