Originally Posted by Myrtledog
It's a very interesting undertaking and I would read the results too. You know it will happen but I really hope a ton of people don't just say it was a bad connection if they don't play well. If people truly cooperate, it could be a big step in the right direction.
Well, two ways they get around that. Either they go the scientific route, and correlate the results to objective measures like connection quality and score. They can try and identify the players that consistently rate it as poor (or good) no matter what, and then test changes to see which changes satisfy which populations. The actual score doesnt matter, just the change in it. I'm sure there's a heavy correlation between losing and perceiving a bad connection - they can measure the strength of that correlation and factor it out. As long as they have enough data, and a competent statistician, they can really get down to the bottom of it, no matter how hard anyone tries to confound it. The people trying to screw it up will inadvertently give them good data, a measure of the number of people that are impossible to satisfy, and who they are.
If they wanted to be *really* clever about it, they could measure the circumstances that individual players rate their games the highest, and work that into matchmaking. Some players seem to despise being host, some love it. They could just prevent those players who hate it from ever being host, and vice versa. Now would also be a great time to see how the number of times host switching affects perception of quality. Some might not mind, some might really hate it. They should prevent constant droppers from ever being host, and match the people most bothered by host switching with the most reliable hosts, even though theyre not the fastest. People who prefer the most responsive connection could be put into matches with the best connection, etc. There's so much potential here if they really do this right. They could take that data, link it to gamercard, and carry it right into every game Activision makes.
The other options is to not track any of that, just the total global response, tweak something, and see how the numbers change. Even if 90% of people say they got a poor connection no matter what, if they can move it up to 85%, its a change in the right direction, which is all that matters. It'll still get them good results over time, but it's the lazy way to do it, and doesnt really inform their design for future games as well.
This kind of study would have industry-wide significance, it'd actually be an incredibly valuable, whether they share it or keep it a trade secret. Even if they're not going to put out a paper, it'd make a great GDC presentation.Edited by bd2003 - 5/1/13 at 4:13pm