lol Metacritic, etc

[GONNAFISTYA]

Fuck it, this deserves its own thread...




Yeah...so it turns out Metacritic doesn't exactly have a "level playing field" when it comes to how they weigh review scores.

Full Sail study attempts to shed light on Metacritic's weighting system

The website Metacritic weights the scores of different video game critics and publications when calculating its aggregate 'metascore' for any particular title.

Each critic/ publication is assigned one of six different weightings of 'importance', with some publications exerting considerably more influence over a game's final 'metascore' than others.

This was revealed by Adams Greenwood-Ericksen of Full Sail University at a talk titled 'A Scientific Assessment of the Validity and Value of Metacritic' delivered at the Game Developer's Conference in San Fransisco this afternoon.

Here's the important part:

Metacritic confirmed to Greenwood-Ericksen during the course of his research that the site applies different weightings to incoming critics and publications' reviews in order to calculate its 'averaged' numerical score for any particular title.

Metacritic's response: "their guesses are wildly, wholly inaccurate"

Today, the website Gamasutra "revealed" the weights that we assign to each gaming publication (for the purpose of calculating our Metascores), based on a presentation given at the Game Developers Conference this morning. There's just one major problem with that: neither that site, nor the person giving the presentation, got those weights from us; rather, they are simply their best guesses based on research (the Gamasutra headline is misleading in this respect).

First off, the Greenwood-Ericksen report on the Gamasutra site was not journalism or someone's opinion, it was the results of a scientific study, thus having no other possible result other than giving "their best guesses based on research". FFS...that's what science is. The fact that Metacritic is doing everything it can to ignore that fact and refusing transparency of their "real" weighting system just makes things worse for them. Face it, Metacritic...your "game" has been exposed. Nobody should trust your results ever again.

I stopped paying attention to Metacritic scores more than 5 years ago because I saw it for what it is, a hammer to be used against the industry and the people who work in it. I'm hoping more people ignore it as well, but I don't hold out much faith...people are just too fucking stupid. It's a hard, uphill slog to change that mindset because 95% of upper management in games companies still see the Metacritic rating as their holy grail of games quality and more importantly...marketing budget approvals. I seriously despise this industry sometimes.

[scared?]

weirdo...

[GONNAFISTYA]

Even from here I can hear the shit loading up your kid's diapers.

[Ryoki]

Intresting reading the article and the comments on that page. Lies, damn lies and statistics eh?

To be honest it sounds like a business opportunity, if MetaCritic is indeed so fundamentally broken and dishonest, someone needs to make something that actually works.

[xer0s]

Metacritic is really a great concept if they would simply give every critic the same clout. But greed is powerful...

[obsidian]

someone needs to make something that actually works.

There is, it's called, "Let some other sucker buy the game first and post a video of them playing it on YouTube".

[GONNAFISTYA]

Metacritic is really a great concept if they would simply give every critic the same clout. But greed is powerful...

It's only a great concept as it applies to the customer...who rarely give a shit about those scores anyways.

However, it is a horrible idea as it relates to developers because that's all upper management focuses on and, as I mentioned before, the scores themselves are used against the development staff as a means to withhold bonuses/advancement/project funding and, at its absolute worst, an easy means to shut an entire studio down.

Fuck Metacritic.

[GONNAFISTYA]

someone needs to make something that actually works.

There is, it's called, "Let some other sucker buy the game first and post a video of them playing it on YouTube".

Stop giving away my secrets.

[obsidian]

While on the subject of secrets, GONNAFISTYA also keeps a girl locked up in his basement. Did you at least remember to feed her and empty out her bucket today?

[GONNAFISTYA]

I have a Mexican doing that.

[Theftbot]

Lil fellow died from starvation though.

[losCHUNK]

Plenty more in the sea

[Transient]

I only ever use Metacritic to look at user scores. I never really cared what "critics" thought.

[Tsakali]

those are proly messed up too.

There is such a thing as author 'authority'.

[seremtan]

someone needs to make something that actually works.

There is, it's called, "Let some other sucker buy the game first and post a video of them playing it on YouTube".

ding ding

[Plan B]

I like watching totalbiscuit's WTF is... play throughs.

This guy covers a lot of games, and he always seems objective and accurate.
I find myself agreeing with him most of the time.
Usually quite funny too, it's like watching Alan Partridge judging games.

[Tsakali]

surely a youtube video can't do the feel of a game justice? what would Q3 be if it wasn't for it's superior feel on the controls?

or is game feel not important anymore? Probably something that can't really be appreciated and trully experienced on some kind of console controller... which saddly seems to be the general game interface choice these days.

[Scourge]

surely a youtube video can't do the feel of a game justice?

You can't really get that without playing it anyway. I find the videos far more helpful deciding on a game than any critic's opinion by far. I've never relied on critics for anything anyway.

[LawL]

Good for you.

[Eraser]

surely a youtube video can't do the feel of a game justice? what would Q3 be if it wasn't for it's superior feel on the controls?

or is game feel not important anymore? Probably something that can't really be appreciated and trully experienced on some kind of console controller... which saddly seems to be the general game interface choice these days.

And a review can do it justice? Now don't say "no, but a demo will" as we've already established that there is no better preview than hands on experience with the game (demo) itself? Unfortunately there's a distinct lack of demos these days.

[DTS]

They don't make demos cause if they did then you'd realise how shit their game was and not buy it.

[seremtan]

i bought the first CoD because of the demo - it was the village assault level that starts with Capt Foley saying "ok men, get to set to move on my command!", right before a load of MG42s open up on you in the middle of a minefield. awesome stuff

[MKJ]

surely a youtube video can't do the feel of a game justice? what would Q3 be if it wasn't for it's superior feel on the controls?

or is game feel not important anymore? Probably something that can't really be appreciated and trully experienced on some kind of console controller... which saddly seems to be the general game interface choice these days.

as always the point should be made; FPS isnt the only genre out there.

[LawL]

Video games.

[Plan B]

Pretending to live "la vida loca", while being on a 20 year old game board 24/7