In retrospect, this was bound to happen. And it makes a lot of sense.
The last few weeks have been a real head-scratcher for me. I haven't been able to dedicate the time I need to this site, and because of that I missed my January 1st deadline for taking SomeAwesome out of private beta (i.e. it is still closed to new user signups). I have a ways to go to make the site work in alignment with the vision that I have for it, and each new week seems to bring another curveball out of left field.
Clearly I also need to work on my baseball metaphors.
On Tuesday, Steam made an announcement that really made me sit up and take notice. In a nutshell, they are ending the points reward system for user reviews. Whereas once before you could spend points to give a review a special "award" (complete with a variety of custom emojis) and pass a percentage of those points on to the reviewer, now you have a mere 12 award types to choose from, and no points are passed on. Their explanation for this change was hard to hear, but I agreed with it 100%, because it hit pretty close to home for me and how I feel about SomeAwesome.
TL;DR, they want to prevent shitty, hastily written reviews. Same bro, same. (If you want to TL;DR this article click here)
SomeAwesome's mission has always been to empower and encourage as many people as possible to write unique, genuine, and comprehensive game reviews. But at this point even I have to admit that, in its current incarnation, it also allows for lazy, race-to-the-bottom game reviews constructed with a bare minimum of effort. Reading this surprisingly (IMO) brave announcement from Valve has inspired me to redouble my efforts on both sides of the coin: not only giving people the tools to write the best reviews they can, but also in preventing low-quality slop from getting published in the first place.
So what does this mean, really?
From a long-term perspective, this means adding more quality checks to the algorithm. We already calculate points based on 21 basic "qualitative factors" from the review structure itself, even before we run it through two completely different AI models to assess whether a review was written using AI. But this is not enough, not even remotely close. So we have a roadmap outlined that will build out additional features, bonuses, and penalties that hew towards a variety of "game review best practices", to be implemented in the coming months.
Side note: For what it's worth, if you want to know what I consider a good example of "game review best practices", just go over to the Game Informer website and read their 30+ years of game reviews.
From a short-term perspective, I had to look at the data and figure out what was the quickest win to get everybody on the same page, while simultaneously allowing me more time, bandwidth and sanity to focus on the work that will push this site forward. And the data told a very simple, pretty obvious story.
GOOD REVIEWS NEED TIME TO COOK
I'm a big believer in the phrase "the numbers don't lie". And there are two very specific numbers that correlate almost 100% of the time: number of days between submitted reviews and average amount of points awarded. In other words, the users who submit reviews with at least 7 days between each review had more than double the amount of points awarded than the users who submit a review less than two days apart. On the surface, this is pretty obvious, because if you're submitting a review every day or two then you are either:
a) Not playing the game enough
b) Rushing the review and leaving fields empty
c) Using AI to construct the majority of your review
d) All of the above
And hey, look, I'm not an "Anti-AI" guy. I use it at work to build reports. I use it to code, I use it to automate rote, repetitive tasks, etc. Plus I obviously use AI to check for AI. I'm absolutely not knocking AI as a technology in-and-of-itself like I know a lot of people tend to do. But the entire reason I built SomeAwesome is because I WANT TO KNOW YOUR ACTUAL OPINION ABOUT GAMES!
Side note: For those of you who use AI just to translate your reviews from your native language into English, I've confirmed that this does not seem to impact your score after the checker does its assessment. So you're safe to continue doing that if you need to.
Up to now I thought my only recourse to this was to train the system's AI-assessment tool and make it completely foolproof. That way I could effectively combat this problem by banning users that used AI all the time for every section of their reviews. What I realized instead was...the data tells enough of a story that making it foolproof just isn't necessary. The best thing I can actually do, is simply to bend the site to adhere to the data pointing at the highest-quality reviews. And there's an easy way to do that. In fact I've already done it.
Say Hello to the 7-Day Review Cooldown
Makes sense, right? If the users creating the reviews with the highest average point totals are submitting them on average only once a week, then that's how long it should take everybody to write their reviews. This gives you enough time to play your game thoroughly, really think about the pros and cons of the game, and ensure that you add everything that the site asks you to fill in to make your review bulletproof (and therefore maximize both your points and the quality of the review). For those of you who are dissatisfied with the amount of points you receive on your reviews, that last point is very important to understand!
If you are skipping fields because you don't think it is important or you are just trying to put in as many reviews as quickly as you can, think about why you are doing that.
If you are just filling in fields with placeholder text or stock images because you are trying to "game the system", then really THINK ABOUT WHY YOU ARE DOING THAT!
The system is literally designed to catch and penalize incomplete, insincere and irrelevant content. The best advice I can give to you is to make sure you fill in as many fields as you can with the most comprehensive and relevant content possible. It's as simple as that.
TL;DR Version
So here's the deal. For now, there's a 7-day cooldown timer that goes into effect once your review is submitted. Even if it takes me some time to get around to publishing that review, the cooldown is calculated from the time it is sent in for moderation. This way you're not unfairly penalized in case I can't be around to publish it in an expedient fashion. I'm mitigating both of my biggest pain points: it levels the playing field for the people really taking their time with their games and reviews, and it prevents a race-to-the-bottom incentive for people rushing their reviews out the door in an attempt to just redeem as many games as they can in the shortest time possible.
So, finally, if you've read through this short novel of a post, I really appreciate that. It means you care about what I think this site can become in the long run. I know it's a lot, but together we can make this site a true gem among all the review sites out there. You guys are what makes all of this worth it.
Sincerely,
~ Serverus