The case against the open Beta

In recent years, an increasing number of games have released as Open Betas, allowing customers to get a sneak preview and publishers to get real feedback from potential buyers while gaining some free marketing in the process. It's expected, to the point that gamers will whine and moan publicly if you don't. There are plenty of good reasons to run an open Beta phase for your game project, not limited to:

• Free testing by real users

• Immediate feedback

• Gain market exposure

But if you look at the potential risks imposed by the public Beta, the picture is very different. Let's look at each of those benefits, and the downside imposed by each.

Free Testing by Real Users

Do you know who your users are? Are they simply enthusiasts or are some of them your competitors? A common tactic in software development - particularly in gaming - competitors will sign up for a Beta program to understand if not outright copy your ideas. It's extremely difficult to enforce IP rights in the video game industry, especially when the game may "look and feel" different, while copying the themes or elements of gameplay that make it unique.

During the early days of World of Warcraft, there were many sites dedicated to reverse-engineering the game files from the Beta version for the sole purpose of sharing information with the user community. For the most part, this was a positive influence on the game and eventually, Blizzard put up their own Armory in response. But the damage caused by these community-run websites was clear; nothing could be a surprise to the players anymore, and the competition would always know what Blizzard was up to.

Nintendo has always taken a unique approach among game producers. Simply put, they don't release anything until it's ready. This is possible in part because of their near-legendary status as game creators, with nearly every one of their franchises holding a special place in the gaming community, but also because Nintendo have a long standing track record for high-quality games. The community knows Nintendo will deliver a high quality gaming experience that we tend to forgive consistent delays, but very few publishers have this luxury (See also: Rockstar Games).

Immediate Feedback

Feedback can a double-edged sword. We absolutely need and want feedback from users and the market, but what happens when that feedback isn't positive? We're now in an age where game development studios are giving access to their projects as early as pre-alpha, even sometimes charging for it, and often the feedback given during these early phases doesn't do the developer any favors in terms of later sales and adoption. The problem being, unless your beta testers are under strict NDA, there's little stopping them from posting any negative feedback to Reddit, YouTube, Twitch, and any other medium they choose. Complaining is very popular on the Internet, it seems.

So while it is fast becoming the standard approach for fundraising and getting customer feedback, it may be better to avoid opening up your game to the public too early.

Market Exposure

This is related to the customer feedback in that getting market exposure too early in development may actually harm the release of your game. Take the very public example of No Man's Sky by Hello Games. No Man's Sky was one of the most hyped releases in the history of gaming, thanks in part to some very good marketing. The 10-man team, which is tiny by game studio standards, promised to change gaming forever with their procedurally generated universe; a term that typically sends us techie types into a fervor. 

Their presence at E3 was remarkable in 2014, and again in 2015 although rumors of a scripted demo started to appear on the web. Overhyped reviews, combined with release delays set the stage for what may have been the most disastrous title launch since Duke Nukem Forever. Needless to say, the launch of No Man's Sky didn't live up to the hype they had generated, and some distribution channels were forced to issue refunds to players. Some of the comments on that article sum up the feelings of customers fairly well.

I actually really liked No Man's Sky, and continue to play it from time to time. The developer has solved a lot of the early issues in free updates, and has gone on to release content that expands the scope of the game considerably, the latest at the time of this blog even producing some very nice graphics updates, which is rare for a game patch. Unfortunately, even though Hello Games solved a lot of their release issues, the potential losses caused by the mismatch between customer expectations set by the marketing hype and the actual released game were... significant.

Meanwhile, The Legend of Zelda: Breath of the Wild was delayed for several years, and not only met customer expectations, but vastly exceeded them. In the end, it's about setting expectations: