I am a lifelong Pokémon fan (Pikachu was my ringbearer) that desperately wants Pokémon GO to survive and thrive. I think the game has tremendous, disruptive potential for the long term (you can read my initial writeup on Forbes.com.) But I’m also a 15-year veteran of the software, gaming and entertainment industries – and I have many thoughts on the changes in the game in the last month.

In the month since Pokémon Go’s initial US release, it has been a rocky road, with spectacular highs and worrisome trends. Certainly the popularity of the game has, for the most part, transcended the recent issues — but the momentum seems to be pointed in the wrong direction when it comes to nurturing a game – and not just another app – that can rally a healthy community. I say all of these things as a lifelong Pokémon fan that desperately wants this game to survive and thrive.

Certainly the short-term popularity of the game has, for the most part, transcended the recent issues – but the momentum seems to be pointed in the wrong direction, and I feel compelled to deconstruct the latest frustrations with the game.

Background

Shortly after I penned my first article (originally published on Quora) – in late July, Niantic unceremoniously released a new update to Pokémon Go. Some of the major changes:

• A major feature – the proximity indicator for nearby Pokémon – was completely removed (for about 2 weeks it had been present but broken.)
• The strength and timing of many Pokémon moves were changed.
• Battery Saver mode, which darkens the screen when the phone is turned upside down to reduce battery drain on long Pokewalks, was removed from iOS with no explanation.
• Pokemon of all levels became significantly harder to catch.
• A feature which awarded extra XP for “skillful” Pokeball throws was removed with no explanation.

This release led to significant outcry from the player community. The outcry was best observed on Reddit, where players eviscerated Niantic for making the game “unplayable”. Some of this was hyperbole, but in games many times perception is reality. Now, to an outside observer, the game still functions – you can turn the game on, catch Pokémon, and fight at gyms, right? The fundamental problem facing the Pokémon Go community is that Niantic is not an experienced game development company. Niantic has yet to learn best practices for developing a game-as-a-service platform from other industry titans.

Thus, Niantic appears to be making decisions like a traditional software company, and not a game company. They are not acknowledging the emotional connection players have with a game, the change in motivation behind purchasing for entertainment versus productivity, or the importance of core game design imperatives like providing paths to mastery.

The Mastery Problem

I touched on the importance of mastery in my initial article. In short, people who play games generally expect to be rewarded more as their skills increase. They want to clearly understand what skills are needed, and they want rewards when those skills are demonstrated successfully. One such mechanic in the launch of Pokémon Go was the experience boost for well-thrown Pokeballs. This clever mechanic provided extra points when a player either curved a ball and returned it to center for a catch (like a pro bowler’s throw) or landed a particularly well-centered straight shot. The bonuses could range from 10 – 100% of the XP for a single catch.

In the July Pokémon Go update, this bonus was unceremoniously removed. For players that had spent time trying to understand and practice the skills required to land a skill shot, this was a low blow. Niantic made this skill useless, and provided no other way for players to feel as if their amassed experience in the wild Pokémon encounters was useful. Without skill shots – and with the vastly increased chance of failure in that update – the wild Pokémon encounters feel like a random number generator. Why force players to sit through the encounter when their success is more determined by random chance than a player’s skill? Gameplay fun often evaporates when the last shred of control is taken from the player. (It is for this reason that a subgroup of board game players vehemently detest any game where dice play a significant role.)

Niantic has been slowly but surely de-gamifying their game in fear of bugs, all with little transparency to their player base.

The other, perhaps more controversial, loss of masterable game mechanics surrounds the “three-step bug”. At original launch, Pokémon Go shipped with a feature that showed the relative proximity of nearby Pokémon, creating an effective mini-game of hot vs. cold. If you see a Pokémon you want is two “steps” away (somewhere around 10m each step), you choose a direction and monitor the steps. If you hit 3 steps, you’re going in the wrong direction. If the Pokémon’s distance decreases to a single “step”, you’re headed in the right direction and should keep going.

About two weeks into the release, players began to observe the “three-step bug”, where all Pokémon displayed a distance of 3 steps AT ALL TIMES, regardless of actual proximity. As a result, you could no longer skillfully or purposefully seek out nearby Pokémon. This is of minimal impact to urban players (unless a rare, desirable Pokémon is somewhere nearby) – but for rural players who may only ever see a few Pokémon an hour, this bug makes the game all but unplayable.

Along came third-party services to bridge the gap. Several enterprising fans found a way to reverse engineer the game’s communications to determine the type and duration of any Pokémon spawned near a specific set of GPS coordinates. This allowed players, both rural and urban, to track Pokémon again – either to limit their search to desirable Pokémon, or to find the few Pokémon present in less populated areas.

Unfortunately, in the July 30 release, Niantic eliminated this game mechanic entirely in one fell swoop – the steps were entirely removed from the game without explanation, creating the sense with players that the important feature would never return. At the same time, all third party locator services were shut down. Niantic claims the services were causing server overloads; which may be true as many were extremely generous, allowing refreshes as little as every 30 seconds.

But by providing no timeline, no explanation, and no alternatives, the game immediately lost its appeal to anyone outside densely populated areas where there would be a good chance of random encounters. Niantic has been slowly but surely de-gamifying their game in fear of bugs, all with little transparency to their player base. And the effect of this lack of communication brings us to the second point – the importance of communication when running a game as a service.

Gameplay as a Service

The advent of games that are provided as a service – rather than as discrete releases with optional additional updates – is still fairly recent. Many players were indoctrinated in a world where games and their rules were unchanging.  Look into how to get free robux the community there is becoming more and more popular. When you purchase a console game on disk, you are guaranteed a certain experience. And before video games reached critical mass, this same model applied to board games.

Now, imagine a world where you launch Super Mario Brothers only to find that the rules have changed – but no one explains to you what the rules are. You can’t stomp on Goombas to kill them any more, and suddenly you can’t get past the first level. How would you feel? How long would you continue to play unless someone came up to you and explained that Goombas can only be stomped if you land dead center on their heads?

In a world where a game is presented and maintained as a service – massively multiplayer games like League of Legends or World or Warcraft – it becomes inevitable that those publishers (Riot and Blizzard, respectively) will need to change the rules within their games. There are many reasons why changes to fundamental game mechanics may be necessary. For example:

1. A game mechanic is too powerful and makes the game too easy.
2. A game mechanic is too hard and players are dropping out.
3. A game mechanic is discouraging players from making in-game purchases, which often keep these games alive
4. There is just a need for change – to give players a reason to keep coming back.

Pokémon Go is too young to “need” rule #4 just yet, but there are plenty of opportunities to do so. For example: The “breeding” mechanic from Pokémon Generation 2 is likely to surface eventually – but it will require a change to the core game to give EVERY Pokémon a gender. Without appropriate communication about that change, how will players react emotionally when they find the Pokémon they named “The Dude” is in fact a lady? Game developers must take into account not just the mathematical implications of their changes, but the emotional and psychological impact on players.

But with ANY of these changes, there are a number of best practices that MMO publishers have demonstrated over the years:

1. Get customer feedback, in an open form if possible.
2. Provide clear expectations and timing of the change, allowing customers to make purchasing decisions if they agree or disagree with the changes
3. Monitor the change and communicate if things did not go as intended
4. Clearly distinguish bugs from gameplay decisions and communicate the difference in a timely fashion.

I cannot overstate how important Step 1 is. No matter how carefully planned your game is, games with long-term replay value will see emergent gameplay behaviors you can’t predict. In my years on the Sims, I saw how players creating their own stories and soap operas – independent of the gameplay necessarily – was honored as a huge new opportunity for the company. New features made it easier to customize characters, environments, and to capture and share stories from the game.

Seasons of Change

With regard to steps 2 and 3, Riot’s League of Legends is an excellent case study. In LoL, they have “seasons” of gameplay. At the end of a season, a series of major changes is implemented. This creates a really clear expectation with players – the major rules will remain unchanged for a specific period of time. It provides a clear timeline for feedback and a “pressure release valve” for those players who don’t like the change – they can hope that the next release will address their concerns. Of course, minor tuning changes may happen along the way, but never without clear communication to players.

As an example, look at the 2016 Season announcement page – major changes, clearly communicated in advance. LoL also has a dedicated “Preseason” period, where customers can test drive changes and provide feedback before they go to a wider crowd. This may be a difficult problem on mobile, where all customers generally must have the same game build, but not an unsolvable problem or an unreasonable goal.

Radio Silence: It’s not very effective

Niantic has been failing almost all of these best practices. They are notoriously bad at customer communication – in fact, their Android feedback mail is unmonitored, leading to customers complaining that Niantic may be violating the Google terms of service. The Niantic Twitter and blog are devoid of specific communications regarding game design changes pre-release. And the updates to Pokemon Go have been released almost without any communication at all; including vague and incomplete changenotes.

For example, look at Niantic’s recent blog posts. First of all, note that there was NO post prior to the major July 30 update, which landed on a Friday making customer service unresponsive all weekend. Then note the tone of the releases – defensive, unapologetic, and dry.

Another example: the removal of iOS Battery Saver mode, which impacts me as a player directly and significantly. They didn’t announce the removal in the build changenotes; then several days later they simply note they removed it because they found it to be ‘buggy’. Players knew it was buggy, but for many of us it was the only thing that made the game playable – surprising as it may seem, any game experience that depletes 1% of my battery a minute on a 4-month old phone is just unplayable without a mitigation. Niantic would have done better to simply communicate to players that bugs could be expected, allowing players to make an intelligent choice based on their lifestyle and risk tolerance.

In general, any software product that removes features without announcing the removal clearly – whether game or productivity software – is creating a worrying precedent that makes me, as a customer, no longer trust the company behind the changes.
As a user experience designer, I strongly suspect that Niantic is making changes as a group of software developers so overworked that they can’t spare the time to reach out to their customers. In many cases, this can lead to wasted time and effort chasing decisions that don’t align with the priorities of paying customers. Why spend time removing Battery Saver on iOS when you were supposedly planning on fixing it anyway in the future? Why not use those resources in another way?

Sidenote: The Culture of Overwork

A common problem in the software and game industries is the “working long hours” refrain. You see this frequently in Niantic’s blog posts, mentioning “working long hours” for players.

From the August 3 blog post: “We want you to know that we have been working crazy hours to keep the game running as we continue to launch globally. If you haven’t heard us Tweeting much it’s because we’ve been heads down working on the game.”

There are a few fallacies in citing overwork to customers as a justification for problems in your product. As a customer, I see this repeated refrain and I wonder why the long hours are needed…

• Isn’t there someone at your company whose entire job it is to reach out to the community? Even smaller game companies recognize the importance of this outreach. If your team is 100% engineering, you weren’t prepared for a global launch.
• Is your company working on the wrong features because they’re being driven by marketing priorities and not customer priorities? (a common problem at Microsoft back in the day that they’ve been addressing for years)
• Is the product poorly engineered? That probably means these bad releases will continue, and that’s not where I want to invest my money.
• Is your company culture built to treat people poorly? Tread carefully there… Amazon can tell you what problems the New York Times article brought their way.

Certainly Niantic’s goals are ambitious – possibly too much so. Yes, the world is clamoring for the game, but plenty of other products with less challenging geographic requirements have released on a much slower schedule. Amazon’s Alexa voice products still exist in the US only after over a year in general availability. Cautious, measured growth often leads to a better experience overall. Not always, but often.

Above all, when communicating with customers, remember that few customers base their purchasing decisions on how many hours a developer may be putting in. Perhaps that’s unfair, and especially so if you’re one of the folks putting in the 60 hour work weeks or more (I’ve been there, especially during my time at Electronic Arts.) But on the Sims, we didn’t get anywhere by defending our decisions with how many hours we worked. It’s about the quality of the product, and if the features you’re working those hours on aren’t exciting to customers on their own… why are you working on them?

The Web They Weave

Any one of these issues on its own would perhaps have caused less of a significant impact within the player base. But these problems are interconnected. If you’re a rural player and lost any ability to track Pokémon AT THE SAME TIME that Pokémon became noticeably harder to catch AND the catching process became less satisfying due to the lack of skill shots – there’s nothing remaining to keep you playing. And without clearly communicated goal dates for new features or fixes from Niantic, that app may well be uninstalled permanently.

If many changes are required, smaller incremental releases are a much better way of accomplishing the same goals. Unfortunately, the way App Stores are currently engineered, any change to the app code is far from iterative, as it requires a full certification and release. Service changes can go out without store approval, but should be clearly communicated to players to avoid frustration. When changes must be rolled out en masse, they should be tested carefully, and not rushed.

When corners are cut, you end up with the situation that unfolded the weekend of the last major update release. Thousands of players requested refunds for in-app purchases to express their frustrations when the game they thought they were paying into changed on a fundamental level. At the very least, app developers caught between a rock and the App Store can time their releases for Mondays, where their customer service reps will be best positioned to hear feedback and troubleshoot issues.

To Be or Not To Be The Very Best?

The net effect of the recent changes has been the de-gamification of Pokemon Go. Right now, it feels very much like a coin flip simulator and buggy step counter with some really cute graphics. Certainly, all hope is not lost. Niantic can improve their situation by taking a few new approaches:

1. Clearly communicate all major changes BEFORE updates hit the app stores
2. Replace their “working long hours” messaging with specific dates for new features and fixes, even if those dates are far in the future
3. Better community outreach and user testing, ideally from full-time resources
4. Treat any changes affecting core game mechanics like Pokeball catch rates as ship-blockers until verified
5. Consider a “season” mechanic (or a “generations” mechanic to ape terms from the original games) where gameplay changes are rolled out at specific times each year.

To all my fellow Trainers – good luck out there. And to all the folks at Niantic – good luck. Take care of yourselves, and trust that time spent engaging your community will pay dividends even if it slows engineering progress in the short term. This is a marathon, not a sprint.