Over the past few months we have seen Jagex return to asking the community for what kind of updates they would like going forward. It’s important to note with this strategy that the updates being polled are not the only updates that are going into the release pipeline. We’re still seeing larger updates such as Elite Dungeons and the Player Owned Farm being spearheaded by Jagex themselves which is great. The updates being polled have been introduced in the “Build-a-Backlog” series in which we’re given a list of 20–30 updates ranging in size from small to large and are asked to vote on them. The vote features an accompanying blog post explaining each update. This is a good idea in theory but falls short due to human nature.
The poll questions are worded as follows: Which of these _________ updates would you be eager for us to work on? (Choose all that apply.) Looking at the May poll we see updates including new Alchemical Onyx items, Shops rework: stocking shops to receive Managing Miscellanea-Style rewards, Elite slayer creature improvement: new, separate drop tables, Dragonkin quest, Raise Summoning to 120: more familiars, new way to train (Zarosian summoning, for example) and Rework Accuracy and Defence: make combat feel more satisfying.
So what’s the problem with this poll and the options that have been laid out?
The problem with this poll is two-fold:
1.) Asking the player base of a game that is updated weekly and has been updated weekly for the better part of 18 years which updates they’d be eager to be worked on is not a strong question. We know that barring major catastrophe there’s always going to be at least 2, 3, or maybe even 4 good sized RuneScape updates in a month. After 18 years (or even a decade plus for many veterans) the question of RuneScape updates is redundant. The question of a new update doesn’t capture the imagination as it should. A question dealing with this poll’s skilling updates would be: We want to revitalize old skills, which of the following updates do you think would accomplish this best? When polling updates addressing a specific issue the player will be wired to think differently about the updates. In this case, solving the problem of revamping old skilling methods. This question jumps out at the voter and forces a brief second of critical thought which will deliver a better result than any preconceived notions (i.e. GP or XP per hour) the player may have.
2.) Some of the updates as listed plant a seed in the players’ head that primes them to vote for them despite their content.
These updates are (with their reasoning for being biased afterwards):
The intentions of this poll are great, however, the wording of the questions and the prepared responses prime players for expecting something that they may not ultimately get. When a person is conditioned to expect some thing and an entirely different outcome happens, that is a recipe for disaster (no pun intended). The poll could go a long with with some reworked questions and options, here’s an example:
We want to revitalize old skills, which of the following updates do you think would accomplish this best? (randomize list)
This list isn’t exhaustive but it gets the point across. The point being that the poll needs to address a specific issue or area rather than just asking which updates players would like to see. Ordinarily when asked what updates we would like to see a player defaults to three options at the most basic level: money per hour, XP per hour, or lore. As we know each of these isn’t absolute, Cireon looked at this earlier in the month in Analysing Skilling Variety. Depending where the player falls on the skilling variety spectrum, that will influence their natural automatic choice that they make without proper questioning.
Managing expectations and knowing how to lead the community to a desired outcome is something that Jagex needs to work on. 2018 so far has been great in not setting expectations too high while failing to deliver. This was talked about last month by yours truly in 5 Steps to Harmony with Jagex and this discussion on polling fits into the Second Act. The important distinction to take away from the Build-a-Backlog polls is that while they are engaging in community based design, expectations and outcomes need to be set. By having lopsided poll questions and options, players are setting the expectations and this has the potential to cause disappointment and many unintended consequences.