Automatic translation of the archived Russian text. Recordings and text within images remain in their original language. Read the Russian original
About this material
Practical advice on collecting and handling feedback: separate the problem from a suggested fix, account for the target audience and prioritise. Feedback is useful only when it is expressed, received and turned into action effectively.
Surprisingly, I cannot recall any articles or talks about working with feedback. Yet it is an integral part of game development. So I will share what I have figured out over my years in the industry. First, a little terminology. As I understand it, feedback is when users describe their experience of interacting with a product. It covers a wide range of situations, from focus tests to forum comments. Its purpose is to confirm hypotheses and identify problems. In a sense, it resembles bug reports, except that it is written by people who are not professional QA testers and usually concerns the overall experience (“I do not know what to do,” “boring”) rather than specific errors (“open and close the door 20 times and it stops opening”). The usefulness of feedback depends on how it is given, received and processed. It is therefore useless if delivered terribly, processed by no one and followed by no action. I will admit upfront that I am not much of an adviser, but this is better than nothing.
Tip No. 1 Players do not ask for what they want, and what they want is not what they actually need. Remember this mantra whenever you see users suggesting balance changes. Such suggestions certainly signal a problem, but what the problem is and how to solve it still needs to be established.
Tip No. 2 Feedback should not contain proposals about what to do. That would be feedback on the game the user has just imagined, rather than on your game. And that game may not be better than yours. Besides, who is the professional here?
Tip No. 3 Weight feedback by how closely its author matches your target audience. Otherwise you risk spending time and effort pleasing people who will not play your game anyway. They will notice things your target audience will not, while missing things that really matter to it.
Tip No. 4 Criticism is feedback too. Do not let emotions cloud your judgment. Everyone gives feedback as best they can. Some people cannot help spewing acid, but that does not mean their feedback should be ignored, much less disputed. Arguing with feedback is foolish because it is pointless. If you disagree, simply do not act on it; there is no need to prove anything to anyone. Accept feedback as inevitable, skipping denial, anger, bargaining and depression.
Tip No. 5 Do not interrupt play to give feedback. You are not making a game for people who spend a couple of minutes doing something else after every minute of gameplay. Yes, spotting a problem makes it tempting to examine it under a microscope, discuss it and look for a solution immediately. That is a good way to miss the forest for the trees. When playing, I keep a notebook nearby for short notes of 1–2 words, and constantly take screenshots. Only after finishing do I sort through the screenshots and notes and turn them into detailed feedback.
Tip No. 6 Players usually feel the need to give feedback when reality fails to meet their expectations. That means you need to find out what those expectations were. Ideally, reduce feedback to three fields: “what I saw,” “what I expected to see,” and “other comments.” Even when reporting an obvious bug, follow the formal procedure and fill in seemingly obvious information. What is obvious to one person can also be obvious to another—but in a different way.
Tip No. 7 Structuring feedback helps both the giver and the recipient. The fields from the previous tip help players formulate their thoughts better than dumping everything into one text box. If you are gathering feedback from 5+ people, definitely use a questionnaire where respondents choose prepared answers rather than invent their own. It is easier to see that 28% of players think “the druid is OP” than to read a couple of hundred essays on “the druid in PvP.” Naturally, this method is primarily suited to testing hypotheses, since developers must understand what to ask and what the possible answers are. A small note: people are much more willing to fill out questionnaires written in a light, friendly tone. But do not include joke answers, or you will have jokers to deal with. Nor should every question be mandatory. Nothing good comes of forcing players to answer something they do not want to answer for whatever reason.
Tip No. 8 Expand feedback into specifics. Knowing that the game is boring does not help much in fixing it. “At higher levels the old activities become tiresome and no new ones appear” is much more useful. You may need several rounds of feedback gathering to clarify the problem.
Tip No. 9 It is all pointless if feedback does not result in concrete tasks in your task-management system. Of course, work plans may already be packed with extremely high-priority tasks, and changing them would mean thrashing around and creating chaos. In that case, feedback is, as they say, taken under advisement. But if you genuinely want to listen to users and make the project serve them, you need concrete actions scheduled in your plans.
Tip No. 10 Set priorities before turning feedback points into tasks. You may suddenly discover that most feedback belongs in the “nice to do someday” category, while some inconspicuous point deserves “drop everything and switch to this immediately.”
There are many ways to establish priorities. I will suggest the two simplest. 1) Define a priority scale with a clear description for each level. This is commonly used in bug tracking. For example: Blocker — Further development is impossible while this problem exists in the game. Drop everything and work on it. Critical — The game cannot ship with this problem. Address it as soon as the current task is finished. Major — A very unpleasant problem you can live with, but should fix as soon as possible. Normal — Ordinary roughness that does not ruin players' lives but leaves an unfinished impression. Fix it someday when there are no urgent tasks. Minor — Something so insignificant that it may not be worth doing, since the result will not justify the effort. 2) Bubble sort. Compare each item with another and decide which of the two is more important. The result is a column with the most important task at the top and the least important at the bottom. This works when deciding what to start first, but is less suitable for integrating with existing tasks; the unified scale from the previous point is better for that. Tip No. 11 Collect feedback in one easily accessible place. Feedback may come from many sources and at different times, so you need somewhere to accumulate and later process it. Ideally, add links to tasks there as well, so that the same page can track their progress.
On the one hand, I may sound like Captain Obvious. On the other, all these tips came from real cases where the opposite was done and the results disappointed.