80 Level Research Team presented a new report dedicated to playtesting, its importance for the game industry, types of playtesting, and how it can be used to improve marketing.
Playtest, or users/player/UX research, is an essential part of the game development process. These co-op activities provide gamers opportunities to explore the mechanics, levels, and ideas, and they help developers to make games in a shorter time with much lower costs.
To learn how developers approach playtesting, what outcomes they expect from them, what difficulties they encounter, and how they overcome these challenges the 80 Level Research team conducted a series of interviews with experts from game dev studios and agencies that specialize in playtests. Below you can find some highlights from our report. Don’t forget to download the full version here.
Playtests and marketing should start at the early stage of development. Ideally – the initial research should be completed before laying down the first line of code or a sketch. Based on the feedback, developers can understand the pain points, barriers, and motivation of users, build their games around these needs and even alter the initial concept.
Playtesting results affect marketing strategies and help to forecast the revenue for the game. It plays a significant role in determining the business model. Testers from the playtest target audience can potentially be the first customers (early adopters) of the game. The marketing department should be somewhat involved in the testing, as it will make the distribution and promotion process easier and more efficient in the future.
Growth marketing associate at Playdew Urooj Iqbal:
Playtests that are focused on the technical side of the game can be done automatically with different types of software. They’re intended to make sure that the core and server systems are responding correctly, with the main focus being on the game engine itself as well as the system integrations. User experience, gameplay mechanics, etc., where the technical and marketing sides of the game meet, are usually tested manually using, for example, surveys, interviews, and hands-on demos.
To test their builds, developers can use in-house testers (and invite colleagues from other departments) or external groups of people can be attracted. The only way to get valuable insights is to attract real players to test the game. Friends and colleagues will always be biased, and people from the gamedev industry as well as research specialists will have modified perceptions. Only players from the target audience or a wider group of potential users will help you build something new and enjoyable.
Playtests, like other research projects, can be split into two categories: qualitative (this type of research answers “how” and “why” questions) and quantitative (which answers “how many/much”). The goal of a quantitative approach in playtesting is to find numeric estimates, and the goal of qualitative playtests is to find out more about emotional and behavioral reactions, players’ motivations, and their perceptions.
In the playtesting field, an example of an instrument for studying reactions and their sources is the dual recording: i.e. recording the respondent's screen as well as their face. Before COVID, hands-on controllers could be recorded as well (how someone is holding a device, how they shake it, hit it, etc). In terms of mobile games, observations are made on where the player tries to click as well as what objects or buttons they perceive as interactive ones (when it could be a part of a background). Other components like motivation and perception can only be studied via different kinds of interview approaches.
CEO at Sense.Vision Alexander Dzyuba:
Big studios assemble research teams and develop their in-house testing services, while others use external test automation software which can be added to the project (like GameDriver, Impactqa, modl AI, etc.) or outsource playtests to an agency (like Sense.Vision, PlaytestCloud, Antidote, etc.). In-house playtests are cheaper and more secure, but only if a company has 10+ projects and 3 – 4 games to test at the same time. Agencies usually have wider expertise and are able to recruit audiences and conduct playtests faster.
The process of finding external game testers can be challenging. Companies can recruit people who have already worked with them or find enthusiasts on Facebook and Twitter. It is also effective to create Discord groups for getting useful feedback from volunteer testers. There are some special platforms that can help gather focus groups. Usertesting.com connects companies and testers that fit a given profile. Developers schedule calls and pay per player. Due to the complexity of conducting stress tests with real users, a studio can also use special tools that simulate them.
CEO at Lila Games Joseph Kim:
Playtests have to be conducted consistently or when a developer needs to make a game-changing decision. Tests with in-house participants can be done weekly or even daily, especially when there’s an in-house software or enough budget for outsourcing, while tests with real players, especially manual tests, require additional team resources or budget for outsourcing.
Operations Lead at Super Evil Megacorp Tamir Nadav:
Among the main problems in conducting playtests, developers note difficulties in selecting the target audience. Developers also fear for security when they provide access to the game to testers. The testing process and results analytics are very resource-intensive, and there is also the human factor with human limitations: blurry eyes, loss of concentration, etc. Many studios don't test games often enough, even though they would like to. External playtesting agencies and platforms can help indie developers by recruiting a target audience or creating a user base for testing, as well as improving security by providing legal support for developers and testers.
Founder of Cykyria Benedikt Engelhard:
Interviewees see the future of playtesting in the automation of technical tests, which will free up testers' time for more creative tests that automated software cannot do. The main goal of playtesting automation in GameDriver, for example, is not to replace testers, but to free up their time so testers can be sure that the core functionality works correctly and focus on other characteristics that make a game enjoyable.
Experts also consider integrating research into the gameplay and creating a special version of a game where players will be informed of an ongoing study while playing. Sense.Vision, for example, is working in the direction of dedicating playtesting analytics to algorithms or AI (artificial intelligence). It doesn’t mean that there will be no people; they will just solve tasks of a higher level and have more opportunities to make much higher quality games.
Co-founder of GameDriver Shane Evans:
Testing was introduced in the mid to late 90s and helped to alleviate some of the pressure from software companies that are trying to ship faster and faster. The development process is very complex but right in the middle of it, there is another manual process of testing driven by people who have only two hands. Either you increase the number of hands or you make those hands more efficient. That's what I think automated testing can do for the industry (more information about automatic playtests could be found here).
The automation of the repetitive tests will become an ordinary thing, the developers will make much higher quality games because their focus will move from the work of the game to its other characteristics like making a game more fun and enjoyable.