How to effectively test mobile games?
51Testing | 17 Years As A Software Testing Service Provider
According to the report of Statista, the global gaming market was valued at USD 162.32 billion in 2020 and is expected to reach a value of USD 295.63 billion by 2026, registering a CAGR of 10.5% over the forecast period (2021–2026). The great potential of the gaming market makes game testing extremely important. Therefore, today we will discuss this topic: how to effectively test mobile games?
- There are differences between mobile game automated testing and App automated testing.
The different development technology of mobile games and apps leads to complete different automated testing technologies. Generally speaking, Android applications are developed using the Android SDK and written in Java.
For APP testing, testers generally complete the test by obtaining information on the current window, searching and operating buttons and other controls. This process can be standardized which makes it easier than mobile game testing.
For mobile games testing, the main difficulty comes from the development requirement of engines (such as cocos2d and unity3d). When the game is running, the engine will render the widgets into a simple picture which hinder testers to find out the buttons in the game through the services coming with Android. Therefore, the testers can not perform regular testing.
2. The biggest difference lies in functional testing.
The mobile game functional testing is very complicated compared with App functional testing. The game is mostly dynamic and random. Whether you play a level repeatedly, hang up, or do nothing, the scene changes all the time. So automated testing for mobile games requires more flexible scripts and use cases, appropriate exploration and adaptive functions.
While most applications are static and repeatable so automated testing can be written and executed completely in accordance with the script.
What’s more, there are levels, PVP, etc. on the game which make the function points to be tested more complicated. Not only do testers pay attention to the subtle difference of level time, but also check all game levels. The repeat and detailed test work are very time-consuming.
In PVP, the mobile game has a very high requirement of network latency. Testers need to simulate a large number of different scenarios to observe whether the status and data of two players are consistent and the players’ experience influenced by the network latency.
3. The focus of mobile game testing and App testing is different.
For most non-payment apps, security is not a particularly big issue. What the company needs to ensure is generally the security of login authentication. The app is designed user-friendly. No one will lock the memory or turn addition operations into multiplication operations when using their calculator.
However, for mobile game testing, almost all products require more attention to business testing, such as payment, security, and profit testing. These tests generally require compatibility testing on a large number of different systems and models of mobile phones to reduce the possibility of errors. At the same time, for the security of the mobile games, testers generally lock and modify game resources and memory to simplify the flow of the game.
User interface
- UI basis
The UI element is displayed completely, and the corresponding elements are used in the correct state. The element level is displayed correctly. No penetration when clicked on the interface.
- UI panel
The UI panel is generally opened by other entrances, and it can be closed or returned to the last step if necessary.
- Interface stack
The interface stack is mainly used to manage interface opening, jumping, and interface resource information storage.
For example, an interface may be opened from interface A→interface B→interface C, and the information of ABC is stored in the stack. If you return from C, it will return to interface B, and interface C in the stack is popped out of the stack.
After the disconnection and reconnection, check the close or return operation of the current interface to see if the stack information will be cleared due to the disconnection and whether there are jump errors of the return of interface.
- Multi-screen adaptation
Screen adaptation is actually part of compatibility. Resolution and screen shape are two important points to consider.
The ratio of mobile devices on the market is from 4:3 to 13:6. Sometimes the text is missing and the mask is not covered completely because the display ratio is not adapted.
And because of the pursuit of a comprehensive screen, the screen has been getting bigger in recent years, the shape is no longer the traditional rectangle, and for the special-shaped screen, the problem that the buttons are blocked and cannot be used exist.
Performance
The key indicator of mobile game performance is CPU usage, memory usage, traffic consumption, FPS (frames per second), and power consumption.
How to do performance testing?
1. Clarify test goals and understand performance test requirements;
2. Compile a performance test plan;
3. Analyze performance test requirements;
4. Compile performance test plans and design test scenarios;
5. Prepare relevant resource (human resources, hardware resources, software resources);
6. Test program development; script maintenance, test data preparation, test monitoring preparation;
7. Execute performance tests and collect test results;
8. Analyze results;
9. System tuning and retesting;
When testers execute performance testing, try to avoid the following misunderstandings.
- The main contradiction of mobile game testing is not the performance but the compatibility.
Developers overestimated the importance of compatibility testing and underestimated performance testing. In fact, many problems are caused by performance. For example,
“A phone call came suddenly when I was playing the game. And the game crashed when I came back.”
or
“After I finally typed a large paragraph on my phone, and a text message came when I was about to send it. The note-taking software had crashed after I read the message and the hard-worked content was gone for a long time.”
This kind of problem can easily arouse the disgust of players and cause loss.
- Performance testing should cover all functions
The main purpose of performance testing is for system tuning. It is impossible to execute performance tests on all functions. In the test design, it is necessary to first analyze the possible bottlenecks based on the actual situation of the system. At this time, the analysis can be based on the 80/20 rule:
Firstly analyze the utilization of system resources, massive data transmission, data conversion, user usage frequency, logic complexity, etc. And then select the functions and scenarios to be tested, and finally performance test plan accordingly.
- Performance testing is mainly about the use of test tools.
Some testers think that they can not do performance testing because they are not familiar with testing tools, and some think that the tools can automatically generate the reports that they need or they can locate the system bottleneck with performance testing tools. The tools only play an auxiliary role in testing. While the test plan, test scenario analysis, and the problem location is the key to performance testing. Don’t expect that test tools can generate everything you want, tools can only provide us as much analysis as possible.
- Performance testing is not important because we can improve performance by improving hardware configuration.
With the expansion of software scale, improving hardware configuration is only a basic method to solve performance problems. If the software itself has a performance problem, no amount of resources may be enough. For example, memory leaks. As time passed by, the memory will eventually be exhausted and cause the system to crash.
- Performance testing can be executed in any environment.
Performance analysis can be executed in a production-like environment and the configuration can be different. The general reasons that performance testing is carried out in a production-like environment are as follows:
- The performance testing should be completed as soon as possible to find out the problem and fix it.
- Testing in a production environment could cause dirty data.
Usability and user experience
- Discover major problems in the early stages of the game.
About 60% of the serious problems can be found in the early stage of the game through the usability test. The optimization of these problems can improve the efficiency of the test after the launch. However, due to the long process of the game, it is impossible for the testers to complete all the test items at once, so usability testing is generally used to find problems in the early stage of the game.
- Find the reason for the abnormal data in the novice stage.
In the novice stage of the online game, there are sometimes data abnormalities, however, the problem can not be found by self-testing. At this time, the tester can find out the problem by usability testing.
- Discover the potential optimization of the game.
By observing the behavior of the tester and collecting their feedback to discover their potential needs and provide a reference for game optimization.
User experience (UX) is a special design discipline that focuses on the players’ psychology, behavior, thinking, and game skills to ensure the game experience designed can be truly reflected in players’ minds. Everyone on the development team is a designer. Every seemingly small choice will affect the player’s gaming experience, no matter who makes the choice.
The programmer chooses the grid UI layout instead of the list layout; the legal department requires players to read the End User License Agreement (EULA) before registering the game; the dubbing staff decides which words need to be emphasized, and so on. In the process of game development, there are too many things to decide. Every small decision may have a small or huge impact on the player’s overall experience, but we can find ways to avoid this from happening.
Before making the final decision, show these designs to your players, observe their behaviors and understand their real thoughts. With this support, the development team can collectively change their ideas, iterate their designs, justify each change, and ensure that every final decision they make is correct.
Multi-players features
Nowadays, the multi-player/user feature is very common in mobile games. A need to test multi-player abilities is there and is naturally more challenging and requires the other end to perform as a real counterpart. For example, a typical case is that the game communicates with the backend server. In these cases, connectivity is essential and the synchronization between backend and devices that need to get information about the gameplay. There are tons of different scenarios you should be testing and many of those will severely impact the game’s experience, resulting in negative feedback and gets uninstalled by users.
Social integration
One of the great things about social media integration is it can fit any of several definitions. For mobile game developers, social media integration can mean using social media:
· As a marketing tool
· To create, build, and foster a community around your game
· As a central hub for all activity concerning your game
· To add social features to your game
· To take advantage of the platform’s analytic and growth tools
Social media integration can offer your mobile game a plethora of built-in benefits you can leverage in a number of ways. These days, the largest social media organizations want you to integrate their platforms into not just your game, but the entire ecosystem surrounding your game.
To this end, many social media companies have tools at the ready to help you market, manage, collect data, and make things easier for your users. Even without implementing social media integration directly into your game, you will still likely use one or more social media platforms for your game and your game’s community.
Security
At present, more and more games are facing various security risks, such as static modification of files, dynamic modification of game logic, tampering protocols, game ID theft, malicious comments, etc.
