51Testing | 17 Years As A Software Testing Service Provider
Live streaming is currently a very popular way of interaction on the internet. As a brand new domain, the live streaming market is widely optimistic and achieved its prosperity due to a huge investment. The unstopped capital investment makes live streaming a phenomenal and top market trend.
With the development of live streaming, live streaming software emerges one after another. TikTok, Periscope, Livestream, StreamNow, Facebook Live, Instagram Live Stories…have hundreds of millions of users, which has also led to the demand for live streaming software testing.
The quality of a large number of live streaming software influence directly the development of this market. No one would be willing to watch a stuck, blurry live video with unsynchronized picture and sound.
For testers, it is necessary to have a comprehensive understanding of live streaming software. Only in this way can we screen out sufficiently detailed and important test points from software activities.
Just as a software test engineer needs to have a very clear understanding of the business under test. Live streaming software testing also requires test engineers to have a certain understanding of its principles, architecture, and even technical details.
Live streaming platform technical architecture
The scene-setting of the live streaming platform is very simple, however, the logic behind is very complicated. The architecture with multi-platforms and multi-channels features supports the quality of live streaming.
The principle of live streaming
The traditional live streaming is to send the video recorded by the host to the server, and then the server will distribute it to the audience. Nowadays, live streaming is more than the host uses the terminal to broadcast the program, and the server takes charge of the user’s access and video distribution. This principle is more live streaming of mobile terminal.
It is precisely because of this development that TV stations also use mobile live streaming for program production and even live broadcast.
Taking this principle, it is generally divided into three parts.
- Live streaming (the host side): the acquisition, beautifying process, encoding, and streaming;
- Server-side processing: transcoding, recording, screenshots, content review;
- Video player: audience side, streaming, decoding, and rendering.
The information flow transmission and cooperation between each other make up the whole function of live streaming.
Test points for live streaming software
The testing points of live streaming software are divided into four main areas, functional testing, performance testing, interface/usability testing, and compatibility testing.
Pay attention to the following aspects when checking the function points of the live streaming software:
1. Personal account: ordinary login, third-party associated login, forget the password, registration, search, view/modify personal information, account settings;
2. Live Stream/Likes/Followers, personal income, follow and unfollow, personal rating, deposit, and withdrawal;
3. Popular, nearby, latest, classified host list, banner, search box;
4. Create / enter / exit / close / live / switch / room, live room settings, room user list view, etc.;
5. Host live video recording, streaming, decoding, playback, beautifying, watermark, front and back switching, etc.;
6. Chat display, scrolling live commenting, loading related function(sending gifts, viewing gift list), viewing various ranking lists, recharge/third-party recharge, business statistics, third-party indicator statistics, etc.;
7. Private chat, public chat, independent chat room, user/program push, white/blacklist, etc.;
8. Super management, banned, hidden, audited, etc.
A great number of live streaming software with diverse quality from each other brings challenges to the development of the live stream market.
Functional testing
For functional testing, it is necessary to pay attention to the following aspects:
1. Whether the launch screen is opened immediately;
2. Whether the sound is distorted, delayed, audio stability under long-term connection;
3. The fluency and clarity of video playback (static/moving scenes), and whether there are decoding errors/delays;
4. Real-time synchronization of audio and picture;
5. The quality of audio and video in different network environments (IPV4/V6, 4G/5G, Wi-Fi);
6. The playback of the video after the reconnection of the network;
7. The switching of front stage and backstage; unlocking the screen; the video is reconnected when returning to the room after sharing the video or when it is interrupted by the third-party application;
8. Switch between Wi-Fi/mobile network. When the Wi-Fi status of some mobile phones is not good, and part of the traffic is transmitted through the mobile network, whether the video is smooth and free of stuck;
9. Whether the live video supports horizontal and vertical screen switching.
The interface of the live stream software is the first part that the users get in touch with. There is a lot of information and testing points in the interface. And many testing domains with different functions.
Performance testing
The popularity of live streaming has brought massive users and massive server concurrency. Live stream applications bring more challenges to the server: larger data volume, higher concurrency, more realistic user login scenarios, and lower latency. Therefore, the pressure of the server is the test focus of the live streaming APP test.
What needs to be focused on is greater data volume, higher concurrency, more realistic and visible user login scenarios, and lower latency. The most difficult part of the server to handle is high-frequency and high-bandwidth scenarios such as video traffic and user interaction, unpredictable user behaviors, and sharp increment of performance indicators.
When organizing some activities, the traffic maybe ten times or even dozens of times higher than usual. In such a scenario, the pressure of the live stream system is very huge. The stress testing under multiple scenarios must completely simulate real user behaviors so that the test results can be closer to reality.
When conducting the test, pay attention to the video quality of the host side and audience side at the same time. Generally, the occurrence of the limit cases is higher than that of the other software. It can also be said that the live software runs in the limit state for most of a lot of time. Therefore, the performance indicators should attract more attention.
When there is no clear performance index requirement, the following scenarios can be used for example tests:
1. Add 5–10 users every 2 minutes and log out 5–8 users at the same time;
2. In the first 5 minutes of the live streaming, add 5–8 users every minute, and at 10 minutes, add 50–70 users at the same time, and then log out users randomly;
3. After the number of audience in the room stabilizes, use the client application under different conditions to access to check the stability.
The software runs on the mobile terminal, so the impact on the operating platform will also be considered in the test.
For iOS, you can use the Activity Monitor of instruments to help. For Android, you can use CPU detection coming with Android Studio to perform CPU tests.
For iOS, you can use Leaks, Activity Monitor, Allocations, and Zombies of instruments to perform memory testing while Android Studio’s built-in memory detection function can be used for memory testing for Android.
For data traffic, you can use a tool to record the traffic consumption during live streaming. In addition to the statistical functions of the mobile phone itself, you can also use third-party statistical software.
Regarding the power, in the same initial environment, you can start the live streaming for 30 or 60 minutes to compare the power consumption of different software. For the phone and CPU temperature, you can use tools such as Castro to record the temperature changes within the above time frame.
Some indicators have relatively clear standards in the industry. For example, the loading time of the launch screen when entering the room is about 1 second, and the delay should be below 500–600ms (the timeout of general communication protocols is mostly concentrated in this order of magnitude).
This interval is considered to be the most serious situation that the user can tolerate, and other indicators may be more dependent on the user’s subjective perception, such as sharpness, contrast, and so on.
Interface/usability testing
This test point is mainly to ensure that the user interface will provide users with corresponding access or browsing functions through the functions of the test object. Ensure that the user interface complies with a company or industry standards, including user-friendly, and usability testing. Test the user interface layout, whether the style meets the requirements, whether the text is correct, whether the page is sightly, whether the combination of text and pictures is perfect, whether the operation is friendly, etc.
The test points that need to focus on are whether the navigation is perspicuous, whether it is easy to navigate, whether the navigation is consistent with the style of the page structure, menu, and connection page; whether the content is adaptive to the window automatically; whether the page tab style is consistent; whether the pictures on the page are practical and slightly; whether the text length is restricted; whether the text content is clear; whether there are typos.
It should be noted that the functions of the audience side and the host side of the same live streaming software are totally different. Therefore, the focus of the host-side differs from that of the audience side.
Host side
The start page: camera, flash, beautify function, cover image, title, sync button, live test, position, share, settings, title, definition, horizontal and vertical screen;
Live room page: profile photo of the host, audience number, usecp, watermark, audience list, chat, camera, flash, beautifying, share, title, voice privacy, image privacy, and setting display are normal;
Last page: the number of audiences, live duration, likes, usercp, delete button is displayed normally.
Audience side
Live room page: the profile photo of the host, follow button, number of audiences, location, usercp, watermark, audience list, source, chatbox, magnified playback, gift, share function, and interface are normal;
Last page: host profile photo, hostname, usercp, number of audiences, live stream duration, follow button, and popular live stream recommendations are displayed normally.
Compatibility test
The compatibility test is the same checkpoint as all other mobile apps, such as compatibility with different mobile phones, operating systems, and screen sizes.
It is worth mentioning that users without installing the live streaming software can use the browser to open and watch the program according to the shared link. Therefore, it is also essential to test different browsers and different kernels.
Live streaming software, as a special mobile terminal application, has its own characteristics different from other software in addition to the commonality of software. With the development of technology, live streaming is no longer a unilateral display, the communication and interaction are gradually mature. Nowadays, we can find live commerce and live court hearings, etc. These newly added scopes greatly expand the function and scope of the live streaming but also put forward new requirements for the function of the live streaming.
