51Testing | Software Testing Service Provider |Wednesday, 9th, Dec. 2020
Last time we have shared the following topics:
- What is the Internet of things?
- What technologies make IoT possible？
- What are the use cases of the Internet of Things?
And for this time, we will talk about the rest three parts:
- IoT, what are we testing?
- What skill does an IoT tester need to know?
- What are the IoT testing tools you should know?
A typical Internet of Things consists of the following parts:
- Device layer
Here is the communication protocol of devices. RF fundamentals may generate diagnostic messages, and sensors generate application data such as temperature and humidity. Device connection and data transmission between devices are completed by the communication protocol. Top IoT communication protocols are Bluetooth, ZigBee, WIFI, MQTT, OPC-UA, Cellular, NFC, LoraWan, Sigfox, etc.
- Data collection layer
This layer usually collects the data reported by the device by the gateway and then converts it into the data format required by the application layer, such as a message queue.
- Data processing layer
If the network size is big and the collected data volume is large, processing such as storage filtering may be required. Storing data in the cloud is a popular method.
If the amount of reported data is small and the application layer can process these real-time data, the data processing layer is not required.
- Data presentation layer (application layer)
The application layer ultimately is presented to the user. The application layer can be a simple dashboard data display to provide users with the required information. It is also possible to integrate some manual trigger behaviors or brake trigger behaviors.
From the above analysis, it can be seen that the Internet of Things is a system that mixes multiple hardware, multiple protocols, and multiple applications.
So how should IoT testing be conducted?
According to the above hierarchy, the following tests are mostly carried out in the R&D department:
- Device layer
Hardware testing: such as the antenna of RF equipment, power consumption, sensor accuracy, etc. However, most IoT company do not produce their own hardware, only purchase or cooperate with hardware/protocol vendors.
Protocol testing: There are many scenarios for the Internet of Things with different requirements. Low power consumption is the focus of multiple transmission protocols in the Internet of Things, but at the same time, many scenarios have high requirements for low latency, so whether power consumption and performance can meet the standards is one of the tests; in addition, multi-node support, routing under different network structures is also an important point. If you use common IoT protocols, you don’t need to test the protocol itself too much.
- Data collection layer
Gateway: When multiple nodes report frequently, the reliability of the gateway is an important point: whether the gateway can handle a large amount of data (including the connection between the gateway and the device and the gateway itself), whether it can avoid the influence of abnormal data, and whether the hardware where the gateway is located can support the uninterrupted operation of the gateway for a long time. Whether the data timestamp can be recorded correctly to ensure the correctness of the business data obtained later.
- Data processing layer
Whether the data filtering process (if any) is correct, for example, storage and access rate.
- Data presentation layer(Application layer)
Interface testing: Like any application, interface testing is necessary for the presentation layer. Manual and automatic testing can be used for interface testing, but manual testing is still an essential part.
Functional test: For the required query and trigger, the desired result can be obtained. Although it is the application layer, testers need to have a very clear understanding of the entire business logic and the origin of data.
In addition to the above conventional test items, what else will customers pay attention to?
- Security
How to ensure the security of data at all layers and in the transmission is a pain point in the development of the Internet of Things: whether the protocol has encryption and authentication, whether the data has access rights restrictions…
No customer wants the following scenario to happen: the smart meter at home is hacked, and the body temperature and heartbeat data are placed in a place that everyone can access…
- Compatibility
Compatibility here includes a wide range, which may be the compatibility of multiple hardware and multiple protocols during deployment; it may also be the compatibility of multiple platforms of applications; or the compatibility between different versions of each layer.
Compatibility strategies usually vary according to customers or company decision-makers.
- Deployment and upgrade
Whether the deployment is simple, whether the device transmission will be affected by other network environments such as WiFi and Bluetooth during deployment, and how much interference will the building reflect on the radio frequency to the network.
The hardware should reserve enough memory as much as possible because embedded systems have limited memory, considering future upgrade requirements to prevent frequent hardware replacement on a large scale. For software, we need to consider how to upgrade on a large scale and compatibility before and after versions.
- Protocol
In cellular communication and the Internet, data collection and storage, and visualization have been sufficiently applied. Under the premise that the upper-layer technology has matured, the quality of an IoT solution is largely due to the matching of the protocol and the requirements.
In complex networking, low power consumption, low latency, and self-organizing networking are always the core of the underlying protocol of the Internet of Things.
What skills do IoT testers need to master?
- Gray Box Testing
In the Internet of Things, various sensors, monitoring equipment, and smart terminals can access the network through multiple protocols. The use way and use environment cannot be determined, so in order to ensure the stability and reliability of the devices, IoT testers need to conduct gray box testing on products.
That is to say, the test team not only needs to verify the function of the application by white box testing but also needs to go deep into its internal structure with black-box testing. Therefore, testers need to collect as much as possible related information of device APIs, third parties, and connection protocol, carefully design test cases to cover as many use case scenarios as possible.
- Embedded systems
Embedded development has always been a relatively low-key technical field. Compared with Web development and enterprise-level development, embedded development seems to seldom make a sound on the Internet. However, with the development of wearable devices such as smart bracelets, smartwatches, and smart glasses, the concept of “embedded” has quickly become one of the hot spots in today’s scientific and technological community.
But it is undeniable that although the embedded concept has undergone considerable development so far, there is still no well-known and well-maintained operating system and supporting tools like iOS and Android. Although Ubuntu and Debian have developed installation packages for ARM-based MMU (Memory Management Unit) devices, developers still need to master basic skills such as cross-compilation to get started with embedded development. If IoT testers do not have certain embedded knowledge, they still cannot do a comprehensive test job.
- Effective and executive emulation technique
The Internet of Things realizes the interconnection of all things with the features of diverse devices, unprecedented scale, different protocols, and complicated environment. This means that testers cannot target every device, every protocol, and every scenario to construct a comprehensive test case, even if it can be realized theoretically, it will cost a great deal. For this reason, it would be better if testers master the emulation technique. Testers often need to create a variety of virtual devices and virtual networks on a single physical device to simulate various use case scenarios to achieve the purpose of testing. Emulation not only saves time and money but also has scalability that the physical environment does not have.
- Ability to deal with traffic peak and sudden traffic peak
There is a phenomenon in the field of virtual desktop infrastructure(VDI) called “Boot Storm”: everyone tries to log in to their VDI at the same time (usually in the morning). As a result, the back-end system responds very slowly and the user experience is very poor. So what is the situation for the Internet of Things?
In many cases, we will face the same problem. For example, in the field of smart homes, various connected devices at home are basically turned off or on standby after the user goes to work. At this time, the data flow is very small and there is almost no pressure on the back-end system. However, when the user goes home, connect to the home WiFi network. Various devices, such as TVs, computers, Pads, and mobile phones, will operate. Just as peaks in power consumption are usually generated at this time, peaks in data will also be generated at this time, which will cause great pressure on the back-end system. The same situation also exists in the field of intelligent transportation, such as peak traffic jams caused by commuting to get off work, and highway paralysis caused by holidays. This is a new challenge for most IoT test teams.
- Data security
The potential data security issues of the Internet of Things are well known, but as Nicholas Evans mentioned in “Computer World”: For Internet of Things suppliers, security is usually a post-consideration. It may be possible to do this before the Internet of Things is popularized and penetrated into people’s daily lives, but as the public recognizes and uses the Internet of Things more and more, this has become an urgent problem. If in the early stage of the development of the Internet of Things, suppliers need to attract users through innovation, then in the later stage they must retain users through stability and security. But the reality is that it is difficult for people to implement an effective and comprehensive protection strategy in the face of such a large scale and a rich variety of equipment, applications, and services. For the IoT testing team, this is undoubtedly a great challenge.
IoT testing tools you should know
- Wireshark
Wireshark is open-source and free network traffic analyzing software. It is used to monitor all the traffic visible on that interface including unicast traffic. It is similar to Tcpdump but has a GUI(Graphical User Interface), also it has some sorting and filtering options. It’s another version without GUI is known as TShark. It runs on Linux, macOS, BSD, Solaris, some other Unix-like operating systems, and Microsoft Windows.
- Shodan
Shodan is an IoT testing tool that you can use to discover which of your devices are connected to the Internet. It allows you to keep track of all the computers which are directly accessible from the Internet.
- Tcpdump
Tcpdump is a data-network packet analyzer computer program that runs under a command-line interface. It allows the user to display TCP/IP and other packets being transmitted or received over a network to which the computer is attached.
Thingful is a search engine for the Internet of Things. It allows secure interoperability between millions of objects via the Internet. This IOT testing tool also to control how data is used and empowers to take more decisive and valuable decisions.
It enables four types of test automation on a single web platform — Mobile Functional & Performance testing and Web-based Functional & Performance testing. It can simulate millions of geographically dispersed concurrent users visiting a website to test the application under huge loads. With real-time analytics & seamless integration between test design, monitoring, and reporting, CloudTest enables convenient and efficient testing.
——
You may also be interested in these topics:
IoT Testing | A complete guide | What it is and how it works?
How to Become A Game Tester? |Tutorial &Beginner’s Guide of Game Testing
Exp. Share | Performance Testing tutorial, step by step, a complete guide
