Top 12 Problems With Mobile Testing

The first iPhone was released over a decade ago. Nowadays smartphones are everywhere. Our smartphones are like Swiss knives to us: they contain maps, address books, calendars, cameras, music players, and, of course, communication tools. Software testing would be incomplete without testing on mobile devices. Today I’ll talk about the challenges of mobile testing.

Now, the challenges! Below are twelve reasons why testing on mobile devices is so difficult. I decided it would be funnier to illustrate each of them by describing the bugs I encountered. Some of them were found in the course of my career, and some – on my personal device, where I was an end-user.

1. Network carriers: The performance of mobile applications may vary depending on the carrier being used. The largest carriers in the US are Verizon and AT&T, and we also have smaller carriers like Sprint and T-Mobile. In Europe, the most common are Deutche Telekom, Telefonica, Vodaphone, and Orange, in Asia – China Mobile, Airtel, NTT, and Softbank. When doing mobile testing, it is important to know which carriers your end users will be using and to test based on this information.

• Bug example: I once tested the map functions within the application and found that when using one carrier, the map was updated depending on my location, but when using another, it wasn’t. It had something to do with the way the location was cached after the cell tower was pinged.

2. Network or WiFi. Users can connect both to the carrier’s network and via WiFi. They can even choose different methods of connection in the middle of using the application, or they can be completely disconnected if they go out of range of the network. It is very important to test the application using both network and WiFi connections, and see what happens if the connection suddenly changes or is completely lost.

• Bug example: I have a WiFi repeater at home. When I change my phone connection to a repeater’s IP, Spotify thinks I’m offline. I have to force-close the app for Spotify to know I’m online.

3. Application type. Mobile apps can be web-based, native, or a hybrid of the two (developed as web, but installed as native). Some of your end users will ignore the native or hybrid app, preferring to use it through a phone browser. Many different applications can be used as a browser – for example, Safari, Chrome, Opera. It is important to make sure that your web application works correctly in different browsers.

• Bug example: Many times I came across the situation when switching to a mobile website, the necessary functions were not present. I had to switch to the full version, where the text is very small and the navigation is too complicated.

4. Operating system. Mobile apps function differently depending on the operating system. The most popular are iOS and Android, but there are others, such as Windows Mobile and Blackberry. It is important to test the operation systems used by your end users and make sure that all application functions are supported regardless of OS.

• Bug example: This is not a bug, this is a key difference between Android and iOS. Android devices have a back button, unlike iOS. Apps built for iOS require a back button embedded on every page to allow users to return to a previous state.

5. Version. Each OS is time to time updated, and the added features push users to install updates. However, not every user will update their device to the most recent version. It is important to use analytics to determine which OS your customers are most likely to use and to ensure that you are testing for all of those versions. In addition, each OS update can potentially cause bugs in your application that were not present before.

• Bug example: often, when updating the OS version, I can no longer use the speaker when making calls. I can hear the person, but they can’t hear me.

6. Manufacturer. All iOS devices are made by Apple, but Android devices are not so simple. Samsung is one of the largest manufacturers of Android devices, but there are many more, for example, Huawei, Motorola, Asus, LG. It is important to note that not every Android user will necessarily have Samsung, and test on other devices too.

• Bug example: I once tested a tablet app where the keyboard function worked fine on some manufacturers’ devices, but not on others. It just didn’t show up and I couldn’t enter anything into the form fields.

7. Model. In a similar way to the versions, new device models appear annually. Some users will update their devices every couple of years, others will not. Moreover, some devices will not be able to update to the latest OS version and will become obsolete. Again, it’s important to figure out what devices your audience is using to make decisions about which models to test on and which models to support.

• Bug example: This is not a bug, but it’s important to keep in mind that when Apple released a new iPad model that allows signing in forms, the software I tested took this feature into account. However, older iPad versions did not support it, so the application had to take into account this situation and not ask users of older versions to sign the document.

8. Tablet or smartphone. Many of your users will interact with the app through a tablet rather than a smartphone. Native apps often come in different versions depending on whether they are for a tablet or phone. An application designed for a smartphone can often be downloaded to a tablet, but not vice versa. If you are using a web application, it is important to remember that tablets and smartphones have different functionality. Test your application on both.

• Bug example: I tested apps that worked fine on a smartphone, but gave a blank screen when I tried to test them on a tablet.

9. Screen size. Mobile devices come in many different sizes. IOS devices have little variation in size, but Android devices are much worse. Of course, it is impossible to test every single size, but it is important to test small, medium, large, and very large screens to ensure that the application renders correctly regardless of the resolution.

• Bug example: I was testing applications on a small screen, and page elements began to overlap, making it difficult to see text fields and click on buttons.

Related: Learn about more types of testing here.

10. Portrait and landscape mode. When testing on smartphones, it’s easy to forget to check landscape mode, because we tend to hold the phone in portrait mode. However, sometimes smartphone users will want to view the app in landscape format and this is especially true for tablet users. It is important not only to check both modes but also to switch between them while working with the application.

• Bug example: I once tested an application that looked great on a tablet in portrait mode, but after switching to landscape, all the fields disappeared.

11. In-App integration. One of the benefits of mobile apps is that they can integrate with other device features, such as a microphone or camera. They can also link to other applications such as Facebook or Twitter. Regardless of the integrations supported, make sure you test them thoroughly.

• Bug example: I tested an application in which the user could take a photo of the device at home and add it to the home warehouse. When I tried to take a photo I was redirected to the camera app, and I could take a photo, but after that, I didn’t go back to the app.

12. Outside of App Integration. Even if your application is not designed to interact with other applications or features, there can still be bugs. What happens if a user receives a phone call, message, or low battery warning while using the app? You need to find out.

• Bug example: the following bug existed on my phone for a long time – if the timer worked when I was talking on the phone, then after the end of the call the timer sounded non-stop.