Blog
Articles to grow your career
Article
Nowadays testers are trying to use test automation to speed up the process of testing, increase test coverage and make the work easier in general.
I think everyone has received messages from users about bugs that were not detected by automated tests. Users are also inattentive, and we ourselves do not always perfectly understand the nuances of the business for which a product is being developed, or we cannot cover all the scenarios. For example, there was such a case in one of the test environments. The test followed the link and checked that the new page opened. However, opening a page that says “You do not have permission to view this page” also counted as a successful scenario, although it was not. As a result, there is a problem, but the test was successful. In any case, the process can be improved, and test automation is a powerful tool for this.
It is best to start test automation based on units and integration tests from developers.
Next, depending on the frequency of the type of testing, its need, and risks. Therefore, the next to automate is the smoke tests, then go functional or regression tests. After that, we can implement test automation at the Continuous Delivery level. But it is important to choose the correct time.
If you already have test cases, this is good, and we build the analysis on their basis. If not — well, it’s time to create them.
Let’s pay attention to the following points.
Is it possible to automate certain scenarios and is it advisable? For example, a record in the database will appear in half an hour or an hour after adding, is there any point for the automated test to wait for this? You can wait, but will it speed up the testing process as a whole? Usually, this is almost the main purpose of automation. It turns out that it is necessary to replace manual testing in such a process only if we want to completely save our Manual QA from the need to test this.
If the script is simple, and the check is one-time, is it necessary to spend time automating it? In theory, yes, especially if the client requires “to automate absolutely everything!” But remember that this requires additional costs. Do you have them? Maybe you should take a closer look at something more important?
Finally, if every time the data for the test needs to be carefully selected by hand, is there any point in automation? For complex financial systems, it is not always possible to create a universal request to obtain information. Is it advisable in your case to manually generate data each time you run automated testing?
There are also tests where a human is faster and more likely to notice an error. Do you need to use scripts to build tests for the required expectations?
Will the resources spent on automation pay off? At first, the automation of a specific test looks simple, but when u start to work, you see that in the current implementation or under certain conditions it will require significant resources. Let’s think about whether we have time and if the client is willing to pay for it?
Do I need to automate simple tests? Why not! Maybe you should shovel the data into JSON of 500 lines. Once upon a time, I had to compare JSON with the data scattered throughout the Excel file the day before the release. I won’t say that the test was too difficult, but with maximum concentration, it took me seven hours. Of course, after that incident, we automated the process!
What about complex ones that require mathematical calculations? Definitely! This will provide the necessary accuracy in calculations and eliminate the human factor.
What do we mean by that? Let’s start with the design.
In many test management systems, you can add an attribute to a test that allows you to identify whether the test needs to be automated (a reason is also given) or if it is already automated. This comes in handy as it makes it easier to filter and determine the test coverage.
In general, writing clear and detailed test cases, as well as document management is a real art. A good practice is to use a test case review that can be performed by one of the colleagues in the testing team, as well as its lead or business analyst. An outside perspective is always useful, with its help you can not only make sure that you have not missed anything but also look at the project from the point of view of BA. This approach confirms that you have covered all user scenarios and met all requirements.
When checking test cases further, I recommend paying attention to the following points:
Leave an application and get a free consultation from our manager.
Often, automated tests generate data for verification and delete it after execution.
This has its pros and cons: on the one hand, we try not to interfere with anything as we created what we needed ourselves, checked the functionality on the data, and removed it afterward. In addition, this is how we guarantee the required coverage. But it is important to conduct testing on the data with which the user is working. If for any reason we cannot create such data, we use what is already there, but after the tests, we do not delete the data.
Why is that? Real data has some peculiarities: for example, it can be imported into the system, formed differently, have more complex logic. It is something that will be difficult to repeat for the purity of the script.
Another caveat is that the data changes frequently. If this is your case, let’s look at a different approach.
At some point in the cases, we add criteria to the data that needs to be used. They can be simple: for example, you need to take a user who registered in the system more than a year ago. One of the solutions is a request to the database with the same criteria, it is even more convenient for automated tests. This approach is suitable if the test needs to be performed in different environments where the data is different. But there is also a drawback: if the data is rather complex, that is, there are many criteria (a user registered a year ago who has not made purchases of goods of a certain category in the last 3 months), then it will be difficult or even impossible to get them in this way. It will probably be easier to select data manually and substitute hardcode.
In order not to interfere with each other during testing, use different environments or separate data for automated testing and manual testing. Then, when checking a specific scenario, you will not face the problem of accidentally changing data.
The quality of tests should not be neglected In the process of automated tests optimization. They are made faster as it is their obvious advantage over manual ones. However, be sure to provide some coverage as well.
Attempt to speed up the test can have a bad effect on quality. For example, sending requests not through the user interface, but directly through the API — of course, it will go faster. But with this approach, the UI is not checked, and this is fraught with consequences. Users will not use the API, but they will notice problems with the interface right away — they will become a blocker for user tasks. But if you have such an idea, it might be the right time to split and combine tests. Then calmly think over the coverage and optimize the process for your convenience.
Let’s summarize what to keep in mind when running test automation on a project:
And the main thing to remember: the purpose of any tests is to find problems and release a quality product. Let automated tests be ‘green’ and users happy.