5 Bugs Missed by Manual QA

Many novice testers desire to learn more about interviews and what questions are often asked there. Questions about knowledge of theory or those that require a specific answer usually do not cause problems, if, of course, you are prepared. But some cannot be answered unequivocally. They are asked to the candidate to understand how they think, how they build logical links, and deal with something that cannot be simply read in a book.

One of these questions is a request to tell about the bugs that the specialist missed in their practice. In this way, the candidate can analyze the sequence of actions that led to the occurrence of the error and describe what steps should have been taken to avoid such a situation in the future. Missed bugs, if they were serious, are like a rake, stepping on which you get a painful but memorable experience.

There is even such a system for assessing experience, in which failures in a person’s career are considered, and not just their success. Success is largely due to luck; a complex and dangerous bug can be reproduced by accident. But missed bugs are already an experience that is difficult to forget, and any sane person will take all measures to avoid such mistakes in the future.

But where can a young specialist get such experience? To do this, I made up my top 5 biggest bugs that were missed in my experience. This is not the same as facing them and their consequences in person, but it is still better than a complete lack of experience and, as a result, inability to answer a common question in a job interview. For my list, I’ve selected bugs from two testing areas: game development and web application development. So let’s get started.

1. The Day the Game Stopped

A bit of background. The bug occurred on a large game project developed by a distributed team from several countries. At some point, the development department, for its internal reasons, decided to change the ID system on the project. On the new build, this did not affect anything, so they did not report the changes to either the testing department or other related departments. There was no changelog or other documentation either. Even though the project was already three years old, only the most basic things were described.

As a result, when a save from the previous build was connected to a new build, the ID that was responsible for marking the completed tutorial was invalid, and the game believed that the player had not yet completed the tutorial. The sequence of the tutorial ran. But the player by this moment could already be at distant levels, where it is impossible to fulfill the training requirement. And it was impossible to miss it, it blocked all controls.

The new release was tied to one of the holiday events, so it was tied to being featured in the app store. It was impossible to postpone it, and it went live. As a result, for all players who made progress (and the game, as I already mentioned, was three years old), the game stopped. Completely. The situation was a little bit saved by the fact that the build became available to players gradually, primarily in non-paying countries. A sleepless night awaited us and a hotfix with which we made the tutorial optional. Fortunately, the build itself allowed such changes to be sent without re-uploading it to the store.

Sadly, the bug was discovered even before the release. The bug was reopened 8 times, but each time it went to the same developer, who swept it aside with a note that it was a broken save. Yes, the developers had the right to fix bugs on their own, without a tester’s verdict. Here, of course, from the QA side, it was necessary to insist and fight for this mistake. Prove that it is present and needs to be corrected. But the team, when there is no opportunity even to come up and talk, did not allow this serious defect to be prevented.

How could this situation be avoided? Well-established communication between teams, documenting changes, creating required update tests, and a change in the life cycle of a bug, when it cannot be closed and forgotten without taking into account the opinion of the testing team.

2. Fix, Which Users Took For a Bug

It happened on a game project that was awarded the title of “Game of the Year” on Facebook. The game’s monetization system was based on the fact that if a player does not complete a level, they lose their life. There are 5 lives in total, and when they run out, you can no longer play, but you can buy them for money or wait for them to recover according to the timer. You could also ask your friends for a life, those who also played on Facebook. This functionality was needed to increase the involvement of players who entered the game to help each other. This is where the bug came in.

A player could send a request to replenish lives as many times as they wanted, and another player could do the same infinite number of times. Thus, having only two linked Facebook accounts, one could throw lives to oneself for an infinitely long time, without spending anything on their replenishment. Not surprisingly, players loved this functionality a lot.

When this bug was discovered, the producer of the project turned a little pale, because, according to the most conservative estimates, it deprived the company of many potential purchases. The reason for missing it was rather simple: the tester created a second account, spent their life on one, sent a request for the second, returned life, made sure that it came, and considered the test passed. Even if they sent themselves several lives in this way, they did not have any suspicions, since there was no documentation that would describe this functionality in the game.

What could have prevented this situation? The requirements specification describing the functionality, a clearly and unambiguously compiled test, and the tester’s ability to look at a project a little wider than just a set of functions, and think about how it makes money. The latter is true in general for all participants in the development, be it a game or other software. Often, behind the picket fence of current tasks, we lose the understanding of why we are working at all, but this is a topic for another discussion.

When the bug was fixed, players flooded the support team with complaints that their favorite free life replenishment functionality stopped working, but it was too late – the hole in monetization was closed.

3. An Item That Destroyed the Game

This bug was not as expensive as the first two, but much more spectacular. When it happened, the game would crash and then would not start. How did it all begin? At first, it was a genre-mixing game project. The main mechanic was time management: you had to keep up with serving customers in the diner. By earning points, the player built new cafes and thus developed the city. The second mechanic was a simple city builder, in which the user was building houses and paving roads so that the diners had customers.

At some point, they decided to abandon the second mechanic and created one large map on which all the houses and diners were already built. They only needed to be opened as the game progressed. From the city builder, there were only decorations left: benches or flowers that could be placed around their cafes. When the second diner was opened, a small item in the form of a bench appeared on the map. It could be moved, rotated, or put into inventory. In the latter case, the game crashed. And not only the first time, but all subsequent times when the save was loaded.

Here is what caused it. The bench near the diner has remained since the days when the game had elements of a city builder, but now it has become a decorative item that was supposed to appear as a reward in the middle of the game. And at the moment when the player added an item to their inventory, an item appeared on the save that could not exist at the moment of the game. This caused the game to crash.

Fortunately, this bug did not reach users, but it required a serious refactoring of the build. Here, as in previous cases, the situation could have been saved by the documentation describing the changes, or at least communication with the development team, which was also missing.

4. A Tiny Detail That Breaks Everything

To diversify our bug chart, let’s add a couple of examples from web development. A client ordered a website redesign for one of Los Angeles’s finest restaurants in Beverly Hills. Everything is very serious and luxurious. There was one peculiarity here – the client asked not to make a backend for the website, as they made money by providing the end client with services to maintain the platform. If there was an opportunity to conveniently edit content through the admin panel, the restaurant owner would not need the help of our client.

Therefore, we did everything on ordinary HTML pages. But there was a nuance: the restaurant often held promotions, events, and all that. Landing pages were created for them, and links to them had to be placed in the website header. Not a problem if you have an admin panel but there isn’t any. But the website contained one and a half hundred pages since they also posted recipes from the chef here.

We have offered an implementation where the header and footer would be separate files and substituted for each page. So it would be possible to edit them from one place. Fast and convenient.

There was no software specification for the website, and we made it simply according to the Photoshop layout. The manager communicated with the client, and neither the developers nor the tester had a direct connection with them. Having completed the project with pixel precision, we submitted it for review. The client seemed to give the go-ahead, or this was what the manager understood, and the project was uploaded to the live server. After congratulating each other on another satisfied client, we went home.

And here it began! The client first wrote, then called, with tears begging to fix what we had broken. It turned out that we missed out on such a tiny thing as the restaurant reservation system. As the client later emotionally explained to us, the restaurant reservations make $ 50,000 a day, as it is famous and very popular. And we broke everything. This bug is, of course, stupid but is a good example.

As I noted above, we often forget about the big picture. The website has been checked for all the requirements of the company in full compliance with all the procedures and stages adopted for such projects. But what seems obvious to the client may not be so to the tester if it is not described. Therefore, it is important not to forget about the big picture, it will save you more than once.

5. Generous Legacy

In the examples above, I have already talked about what happens when there is no good communication. This case will be no exception. We had a client who was developing a large workforce management application for a corporation. The program was able to do everything: take into account working hours, record overtime, vacations, sick leaves, keep a schedule of internal classes and trainings, collect feedback from employees, and submit requests for financing advanced training courses. It even considered the costs of cleaning office premises.

We got to develop the frontend, which was a bunch of tables. The client was from France, the development was like sprints, and he just sent several new layouts for development every couple of months. We made them, sent them back, and never even received feedback. The team joked that the Frenchman, probably, simply does not know English and cannot write well.

Time passed, and some of the developers quit, some were transferred to other departments. It turned out that there was not a single person left in the team who would have had experience working with this project. This is not so rare. And so, a new, young, and promising developer received another batch of tables for layout. Everything was done according to the layout. The tester checked the layout and made sure that everything was in line with the design. Only after the project was handed over to the client it became clear how bad everything was.

Not knowing that the project has existed for a long time, the developer created all the styles from scratch and thus disrupted the old ones. When his styles were added to the existing pages, they fell apart and the whole project stopped working as expected. Thus, we found out that the client speaks English very well, in particular, in the field of emphatic expressions.

The requirements specification could have saved everyone. Even the most simple one, which would inform subsequent generations of developers about what happened here before them. But, as often happens, the development participants carry all the knowledge of the project in their heads. And when this head goes to another department or company, all its knowledge goes with it. Documentation may not be the most exciting thing in IT but it is like a lifeline to help in a storm.

It’s time to wrap it up. Of course, these are not such epic bugs as errors in the operation of the Boeing 737 MAX control system, which led to the crash of two scheduled flights with passengers on board. Or a bug in the Patriot missile guidance system, which led to misses during the Gulf War, due to which the Scud missile was missed, which killed 28 people in an American military camp. But these are all living, real examples that may not be as memorable as your personal failures but they will still help you get around a couple of traps on your way.