It's Not You; A Big Chunk of Being a QA IS Defect Localization

A big chunk of being a QA is defect localization.

Sure, test design techniques help us choose test scenarios and make things more efficient. But what exactly is defect localization, and how can we make it less painful?

Let's start with the basics

Localization is like playing detective: “Where and when did things go wrong?” Without proper localization, a defect can become a hot potato tossed between frontend, backend, and any development team. Time gets wasted, and, potentially, even the context.

Think of defect localization as navigating a labyrinth, with application requests and logs as your ball of yarn. But wouldn't it be easier to have a map of this labyrinth, even a sketchy one, instead of just stumbling around with yarn? That's where the application's architecture comes in.

What is application architecture?

It's how the different parts of the system work together. In terms of our labyrinth metaphor, it’s how one section connects to another, which passageways lead where.

I distinguish two main architectures: client-server and backend.

• Client-server architecture shows us how the client and server interact.
• Backend architecture determines how the backend handles client requests.

A few words about client-server architecture

There are generally two types:

• Thin client
• Thick client

The type affects how much information the client keeps and processes on its own. There are other ways to set this up, but I'll stick to what I've actually worked with.

Most mobile and web apps are thin clients. All information is stored on the server, and the client application requests data or asks for it to be processed. Registering, logging in, subscribing to notifications – all these are calls to the server. The entire processing on the server is hidden from the client. In response to the request, the client receives collected and processed information from the database or a confirmation that the request was successfully completed.

In thick client applications, the client performs most of the processing itself: adding data to the database, generating reports, calculating sums, and creating documents. They are often installed locally, but not always. Examples of thick clients include offline games, AutoCAD, and some versions of 1C.

Now, let's explore the backend architecture

Two common approaches are:

• Monolithic
• Microservices

When almost everything is processed in one place, it's a monolith.

If requests for processing are sent to other services within the system, you're likely dealing with a microservice architecture.

In a monolithic architecture, pinpointing the source of a defect can be tricky, as different teams and services typically share the same codebase, meaning changes can have unexpected consequences.

In the second case, services are separated, each with its own codebase, meaning changes in one service have little impact on others.

Organizational structure and areas of responsibility

The title sounds scary, but it just tells you who does what, and who is responsible for which part of the labyrinth (application). Imagine we have a large company: a bank, a marketplace, a food delivery service – you name it. The bigger and more complex our application, the more people work on it. And the more people there are, the more you need to divide them into teams, each responsible for their own development area.

For example, one team might handle promotions, while another is responsible for payments. If our application offers different services, teams might be responsible for individual services, such as electronic document management, accounting, or government procurement.

You don't need to know everything and everyone, but if there's documentation outlining which team is responsible for which area, it's best to keep it bookmarked.

Putting it all together

Map in hand, yarn at the ready, let's delve into our labyrinth and hunt down the source of a defect. Let's imagine a few scenarios.

Scenario 1

Picture this: We're testing a website for a conversation club.

We're browsing the class schedule, reading about upcoming sessions, when at some point, we spot a typo.

Now, how do we figure out where it originated? Let the adventure begin!

We open devTools, refresh the page, and look at the requests and responses. Since we have a thin client, we find our typo in one of the responses – it came from the backend.

Now, we open the logs and search for the processing of the backend's request or response – this is our yarn from the magic ball. We can search the logs using any information from the request and response, but it's better to use unique values: request xiid, ID from the request, phone number, and so on.

We find the entry and check: did we get the class information from the database or from another service?

If the information came from the database, we can pass the issue to tech support to fix the typo in the database.

If the information came from another service, we can pass the defect to them.

Congratulations! We've conquered our first labyrinth: the defect is localized and reported.

Scenario 2

Now picture we're testing a registration form.

We enter an email, some data, and a made-up password. We click the registration button and unexpectedly get an error.

It’s time to unravel our magic ball! We head to our beloved Network tab in devTools and see what went wrong: we repeat all the steps and check the server's response.

In response to the request, we get a 400 code with an empty response body. Should we run off and file a defect against the frontend? But we still don’t know what exactly went wrong and what needs to be fixed. Often a 400 error occurs when there's a discrepancy between what the client sent and what the server expected. There could be many reasons for this, including:

• Outdated documentation in the task
• Changes made without documentation
• Typos

Let's check the client's request

If we have documentation, written manually or generated in Swagger or OpenAPI, let’s use it to verify that:

• We're sending all required parameters
• Parameter values have correct data types (e.g., numeric values for int parameters)

How else can we check the request?

Even if we don’t have documentation, we can verify:

• Syntax compliance (e.g., if the request is sent in JSON format, it should follow JSON syntax)
• Typos in parameter names (e.g., “mony” instead of “money”, “doby” instead of “body”)
• Cyrillic characters among Latin characters (e.g., “namе” written with a Cyrillic "e")

Everything’s in order? Then it’s time to continue our journey through the labyrinth to find the answer. We take our map and “descend” into the logs.

Log analysis

Here, two scenarios are possible:

• An error is logged during request processing
• No error is logged, but requests are recorded

In the latter case, we'll have to continue our journey through the microservice labyrinth and look for the place where our request was processed.

Upon locating the error log, we'll know what exactly went wrong, which means our localization and our journey are complete! All that remains is to gather the following information for the defect report:

• Backend request
• Error log
• Recommended fix

Conclusion

Defect localization can be challenging. Sometimes you'll hit a wall: the log you were following doesn't lead to the error or makes things more confusing. In such situations, I usually take a couple of steps back or start from the beginning.

It may take a lot of time to explore the labyrinth. The journey may be difficult, and fraught with danger: the processing of some requests can be convoluted and send requests to several different services. Sometimes it makes sense to simplify the task and contact the labyrinth's architects – the developers.