Controlling the Quality of Your Software Product

Quality is one of the most important features that applies not only to the product but to every stage of its delivery. If you want to create a successful product, you have to be professional in everything you do. Testing the software product is an important part of development because even minor errors can affect the effectiveness and success of a whole project. 

Poor quality assurance can lead to the main causes of software startup failures, so the purpose of executing quality assurance tests is to avoid the delivery of poor-quality products. In this article, we are going to share with you our ideas for quality measurements.

Quality Characteristics

1. Functionality – is determined by the ability of the software to solve problems that correspond to the fixed and anticipated needs of the user under the given conditions of software. This characteristic ensures that the software operates correctly and accurately, is interoperable, meets industry standards, and is protected from unauthorized access.
2. Reliability – the ability of software to perform the required tasks under specified conditions for a specified period of time or a specified number of operations. The attributes of this characteristic are the completeness and integrity of the entire system, the ability to independently and correctly recover from operational failures, and resiliency.
3. Usability is the user’s ability to easily understand, study, use the software.
4. Efficiency is the ability of the software to provide the required level of performance in accordance with the allocated resources, time, and other specified conditions.
5. Maintainability provides the ease to analyze and test the software, change and fix defects, meet the new requirements, facilitate further maintenance, and adapt to the existing environment.
6. Portability characterizes software in terms of ease of portability from one environment (software/hardware) to another.

Quality Measurements 

The introduction and use of metrics are essential for improving control over the development process, and in particular over the testing process. If we choose the Bug/Defect Metrics, there are the following types:

• Open/Closed Bugs (the ratio of the number of open bugs to closed (corrected and rechecked)
• Reopened/Closed Bugs (the ratio of the number of reopened bugs to closed (fixed and rechecked)
• Rejected/Opened Bugs (the ratio of the number of rejected bugs to open)
• Bugs by Severity
• Bugs by Priority

Before delivering a software product, we need to measure its quality to ensure that it is bug-free. The bugs need to be reported per month on all possible environments (Staging, Development, Production) in components. The goal is to minimize the number of bugs on the production on a monthly basis reviews. In case of bugs, the QA needs to meet and find out the reason to fix the problems. The team can use the Jira tool and create the dashboard gadget “Filter Results.”

QA: Bugs reported on staging

The bugs can be reported per feature or epic. The goal is to maximize the quality of every feature/epic on the weekly basis. If needed, this can be followed by the discussion on the code quality lessons learned session with the development team and tech lead. 

The open or resolved bugs per sprint have the goal to minimize the number of open bugs and manage the backlog efficiently on a weekly basis. If needed, the team can discuss the code quality lessons learned session and create the Jira Dashboard gadget “Created vs. Resolved Chart.”

Backlog Management Index (BMI) is used to manage the backlog of open and unresolved problems.

If BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100, then the backlog increased.

Created vs Resolved Chart: OD bugs

The average time in status and the average number of times in the status are used to identify bottlenecks and improve the development process on a daily basis. The team has to discuss the average time with the responsible for blocked items to unblock the team, so everyone can be updated on the process. For these purposes, we can use the Jira dashboard gadgets “Average Number of Times in Status” and “Average Time in Status.”

Average Time in Status

Quality Requirements

Standards should be applied when writing test plans, specifications, user interfaces, documentation, training materials, and other products. A shared project’s vision can help ensure its quality. But along with the standards, it is necessary to determine the situations of their use and develop guidelines and requirements for adopting the standards to the needs of the team and the product, if necessary. Any standard you adopt should help you do your best job.

Most of the requirements could be used directly from the technical reports and all of them had to be for the project’s definitions. Bugs reported as a result of poor requirements have to minimize the number of bugs caused by poor requirements or the lack of them. The team usually can perform such reports per sprint after the demo. If needed, the PM has to outline action items with the team and feature owner to avoid poor/lack of requirements in the future.

Pull Request

The quality metric can give you an idea of the amount of time (usually in days) needed for the pull requests to be merged or closed. You can calculate the lead time of all repositories and get an understanding of the team dynamics.

To get this number, it’s important to track every pull request and save the date and time for each pull request when it’s opened and it’s merged. For the quality measurement we need to calculate:

1. Pull requests that did not pass the test suite
2. Pull requests that broke the build
3. The number of rejected and merged requests
4. The number of pull requests comments

Preventative Measures

In a management review meeting, the quality manager gives a valuation of the product’s quality.  The information should correspond to satisfaction surveys. In order to prevent a potential problem, the management team chooses the responsible parties to review code reviews and implement actions that will help to analyze the data. Among the preventative measure we can highlight:

• Internal code reviews
  performed by one developer and Tech Lead on a daily basis, if there is a ready task. If needed, the responsible developer should provide code improvement. If it is a common issue, it should be discussed in the lessons learned session. 
• Code quality lessons learned sessions
  to review the feedback in PRs per sprint to raise team awareness, improve code quality and avoid the same mistakes in the future. If needed, the responsible developer should provide code improvements.
• Story opening sessions
  to confirm technical approach and prevent rework in future performed by developers and Tech lead with the Client’s developers. If needed, the team should research the suitable solutions and review the areas for related components.

To analyze the preventative measures, we can use such trusted best practices, as:

1. Lessons Learned, Post Project Analysis – is one of the most powerful tools for proactive improving the quality of your work.
2. The retrospective – is a separately allocated period of time in order to study the experience gained and ask yourself the following questions: “What worked for this product and how to do the same in the future?” and “What went wrong for this product and how to avoid this?”

Despite the fact that retrospectives are classified as Best Practices, they are rarely used, because it is difficult to gather an entire team: retrospectives take place at the end of project development. Most of the team members are already working on other projects, and those who remain are busy releasing the project or supporting it.

TYPES OF QUALITY ENGINEERING AND TESTING SERVICES

To deal with quality engineering and testing processes you have to cover all stages starting with test management and automation to such categories of testing software services as: 

• Technical Consulting
• Automation Testing
• Performance Testing
• Security QA
• Regression Testing
• Smoke Testing
• Functional Testing
• Automation Services for Web/Mobile Apps
• API Testing
• Integration and unit testing
• Review of product automation framework and process

Thanks for reading!