Deepening our understanding along with our lessons learned.

In our previous blog 7 Software Testing Principles | Practical Considerations, we discussed principles 1-3, in this article, we will address number 4 to 7.
Let’s dive in.

Principle 4 | Uneven distribution of defects

Most of the defects and operational failures found in pre-release testing are over-concentrated in a few specific modules – usually minor. As previously mentioned in Principle 2, it is smart to predict the uneven distribution of defects in order to concentrate testing efforts and perform risk analysis based on actual observations in testing and operations.

♦Common Failure
Due to various reasons like being unable to allocate members because of a lack of engineer skills, complicated business requirements, implementation difficulty, insufficient development resources due to other projects working in parallel, etc. problems may repeatedly occur in a specific area.

♦Solution Example
We constantly examine and verify what kind of software test should be performed in each phase based on the importance of each function and bug detection rate. By doing so, it enables you to optimize resource allocation and adjust test scenarios for each function.

Principle 5 | Beware of pesticide paradox

If you repeat the same test over and over, the test will eventually fail to find new defects. To avoid this “insecticide paradox”, tests and test data need to be reviewed regularly to revise or create new tests. It is simply because, repeated use of insecticides reduces its effectiveness over time and similarly, the ability to find defects in testing will decrease. Except, in the case of an automated regression test, repeating the same test can indicate a beneficial result that there is less regression.

♦Common Failure
If the person who developed also supports the test, depending on the experience of the developer in question, there may be a lack of perspectives and variations of the test. In addition, appropriate validation steps may be neglected due to bias, pressure on schedule, and the belief that it will be all right.

♦Solution Example
Risks can be reduced by establishing a review and validation system with multiple people, such as separating developers and QA personnel or defining a review process.

Principle 6 | The right kind of testing is subjective

Different situations and products require different testing methods. For example, testing industrial safety-critical software is different from testing e-commerce mobile applications. Moreover, the test execution method is different between the Agile project and the sequential software development life cycle project.

♦Common Failure
It may be difficult to instantly assess the right kind of testing according to the development method, especially when testing products in agile development.

♦Solution Example
In agile development, QA personnel should be put in charge during the sprint period, and testing (verification) is performed in parallel with development.

Principle 7 | Pitfalls of "zero bugs"

While some organizations expect testers to be able to perform all possible tests and detect all potential bugs, Principle 2 and Principle 1 make this impossible. It is also a false assumption to expect a system to be built perfectly, simply through the process of detecting and fixing a large number of bugs. For example, even after thoroughly testing all specified requirements and fixing all detected bugs, a system that is difficult to use, does not meet user needs or expectations. So theoretically a bug-proof application doesn’t guarantee success if it has poor usability and may be inferior to other competing systems.

♦Common Failure
If you focus too much on achieving zero defects, you may be conservative when defining requirements and selecting solutions, or you may have excessive checkpoints before release. This may conflict with meeting business requirements, result in a slow time-to-market cycle, or take away resources from core business development. Despite some apparent advantages, having an impartial perspective either on performance or testing, you must be careful not to fall into the trap and pride thinking “it’s okay because I tested it”.

♦Solution Example
The aforementioned problem is a very difficult one, and it tends to occur in large-scale projects. It is necessary for an organization to be creative enough to devise various resolutions such as how the business and IT departments communicate to form a consensus at each phase.
In addition, it is essential to manage the project transition and regulate the operation process after production is started, with the assumption that the defect is never zero. Also, system design requires input from the CI perspective, especially for tests linked to release judgment.

Conclusion:

Just as a such thing as perfect software cannot exist, it is extremely difficult for perfect software testing to exist.
Software development demands precision and control. Software quality assurance measures ensure developers, QA teams, man-hours, and delivery dates remain on track; leading to successful project completion with all stakeholders satisfied!

In order to satisfy that purpose, let’s remind ourselves that test design and execution according to the test plan for each phase are important. And how complex your projects get, they are all based on these seven principles.

We are live on https://medium.com/@shiftasia & https://www.linkedin.com/company/shiftasia
Please feel free to interact with us there or drop us a line through contact form 🙂