Jul 22, 2020 SA Blog Team
ISTQB Testing Principles | Lessons Learned
Deepening our understanding along with our lessons learned.
Previously we discussed principle 1-3, in this article we will address 4-7.
Let's dive in.
Principle 4 | Uneven distribution of defects
Most of the defects and operational failures found in pre-release testing are over-concentrated in a few specific modules – usually minor. As previously mentioned in Principle 2, it is smart to predict the uneven distribution of defects in order to concentrate testing efforts and perform risk analysis based on actual observations in testing and operations.
Due to various reasons like being unable to allocate members because of lack of engineer skills, complicated business requirements, implementation difficulty, insufficient development resources due to other project working in parallel etc. problems may repeatedly occur in a specific area.
We constantly examine and verify what kind of software test should be performed in each phase based on the importance of each function and bug detection rate. By doing so, it enables you to optimize resource allocation and adjust test scenarios for each function.
Principle 5 | Beware of pesticide paradox
If you repeat the same test over and over, the test will eventually fail to find new defects. To avoid this "insecticide paradox", tests and test data need to be reviewed regularly to revise or create new tests. It is simply because, repeated use of insecticides reduces its effectiveness over time and similarly, the ability to find defects in testing will decrease. Except, in the case of an automated regression test, repeating the same test can indicate a beneficial result that there is less regression.
If the person who developed also supports the test, depending on the experience of the developer in question, there may be a lack of perspectives and variations of the test. In addition, appropriate validation steps may be neglected due to bias, pressure on schedule and the belief that it will be all right.
Risks can be reduced by establishing a review and validation system with multiple people, such as separating developers and QA personnel or defining a review process.
Principle 6 | The right kind of testing is subjective
Different situations and products require different testing methods. For example, testing industrial safety-critical software is different from testing e-commerce mobile applications. Moreover, the test execution method is different between the Agile project and the sequential software development life cycle project.
It may be difficult to instantly assess the right kind of testing according to the development method, especially when testing products in agile development.
In agile development, QA personnel should put in charge during the sprint period, and testing (verification) is performed in parallel with development.
Principle 7 | Pitfalls of "zero bugs"
While some organizations expect testers to be able to perform all possible tests and detect all potential bugs, Principle 2 and Principle 1 make this impossible. It is also a false assumption to expect a system to be built perfectly, simply through the process of detecting and fixing a large number of bugs. For example, even after thoroughly testing all specified requirements and fixing all detected bugs, a system that is difficult to use, does not meet user needs or expectations. So theoretically a bug proof application doesn’t guarantee success if it has poor usability and may be inferior to other competing systems.
If you focus too much on achieving zero defects, you may be conservative when defining requirements and selecting solutions, or you may have excessive checkpoints before release. This may conflict with meeting business requirements, result in slow time-to-market cycle or take away resource from core business development. Despite some apparent advantage, having impartial perspective either on performance or testing, you must be careful not to fall into the trap and pride thinking "it's okay because I tested it".
The aforementioned problem is a very difficult one, and it tends to occur in larger scale projects. It is necessary for organization to be creative enough to devise various resolutions such as how business and IT department communicate to form consensus at each phase.
In addition, it is essential to manage the project transition and regulate operation process after the production is started, with the assumption that the defect is never zero. Also, system design requires input from the CI perspective especially for tests linked to release judgment.
Just as such thing as perfect software cannot exist, it is extremely difficult for perfect software testing to exist.
On the other hand, by taking appropriate quality assurance measures at each phase of software development, it becomes dramatically easier to control the quality of deliverables. Who doesn’t want to manage the man-hours and delivery dates of projects with happy developers and QA team?
In order to satisfy that purpose, let’s remind ourselves that test design and execution according to test plan for each phase are important. And how much ever complex your projects get, they are all based on these seven principles.
We are live on https://medium.com/@shiftasia & https://www.linkedin.com/company/shiftasia
Please feel free to interact with us there or drop us a line through contact form :)
ISTQB Testing Principles | Lessons Learned
Stay in touch with Us