7 Testing Principles and Practical Considerations
Are the "Seven Principles of Testing" widely used?
Yes. “Seven Principles of Testing” are the general guidelines described in the ISTQB test engineer qualification system - Foundation Level - syllabus that must be considered when conducting software testing..
In this paper, while citing the seven principles of testing described in the syllabus, I will exemplify issues that are likely to occur in real practice and other points to be noted when performing testing.
Principle 1 | Testing can show defects, but absence of errors is a fallacy
In another word, testing can show that there is a defect, but cannot prove that there is no defect. Testing reduces the number of undetected defects left in the software, but finding no defects is never a proof of absolute correctness.
Digesting test cases that we consider necessary and sufficient does not yet prove that there is no defect (bug) in the scope.
For example, there may be cases where unexpected process runs after production is started, or defects are detected only under extremely rare conditions under specific circumstances.
While devising a test plan to improve test coverage, it is also necessary to plan in advance for failure response at the time of release and after production operation.
Apart from testing of the relevant part at the time of additional function implementation or debugging, I can suggest that it is important to take measures to maximize the possibility of bug detection in advance by performing regression test for example.
Testing can show defects, but absence of errors is a fallacy
Principle 2 | 100% exhaustive testing is impossible
Testing everything (all combinations of inputs and preconditions) is impractical except for very simple software. Instead of 100% exhaustive testing, testing efforts should be focused on risk analysis, testing techniques, and priorities.
Testing team members unreasonably judge "we have performed sufficient tests", but on the contrary, stakeholders request back to "prove that there is no problem."
Since there are restrictions such as project schedule and limited man-hours, it is important to formulate and execute an effective test plan. In order to do so, always consider and verify what kind of software test should be performed in each phase based on the importance of functions and the bug detection rate in each process. Also, by managing appropriate data disclosure and consensus among the project stakeholders, you are likely to prevent rework and pushback across development phases.
Principle 3 | Early testing saves time and money
Both static and dynamic testing activities should be initiated as early in the software development life cycle as possible, under the mission to find bugs early. Test at a starting phase is also called shift left. Costs can often be reduced by testing early in the software development life cycle.
When entrusting part of the software development work to an external company, responsibilities are usually defined and shared as the following.
- Perform requirements defining, basic design and acceptance test in-house
- Detailed design, coding, unit / integration / system testing is outsourced to a development firm
Very often, problems that affect project schedule and man-hours arise and become apparent in the integration and system testing phase, which requires changes to the scope, schedule and additional man hours at the time of executing acceptance testing later.
Ideally, reduce requirements omission, discrepancies and human errors by conducting inspections and performing static tests at the basic design and detailed design phase.
In the next article, we will discuss number 4 ~ 7 of the principles to deepen understanding of countermeasures to avoid costly mistakes.
We are live on https://medium.com/@shiftasia & https://www.linkedin.com/company/shiftasia
Please feel free to interact with us there or drop us a line through contact form :)