The term “ad hoc” in ad hoc testing comes from Latin, meaning “for this purpose only” or “on the spot,” referring to conducting tests that are thought up on the spot.

While the idea of ad hoc testing being spontaneous may raise doubts about its effectiveness as a testing method, it can uncover defects precisely because of its nature.

Unlike formal software testing, ad hoc testing does not require preparation such as test design or document, allowing it to be flexible and could start anytime at any stage. This makes it an effective testing method for improving quality, particularly when time and resources are limited or during the final stages of development and testing.

In this article, we will provide detailed information about ad hoc testing, including its overview, effectiveness, considerations, and important points to consider during implementation.

What is Ad hoc testing?

According to ISTQB, an organization that operates international software testing engineer certification, ad hoc testing is defined as “informal testing performed without test analysis or test design.”

In formal software testing, preparation for test execution, such as test design, is done in advance. However, ad hoc testing omits such preparations, does not use a test plan, or sets expectations or predictions for test results. The ad hoc testing approach involves testing features spontaneously and looking for hidden bugs or defects not included in the test plan.

In ad hoc testing, testers rely on their experience and knowledge of the software to test. Testers may deliberately perform ‘error guessing’ techniques to predict and find defects.

Different types of ad hoc testing

There are three primary forms of ad hoc testing:

• Monkey Testing: This involves randomly providing invalid inputs, unexpected actions, and nonsensical user interactions to the software. The tester acts like a metaphorical monkey, randomly pressing buttons and exploring the application in an illogical way. The goal is to see how the software behaves under stress and identify potential crashes or unexpected outputs.
• Buddy Testing: A collaborative approach in which two testers, typically a developer and a tester, work together to explore the software and identify defects. They can bounce ideas off each other, explore different functionalities, and share their perspectives for a more comprehensive evaluation.
• Pair Testing: Similar to buddy testing, pair testing involves two testers working together. However, in pair testing, the roles are more defined. One tester acts as the driver, actively using the software and executing test steps. The other tester observes the process, thinks critically, and notes any issues or areas for improvement. They then switch roles to gain a well-rounded understanding of the software.

Ad hoc testing vs. Exploratory testing

Exploratory testing is a software testing approach in which testers simultaneously learn about the product being tested and its defects, plan the test, and design and execute the test content. While this sounds similar to ad hoc testing, there are some differences between these two testing methods:

Why Ad hoc testing?

Benefits of ad hoc testing

In regular testing, tests are designed based on specifications and executed according to the test design. However, formal testing doesn’t have 100% bug coverage, and there might be loopholes and hidden bugs. Furthermore, bugs that occur due to “special procedures” cannot be documented in the design specifications and cannot be detected even with careful testing. This is where ad hoc testing comes to rescue the day.

• Uncovers hidden defects: Ad hoc testing, by its random nature, can explore the software unexpectedly, potentially revealing bugs that might slip through the cracks of formal testing procedures.
• Faster execution: Since elaborate test plans and documentation are unnecessary, ad hoc testing can be implemented quickly, making it ideal for situations where time is tight.
• Complements other testing methods: Ad hoc testing acts as a valuable supplement to structured testing approaches. It can identify issues that rigid test plans might miss.
• Diverse perspectives: Ad hoc testing’s unstructured nature encourages testers to think outside the box and explore the software from different angles, leading to a more well-rounded evaluation.
• Improved collaboration: Ad hoc testing can foster better collaboration between testers and developers. Developers can readily jump in and perform ad hoc tests, leading to a more efficient bug-fixing process.
• Reduced documentation: Ad hoc testing typically involves minimal documentation, allowing testers to focus their energy on actual testing rather than paperwork.
• Early Feedback: Ad hoc testing can provide early feedback on the software’s behavior and quality, allowing developers to address issues promptly before they escalate. This early detection of bugs can lead to faster resolution and improve the software’s overall quality.
• Agile Compatibility: Ad hoc testing aligns well with agile development methodologies, where rapid iteration and continuous feedback are vital principles. Its spontaneous and exploratory nature makes it particularly suitable for agile teams looking to validate changes and iterate quickly on their software development process.

Drawbacks of ad hoc testing:

While there is an overwhelming benefit of ad hoc testing, here are some drawbacks of this testing method:

• Difficulty in reproducing defects: Since ad hoc testing often involves unplanned and unrecorded test scenarios, reproducing defects can be challenging. Testers may not document their testing activities thoroughly, making it difficult to recreate the steps that led to a particular issue. This can hinder the debugging and resolution process, delaying the identification and correction of defects.
• Limited coverage: Ad hoc testing lacks a systematic approach and predefined test cases, which can result in limited test coverage. Testers may overlook certain functionalities, scenarios, or edge cases, leading to gaps in testing. Ensuring comprehensive coverage of the software under test without a structured plan is challenging.
• Unpredictable results: Ad hoc testing can yield unpredictable results due to its spontaneous and exploratory nature. Testers may uncover defects that were not anticipated or miss critical issues due to the lack of consistency in the testing approach. This unpredictability makes it difficult to assess the effectiveness of ad hoc testing and can lead to inconsistent quality assurance outcomes.
• Difficulty in evaluating quality: Due to the highly impromptu nature of ad hoc testing, evaluating the quality of the tested functions can be challenging, making it difficult to assess whether the quality is “high” or “low.” Even if ad hoc testing is thoroughly conducted on specific functions, resulting in no defects found, it cannot be concluded that the quality of the functions is “good.”
• Reliance on tester expertise: The effectiveness of ad hoc testing heavily relies on the tester’s experience and knowledge of the system. New testers or those unfamiliar with the software might not be as effective in uncovering critical issues through this approach.

When to perform Ad hoc testing?

The ad hoc method is flexible and can be beneficial throughout the Software Development Life Cycle (SDLC). While ad hoc testing is valuable throughout the SDLC, it’s generally recommended to be conducted after some formal testing. This is because formal testing helps establish a baseline understanding of the software’s functionalities and expected behavior. With this foundation, ad hoc testing can be more focused and effective in uncovering unexpected issues.

Best practices for Ad hoc testing

Ad hoc testing is considered an impromptu testing approach. However, it is often helpful to define the purpose, scope, and perspective of ad hoc testing to some extent, depending on the test objectives and the situation.

For instance, when aiming to maximize the remaining time and budget to mitigate significant bugs, strategically targeting crucial functions or areas with a history of defects and adjacent functionalities can prove highly effective. Drawing from past defect information, testers can conduct ad hoc testing with a targeted focus.

Alternatively, adopting specific focal points, such as CRUD operations or validation checks, as perspectives for ad hoc testing may elevate the chances of identifying defects in critical areas.

At SHIFT ASIA, practical ad hoc testing approaches are frequently employed grounded in extensive testing experiences. This approach empowers testers to pinpoint specific areas of interest and comprehensively understand the quality landscape within those domains.

However, conducting ad hoc testing without due consideration, akin to monkey testing, may only uncover less critical defects or complicated defect reproduction. Hence, exercising caution is paramount.

Furthermore, a deep understanding of not only the general specifications and requirements of the software under testing but also of the associated products, services, and business domains is essential during ad hoc testing. With testers equipped with insights into the software’s purpose, target audience, and specific use cases, ad hoc testing can be hypothesis-driven, significantly increasing the likelihood of identifying more crucial defects.

Sum up

Ad hoc testing can be a highly effective testing method, provided it is conducted with an understanding of its merits, drawbacks, and best practices. However, it should not be used as a substitute for design-based testing; instead, it should be used in conjunction with it. Understanding this complementary relationship and combining both testing types is crucial for optimal results.