Generative AI in software testing: a great combination

Jisoo Nguyen -

Generative AI has been a hot technology topic for the past decade, but generative AI, and specifically the arrival of ChatGPT in 2022, has thrust AI into worldwide headlines and launched an unprecedented surge of AI innovation and adoption. Generative AI offers enormous productivity benefits for individuals and organizations. While it also presents  real challenges and risks, businesses are forging ahead. In this blog, we'll explore generative in testing.

What is Generative AI?

Generative artificial intelligence, also known as generative AI or gen AI - is a type of AI that can create new content and ideas. This includes generating conversations, stories, images, videos, and music. It can learn human languages, programming languages, art, chemistry, biology, or any complex subject matter. Then, it reuses what it knows to solve new problems.

Generative AI operates in three phases:

  • Training: to create a foundation model that serves as the basis of multiple gen AI applications. Generative AI models are trained on massive datasets of text, images, audio, or video. Through this process, model learn to recognize patterns, relationships, and structures within this data.
  • Tuning: to tailor the foundation model to a specific gen AI application.
  • Generation, evaluation and retuning, to assess the gen AI application's output and continually improve its quality and accuracy.

For more detail explanation, you might to explore resources on https://www.microsoft.com/en-us/ai/ai-101/how-does-generative-ai-work

Generative AI in testing

Generative AI for testing using AI to assist or automate various tasks within the software testing lifecycle. Large Language Models (LLMs) -  a core part of generative AI, can generate test cases, test scripts, and even mock data based on existing requirements or code. QA engineers can now use tools powered by LLMs to accelerate test automation, identify edge cases, and write documentation. This significantly speeds up the development cycle and enhances overall test coverage.

The evolution of AI in software testing

https://dzone.com/articles/the-impact-of-ai-on-software-testing

Impact of AI of Software Testing

As show in the image above, AI has a significant impact on testing - especially in test case generation. AI bring a lot of benefits but also challenges

Benefits of using Generative AI in Testing

  • Improved Efficiency: AI automates the creation of testcases by analyzing specifications, code, and historical data to automatically create test cases This saves time and effort while ensuring comprehensive coverage. Especially when time is limited, using AI to create testcase will help you save of time.
  • Enhanced Coverage: Generates diverse scenarios to ensure comprehensive testing. After generative testcase by AI, tester can interact like client - asking questions and refining outputs to improve quality.
  • Cost reduction: AI minimizes the need for extensive human resources, reducing manual work in both creation and execution.

Challenge of using Generative AI in testing

  • Quality and Unpredictable: AI might generate incorrect, irrelevant or incomplete test case, test data or scripts. You need to review and refind the output frequently to make sure the testcases work correctly. One prompt send twice may receive different responses which makes user confuse.
  • Integration Complexity: using AI to write test script is very difficult. While AI can generate test script, it might not fully understand your project's structure, library necessary or logic in your code. It may to clean code but it can't work or fail to run as expect.
  • Data Confidentiality: When you submitting your prompts or sharing your project content, always avoid exposing data. Even with pro account, you must to be cautions with any confidential information before sharing.
  • Cost: using AI pro to use more function but it's very expensive. You need take time and big model to train your AI

Example: using AI to generate testcases.

At first, you might face some challenges. Because AI using research and response for you that it doesn't follow the rules you expect. That's why it's important to spend time training AI and refining your prompts.

One of the key tips is to use to well-crafted prompts to guide AI behavior. For example, this link provides a some prompt for training your AI https://faqprime.com/en/ai-prompts-for-test-case-generation/. Tools like Gemini allow you to use features like GEM to create a custom channel for generating testcases consistently or create project in ChatGPT.

Click "Add instruction" and input your required in this project to make sure all response will follow your instrution.

I have example prompt for login feature 

## Persona: Act as a professional software tester on E-commerce
 
## Task: create test plan for login feature
 
## Context:
All ideas should be foucused on discovering risks that impact the quality characteristics: Functionality.
Focus on chrome broswer on Windows, handling both valid and invalid
Login form submission (email, password feilds)
## Format output
Generative table of test case with format columns ID, Test Case Name, Tcs Descriptions, Test data, Test case steps, Expected results

Response for example prompt

Conclusion

Generative AI is not just a passing trend; it's a powerful force reshaping industries and software testing id no expectation. Understanding and maximizing the benefits of AI with the right approach, it can become a powerful assistant, helping our save time and focus more on critical testing tasks.