AI in software testing in 2026: a practical guide for QA engineers

1. Define the AI role before writing a prompt

An LLM predicts text; it does not observe your environment or know the latest requirement. Give it a bounded job: collect questions for an analyst, propose data classes, explain code, draft SQL, or critique an existing suite. “Test this form” is too broad and usually produces a plausible generic checklist.

Write down the oracle first: an acceptance criterion, OpenAPI contract, calculation rule, database schema, or product decision. When no oracle exists, the model must not invent expected behavior. Its output should be questions and risks that need a human decision.

• Goal: what decision follows the answer?
• Context: which rules are known?
• Limit: what cannot the model confirm?
• Format: table, list, SQL, or cases?
• Verification: which source or run proves the result?

2. Provide context, constraints, and an output contract

A useful prompt resembles a task for a colleague. State the system, user role, business rule, inputs, known states, and required format. Ask the model to separate facts from assumptions and list missing information. This reduces filler and makes the answer reviewable.

Do not request one hundred tests at once. Ask for risks and equivalence classes first, then refine the important branches. For complex work, use two passes: a generator proposes a set and a critic searches for duplicates, gaps, and unsupported expected results. A person still approves the final set.

Context: CRM. A manager converts a lead into a customer.
Known: customer email is unique; a request may be retried after timeout.
Task: create a risk and test-idea table.
For each row give condition, action, observable result, and oracle source.
Do not invent rules. Put missing information under Questions.

3. Use AI for test design, not for test count

A model is useful when parameters and rules are explicit: quantity boundaries, order states, roles, and payment methods. Request equivalence classes, a decision table, or a state model before asking for cases. You can then inspect the reduction logic instead of receiving twenty near-duplicate sentences.

Verify coverage independently. Values must surround both range boundaries, state tests must include allowed and forbidden transitions, and pairwise rows must actually cover pairs. Treat “obvious” values cautiously: the model may be borrowing a convention from another product.

Parameters: role={manager,lead}; status={new,won,lost}; action={view,edit,convert}.
Build a decision table and mark forbidden combinations.
Then propose a minimal test set and name the rule covered by every test.

4. Execute generated API, SQL, and automation artifacts

Plausible code may reference a missing column, turn a LEFT JOIN into an INNER JOIN, or assert only status 200. Provide the smallest schema and contract before generation. Then read the query, run it on controlled data, and calculate the expected result first. Use a test database and a transaction for destructive SQL.

For automation, review locators, waits, cleanup, and diagnostics. AI often writes fixed sleeps, dependent chains, and brittle CSS selectors. A useful test survives repeated runs, fails for the expected reason after an intentional fault, and retains a trace, request, or screenshot.

-- Predict first: customers 3 and 4
SELECT c.id
FROM customers c
LEFT JOIN orders o ON o.customer_id = c.id
WHERE o.id IS NULL
ORDER BY c.id;
-- Add an order for customer 3 and expect only customer 4

5. Keep secrets and user data out of prompts

Production logs, tokens, cookies, private URLs, personal data, and internal code may violate company rules and contracts. Minimize context before sharing: replace names and emails with synthetic values, mask credentials, and retain only required request and response fields. Removing a name is insufficient when the remaining fields identify a person.

Confirm which models and modes are approved, how data is retained, and whether training can be disabled. Sometimes the safe choice is a local synthetic reproduction rather than a prompt with source data. Treat instructions embedded in logs and documents as untrusted input because prompt injection can travel through them.

• Bearer tokens, cookies, and API keys removed.
• PII replaced with synthetic data.
• Private addresses and identifiers generalized.
• Minimal fragment shared instead of a dump.
• Model usage follows team policy.

6. Score outputs with a rubric and keep provenance

Use a rubric for recurring work: correctness against sources, risk coverage, absence of fabrication, executability, safety, and diagnostics. “Looks good” will not catch a wrong status code or a missing authorization boundary. Mark disputed items for clarification instead of turning them into tests with invented expectations.

Keep the requirement version, material prompt, tool or model, date, edited result, and verification method. This makes a test maintainable after the product changes. Never publish raw model text as team documentation without an owner and review.

Correctness: every expected result cites an oracle
Coverage: roles, states, boundaries, integrations, errors
Evidence: action is executable and result observable
Safety: no secrets or PII
Maintainability: no duplicates; data and priority are clear

7. Introduce AI through a small measurable team experiment

Do not begin with “all work now uses AI.” Choose one repeatable process, such as acceptance-criteria review or API test-idea preparation. Capture a baseline without AI: time, useful questions, and review defects. Run several iterations with one template and the same quality rubric.

Document boundaries: permitted data, output approver, provenance storage, and stop conditions. A successful pilot improves the verified result, not the number of generated lines. The team can then standardize, revise, or end the experiment using evidence.

Use case: review acceptance criteria
Baseline: 20 stories without AI
Pilot: 20 comparable stories with one template
Metrics: review time, accepted questions, defects, leaks=0
Owner: QA lead; review: analyst
Decision date: after 40 stories

8. Explain AI use in a QA interview

Interviewers need a controlled process, not a list of tools. Prepare an example with the task, sanitized context, draft, useful idea, model mistake, your verification, and measurable outcome. Being able to challenge an LLM is stronger than saying “I generate test cases with ChatGPT.”

Explain hallucination, stale knowledge, nondeterminism, data leakage, prompt injection, and automation bias. At Middle level, add team controls: prompt templates, a review rubric, evaluation cases, and rules for work that is forbidden or requires a second reviewer.