Regression testing by risk: what to run when there is not enough time
Regression testing is not “run everything again”. In real releases there is rarely enough time, so the valuable skill is choosing what must be checked first and what risk remains.
- Read what changed
- Map affected flows
- Pick high-impact checks
- Name what was not covered
Start from change, not from habit
A regression suite often grows into a ritual. Risk-based regression starts with the release notes, commits, tickets or a conversation with the developer: what changed and what can it touch?
The same checkout bug is more important after a payment refactor than after a text change in the footer.
What is high risk
High risk means high impact or high probability. Money, login, permissions, personal data, orders, payments and integrations usually go first.
- Flows that block users from doing the main job.
- Flows that move money or irreversible data.
- Areas touched by the current release.
- Areas with recent or repeated defects.
- Integrations where failure is hard to see in UI.
How to say no safely
A middle QA does not silently skip checks. They say what was not covered and why. That turns an invisible gap into a shared release decision.
The phrase “not tested” is not shameful when it is explicit. It is dangerous only when everyone assumes someone else checked it.
A compact regression checklist
For a small release, cover smoke flows, changed functionality, nearby functionality, permissions, key negative cases and one end-to-end business path.
For an API release, add schema compatibility, error handling, idempotency and old clients if they still exist.
What to improve after release
Every escaped bug teaches the regression suite. Ask which signal was missing: no test case, weak test data, unclear owner, missing log or no automated guard.
Regression gets better when it is edited, not when it only grows.
Likelihood × impact matrix
Use a simple 1–3 scale. Likelihood rises for changed, complex, historically faulty or unstable areas. Impact rises for money or data loss, security, blocked core journeys and broad user reach. The score aids comparison but does not replace discussion: 3×3 comes before 1×3.
Add confidence. An unfamiliar integration may move upward even without defect history. Record the reason for every score or the matrix becomes decoration.
Scenario P I Score Decision\nRepeated payment callback 3 3 9 test first\nOld mobile client 2 3 6 contract + smoke\nOrder-history filter 2 1 2 sample\nFooter copy 1 1 1 out of scope
Example: 90 minutes for payment regression
Spend 10 minutes on the diff, ticket and rollout plan; 15 on login, catalogue and checkout smoke; 40 on success, decline, timeout and retry, duplicate callback, idempotency key and reconciliation; 15 on an old client and permissions; 10 on logs, metrics and summary.
Rare currencies and the full promotion set remain uncovered. State why they rank lower, who accepts the risk and what monitoring can detect trouble during rollout.
A release-ready test summary
Include build and environment, scope, critical results, open defects, untested areas, metric observations and a QA recommendation. QA does not magically authorise release; the team or risk owner decides.
Useful wording: “Critical payment flows passed on build 1842. Blocker PAY-91 remains: a repeated callback charges twice. Stop rollout. JPY and Android 11 are untested; the release owner accepts that risk if rollout continues.”
Common regression questions
How is smoke different from regression?
Smoke quickly decides whether the build is testable and core functions are available. Regression searches for change side effects. Smoke is usually part of regression, not a replacement.
Should the entire automated suite always run?
Often yes when it is fast, stable and cheap, but green automation does not replace impact analysis, new-risk exploration or repair of slow and flaky checks.
Who accepts residual risk?
Not QA alone. QA makes evidence and gaps visible; product, engineering or a release owner makes the decision under the team process.
Good regression testing is visible prioritisation. It protects the product and tells the team exactly what risk is still being shipped.