IMPACT ANALYSIS

How to know which tests a PR will break — before CI

Updated June 2026 · 6 min read

Every team has lived this: a pull request looks clean, gets approved, merges — and twenty minutes later the test suite is red. The change was in src/. The failure is in tests/. And nothing in the review surfaced the connection.

Why code review misses it

Code review shows you the diff. It does not show you what depends on the diff. A reviewer reading a one-line selector rename has no way to know that three end-to-end specs select on that exact attribute — especially if those specs live in a folder they never open, or a repo they don't have checked out.

So the breakage is discovered by the most expensive possible detector: a full CI run, after merge, by whoever is on call for the red build. The person with the most context — the author, mid-review — never saw it coming.

The three ways teams try to close the gap

1. Run the whole suite on every PR. Correct, but slow and expensive, and it tells you after the fact — you still wait for the red. On large suites it's minutes-to-hours of feedback latency per push.

2. Coverage-based test selection. Map code to the tests that execute it and run only those. Powerful for unit tests, but it needs instrumentation, breaks down for E2E (where the "coverage" is a running browser), and says nothing across repo boundaries.

3. Static impact analysis at review time. Read the diff, extract what it changes that tests depend on — selectors, routes, ids, visible text — and match those against the test files. This is cheap, needs no instrumentation, and runs the moment the PR opens.

What "what tests depend on" actually means

End-to-end and integration tests are coupled to the app through a small, identifiable set of anchors:

• Selectors / test ids — data-testid, data-cy, getByRole, CSS, XPath
• Routes — URL literals the test navigates to or asserts on
• Visible text & labels — button copy, headings, aria-labels the test queries
• Element ids and names — the locators page objects hang off

If a PR changes one of these, any test referencing it is a break candidate. That's a tractable matching problem you can run in seconds — no test execution required. Want to see it on a real diff? Paste one into the free diff scanner — it runs the same extraction in your browser.

How Testward does it

Testward is a GitHub App that runs this analysis automatically on every pull request:

1. Reads the PR diff and extracts the anchors the change touches.
2. Scans your test files — in the same repo, or in a separately-linked automation repo — for specs referencing those anchors.
3. Runs an LLM confirmation pass that names the specific specs likely to break and explains why.
4. Posts one sticky PR comment: the risk level, the affected specs, and the reasons.

The reviewer now sees the consequence next to the cause, while the author still has the context to fix it — or update the test in the same PR.

It works for Playwright, Cypress and Selenium

The anchor model is framework-agnostic. See the framework-specific writeups for Playwright, Cypress, and Selenium.