Case Study

Reducing Flaky Tests

Turning a suite that failed randomly into one the team could trust — by fixing waiting strategy and locators, not by adding retries.

Problem

Tests were failing intermittently with no code change and no real bug behind them — the classic symptom of a flaky suite. The immediate temptation was to add retries everywhere and move on, but that only hides the underlying timing and locator problems.

Solution

Replaced every `waitForTimeout()` with condition-based waits tied to actual application state (element visible, network idle, response received).
Audited locators and replaced brittle CSS/XPath chains with role- and text-based locators that survive markup changes.
Leaned on Playwright's built-in auto-waiting instead of manual polling wherever the assertion already implied a wait.
Limited retries to genuine environment-level flakiness (e.g. shared test-environment latency), and treated any test that only passed on retry as a bug in the test itself.

Approach in Practice

Used Trace Viewer on every failure to see exactly what the page looked like at the moment of failure, instead of guessing.
Grouped recurring failures by root cause (timing vs. locator vs. real bug) before fixing anything, to avoid treating symptoms.
Re-ran the suite in parallel across multiple workers to confirm fixes held under real CI conditions, not just locally.

Result

— More stable, predictable execution across repeated CI runs.
— Cleaner, more readable test code — explicit conditions instead of arbitrary timeouts.
— Retry count became a signal worth watching again, instead of noise everyone ignored.