Reducing Flaky Tests: A Practical Checklist

Introduction

A flaky test is a test that sometimes passes and sometimes fails without any meaningful change to the application.

The same code runs.

The same test data is used.

The same environment is targeted.

Yet the result is inconsistent.

Flaky tests are one of the most damaging problems in test automation because they reduce confidence in the entire test suite.

When failures become common and unpredictable, teams start to ignore them.

Test fails

↓

Rerun

↓

Test passes

↓

Failure is ignored

Over time, real defects can be hidden inside automation noise.

Reducing flakiness is not about adding more retries or increasing every timeout. It requires identifying the real source of instability and fixing it systematically.

This article provides a practical checklist for diagnosing, preventing, and reducing flaky tests in Playwright automation.

---

What Is a Flaky Test?

A flaky test produces inconsistent results under apparently identical conditions.

For example:

Run 1: Passed

Run 2: Failed

Run 3: Passed

Run 4: Passed

Run 5: Failed

A stable test should produce the same result when the application state and inputs remain unchanged.

Flakiness may come from:

• The test code
• The application
• Test data
• The environment
• External services
• Parallel execution
• Network instability

The first step is to determine which category caused the failure.

---

Why Flaky Tests Are Dangerous

Flaky tests create more than technical inconvenience.

They affect the entire delivery process.

Common consequences include:

• Developers losing confidence in automation
• Real defects being dismissed
• CI pipelines being rerun repeatedly
• Release decisions being delayed
• Debugging time increasing
• Regression suites becoming unreliable

A test suite with a high pass count is not valuable if the team does not trust the results.

Reliability matters more than the number of automated tests.

---

Do Not Treat Retries as the Fix

Playwright supports automatic retries.

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
});

Retries can help collect additional evidence and reduce the effect of temporary infrastructure problems.

However, retries do not remove the cause of flakiness.

Consider this result:

First attempt: Failed

Retry 1: Failed

Retry 2: Passed

The final status may be green, but the test is still unstable.

Retries should be treated as:

• A temporary safety net
• A diagnostic signal
• A CI resilience mechanism

They should not become a permanent replacement for root-cause analysis.

---

The Flaky Test Investigation Workflow

When a test fails intermittently, use a consistent investigation process.

Reproduce

↓

Collect evidence

↓

Classify the failure

↓

Identify the root cause

↓

Apply the smallest reliable fix

↓

Run repeatedly

↓

Monitor in CI

Avoid immediately adding:

await page.waitForTimeout(5000);

or:

timeout: 100000

These changes often hide the problem rather than solving it.

---

Checklist 1: Verify the Locator

Weak locators are one of the most common causes of flaky UI tests.

Avoid Dynamic CSS Classes

Fragile:

page.locator(
  '.css-1a2b3c-button',
);

Generated class names may change between builds.

Prefer:

page.getByRole('button', {
  name: 'Submit',
});

---

Avoid Position-Based Locators

Fragile:

page
  .locator('button')
  .nth(2);

The test depends on the button's position in the DOM.

Prefer scoping the locator to a meaningful container:

const checkoutForm =
  page.getByTestId('checkout-form');

const submitButton =
  checkoutForm.getByRole(
    'button',
    {
      name: 'Submit',
    },
  );

---

Check for Duplicate Matches

This locator may match both a visible and hidden element:

page.getByText('Login');

Use strict, scoped locators:

page
  .getByRole('dialog')
  .getByRole('button', {
    name: 'Login',
  });

When investigating, check the count:

const count =
  await loginButton.count();

console.log(
  `Login button count: ${count}`,
);

A locator that unexpectedly matches multiple elements may cause intermittent interaction with the wrong target.

---

Prefer Semantic Locators

A practical locator priority is:

1. getByRole()
2. getByLabel()
3. getByPlaceholder()
4. getByText()
5. getByTestId()
6. Stable CSS selectors
7. XPath as a last resort

Semantic locators are usually more readable and resilient to layout changes.

---

Checklist 2: Remove Hardcoded Waits

Hardcoded waits create false confidence.

await page.waitForTimeout(3000);

Three seconds may be:

• Too long on a fast machine
• Too short in CI
• Completely unrelated to the actual application state

Wait for a real condition.

Before:

await searchButton.click();

await page.waitForTimeout(5000);

await result.click();

After:

await searchButton.click();

await expect(result).toBeVisible();

await result.click();

Other useful conditions include:

await expect(locator).toBeEnabled();

await expect(locator).toHaveText(
  'Completed',
);

await locator.waitFor({
  state: 'hidden',
});

await expect(page).toHaveURL(
  /dashboard/,
);

The test should continue when the application is ready, not when an arbitrary delay expires.

---

Checklist 3: Use Web-First Assertions

Immediate state checks can fail before the UI finishes updating.

Fragile:

const visible =
  await successMessage.isVisible();

expect(visible).toBe(true);

isVisible() returns the current state immediately.

Prefer:

await expect(
  successMessage,
).toBeVisible();

Playwright retries the assertion until it passes or reaches the timeout.

Other useful web-first assertions include:

await expect(locator).toBeHidden();

await expect(locator).toHaveText(
  'Success',
);

await expect(locator).toContainText(
  'Completed',
);

await expect(locator).toHaveCount(5);

await expect(locator).toHaveValue(
  'John',
);

await expect(page).toHaveURL(
  /orders/,
);

Web-first assertions are one of the simplest ways to improve test stability.

---

Checklist 4: Wait for the Correct Application State

An element may be visible but not ready.

For example:

• A balance displays an old value
• A table shows stale rows
• A button is visible but disabled
• A modal is visible but still animating
• A canvas is rendered but the game is not ready

Wait for the exact state required by the test.

Example: wait for a balance to change.

const previousBalance =
  await balance.textContent();

await depositButton.click();

await expect(balance).not.toHaveText(
  previousBalance ?? '',
);

Example: wait for a button to become enabled.

await expect(
  submitButton,
).toBeEnabled();

await submitButton.click();

Example: wait for loading to finish.

await loadingSpinner.waitFor({
  state: 'hidden',
});

Visibility alone is not always sufficient.

---

Checklist 5: Synchronize Network Activity Correctly

A test may continue before the backend operation completes.

Avoid:

await saveButton.click();

await page.waitForTimeout(3000);

Wait for the expected response:

const responsePromise =
  page.waitForResponse(
    response =>
      response.url().includes(
        '/api/profile',
      ) &&
      response.request().method() ===
        'PUT' &&
      response.status() === 200,
  );

await saveButton.click();

const response =
  await responsePromise;

expect(response.ok()).toBeTruthy();

Create the waiting promise before triggering the request.

Correct:

const responsePromise =
  page.waitForResponse(
    '**/api/orders',
  );

await submitButton.click();

await responsePromise;

Risky:

await submitButton.click();

await page.waitForResponse(
  '**/api/orders',
);

The response may complete before the listener is registered.

---

Checklist 6: Verify the Backend Response

Sometimes the UI appears slow because the backend actually failed.

A test may report:

Success message not visible

But the true cause may be:

POST /api/orders

↓

500 Internal Server Error

Capture and validate important responses.

const response =
  await responsePromise;

if (!response.ok()) {
  console.error(
    await response.text(),
  );
}

expect(response.ok()).toBeTruthy();

This produces better failure information than simply increasing the UI timeout.

---

Checklist 7: Eliminate Shared Test Data

Parallel tests can interfere with each other when they use the same data.

Examples include:

• The same user account
• The same wallet balance
• The same shopping cart
• The same report file
• The same game session
• The same database record

Imagine two tests running simultaneously:

Worker 1 changes account language to English

Worker 2 changes the same account language to Thai

Both tests may fail unpredictably.

Use unique test data.

const uniqueEmail =
  [
    'tester',
    test.info().workerIndex,
    Date.now(),
  ].join('-') +
  '@example.com';

For files:

const exportPath =
  test.info().outputPath(
    'report.xlsx',
  );

Each test should own its data whenever possible.

---

Checklist 8: Avoid Global Mutable Variables

Global variables can leak state between tests.

Fragile:

let currentUser;
let startBalance;
let orderId;

One test may overwrite values used by another test.

Prefer local variables:

test(
  'user can place an order',
  async ({ page }) => {
    const currentUser =
      await createUser();

    const orderId =
      await createOrder(
        currentUser.id,
      );

    // Test logic
  },
);

If shared setup is required, use Playwright fixtures with the correct scope.

---

Checklist 9: Keep Tests Independent

Tests should not depend on execution order.

Fragile:

Test 1 creates a customer

Test 2 updates the customer

Test 3 deletes the customer

If Test 1 fails, the remaining tests fail.

Instead:

Test 1 creates and uses its own customer

Test 2 creates and uses its own customer

Test 3 creates and uses its own customer

Independent tests support:

• Parallel execution
• Individual debugging
• Random execution order
• Selective execution
• Easier maintenance

---

Checklist 10: Clean Up Test Data Reliably

Poor cleanup can make future tests unstable.

Use try and finally:

const user =
  await userApi.createUser();

try {
  await loginPage.login(
    user.email,
    user.password,
  );

  await expect(
    dashboardPage.heading,
  ).toBeVisible();
} finally {
  await userApi.deleteUser(
    user.id,
  );
}

The finally block runs even when the assertion fails.

For complex projects, cleanup can also be handled through fixtures.

---

Checklist 11: Avoid Reusing a Dirty Browser State

A previous test may leave behind:

• Cookies
• Local storage
• Session storage
• Open dialogs
• Modified language settings
• Cached data

Playwright creates isolated browser contexts for tests by default.

Avoid manually sharing one page or context across unrelated tests unless there is a strong reason.

When authentication state is reused, ensure it contains only the expected data.

use: {
  storageState:
    'playwright/.auth/user.json',
}

Do not allow tests to permanently modify shared authentication state.

---

Checklist 12: Handle Animations and Transitions

Animations can cause an element to be visible but unstable.

Playwright waits for stability before clicking, but application-specific animations may still create problems.

Possible solutions include:

• Waiting for the final UI state
• Disabling animations in the test environment
• Using reduced motion
• Waiting for an overlay to disappear

Example configuration:

use: {
  reducedMotion: 'reduce',
}

For a modal:

await expect(modal).toBeVisible();

await expect(
  modal.getByRole('button', {
    name: 'Confirm',
  }),
).toBeEnabled();

Do not solve animation problems by forcing every click.

---

Checklist 13: Avoid force: true as a Default

This is tempting:

await button.click({
  force: true,
});

Forced clicks bypass some actionability checks.

They may hide problems such as:

• An overlay covering the button
• A hidden duplicate element
• An unfinished animation
• A disabled control
• An incorrect locator

Before using force, inspect why the normal action fails.

Use it only when the application design genuinely requires it.

---

Checklist 14: Handle Dropdowns and Overlays

Dropdown menus frequently cause pointer interception issues.

A failure may report:

Element is visible

but another element intercepts pointer events

Possible causes include:

• The menu closed unexpectedly
• Another dropdown opened
• An invisible overlay remains
• The locator matches a hidden duplicate
• The target moved during animation

A stable pattern is:

await menuButton.hover();

await expect(
  menuItem,
).toBeVisible();

await menuItem.click();

If clicking one menu item changes the page, wait for the expected result:

await Promise.all([
  page.waitForURL(
    /category=fishing/,
  ),
  menuItem.click(),
]);

---

Checklist 15: Handle Frames Correctly

Elements inside an iframe cannot be accessed directly through the main page.

Use a frame locator:

const gameFrame =
  page.frameLocator(
    '#gameFrame',
  );

const betButton =
  gameFrame.getByRole(
    'button',
    {
      name: 'BET',
    },
  );

await betButton.click();

Flakiness may occur when:

• The iframe reloads
• The frame URL changes
• The frame has not attached
• The target exists in multiple frames

Wait for the iframe to be visible before interacting:

await expect(
  page.locator('#gameFrame'),
).toBeVisible();

Then use frameLocator() so Playwright resolves the current frame state automatically.

---

Checklist 16: Handle Canvas-Based Applications Carefully

Canvas applications do not provide normal DOM elements for every control.

Tests may rely on:

• Coordinates
• Screenshots
• Pixel states
• Game status elements outside the canvas
• Backend events

Coordinate-based automation can become flaky when:

• The viewport changes
• The canvas resizes
• The page scrolls
• Device scaling changes
• The game enters fullscreen
• Animations move the target

Use coordinates relative to the canvas.

const box =
  await canvas.boundingBox();

if (!box) {
  throw new Error(
    'Canvas is not visible',
  );
}

await page.mouse.click(
  box.x + box.width * 0.5,
  box.y + box.height * 0.8,
);

Standardize:

• Viewport size
• Browser zoom
• Device scale factor
• Canvas dimensions
• Scroll position

Coordinate tests require stricter environment control than normal DOM tests.

---

Checklist 17: Use API Setup Instead of Long UI Setup

Long UI setup increases the number of failure points.

For example:

Login

↓

Open products

↓

Search

↓

Add item

↓

Open cart

↓

Checkout

↓

Open order history

↓

Run actual verification

If the test is about order history, create the order through an API.

const order =
  await orderApi.createOrder(
    user.id,
  );

await orderHistoryPage.open();

await expect(
  orderHistoryPage.getOrderRow(
    order.id,
  ),
).toBeVisible();

Shorter tests are usually more stable and easier to diagnose.

---

Checklist 18: Control External Dependencies

Third-party services can make tests unstable.

Examples include:

• Payment providers
• Email services
• Analytics systems
• External game providers
• Captcha
• File storage systems

Possible strategies include:

• Mocking external APIs
• Using a sandbox environment
• Using test doubles
• Validating only your integration boundary
• Separating external dependency tests from core regression tests

Example API mocking:

await page.route(
  '**/api/payment-provider',
  async route => {
    await route.fulfill({
      status: 200,
      contentType:
        'application/json',
      body: JSON.stringify({
        status: 'approved',
      }),
    });
  },
);

Do not allow a third-party outage to make the entire core regression suite unreliable.

---

Checklist 19: Review Test Timeouts

Very short timeouts cause false failures.

Very long timeouts make real failures slow.

Use sensible defaults:

export default defineConfig({
  timeout: 60_000,

  expect: {
    timeout: 10_000,
  },

  use: {
    actionTimeout: 15_000,
    navigationTimeout: 30_000,
  },
});

Override timeouts only for known slow operations.

await expect(
  reportCompleteMessage,
).toBeVisible({
  timeout: 60_000,
});

Avoid:

await everyLocator.click({
  timeout: 100_000,
});

A large timeout applied everywhere can hide performance regressions and make failures painful to investigate.

---

Checklist 20: Check Parallel Execution

A test may pass with one worker and fail with two or more workers.

Run:

npx playwright test \
  --workers=1

Then:

npx playwright test \
  --workers=2

If failures appear only in parallel mode, investigate:

• Shared accounts
• Shared files
• Shared database records
• Shared browser state
• Rate limits
• Environment capacity
• Global variables
• Cleanup collisions

A stable framework should not depend on test execution order.

---

Checklist 21: Use Unique Artifact Paths

Multiple workers may overwrite the same screenshot or report file.

Fragile:

await page.screenshot({
  path: 'screenshots/error.png',
});

Safer:

const screenshotPath =
  test.info().outputPath(
    'error.png',
  );

await page.screenshot({
  path: screenshotPath,
});

Playwright creates an isolated output directory for each test result.

This prevents filename collisions.

---

Checklist 22: Enable Traces, Screenshots, and Videos

Flaky tests are difficult to fix without evidence.

Recommended configuration:

use: {
  trace: 'retain-on-failure',
  screenshot: 'only-on-failure',
  video: 'retain-on-failure',
}

The trace can show:

• The selected locator
• DOM snapshots
• Network requests
• Console logs
• Screenshots
• Timing
• Actionability checks

A screenshot shows only one moment.

A trace shows the sequence leading to the failure.

---

Checklist 23: Add Meaningful Logs

Logs should explain the current business state.

Weak:

Clicking button

Better:

Submitting withdrawal request for user test-1042

Include relevant information such as:

• Environment
• Browser
• Worker index
• User ID
• Game code
• Transaction ID
• Current balance
• Expected state
• Actual state

Example:

console.log({
  worker:
    test.info().workerIndex,
  userId: user.id,
  orderId: order.id,
  environment:
    process.env.TEST_ENV,
});

Avoid logging passwords, tokens, or sensitive information.

---

Checklist 24: Investigate Console Errors

Frontend errors may explain intermittent UI behaviour.

Capture console messages:

page.on(
  'console',
  message => {
    if (
      message.type() ===
      'error'
    ) {
      console.error(
        `Browser error: ${message.text()}`,
      );
    }
  },
);

Also capture uncaught page errors:

page.on(
  'pageerror',
  error => {
    console.error(
      `Page error: ${error.message}`,
    );
  },
);

A failing locator may only be a symptom of a JavaScript error that prevented the page from rendering.

---

Checklist 25: Verify Environment Health

Not every flaky failure is caused by test code.

Environment instability may include:

• Slow APIs
• Database locks
• Deployment in progress
• Insufficient CPU or memory
• Expired credentials
• Shared test accounts
• Service restarts
• Network interruptions

Track whether failures correlate with:

• Specific environments
• Specific execution times
• Specific workers
• Deployment windows
• High system load

If many unrelated tests fail simultaneously, investigate the environment before changing every test.

---

Checklist 26: Separate Product Flakiness From Test Flakiness

Sometimes the application itself behaves inconsistently.

For example:

Correct username and password

↓

Login succeeds sometimes

↓

Login fails sometimes

Possible product causes include:

• Authentication service instability
• Session conflicts
• Rate limiting
• Load balancer issues
• Delayed database replication

The automation test should not automatically be blamed.

Use API logs, network responses, screenshots, and backend evidence to determine whether the failure belongs to:

• The application
• The test
• The environment

---

Checklist 27: Avoid Catching and Ignoring Errors

This pattern hides failures:

try {
  await button.click();
} catch {
  // Continue test
}

The test may continue in an invalid state and fail later with a misleading message.

If the action is optional, verify the condition explicitly.

if (
  await closeButton.isVisible()
) {
  await closeButton.click();
}

For required actions, rethrow the error.

try {
  await submitButton.click();
} catch (error) {
  await captureDiagnostics();
  throw error;
}

Never silently ignore required failures.

---

Checklist 28: Be Careful With Optional Elements

Using a short timeout inside try/catch can be useful for optional UI states.

try {
  await closeButton.waitFor({
    state: 'visible',
    timeout: 1000,
  });

  await closeButton.click();
} catch {
  // Button was not present.
}

However, this should only be used when both states are valid.

Do not use optional handling for an element that is required for the scenario.

Document why the optional state exists.

---

Checklist 29: Avoid Overly Complex Recovery Logic

A flaky test is sometimes patched with many fallback actions.

Try click

↓

Reload

↓

Try click again

↓

Open menu

↓

Try another locator

↓

Wait

↓

Retry

This may make the test pass, but it also makes failures difficult to interpret.

Recovery logic should be:

• Intentional
• Limited
• Based on known application states
• Logged clearly
• Used only when multiple states are legitimate

If the application should always be in one state, fix the synchronization rather than adding fallback paths.

---

Checklist 30: Repeat the Test Locally

Playwright supports repeated execution:

npx playwright test \
  tests/checkout.spec.ts \
  --repeat-each=20

Run with one worker:

npx playwright test \
  tests/checkout.spec.ts \
  --repeat-each=20 \
  --workers=1

Then run in parallel:

npx playwright test \
  tests/checkout.spec.ts \
  --repeat-each=20 \
  --workers=4

This helps determine whether the failure is related to:

• Timing
• Shared state
• Parallel execution
• Environment load

A test that passes once is not necessarily stable.

---

Checklist 31: Use Stress Execution Strategically

For a suspected flaky test:

npx playwright test \
  tests/payment.spec.ts \
  --repeat-each=50 \
  --retries=0

Disabling retries exposes the true failure rate.

For example:

50 executions

47 passed

3 failed

Flake rate: 6%

After applying a fix, repeat the same execution pattern and compare the result.

---

Checklist 32: Track Flaky Test Rate

A practical metric is:

Flaky tests

÷

Total executed tests

×

100

Example:

8 flaky tests

÷

400 executed tests

×

100

=

2% flaky test rate

Other useful metrics include:

• First-attempt pass rate
• Retry recovery rate
• Most frequently retried tests
• Failures by environment
• Failures by browser
• Failures by worker count
• Mean time to resolve flaky tests

Tracking trends is more useful than reviewing isolated incidents.

---

Checklist 33: Quarantine Carefully

Quarantining a test can protect the main pipeline while the issue is investigated.

For example:

test.fixme(
  true,
  'Flaky due to issue QA-184',
);

Or assign a tag:

test(
  'external payment @quarantine',
  async () => {
    // Test
  },
);

Exclude quarantined tests from the blocking pipeline:

npx playwright test \
  --grep-invert @quarantine

A quarantined test should have:

• A documented reason
• An owner
• A tracking ticket
• A review date

Quarantine should not become permanent storage for ignored tests.

---

Checklist 34: Delete Tests That Provide No Value

Some flaky tests exist because they automate unsuitable scenarios.

Consider removing or redesigning a test when:

• It duplicates lower-level coverage
• It depends heavily on unstable third parties
• It verifies implementation details
• Its maintenance cost exceeds its value
• The same behaviour can be tested reliably through an API or unit test

More automated tests do not always mean better coverage.

A smaller, reliable suite is more valuable than a large, noisy suite.

---

A Practical Failure Classification

When a flaky test fails, classify it before fixing it.

Automation Defect

Examples:

• Weak locator
• Missing synchronization
• Shared variable
• Incorrect assertion
• Wrong frame
• File collision

---

Product Defect

Examples:

• Intermittent API error
• Race condition
• UI overlay not closing
• Session conflict
• Inconsistent rendering

---

Environment Issue

Examples:

• Service unavailable
• Slow database
• Deployment in progress
• Network failure
• Insufficient resources

---

Test Data Issue

Examples:

• Account already exists
• Balance is insufficient
• Data was deleted
• Record is locked
• Shared account state changed

---

External Dependency Issue

Examples:

• Payment sandbox outage
• Email delivery delay
• Third-party API timeout
• Game provider unavailable

This classification prevents random changes to the test code.

---

A Practical Playwright Flakiness Checklist

Before marking a flaky test as fixed, verify the following.

Locator

• Does the locator match exactly one element?
• Is it based on user-facing semantics?
• Does it avoid dynamic classes?
• Is it scoped to the correct container?
• Does it avoid unnecessary .first() or .nth()?

---

Synchronization

• Does the test avoid waitForTimeout()?
• Does it use web-first assertions?
• Does it wait for the correct UI state?
• Does it wait for the relevant API response?
• Is the wait registered before the triggering action?

---

Test Data

• Does the test create unique data?
• Does it avoid shared accounts?
• Does it clean up after itself?
• Can it run repeatedly?
• Can it run in parallel?

---

State Isolation

• Does the test avoid global mutable variables?
• Does it start from a known browser state?
• Does it avoid dependency on previous tests?
• Does it own its output files?

---

Environment

• Is the environment healthy?
• Are backend responses successful?
• Are credentials valid?
• Are external services available?
• Is the infrastructure large enough for the worker count?

---

Diagnostics

• Is trace enabled?
• Is a failure screenshot available?
• Is video retained when useful?
• Are console and page errors captured?
• Are important IDs and states logged?

---

Validation

• Has the test been repeated multiple times?
• Has it been tested with one worker?
• Has it been tested in parallel?
• Has it been tested in CI?
• Are retries disabled during stability verification?

---

Example: Refactoring a Flaky Test

Before

test(
  'user can place a bet',
  async ({ page }) => {
    await page.goto('/game');

    await page.waitForTimeout(
      5000,
    );

    await page
      .locator('.bet-button')
      .first()
      .click({
        force: true,
      });

    await page.waitForTimeout(
      7000,
    );

    await page
      .locator('.cashout-button')
      .click();

    await page.waitForTimeout(
      3000,
    );

    const balance =
      await page
        .locator('.balance')
        .textContent();

    expect(balance).not.toBe(
      '0',
    );
  },
);

Problems:

• Hardcoded waits
• Class-based locators
• .first() without clear intent
• Forced click
• Immediate balance check
• No verification of game state

---

After

test(
  'user can place and cash out a bet',
  async ({
    gamePage,
  }) => {
    const initialBalance =
      await gamePage
        .getBalanceValue();

    await test.step(
      'Wait for betting round',
      async () => {
        await expect(
          gamePage.roundStatus,
        ).toHaveText(
          'Betting Open',
        );
      },
    );

    await test.step(
      'Place the bet',
      async () => {
        await expect(
          gamePage.betButton,
        ).toBeEnabled();

        await gamePage
          .betButton
          .click();

        await expect(
          gamePage.cancelButton,
        ).toBeVisible();
      },
    );

    await test.step(
      'Cash out',
      async () => {
        await expect(
          gamePage.cashOutButton,
        ).toBeVisible({
          timeout: 15_000,
        });

        await gamePage
          .cashOutButton
          .click();
      },
    );

    await test.step(
      'Verify balance update',
      async () => {
        await expect
          .poll(
            async () =>
              gamePage
                .getBalanceValue(),
            {
              timeout: 15_000,
            },
          )
          .not.toBe(
            initialBalance,
          );
      },
    );
  },
);

The improved test waits for actual business states rather than fixed delays.

---

Building a Team Process for Flaky Tests

Flakiness should be managed as an engineering problem.

A useful workflow is:

Failure detected

↓

Failure classified

↓

Ticket created

↓

Owner assigned

↓

Root cause investigated

↓

Fix reviewed

↓

Test stress-run

↓

Monitoring continued

Teams should define:

• What qualifies as flaky
• When a test should be quarantined
• Who owns the investigation
• How stability is validated
• Which metrics are tracked

Without ownership, flaky tests accumulate quickly.

---

What Not to Do

Avoid these common responses to flaky tests:

• Increasing every timeout
• Adding multiple fixed waits
• Setting force: true everywhere
• Retrying until the test passes
• Catching and ignoring errors
• Running all tests serially permanently
• Disabling tests without tracking them
• Assuming every failure is an automation issue
• Blaming the environment without evidence

These approaches reduce visibility without improving reliability.

---

Final Checklist

Before merging an automated test, ask:

• Is the locator stable and unique?
• Does the test wait for real conditions?
• Are assertions retryable?
• Is test data isolated?
• Can the test run independently?
• Can it run safely in parallel?
• Are API responses validated where necessary?
• Are external dependencies controlled?
• Are traces and screenshots available?
• Has the test passed repeated execution without retries?
• Is the failure message useful?
• Can another tester understand the scenario quickly?

If several answers are no, the test is likely to become flaky later.

---

Conclusion

Flaky tests are rarely caused by one single issue.

They usually emerge from a combination of weak locators, missing synchronization, shared state, unstable environments, and unclear test design.

The solution is not to make tests wait longer or retry more often.

The solution is to make them more deterministic.

Use semantic locators, wait for meaningful application states, isolate test data, control external dependencies, collect diagnostic evidence, and validate stability through repeated execution.

Most importantly, treat flaky tests as real defects.

A trustworthy test suite should provide a clear signal:

Passed

means

The feature works

and:

Failed

means

Something requires investigation

Reducing flakiness is not only about improving test code. It is about restoring confidence in the entire software delivery process.