skip to content

docs / Features

Caching

Replay verified agent actions without model calls, and let the agent take over when the app changes.

The replay cache records the actions of an agent.act() step after a later check verifies the result. On the next run, the runner replays those actions without a model call. When a control or the expected end state no longer matches, the agent takes over from the current screen. See Self-healing tests.

agent.assert, agent.waitFor, and agent.extract always run live. The run summary counts the outcomes:

           AI  4.1k tokens · 2 model calls · openai/gpt-6-luna-fast
        Cache  4 replayed · 1 handed off · 1 missed · 1 not saved

not saved counts the steps of passing and flaky tests whose step.cache.write is unconfirmed, no-change, or not-written in the attempt that passed.

When a recording is written

End the test with a check of its outcome. In read-write mode, the runner saves the recording of each act at the end of the attempt when a later verification step passed. One check at the end confirms every act before it.

tests/cart.e2e.ts
import { expect, test } from 'e2e';

test('adds an item', async ({ app, agent, screen }) => {
  await app.open('/cart');

  await agent.act('add one item to the cart');

  await expect(screen.getByRole('status')).toHaveText('1 item');
});
Counts as verificationDoes not count
expect(locator).toHaveText(...) and other locator matchersexpect(value).toBe(...) on a plain value
expect(browser).toHaveURL(...) and other engine assertionsexpect.poll(...)
locator.waitFor(), browser.waitForURL()locator.textContent() and other reads
agent.assert(...), agent.waitFor(...)agent.extract(...), another act, the attempt passing

step.cache.write, in the report only, says what happened to the recording when the attempt ended:

step.cache.writeMeaning
savedRecorded for the next run.
keptThe entry stays as it is.
unconfirmedNot saved: no check passed after the step. End the test with a locator or engine assertion or agent.assert, or wait for the state with locator.waitFor() or agent.waitFor.
evictedThe entry was deleted: its replay ended in a failure or a repair, or no check confirmed it before the attempt failed.
no-changeNothing recorded: no control appeared, went away, or changed state, and the route stayed the same. That includes a step whose only change is inside a frame from another site. Such a step runs live on every run.
not-writtenNot saved: the store rejected the write, or refused the recording, for example one over the file store’s size limit.

A step that took no action records nothing and reports no write.

A failing attempt still saves the steps verified before the failure. It deletes unverified entries and the replays involved in the failure. To keep setup goals recorded while you fix a failing assertion, end the setup with a wait for its state, such as locator.waitFor(). A custom store needs a delete method for this. Cancelling a run leaves the cache unchanged.

Failures where no model answered

An attempt whose only failures are ones where no model answered evicts nothing, since these say nothing about the app:

  • MODEL_UNAVAILABLE: no model is configured, or GitHub Copilot doesn’t serve it to the account.
  • MODEL_PROVIDER_FAILED: the provider returned an error, such as a missing or rejected key, a rate limit, or a 5xx, or a request got no response within 120 seconds.

So a read-write run without model access keeps its recordings, including a step that replayed before an agent.assert that could not reach its model.

These count as a failure of the flow: an unusable model answer (MODEL_OUTPUT_INVALID, CONTEXT_OVERFLOW), a refusal (MODEL_REFUSED), STEP_TIMEOUT (even with a request still open), and any other failure in the same attempt, such as an afterEach assertion.

What must match

A cache entry belongs to one test, target, instruction, set of params, and agent. Each of these changes causes a miss:

  • renaming the test, its describe group, or the target, or moving the test file,
  • changing the instruction or an ordinary param,
  • running with another configured agent (agent: ['buyer', 'admin'] records one entry each), or changing the agent’s context or the test’s agentContext (a secret in it enters the key as its name only),
  • switching to another engine, or changing the project id or the app’s identity.

Changing the model doesn’t cause a miss, and neither does upgrading the engine: a node the new version resolves differently misses at replay.

Use --no-cache to evaluate a new model. Replays expose step.cache.recordedBy when the entry has provenance; older or unattributed entries omit it.

A timestamp or a fresh email address would miss on every run. Wrap such values in unique(), and the runner puts in the current value at replay:

ts
import { unique } from 'e2e';

const name = `E2E ${Date.now()} Company`;
await agent.act('create a company named {name}', { params: { name: unique(name) } });
await expect(screen.getByRole('heading', { name })).toBeVisible();

Use unique() for data the flow enters or checks, and ordinary params for choices that change the flow, such as a plan name. The substitution also covers the value as a URL spells it (%20, +) and a multi-line value as the screen shows it, on one line. When a unique() value equals or sits inside another param’s value, the step isn’t recorded: step.cache.notRecorded reads param-collision, and the next run records it.

For preview URLs that change on each deploy, set app.identity to keep one cache identity. On devices, replay uses the screen title as the path.

Rerun a test while you fix it

While you fix a failing test, keep the cache on in read-write mode, the local default, and outside strict mode, which never writes. With CI set, an unset mode is read-only. A failed run keeps the recordings a check confirmed before the failure. The next run replays those goals without model calls, as long as each replay still matches the screen.

  1. Run only that test, by the line its test( opens on: npx e2e run tests/cart.e2e.ts:3.
  2. Don’t rename the test or reword a goal that passes. Each causes a miss: a rename for every goal in the test, a reworded goal for itself.
  3. When retries are on (retries in the config, or CI set), add --retries 0. A retries option on the test or its describe wins over the flag. A retry doesn’t replay, so it runs every goal live.
  4. When the test passes, run it one time with --no-cache. Every goal runs live, which shows the agent still reaches it without a recording.

How a replay works

  1. Check the starting screen: it must be the same route.
  2. Find each recorded control by role, name, test id, and surrounding context, and repeat its action. The runner waits for missing controls and stops when a match is ambiguous.
  3. Check the final route and what the original step changed: the controls that appeared, with their checked, selected, expanded, or pressed state, and the ones that went away.
  4. Finish without a model call, or let the agent continue with a record of what ran and why replay stopped.

Typed values replay as recorded. A secret fill resolves the current value by name and goes through authorization again.

A replay finishes as soon as the recorded changes show, which can be before content below them has loaded. To check content that loads after the step, use agent.waitFor, which waits for it, rather than agent.assert, which judges the screen once.

What counts as the same route

A route is the origin, the path, and the query. The record id and the fragment don’t count. Ids, tokens, timestamps, and cache busters in the path or the query read as an id placeholder, so /orders/42?t=1727780000 and /orders/7?t=1727780999 are one route.

Every other query value is part of the route: ?mode=safe and ?mode=unsafe differ, and so do /companies and /companies?tab=notes, or a new tracking parameter. /products/summer-sneaker and /products/winter-boot are different routes too.

A location on the app’s own origin is compared by its path, so a recording follows the app to a preview deployment that shares its app.identity. A page on any other origin is another route.

How the end check reads the screen
  • At least one recorded change must happen during the replay. An outcome already on screen before the actions proves nothing. When the step’s last action moved to another route, that move is the evidence.
  • An alert that the replay raised and the recording never saw stops the replay. The web engine does not observe Next.js App Router’s route announcer, so its title echo never counts.
  • Text that differs on every run, such as a date, a duration, or an id, is skipped while a stable change exists on either side of the step’s delta, whether something appeared or disappeared. With nothing else, it is kept, and a changed value hands the replay off. In an alert, it reads as a placeholder. A node known only by a test id the app generates (node-<uuid>) is skipped the same way. When kept, a node that appeared may carry any id with that pattern; one that went away is still checked by its exact id.
  • A count that names what it counts, such as 3 records imported, must read the same on replay when the step made it appear on the screen it acted on. A count that only moved (41 items to 42 items), or one on a screen the step navigated to, is skipped only when its own side of the delta has a stable change. Otherwise the count stays, even when the opposite side has a stable change. A bare number such as a badge follows the text rule above.
  • A control with no name is found by its place among the unnamed controls of its kind inside the same named row, list item, or group. A control known only by a generated test id is found by the id pattern the same way, or when it was the only one with that pattern.

Why a step was not replayed

Each act step reports step.cache, absent when caching is off for the step. agent.act() returns it as result.cache, without write, and the trace page shows it under the step.

Run summarystep.cache.modeMeaning
replayedself-finalizedThe recording completed without a model call.
handed offagent-concludedReplay started, then the agent took over.
missedmissedThe agent ran the step from the beginning.

Both carry step.cache.reason, and step.cache.detail says where the replay stopped and why (at action 2 of 3, tap button "Apply": ...). After an end-mismatch it names the recorded nodes that did not show, the ones that did not go away, and any alert the recording never saw, or says the outcome already showed before the replay acted. step.cache.entry is the entry’s key and its file name (<entry>.json) in the file store.

ReasonWhat happenedWhat to do
no-entryNo recording matches this step and its params.The next verified pass records it, unless the step’s step.cache.write is no-change.
retryRetries never replay, except under --strict-cache or cache.strict.Nothing.
invalid-entryThe file is unreadable, over 1 MiB, or from another schema version.npx e2e cache clear.
truncatedThe recording hit a limit, see below.Split the step, or pass the secret as a Secret.
wrong-contextThe app was on another route than when the step was recorded. See How a replay works.Open the same screen before the step.
target-not-foundA recorded control did not appear within 15 seconds, or it has no name and no named container to tell it from its twins.The UI changed: the next passing run records it again. For an unnamed control, give it a label or put it in a named row or group.
target-ambiguousThe number of matching controls changed.Give repeated controls distinct names or use unique test data.
gapAn action can’t be replayed, see below.The agent does this part live. Put stable values in params.
action-failedThe app rejected a replayed action.Check the app.
action-uncertainA replayed action failed, and it’s unknown whether it took effect. The agent is told to check before it acts again.Check the app.
viewport-changedA bare-point tap, hover, or drag was recorded on a viewport of another size.Keep the viewport fixed.
end-mismatchEvery action ran, but the end check failed.Leftover data from an earlier run, a banner that comes and goes, or a regression. Unique test data and cleanup in afterEach help.

To rule the cache out of a failure, run one time without it:

npx e2e run --no-cache

Limits and gaps

A recording is truncated when the step:

  • took more than 50 actions, which is possible only with maxSteps above 50,
  • typed a value or URL over 4,096 characters, or one that holds a registered secret,
  • acted on a control with no usable description.

A gap is an action the cache can’t repeat: a mutating project tool, or a value the agent read off the screen, typed after it saw a screenshot, or worked out from the date. step.cache.derived names the rule: whole-node (a control’s whole name or text), minted-token (a code or id the app showed as its own word), date, or pixels. Values from the instruction or params, and values the agent composed itself, replay.

What a recording holds

Each entry is a JSON file under .e2e/cache/, named by a hash of its key. It holds actions, target descriptions, typed text, a step summary, and the end checks: the route and up to eight controls that appeared and eight that went away. Entries hold no prompts, conversations, screenshots, observed trees, or secret values. A secret fill stores only the secret’s name.

Other typed values are stored as typed. Review recordings before you share them. See cache trust.

Modes

ModeReplaysRecordsDefault
'read-write'yesyeslocally
'read-only'yesnoin CI when cache is unset
'off'nono--no-cache for one run
e2e.config.ts fragment
cache: 'off',
// or
cache: { mode: 'read-write', dir: '.e2e/cache' },

--no-cache overrides the config. In read-only mode, stale entries stay until a read-write run replaces them. A custom cache.store replaces the file store and doesn’t default to read-only in CI. e2e explore and e2e mcp sessions don’t use the cache. A test a session runs (open_session {test}) uses it as a run’s first attempt does.

Share the cache across checkouts

Point every checkout of a project at one directory with E2E_CACHE_DIR, or --cache-dir for one run. Either replaces cache.dir, resolves from the project root (~ or a leading ~/ is your home directory), and leaves the mode alone. With a custom cache.store, --cache-dir is an error and E2E_CACHE_DIR is ignored. Parallel runs can share the directory: each write replaces its file atomically. Entries are keyed by project, so give every checkout the same projectId (or package.json name); without either, each checkout’s path becomes its identity and nothing replays across them.

  • Checkouts share entries: a failing run in any of them evicts the entry, and checkouts with the same app identity but different app code overwrite each other’s entries (the last write wins).
  • While the override is set, a cache committed under cache.dir isn’t read.
  • Use one directory per project. Two projects in one directory can fail with a false REPLAY_STALE under cache.strict.
bash
E2E_CACHE_DIR=~/.cache/my-app/e2e npx e2e run

Commit the replay cache

e2e init adds .e2e/cache/ to .gitignore, so each machine records its own and CI starts with none. To share replays, remove that line and commit the directory. CI stays read-only unless you set cache: 'read-write'. Treat committed entries as test data, and review them in the pull request that recorded them again.

Local runs leave a committed directory clean unless something changed. A replayed step never rewrites its entry, and a live recording with the same actions, paths, and anchors isn’t written. So an entry for a UI that drifts is repaired only when a control can no longer be found. A file changes when:

  • a step records for the first time, or after a hand-off,
  • a step fails after a replay, ends in an end-mismatch the agent had to repair, passes with nothing to record, or isn’t confirmed by the end of the attempt, which deletes the entry,
  • an e2e upgrade changes how entries are matched, which gives every entry a new key, so the next run records them again.

Fail on a stale recording

By default, a committed recording that stops replaying fails nothing: the agent takes the step over, and every CI run pays for model calls until someone records it again. With --strict-cache (or cache: { strict: true }), the step fails with REPLAY_STALE and makes no model call:

CI=1 npx e2e run --strict-cache

REPLAY_STALE exits 2 and is not retried.

Fails under strictStill runs live
invalid-entry, wrong-context, target-not-found, target-ambiguous, viewport-changed, action-failed, action-uncertain, end-mismatch: the app or the entry changedA step never recorded (a new or edited instruction or params)
no-entry, when the cache directory holds a recording of the same step under another keyA gap or a truncated recording
A step whose store read fails

A recording under another key comes from an e2e upgrade that changed the keys, or a change to the project id, the app’s identity, or the agent’s context. The message names that file and, for a recording made by a current version, what changed. Delete the file when nothing replays it. This check doesn’t cover a custom cache.store.

A strict run never writes or deletes an entry, even in read-write mode, so a new step records only on a run without strict. Prefer --strict-cache in CI over cache.strict in the config. To fix a stale entry:

  1. Run the test in read-write mode without the flag. Turn cache.strict off if the config sets it.
  2. Commit the changed entry. A custom cache.store gets the new entry from that run instead.

Inspect the cache

npx e2e cache ls      # one row per entry: test, target, instruction digest, age, actions
npx e2e cache stats   # entry count and size
npx e2e cache clear   # delete every entry

The cached percentage in AI usage is the provider’s prompt cache. It lowers the cost of a model call. The replay cache skips the call.

Next

was this page helpful?