> ## Documentation Index > Fetch the complete documentation index at: https://e2e.tester.army/docs/llms.txt > Use this file to discover all available pages before exploring further. # Debugging a run > Read the trace page, open the evidence, watch the browser, and see what the model was sent. A failed test prints its error and the path to its [trace page](#traces). For a summary of the whole run, add the `markdown` reporter. It writes `.e2e/summary.md`. ## Read the failure The `list` reporter's `Failed Tests` section shows the error, the failing line, nearby matches for a failed locator, what the app logged, and the trace page. It quotes the app's first 3 errors (console errors, uncaught exceptions, failed requests, 5xx responses), redacted like the trace page. `.e2e/summary.md` quotes them too: ```text FAIL |web| tests/todos.e2e.ts > shows the remaining count LOCATOR_NOT_FOUND: locator did not become visible: getByRole("heading", name: "Todos") ❯ tests/todos.e2e.ts:31:3 ❯ at http://127.0.0.1:4271/nope ❯ app 2 errors, 2 warnings logged × console error: Error occurred prerendering page "/todos" × network error: GET /api/todos 500 Internal Server Error ❯ trace .e2e/results/todos-shows-the-remaining-count-796fac5f12ab34cd/trace.md ``` The exit code tells the type of failure: | Exit | Meaning | Retry the job? | | ---: | --- | --- | | 1 | A test failed | Only if the test is flaky | | 2 | Configuration, collection, or policy | Fix the reported problem first | | 3 | Engine, app process, model provider, artifact, or cleanup infrastructure failure | If the cause is temporary | | 4 | An internal runner error | Report it | | 130 | The run was interrupted | | ## Traces By default, each failed, timed out, interrupted, or flaky test gets a trace page, `trace.md`, in its [results directory](#artifacts). Open it in an editor's markdown preview so its links work. After the browser or device the test ran on, it shows: - **Steps.** The node an action landed on, the values an assertion read while it waited, the replay cache's decision, what the app logged, where the page went, and how the screen changed since the step before. - **App log.** Console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, each with its step: up to 50 lines, errors and warnings first. An attempt keeps at most 200 errors and warnings and 100 other lines. - **The agent's last turns.** - **Screen at failure.** The screenshot and the semantic tree, captured in at most 5 s. An app that doesn't answer in time gets no screen section. ````markdown # ✗ todos › shows the remaining count `tests/todos.e2e.ts` · failed · 2.2s **ASSERTION_FAILED** ```text expect.toHaveText failed locator: getByRole("status", name: "Remaining") expected: text "1 remaining" observed: text "0 remaining" (match count 1) ``` Ran on: browser `chromium 141.0.7390.37` · user agent `Mozilla/5.0 (Macintosh; …) HeadlessChrome/141.0.7390.37 Safari/537.36` Look at: `tests/todos.e2e.ts:15` ## Steps 1. ✓ `app.open` `/todos` (37ms in beforeEach) — `tests/todos.e2e.ts:4` 2. ✓ `agent.act` "Add a todo named Milk." (429ms, 2 model calls) — `tests/todos.e2e.ts:12` - cache: no replay: nothing recorded for this step yet; recording saved (`.e2e/cache/1227d5a6….json`) - type "Milk" into textbox "New todo" (17ms) - press "Enter" on the focused field (5ms) - ✗ network error: `POST /api/todos 500 Internal Server Error` - screen: 6 changes since step 1 at `/todos`, 38 nodes - `added listitem "Milk Delete Milk" testid="todo"` - `added checkbox "Milk"` - `changed status "Remaining" text="1 remaining" (was: status "Remaining" text="0 remaining")` - 3 more 3. ✓ `locator.check` `getByLabel("Milk")` (13ms) — `tests/todos.e2e.ts:14` - check checkbox "Milk" (13ms) - screen: 3 changes since step 2 at `/todos`, 38 nodes - `added checkbox "Milk" [checked]` - `changed status "Remaining" text="0 remaining" (was: status "Remaining" text="1 remaining")` - `removed checkbox "Milk"` 4. ✗ `expect.toHaveText` `getByRole("status", name: "Remaining")` (1.6s) — **ASSERTION_FAILED** — `tests/todos.e2e.ts:15` - expect gave up after 16 reads in 1.6s: text "0 remaining" (1 match) x16 ## App log What the app logged, oldest first, with the step it happened in. - step 2 · ✗ network error: `POST /api/todos 500 Internal Server Error` ## Screen at failure `http://127.0.0.1:4271/todos` · viewport 1280x720 · 38 nodes ![screenshot at failure](attempt-1/screenshots/001-failure.png) ```text #n29 textbox "New todo" #n30 button "Add" #n35 listitem "Milk Delete Milk" testid="todo" #n36 checkbox "Milk" [checked] #n40 status "Remaining" text="0 remaining" ``` ```` An assertion that read one value the whole time has a wrong expectation, or the app never reached that state: look in the app log for a failed request. An assertion whose readings changed (`"2 remaining" x3 -> at 410ms "1 remaining"`) raced the app. `screen: unchanged` after an action means the action had no visible effect. On web, a step's screen is the one an action left. On a device, it's the screen the step acted on. The [`trace`](https://e2e.tester.army/docs/reference/config#trace) mode sets which tests keep a page. `--trace` keeps one for every test of a run, passing ones included. `--trace off` writes none. A test's own `trace` option wins over both. ## Artifacts Each test gets one directory under `.e2e/results/` (or [``](https://e2e.tester.army/docs/reference/config#output)`/results/`), with its trace page and one `attempt-` per attempt. A serial group's tests share one directory: ```text .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/trace.md .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/attempt-1/screen-at-failure.txt .e2e/results/todos-adds-a-todo-1a2b3c4d5e6f7a8b/attempt-1/screenshots/001-failure.png ``` Each run that reaches its tests empties `results/` first. A `--last-failed` rerun keeps the files that the report it reruns names, and writes its own attempts under each test's `rerun-/`. | Artifact | When | | --- | --- | | Screenshots | A failed attempt captures one when the engine can. The built-in `agent.assert` captures one after either verdict, unless `screenshot: false`. `app.screenshot('label')` captures one on request. They mask secure fields only. See [screenshots after a secret fill](https://e2e.tester.army/docs/reference/security#screenshots-after-a-secret-fill). | | Downloads | Files saved through `browser.waitForDownload`, kind `download`, as the app served them. [Downloads](https://e2e.tester.army/docs/reference/security#downloads) says when the runner masks secrets in them. | | Videos | When you [record them](#watch-it-happen). Videos mask nothing. See [where redaction stops](https://e2e.tester.army/docs/reference/security#where-redaction-stops). | | Agent transcripts | With [`--debug`](#agent-steps). | ## For scripts: report.json Every run that reaches its tests also writes `report.json` to the [output directory](https://e2e.tester.army/docs/reference/config#output), `.e2e` by default. It holds every result of the run in a versioned schema. `--reporter json` prints it to stdout: ```bash jq '.run | {status, exitCode, errors}' .e2e/report.json jq '.run.results[] | select(.status != "passed") | .attempts[-1] | {status, error, failure, steps: [.steps[] | select(.status != "passed") | {api, label, source, status, error}], artifacts}' .e2e/report.json ``` `run.errors` holds run-level failures. The last attempt of each failed result holds: | Field | Holds | | --- | --- | | `error` | The code, message, source line, and assertion details | | `failure` | The screen and screenshot artifacts, and any locator candidates | | `steps` | The calls that led to the failure. Each step's `events` hold what it did (`engine`), what it polled (`poll`), and where the app went (`navigation`). Agent steps add `cache` and recent model turns. `observationTruncated: true` means at least one semantic observation was incomplete. | | `appLog` | What the app logged. Each line's `step` is the `index` of its step, absent before the first step. | | `artifacts` | The captured files | A serial group's tests have no `attempts` of their own. Their results are in `run.serialGroups`, one member for each test in each attempt. A retry the run interrupted or that never reached its members has `cutRetry: true` and does not hold the verdicts, so skip it: ```bash jq '.run.serialGroups[] | (.attempts | map(select(.cutRetry != true)) | last).members[]? | select(.status != "passed") | {testId, status, error, failure}' .e2e/report.json ``` ## Watch it happen ```bash npm npx e2e run --headed # a visible browser npx e2e run --video # record every attempt npx e2e run --video=retain-on-failure # record every attempt, keep the failures ``` ```bash pnpm pnpm exec e2e run --headed # a visible browser pnpm exec e2e run --video # record every attempt pnpm exec e2e run --video=retain-on-failure # record every attempt, keep the failures ``` ```bash bun bunx e2e run --headed # a visible browser bunx e2e run --video # record every attempt bunx e2e run --video=retain-on-failure # record every attempt, keep the failures ``` `--video` takes an optional mode. Put test files before it, or write `--video=`. Each recording goes into its attempt's directory: WebM per page on web, MP4 on a device. The failure recap prints the path, or the URL when a hosted browser keeps the recording. Subtract the recording's `startedAt` from a step's `startedAt` to find that step in the video. In CI, set `video: 'on-first-retry'` in the config. A test that passes the first time records nothing, and a flaky test leaves a recording of its retry. A target's `video` wins over the config's, and `--video` wins over both. A test's own option, such as `test('checkout', { video: 'on' }, ...)`, wins over all of them. The config's `video` and `--video` apply only to targets whose engine can record, and one notice names the others. When a target or a test asks for a recording its engine can't make, the run fails with `UNSUPPORTED_ARTIFACT` before any test starts. [Recording](https://e2e.tester.army/docs/reference/config#recording) has every mode and file. ## Console output `console.log` and `console.error` in a test print while it runs, under a heading that names the stream, target, file, and test: ```text stdout | |web| tests/todos.e2e.ts > adds a todo hello from the test ``` Output from outside a test, such as a file's top level or a reporter, prints under `stdout | runner` or `stderr | runner`. Both streams print on stdout. Without the `list` reporter, output goes to its own stream as written, except with the `json` reporter (`--reporter json` or in `reporters`), which keeps stdout for the report and prints output on stderr. Output from the config load prints before the run header, or a warning replaces it if the load fails. The runner [redacts](https://e2e.tester.army/docs/reference/security#redaction) secret values. The report doesn't include console output, so use `expect` or an artifact for anything a later reader needs. ## Agent steps These flags show what the agent did: ```bash npm npx e2e run --debug # timings and a step table on stderr npx e2e run --ai-trace # every model call, for a trace viewer npx e2e run --no-cache # run the agent live instead of replaying ``` ```bash pnpm pnpm exec e2e run --debug # timings and a step table on stderr pnpm exec e2e run --ai-trace # every model call, for a trace viewer pnpm exec e2e run --no-cache # run the agent live instead of replaying ``` ```bash bun bunx e2e run --debug # timings and a step table on stderr bunx e2e run --ai-trace # every model call, for a trace viewer bunx e2e run --no-cache # run the agent live instead of replaying ``` `--debug` adds the run's phase timings and a table with one row per agent step: duration, model calls, tokens, prompt-cache share, and cost when the provider reports one. Each planned step also saves its model transcript as an artifact. ```console [e2e debug] agent steps (execution order, model openai/gpt-6-luna-fast, total $0.0312) step total model observe action calls tokens in/out cached cost act "add a todo named groceries" 3.9s 2.8s 410ms 620ms 3 9412/388 61% (5740) $0.0198 assert "the list shows groceries" 1.2s 1.1s 88ms 0ms 1 3120/41 0% (0) $0.0114 ``` `--ai-trace` writes `.e2e/ai-trace.json`: one run per agent step, one entry per model call, with the redacted input and image bytes replaced by counts. Read it with [unbox-ai](https://github.com/tester-army/unbox-ai) to see prompts, tool calls, and token usage: ```bash npm npx unbox-ai runs .e2e/ai-trace.json npx unbox-ai summary .e2e/ai-trace.json --run 0 ``` ```bash pnpm pnpm dlx unbox-ai runs .e2e/ai-trace.json pnpm dlx unbox-ai summary .e2e/ai-trace.json --run 0 ``` ```bash bun bunx unbox-ai runs .e2e/ai-trace.json bunx unbox-ai summary .e2e/ai-trace.json --run 0 ``` A replayed step makes no model call and leaves no AI trace. Add `--no-cache` to trace the whole flow or rule out a stale [recording](https://e2e.tester.army/docs/cache). ## Common errors | Code | Usual cause | Fix | | --- | --- | --- | | `APP_UNREACHABLE` | Nothing answers at `app.url`, or `app.command` never became ready. On iOS, the message may name the automation runner instead | Set `app.command.log` and read it; check the port; raise `app.command.startupTimeout`. For the iOS runner, follow the message (see [Mobile](https://e2e.tester.army/docs/mobile#troubleshooting)) | | `APP_ALREADY_RUNNING` | Something already serves `app.url` when the runner wanted to start `app.command` | Stop it, or set `reuseExisting: true` for local runs | | `LOCATOR_NOT_FOUND` | Wrong role or name, text not exact, element inside an iframe, page not open | Read the markup for the accessible name; try `exact: false`; `browser.frameLocator` for iframes; `--headed` to look | | `LOCATOR_AMBIGUOUS` | Two matches: a hidden duplicate, a repeated label | Add `{ name }`, scope under a container, `filter`, `first()`, or `{ visible: true }` | | `ASSERTION_FAILED` | The expectation is wrong, or the state settles after 5 s; for `agent.assert`, the judgment was false | Compare with the screenshot; `{ timeout }` on the matcher; rewrite the question | | `ASSERTION_INCONCLUSIVE` | `agent.assert` or `agent.extract` asked about something the screen doesn't show: on another page, still loading, or only in pixels | Open or wait for the right screen first; add `vision: true` when the answer is in pixels (the message says so) | | `ACTION_FAILED` | Element covered, disabled, or detached, or an operation timed out | Wait on the right condition with `expect` first; close overlays | | `STEP_BUDGET_EXHAUSTED`, `STEP_TIMEOUT` | The agent ran out of actions, model calls, or time | Scope the goal to one step. Raise the agent entry's `maxSteps` or `maxModelCalls` for the budget, the config `timeout` for an `act` step, or the agent's `judgmentTimeout` for `assert`, `waitFor`, and `extract` | | `MODEL_UNAVAILABLE` | No model constructed in the config, or GitHub Copilot doesn't serve the configured model to your account | See [Models](https://e2e.tester.army/docs/models); for Copilot, [Pick a model](https://e2e.tester.army/docs/subscriptions#pick-a-model) | | `MODEL_PROVIDER_FAILED` | Bad key, rate limit, no credits, or every try of one request went 120 s without an answer | Check the key the provider reads and the quota; for a slow local model, a smaller screen (`agents..maxObservationBytes`) or a faster model | | `MODEL_REFUSED` | The model refused the prompt (finish reason `content-filter`). Exit 1 | Rephrase the instruction, or switch to another model | The [errors reference](https://e2e.tester.army/docs/reference/errors) lists every code, its class, and its exit code. ## Next Every flag of `e2e run`. Every error code.