tester-army/e2e: A Practical Guide to Agentic E2E Testing

End-to-end tests often break when a user journey changes slightly, even though the outcome still works. The open-source tester-army/e2e project takes a hybrid approach: an agent can carry out a bounded app task, while ordinary assertions check exact results. This guide covers where that pattern helps, how to start, and what to validate before relying on it in CI.

As checked October 26, 2026, GitHub lists [email protected], released October 23. That release adds an onAction callback after each settled action, including replayed actions, returns absolute paths from app.screenshot(), keeps masked screenshots available after secret fills, and adds per-run action and assertion timeout flags. The [email protected] release (October 12) added e2e run --watch; its release notes say watch mode is refused in CI and with the JSON reporter. The project remains in active development toward 1.0, so pin the version and recheck its configuration and APIs before upgrading.

What tester-army/e2e does

The framework combines a TypeScript test runner with web and mobile engines. A test can ask an agent to perform a user-level action such as selecting a plan, then use a semantic agent check and a precise locator assertion to verify the result. The agent is not a replacement for assertions: exact business values should still be checked in code.

The repository separates the main e2e package from engine packages such as @e2e-dev/web and @e2e-dev/mobile. The web engine uses Playwright; mobile tests run against iOS simulators or Android emulators. Tests that contain no agent steps do not need a model. Agent steps use a provider or subscription that you configure, so model access and usage are part of the test design rather than bundled test infrastructure.

Where the hybrid approach fits

• A checkout or account flow with variable navigation. Let the agent follow the visible flow, then assert the exact confirmation status, account name, or total with a locator.
• A mobile journey with changing screens. Use an agent for a user-level goal such as opening a settings section, then verify a stable screen title or control with a device selector.
• A workflow that should be exercised in CI but is awkward to encode as a long chain of brittle selectors. Keep the natural-language action small and preserve deterministic checks around the important outcome.

Use a staging environment and dedicated test accounts for flows that create or change data. Agentic actions can make mistakes, and a natural-language description alone is not a safe authorization or cleanup mechanism.

Set up a small test

The current quickstart requires Node.js 24.8 or later, or Node.js 22.22.3 or later. On Windows, the project recommends running in WSL. Mobile testing additionally needs the relevant Xcode simulator runtime or Android SDK and emulator.

In the application directory, let the setup wizard select Web or Mobile, configure a model provider if the test needs agent steps, and create its example project:

npx e2e init
npx e2e run

Here is a small web example. Run a journey like this only against seeded test data:

import { test } from '@e2e-dev/web';
import { expect } from 'e2e';

test('a member selects a plan', async ({ app, agent, screen }) => {
  await app.open('/settings/billing');

  await agent.act('select the {plan} plan for the test workspace', {
    params: { plan: 'Pro' },
  });

  await agent.assert('the plan selection confirmation is visible');
  await expect(screen.getByRole('status')).toContainText('Pro');
});

agent.act expresses one bounded goal. agent.assert checks a semantic condition; the locator assertion then verifies a specific visible value. Keep generated or user-specific data out of test instructions where practical, use parameters for deliberate test inputs, and avoid live customer accounts.

Keep verification live

The framework's replay cache can reuse recorded actions after a later check verifies their outcome. It is an action-replay optimization, not a reason to remove the check: agent.assert, agent.waitFor, and agent.extract run live. In CI, the cache is read-only by default when no cache mode is configured. A cache entry can contain actions and typed text, so review it as test data before sharing or committing it; do not put real credentials or customer data in a test journey.

For a GitHub Actions pilot, use a lockfile, install the browser version required by the selected engine, and run npx e2e run --reporter list,junit. Keep model keys and test-account credentials in repository secrets and upload the generated report and artifacts on failure. The official workflow examples also separate mobile jobs because iOS and Android runners have different simulator or emulator requirements.

The new e2e run --watch command is useful for local iteration. The 0.20.0 release notes say watch mode is refused in CI and with the JSON reporter, so it is not a CI substitute.

Limits and a sensible pilot

Agent steps add model latency and usage, and their behavior can vary with the UI, prompt, model, and test data. A passing agent assertion is not proof of a correct transaction. Keep exact checks for amounts, identity, permissions, and persisted state; use a follow-up read or API assertion when the result must exist beyond the screen.

Start with one low-risk journey. Seed data, run the test locally, inspect the report and trace, then run it on a pull request. Include a negative case, a changed UI label, a failed model request, and a run with --no-cache. Compare the resulting evidence with the assertions you would accept from a conventional test. Decide in advance which failures should block a merge and who reviews a rerun.

The project is Apache-2.0 licensed, but active development before 1.0 means teams should pin dependencies and review release notes instead of assuming minor-version stability. No tester-army/e2e test was run for this article.

Further reading

• tester-army/e2e on GitHub
• Release [email protected] (October 23, 2026)
• Release [email protected] (watch mode)
• All releases
• Official quickstart
• Caching and replay behavior
• GitHub Actions examples
• Model and subscription setup

Editorial note: AI-assisted independent guide based on the public repository, release notes, and project documentation, checked October 26, 2026. No tester-army/e2e test was run for this article. The project did not commission or control this guide; there are no affiliate links or sponsorship. The cover artwork is adapted from a pinned upstream file under Apache-2.0, with attribution and the required LICENSE/NOTICE copies included.