AgentReplay
A TypeScript library that records every model call and tool call an agent makes, so a failed run can be replayed exactly, forked with a fix, and compared, without repeating side effects like a refund.
record log every call
replay all from the log
fork model live,
tools from logWhat it does
Agents fail in the middle of a run, and re-running them is slow, non-deterministic and sometimes harmful. AgentReplay wraps the model and the tools, records each call, and lets you replay the run from the log. You can fork it with changed instructions and see the first point where the two runs diverge.
How it is built
- An SDK with wrapModel, wrapTool and startRun, plus a small CLI to list runs, show one, and diff two.
- A local dashboard with a timeline, a context-window inspector and a run diff, reading the same trace store.
- A demo support agent with a planted failure at step 5, where a refund goes wrong, used to test replay and fork end to end.
- The SDK and dashboard share only the trace store format, so the dashboard can later read from S3, Postgres or a hosted collector without SDK changes.
What I cared about
- Replay must never repeat a side effect. Write tools are blocked by default during replay, and can be stubbed or allowed explicitly.
- Calls are matched on span path, target, call number and request hash, ignoring timestamps and ids. A hash mismatch at the same key is the divergence.
- Divergence is an event, not just an error: strict replay throws, while fork records it and carries on live.
- Payloads are stored as exact bytes and matched on canonical hashes, because re-serialising a payload changes every request hash after it.
Where it is
Early and run from source. Next on the list: a Vercel AI SDK adapter, streaming capture, forking at a chosen step, and redaction at record time.
Stack: TypeScript, Node 22, npm workspaces.