Pi - Bitfab

Bitfab integrates with the Pi agent harness. Pass your Pi Agent to instrument and Bitfab records every run as a trace. Each prompt becomes a replayable root span. Each model request and each tool call becomes a span beneath it. You write no withSpan wrappers. Canonical signatures: TypeScript reference

Experimental. getPiIntegration may change before it is stable.

Client capture off applies to the integration. With captureEnabled: false, the instrumented agent runs exactly as before and sends nothing. Keep it instrumented: replay items and seedTrace still record when explicitly invoked.

Supported Languages

Language	Method	Status
TypeScript	`getPiIntegration()`	Experimental
Python	-	Pi is a TypeScript harness
Ruby	-	Pi is a TypeScript harness
Go	-	Pi is a TypeScript harness

The integration works with the current @earendil-works/pi-agent-core 1.x packages. It also works with the earlier @mariozechner/pi-agent-core 0.x packages. It imports neither, so it adds no dependency.

Recent pnpm versions may refuse to add @bitfab/sdk to a Pi project. @earendil-works/pi-ai depends on @google/genai, protobufjs, and esbuild, which have build scripts. pnpm stops any pnpm add with ERR_PNPM_IGNORED_BUILDS until each is approved or denied. Run pnpm approve-builds, or list them under allowBuilds in pnpm-workspace.yaml, then add the SDK again.

Quick Start

import { Bitfab } from "@bitfab/sdk"
import { Agent } from "@earendil-works/pi-agent-core"
import { createModels } from "@earendil-works/pi-ai"
import { anthropicProvider } from "@earendil-works/pi-ai/providers/anthropic"

const bitfab = new Bitfab({ apiKey: process.env.BITFAB_API_KEY })

const models = createModels()
models.setProvider(anthropicProvider())
const model = models.getModel("anthropic", "claude-sonnet-4-6")!

const agent = bitfab.getPiIntegration("admin-agent").instrument(
  new Agent({
    initialState: { systemPrompt: "You are an internal admin agent.", model, tools },
    streamFn: models.streamSimple.bind(models),
  }),
)

await agent.prompt("Disable the account for jane@example.com")

instrument returns the same agent. Call it exactly as before. Runs through the returned agent are traced under the admin-agent key. Runs through the original, uninstrumented agent are not.

What Gets Captured

Each agent.prompt() or agent.continue() records one trace.

Span	Type	Input	Output
The run, named after the key	`agent`	The arguments passed to `prompt`	The final assistant reply text
Each model request, named `provider/model` (for example `anthropic/claude-sonnet-4-6`)	`llm`	The model, the system prompt, the transcript sent, and the reasoning, token, and temperature settings	The assistant message Pi received, with its content blocks and stop reason
Each tool call, named after the tool	`function`	The validated tool arguments	The tool result, with its `content` and `details`

Each llm span also records the model, its input, output, and cached token counts, and the cost Pi computed. Bitfab shows these in its cost and token views. Bitfab’s input token count is the whole prompt, so it includes Pi’s cache reads and cache writes. A model request that ends with Pi’s error or aborted stop reason marks its llm span and the run’s root span with Pi’s error message. A tool that throws marks its function span with the error. The integration never records API keys, request headers, or callbacks from Pi’s request options. Tools added after instrument are traced too, whether you assign agent.state.tools or push onto it. Each prompt’s trace is grouped under the agent’s sessionId when it has one, so a conversation reads as one session in Bitfab. Calls made outside a run through the instrumented agent are not recorded. A model request or tool call that reaches the integration with no run open logs one warning naming the fix.

Coding-agent sessions

An AgentSession from createAgentSession in @earendil-works/pi-coding-agent instruments the same way.

import { createAgentSession } from "@earendil-works/pi-coding-agent"

const { session } = await createAgentSession({ customTools })
const traced = bitfab.getPiIntegration("support-agent").instrument(session)

await traced.prompt("Summarize the open tickets")

Each session.prompt() records one root span. The session’s own retries and follow-up runs nest inside it rather than starting new traces. Its model requests and tool calls are recorded beneath that root, including the custom tools you pass to createAgentSession. Each trace is grouped under the session’s id. traced.agent is the instrumented agent the session runs.

Replay

The root span carries the prompt, so a recorded run replays by key with no hand-written root. Pass replay a function that builds a fresh agent, prompts it, and returns the reply.

const pi = bitfab.getPiIntegration("admin-agent")

async function runAdminAgent(request: string): Promise<string> {
  const agent = pi.instrument(buildAgent())
  await agent.prompt(request)
  const last = agent.state.messages[agent.state.messages.length - 1]
  return last?.role === "assistant"
    ? last.content.map((block) => (block.type === "text" ? block.text : "")).join("")
    : ""
}

await bitfab.replay("admin-agent", runAdminAgent)

Tool calls are marked for replay mocking by default. Replay answers each one with the result recorded in the original trace. A tool that writes to a database or calls an external API never runs again during replay. Model requests run live, so a replay shows how your current prompt and model answer the recorded inputs. Choose which tools replay answers from the recording with mockToolsOnReplay.

bitfab.getPiIntegration("admin-agent", { mockToolsOnReplay: ["disable_account"] })

true (the default) answers every tool call from the recording.
false runs every tool live.
A list of tool names answers only those tools from the recording. The rest run live.

When replay cannot answer a tool call from the recording, the replay item fails with the reason. This holds even though Pi turns a tool’s error into an error result and keeps the run going.

Replaying a later turn of a conversation

A Pi agent keeps its conversation in agent.state.messages. When a prompt runs on an agent that already has a conversation, the root span also records that conversation. Replay restores it into the fresh agent before the prompt runs, so a later turn replays with the turns before it.

The fresh agent keeps its own system prompt and tools. Only the user, assistant, and tool result messages are restored, so a replay tests your current prompt.
A recorded continue() run replays as continue(). Your replay function can call agent.prompt(request) for every item. When the recorded run was a continue(), request is undefined and the integration calls continue() instead.
A conversation larger than one span can carry (about 2 MB, usually from large tool results or images) is recorded from its most recent turns, as many as fit, always starting at a user message. The SDK logs a warning when it records such a turn, and again when a replay starts from only those turns. Pruning agent.state.messages keeps long conversations replayable in full.
For an AgentSession, build it in your replay function with SessionManager.inMemory(). The conversation is restored into that in-memory session. A session saved to disk is never written to by replay. Bitfab logs one warning and replays the turn without its history instead.

Grading

Pi traces work with every Bitfab grader, dataset, and experiment, with nothing specific to Pi. A grader can read the root’s prompt and reply, every tool call with its arguments and result, and every model request. For example, a grader for an admin agent can check that a destructive tool only runs when the user asked for it. See Datasets and Graders.

Nesting with core tracing

Inside a withSpan or withTrace root of the same key, the integration records no second root. Its model and tool spans nest under your root.

const handleRequest = bitfab.withSpan("admin-agent", { type: "agent" }, async (request: string) => {
  const context = await loadAccountContext(request)
  await agent.prompt(`${context}\n\n${request}`)
  return agent.state.messages.length
})

Inside a root of a different key, the integration opens its own root, nested under yours. Under a withTrace root, the calls your tool code makes are recorded beneath that tool’s span. The tool records once, not once as the tool and again as its own function.

TypeScript

Method Signature

bitfab.getPiIntegration(
  traceFunctionKey: string,
  options?: { mockToolsOnReplay?: boolean | readonly string[] },
): BitfabPiIntegration

integration.instrument<T>(agentOrSession: T): T

Parameters:

traceFunctionKey (string, required). Groups every trace from this integration under one key in Bitfab.
options.mockToolsOnReplay (optional). Which tool calls replay answers from the recording. Defaults to true.

Returns: instrument returns the agent or session it was given, traced. Calling it again with the same agent returns the same traced agent. Passing an agent that any Pi integration already instrumented returns it unchanged, so each call is recorded once. Throws: instrument throws a BitfabError when the value is neither a Pi Agent nor an AgentSession. bitfab.getFunction(key).getPiIntegration(options) is the same integration, bound to that key.

Getting Started

Primitives

Account

SDKs

Framework Integrations

Integrations

Web Portal

Reference

Changelog

Pi

Supported Languages

Quick Start

What Gets Captured

Coding-agent sessions

Replay

Replaying a later turn of a conversation

Grading

Nesting with core tracing

TypeScript

Method Signature

​Supported Languages

​Quick Start

​What Gets Captured

​Coding-agent sessions

​Replay

​Replaying a later turn of a conversation

​Grading

​Nesting with core tracing

​TypeScript

​Method Signature

Supported Languages

Quick Start

What Gets Captured

Coding-agent sessions

Replay

Replaying a later turn of a conversation

Grading

Nesting with core tracing

TypeScript

Method Signature