Instrumentation

Start by choosing how a trace decides what to record.

	Opt-out (recommended)	Opt-in
The idea	Capture everything, then tune it down to what you want	Capture exactly what you want
What you write	One root, once: `trace()` / `withTrace()`	One per function: the `span()` decorator, or `withSpan()` to wrap
What it records	The root and every first-party call beneath it, unless excluded	Only the functions you mark
Tuning one step	`node()` / `withNode()` names, types, or mocks a discovered call without making it a boundary	Options on that function’s own span
Replay boundaries	The root	Every span
Status	Experimental. TypeScript needs a `@bitfab/transform` adapter, Python needs 3.12+	Stable. TypeScript, Python, Ruby, and Go

Start with opt-out. One root records the whole subtree and you narrow from there, which beats deciding every boundary up front. Use opt-in for a small exact set of spans, on Ruby or Go, on a runtime without subtree capture, or for a live streaming root whose output opt-out tracing cannot finalize without changing its behavior. Keep the entire selected call stack on the same tracing surface. Do not mix the two in one call stack. Both SDKs raise MixedTracingError where they meet, naming which surface was entered inside which. Either way you end up with spans, and each carries two decisions: whether you can see and evaluate that step, and whether it runs or serves its recording on replay. One span around a whole workflow makes neither. One input, one output, one duration, and nothing inside it readable, gradable, or mockable.

Goal 1: capture enough to see and evaluate

Opt-out captures these for you, so the list is what to check for in the trace and what deserves a node. Opt-in is the list of spans to write. Give a step its own span when any of these is true:

It calls a model. Always. This is the span you iterate on, compare across experiments, and attach graders to.
It reads external mutable state (DB query, HTTP GET, object storage, vector search, cache). These are the spans you will want to mock on replay.
It writes external state (DB write, queue publish, email, charge, file write). Mark these to mock on replay so a replayed trace does not repeat the side effect.
It transforms the model output (parsing, validation, ranking, formatting), so a quality regression points at the model or at your post-processing.
It retries or loops, one span per attempt or iteration, so a trace shows how many attempts it really took.

Skip trivial in-memory helpers, per-item work inside a large loop (wrap the loop or the batch), and internals a framework integration already captures.

What one span costs you

You lose	Why
Failure localization	You see that the workflow failed, not which step failed or what it was handed.
Latency and token attribution	Model time and tokens are not separable from storage, parsing, and persistence.
Prompt iteration and grading	With no `llm` span there is no recorded model input and output to compare across runs or attach a grader to.
Replay mocking	Mocking applies only to descendants, so a trace with none can never mock anything.

Example

One trace on the root records everything it calls. Add node only where a step needs what the default did not give it, here typing the model call as llm:

export class DocSummarizer {
  @bitfab.trace("summarize-doc", { type: "agent" })
  async summarizeDoc(id: string): Promise<Summary> {
    const doc = await this.readDoc(id)
    const summary = await this.summarize(doc)
    await this.persist(id, summary)
    return summary
  }

  async readDoc(id: string): Promise<Doc> {
    return storage.read(id)
  }

  @bitfab.node({ type: "llm" })
  async summarize(doc: Doc): Promise<Summary> {
    return generateObject({ model, schema, prompt: buildPrompt(doc) })
  }

  async persist(id: string, summary: Summary): Promise<void> {
    await db.summaries.insert(id, summary)
  }
}

@bitfab.trace("summarize-doc", type="agent")
def summarize_doc(doc_id: str) -> Summary:
    doc = read_doc(doc_id)
    summary = summarize(doc)
    persist(doc_id, summary)
    return summary

def read_doc(doc_id: str) -> Doc:
    return storage.read(doc_id)

@bitfab.node(type="llm")
def summarize(doc: Doc) -> Summary:
    return client.responses.parse(model=MODEL, input=build_prompt(doc), text_format=Summary)

def persist(doc_id: str, summary: Summary) -> None:
    db.summaries.insert(doc_id, summary)

summarize_doc
├── read_doc
├── summarize
└── persist

read_doc and persist are recorded without a decorator. Spans take their function’s name, so TypeScript records readDoc where Python records read_doc. Pass name to node to rename one. Spans nest by call stack in every SDK, so you never wire parents and children by hand.

trace and node are experimental. TypeScript needs a @bitfab/transform adapter wired into your build (one config wrapper for Next.js, a plugin for Vite, webpack, esbuild and friends, or --import @bitfab/transform/register for direct Node). Python needs 3.12+. Without them the root still records and descendants are skipped, so a workflow degrades to a single-node trace rather than breaking.The decorator forms need TypeScript 5+ with standard ECMAScript decorators and apply to class methods. Use withTrace and withNode for TypeScript 4.x, standalone functions, or functions from another library; same options.

Goal 2: decide what gets mocked on replay

Replay re-runs recorded inputs through your current code. The root always runs for real; each descendant either runs for real or serves its recorded output. Steps that serve recordings are mocks, and you choose them at definition time with mockOnReplay / mock_on_replay, on a node under opt-out or on the span itself under opt-in. Only descendants can be mocks: a root always executes. Mock the world, run your code. The point of a replay is to test a change against production scenarios, so anything that is not the change should be held fixed and cheap:

Step	Replay	Why
External reads (DB, HTTP `GET`, storage, vector search)	Mark to mock	The recorded rows are the scenario. Re-reading gives you today’s data, or fails without prod credentials.
External writes (DB write, queue, email, charge)	Mark to mock	Replaying a trace must not re-send the email or re-charge the card.
The model call	Leave live	Improving it is usually the point. Mock it and you are testing nothing.
Your own logic (parsing, validation, orchestration)	Leave live	This is the code under test.

Mark a step when you define it and the default "marked" strategy does the rest. The two IO steps needed no decorator to be captured; they need one now, because marking is what the default did not give them:

@bitfab.node({ mockOnReplay: true })
async readDoc(id: string): Promise<Doc> {
  return storage.read(id)
}

@bitfab.node({ mockOnReplay: true })
async persist(id: string, summary: Summary): Promise<void> {
  await db.summaries.insert(id, summary)
}

@bitfab.node(mock_on_replay=True)
def read_doc(doc_id: str) -> Doc:
    return storage.read(doc_id)

@bitfab.node(mock_on_replay=True)
def persist(doc_id: str, summary: Summary) -> None:
    db.summaries.insert(doc_id, summary)

Now a replay of summarize-doc feeds each historical document straight from its recording, runs your new prompt for real, and writes nothing. To mock by default instead, set mockOnReplayDefault / mock_on_replay_default on the trace. Every configured node then mocks under "marked", and the step under test opts back out with mockOnReplay: false / mock_on_replay=False. Either way, goal 2 only covers steps you captured in goal 1. An excluded step, or one set to capture: false, has no recording to serve.

Replay also needs a root whose own arguments are JSON-serializable, since that is what it re-runs the trace against. Prefer an id, a request object, or a message list over a live connection, a stream, or a class instance.

Pitfalls

One span around the whole workflow

An opt-in mistake, and the most common one by a wide margin.

// ❌ Every step happens inside this span, so none of them is a span
@bitfab.span("summarize-doc", { type: "agent" })
async summarizeDoc(id: string): Promise<Summary> {
  const doc = await storage.read(id)
  const summary = await generateObject({ model, schema, prompt: buildPrompt(doc) })
  await db.summaries.insert(id, summary)
  return summary
}

One node. No prompt recorded, no per-step timing, and replay re-reads storage and re-runs the insert every time. Swapping span for trace fixes it outright, which is the main reason to start with opt-out: the shape you get by default is the one you wanted.

Calling the Vercel AI SDK without wrapping the model

// ❌ The model call is inside the span, but it is not a span
const chatTurn = bitfab.withSpan("chat-turn", () => generateText({ model, messages }))

// ✅ Wrap the model too, under the same key
const model = wrapLanguageModel({
  model: openai("gpt-4o"),
  middleware: bitfab.getVercelAiMiddleware("chat-turn"),
})

A withSpan around a function that calls a model does not create a span for the model call. See the Vercel AI SDK integration.

Wrapping in an anonymous function

Applies to withSpan, which you reach for when the boundary is not a class method.

// ❌ The span records no input: the arrow function takes no arguments
const result = await bitfab.withSpan("summarize-doc", { name: "Summarize" }, async () => summarize(doc))()

// ✅ Pass the function directly so its arguments are captured
const result = await bitfab.withSpan("summarize-doc", { name: "Summarize" }, summarize)(doc)

An input-less root is not replayable, because replay has nothing to re-run the trace against.

An unserializable root

// ❌ A live Request object cannot be recorded, so the trace cannot be replayed
export const handler = bitfab.withSpan("summarize-doc", { name: "handler" }, async (req: Request) => { ... })

// ✅ Move the boundary inward to the values the workflow actually needs
export const handler = async (req: Request) => summarizeDoc(await req.json())

The trace is still observable, but replay needs recorded inputs that round-trip.

Mocking the thing you are changing

Mocking the model call, or running with mock: "all", makes a replay return its recorded outputs and prove nothing about your change. Mock the setup around the step you are iterating on, never the step itself.

Marking nothing, then replaying against production

With no span marked, the default strategy mocks nothing and every descendant runs for real. If the workflow writes, a replay repeats the write for every trace in the run. Mark writes before the first replay, not after.

Hand-wrapping framework internals

If you use a framework integration, the handler, processor, or middleware already records the graph nodes, tools, and model calls. Wrapping them yourself produces duplicate spans. Put an opt-out trace root around meaningful application work above, alongside, or below the framework call when the runtime supports it. Use manual spans there only on an opt-in fallback path.

Self-check

Open a trace in the web portal:

Is there more than one node? Under opt-out, a single-node trace usually means the transform or Python version is missing and descendants were skipped.
Is there a span for the model call, carrying the prompt and the output, and typed llm?
Is every external read and write its own span, marked to mock on replay?

Getting Started

Primitives

Account

SDKs

Framework Integrations

Integrations

Web Portal

Reference

Changelog

Instrumentation

Goal 1: capture enough to see and evaluate

What one span costs you

Example

Goal 2: decide what gets mocked on replay

Pitfalls

One span around the whole workflow

Calling the Vercel AI SDK without wrapping the model

Wrapping in an anonymous function

An unserializable root

Mocking the thing you are changing

Marking nothing, then replaying against production

Hand-wrapping framework internals

Self-check

​Goal 1: capture enough to see and evaluate

​What one span costs you

​Example

​Goal 2: decide what gets mocked on replay

​Pitfalls

​One span around the whole workflow

​Calling the Vercel AI SDK without wrapping the model

​Wrapping in an anonymous function

​An unserializable root

​Mocking the thing you are changing

​Marking nothing, then replaying against production

​Hand-wrapping framework internals

​Self-check

Goal 1: capture enough to see and evaluate

What one span costs you

Example

Goal 2: decide what gets mocked on replay

Pitfalls

One span around the whole workflow

Calling the Vercel AI SDK without wrapping the model

Wrapping in an anonymous function

An unserializable root

Mocking the thing you are changing

Marking nothing, then replaying against production

Hand-wrapping framework internals

Self-check