Instrumentation

Instrumenting a workflow is two decisions, and you make both by choosing spans:

1. See and evaluate

Capture enough that you can read what happened at each step, time it, and grade it.

2. Replay

Decide which steps become mocks: spans that serve their recorded output instead of running again.

Both are per-span decisions, so a workflow wrapped in one span has made neither. It records one input, one output, and one duration, and nothing inside it can be read, graded, or mocked.

Goal 1: capture enough to see and evaluate

Give a step its own span when any of these is true:

It calls a model. Always. This is the span you iterate on, compare across experiments, and attach graders to.
It reads external mutable state (DB query, HTTP GET, object storage, vector search, cache). These are the spans you will want to mock on replay.
It writes external state (DB write, queue publish, email, charge, file write). Mark these to mock on replay so a replayed trace does not repeat the side effect.
It transforms the model output (parsing, validation, ranking, formatting), so a quality regression points at the model or at your post-processing.
It retries or loops, one span per attempt or iteration, so a trace shows how many attempts it really took.

Skip trivial in-memory helpers, per-item work inside a large loop (wrap the loop or the batch), and internals a framework integration already captures.

What one span costs you

You lose	Why
Failure localization	You see that the workflow failed, not which step failed or what it was handed.
Latency and token attribution	Model time and tokens are not separable from storage, parsing, and persistence.
Prompt iteration and grading	With no `llm` span there is no recorded model input and output to compare across runs or attach a grader to.
Replay mocking	Mocking applies only to descendants, so a trace with none can never mock anything.

Example

const service = bitfab.getFunction("summarize-doc")

const readDoc = service.withSpan({ name: "read-doc" }, async (id: string) => storage.read(id))
const summarize = service.withSpan(
  { name: "summarize", type: "llm" },
  async (doc: Doc) => generateObject({ model, schema, prompt: buildPrompt(doc) }),
)
const persist = service.withSpan(
  { name: "persist" },
  async (id: string, s: Summary) => db.summaries.insert(id, s),
)

export const summarizeDoc = service.withSpan(
  { name: "summarize-doc", type: "agent" },
  async (id: string) => {
    const doc = await readDoc(id)
    const summary = await summarize(doc)
    await persist(id, summary)
    return summary
  },
)

service = bitfab.get_function("summarize-doc")

@service.span(name="read-doc")
def read_doc(doc_id: str) -> Doc:
    return storage.read(doc_id)

@service.span(name="summarize", type="llm")
def summarize(doc: Doc) -> Summary:
    return client.responses.parse(model=MODEL, input=build_prompt(doc), text_format=Summary)

@service.span(name="persist")
def persist(doc_id: str, summary: Summary) -> None:
    db.summaries.insert(doc_id, summary)

@service.span(name="summarize-doc", type="agent")
def summarize_doc(doc_id: str) -> Summary:
    doc = read_doc(doc_id)
    summary = summarize(doc)
    persist(doc_id, summary)
    return summary

summarize-doc
├── read-doc
├── summarize
└── persist

Spans nest by call stack in every SDK, so you never wire parents and children by hand.

Goal 2: decide what gets mocked on replay

Replay re-runs recorded inputs through your current code. The root always runs for real; each descendant either runs for real or serves its recorded output. Spans that serve recordings are mocks, and you choose them per span, at definition time, with mockOnReplay. Only descendants can be mocks: a root always executes. Mock the world, run your code. The point of a replay is to test a change against production scenarios, so anything that is not the change should be held fixed and cheap:

Step	Replay	Why
External reads (DB, HTTP `GET`, storage, vector search)	Mark to mock	The recorded rows are the scenario. Re-reading gives you today’s data, or fails without prod credentials.
External writes (DB write, queue, email, charge)	Mark to mock	Replaying a trace must not re-send the email or re-charge the card.
The model call	Leave live	Improving it is usually the point. Mock it and you are testing nothing.
Your own logic (parsing, validation, orchestration)	Leave live	This is the code under test.

Mark a span when you define it, and the default "marked" strategy does the rest:

const readDoc = service.withSpan(
  { name: "read-doc", mockOnReplay: true },
  async (id: string) => storage.read(id),
)
const persist = service.withSpan(
  { name: "persist", mockOnReplay: true },
  async (id: string, s: Summary) => db.summaries.insert(id, s),
)

@service.span(name="read-doc", mock_on_replay=True)
def read_doc(doc_id: str) -> Doc:
    return storage.read(doc_id)

@service.span(name="persist", mock_on_replay=True)
def persist(doc_id: str, summary: Summary) -> None:
    db.summaries.insert(doc_id, summary)

Now a replay of summarize-doc feeds each historical document straight from its recording, runs your new prompt for real, and writes nothing. Note that this only works because read-doc and persist are spans: goal 2 is only available for steps you captured in goal 1.

Replay also needs a root whose own arguments are JSON-serializable, since that is what it re-runs the trace against. Prefer an id, a request object, or a message list over a live connection, a stream, or a class instance.

Pitfalls

One span around the whole workflow

// ❌ Every step happens inside this span, so none of them is a span
const summarizeDoc = bitfab.withSpan("summarize-doc", async (id: string) => {
  const doc = await storage.read(id)
  const summary = await generateObject({ model, schema, prompt: buildPrompt(doc) })
  await db.summaries.insert(id, summary)
  return summary
})

One node. No prompt recorded, no per-step timing, and replay re-reads storage and re-runs the insert every time. This is the most common instrumentation mistake by a wide margin.

Calling the Vercel AI SDK without wrapping the model

// ❌ The model call is inside the span, but it is not a span
const chatTurn = bitfab.withSpan("chat-turn", () => generateText({ model, messages }))

// ✅ Wrap the model too, under the same key
const model = wrapLanguageModel({
  model: openai("gpt-4o"),
  middleware: bitfab.getVercelAiMiddleware("chat-turn"),
})

A withSpan around a function that calls a model does not create a span for the model call. See the Vercel AI SDK integration.

Wrapping in an anonymous function

// ❌ The span records no input: the arrow function takes no arguments
const result = await service.withSpan({ name: "Summarize" }, async () => summarize(doc))()

// ✅ Pass the function directly so its arguments are captured
const result = await service.withSpan({ name: "Summarize" }, summarize)(doc)

An input-less root is not replayable, because replay has nothing to re-run the trace against.

An unserializable root

// ❌ A live Request object cannot be recorded, so the trace cannot be replayed
export const handler = service.withSpan({ name: "handler" }, async (req: Request) => { ... })

// ✅ Move the boundary inward to the values the workflow actually needs
export const handler = async (req: Request) => summarizeDoc(await req.json())

The trace is still observable, but replay needs recorded inputs that round-trip.

Mocking the thing you are changing

Mocking the model call, or running with mock: "all", makes a replay return its recorded outputs and prove nothing about your change. Mock the setup around the step you are iterating on, never the step itself.

Marking nothing, then replaying against production

With no span marked, the default strategy mocks nothing and every descendant runs for real. If the workflow writes, a replay repeats the write for every trace in the run. Mark writes before the first replay, not after.

Hand-wrapping framework internals

If you use a framework integration, the handler, processor, or middleware already records the graph nodes, tools, and model calls. Wrapping them yourself produces duplicate spans. Add manual spans only for work above, alongside, or below the framework call.

Self-check

Open a trace in the web portal:

Is there more than one node?
Is there a span for the model call, carrying the prompt and the output?
Is every external read and write its own span, marked to mock on replay?

Getting Started

Primitives

Account

SDKs

Framework Integrations

Integrations

Web Portal

Reference

Changelog

Instrumentation

1. See and evaluate

2. Replay

Goal 1: capture enough to see and evaluate

What one span costs you

Example

Goal 2: decide what gets mocked on replay

Pitfalls

One span around the whole workflow

Calling the Vercel AI SDK without wrapping the model

Wrapping in an anonymous function

An unserializable root

Mocking the thing you are changing

Marking nothing, then replaying against production

Hand-wrapping framework internals

Self-check

1. See and evaluate

2. Replay

​Goal 1: capture enough to see and evaluate

​What one span costs you

​Example

​Goal 2: decide what gets mocked on replay

​Pitfalls

​One span around the whole workflow

​Calling the Vercel AI SDK without wrapping the model

​Wrapping in an anonymous function

​An unserializable root

​Mocking the thing you are changing

​Marking nothing, then replaying against production

​Hand-wrapping framework internals

​Self-check

Goal 1: capture enough to see and evaluate

What one span costs you

Example

Goal 2: decide what gets mocked on replay

Pitfalls

One span around the whole workflow

Calling the Vercel AI SDK without wrapping the model

Wrapping in an anonymous function

An unserializable root

Mocking the thing you are changing

Marking nothing, then replaying against production

Hand-wrapping framework internals

Self-check