> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bitfab.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Instrumentation

> Choosing your spans: capture enough to see and evaluate what happened, and decide what gets mocked on replay

Instrumenting a workflow is two decisions, and you make both by choosing spans:

<CardGroup cols={2}>
  <Card title="1. See and evaluate" icon="eye">
    Capture enough that you can read what happened at each step, time it, and grade it.
  </Card>

  <Card title="2. Replay" icon="clone">
    Decide which steps become mocks: spans that serve their recorded output instead of running again.
  </Card>
</CardGroup>

Both are per-span decisions, so a workflow wrapped in one span has made neither. It records one input, one output, and one duration, and nothing inside it can be read, graded, or mocked.

## Goal 1: capture enough to see and evaluate

Give a step its own span when any of these is true:

* **It calls a model.** Always. This is the span you iterate on, compare across experiments, and attach graders to.
* **It reads external mutable state** (DB query, HTTP `GET`, object storage, vector search, cache). These are the spans you will want to mock on replay.
* **It writes external state** (DB write, queue publish, email, charge, file write). Mark these to mock on replay so a replayed trace does not repeat the side effect.
* **It transforms the model output** (parsing, validation, ranking, formatting), so a quality regression points at the model or at your post-processing.
* **It retries or loops**, one span per attempt or iteration, so a trace shows how many attempts it really took.

Skip trivial in-memory helpers, per-item work inside a large loop (wrap the loop or the batch), and internals a [framework integration](/frameworks/overview) already captures.

### What one span costs you

| You lose                      | Why                                                                                                          |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Failure localization          | You see that the workflow failed, not which step failed or what it was handed.                               |
| Latency and token attribution | Model time and tokens are not separable from storage, parsing, and persistence.                              |
| Prompt iteration and grading  | With no `llm` span there is no recorded model input and output to compare across runs or attach a grader to. |
| Replay mocking                | Mocking applies only to descendants, so a trace with none can never mock anything.                           |

### Example

<CodeGroup>
  ```typescript TypeScript theme={null}
  const service = bitfab.getFunction("summarize-doc")

  const readDoc = service.withSpan({ name: "read-doc" }, async (id: string) => storage.read(id))
  const summarize = service.withSpan(
    { name: "summarize", type: "llm" },
    async (doc: Doc) => generateObject({ model, schema, prompt: buildPrompt(doc) }),
  )
  const persist = service.withSpan(
    { name: "persist" },
    async (id: string, s: Summary) => db.summaries.insert(id, s),
  )

  export const summarizeDoc = service.withSpan(
    { name: "summarize-doc", type: "agent" },
    async (id: string) => {
      const doc = await readDoc(id)
      const summary = await summarize(doc)
      await persist(id, summary)
      return summary
    },
  )
  ```

  ```python Python theme={null}
  service = bitfab.get_function("summarize-doc")

  @service.span(name="read-doc")
  def read_doc(doc_id: str) -> Doc:
      return storage.read(doc_id)

  @service.span(name="summarize", type="llm")
  def summarize(doc: Doc) -> Summary:
      return client.responses.parse(model=MODEL, input=build_prompt(doc), text_format=Summary)

  @service.span(name="persist")
  def persist(doc_id: str, summary: Summary) -> None:
      db.summaries.insert(doc_id, summary)

  @service.span(name="summarize-doc", type="agent")
  def summarize_doc(doc_id: str) -> Summary:
      doc = read_doc(doc_id)
      summary = summarize(doc)
      persist(doc_id, summary)
      return summary
  ```
</CodeGroup>

```
summarize-doc
├── read-doc
├── summarize
└── persist
```

Spans nest by call stack in every SDK, so you never wire parents and children by hand.

## Goal 2: decide what gets mocked on replay

[Replay](/replay-mocking) re-runs recorded inputs through your current code. The root always runs for real; each descendant either runs for real or serves its recorded output. Spans that serve recordings are **mocks**, and you choose them per span, at definition time, with `mockOnReplay`. Only descendants can be mocks: a root always executes.

**Mock the world, run your code.** The point of a replay is to test a change against production scenarios, so anything that is not the change should be held fixed and cheap:

| Step                                                    | Replay           | Why                                                                                                       |
| ------------------------------------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------- |
| External reads (DB, HTTP `GET`, storage, vector search) | **Mark to mock** | The recorded rows are the scenario. Re-reading gives you today's data, or fails without prod credentials. |
| External writes (DB write, queue, email, charge)        | **Mark to mock** | Replaying a trace must not re-send the email or re-charge the card.                                       |
| The model call                                          | **Leave live**   | Improving it is usually the point. Mock it and you are testing nothing.                                   |
| Your own logic (parsing, validation, orchestration)     | **Leave live**   | This is the code under test.                                                                              |

Mark a span when you define it, and the default `"marked"` strategy does the rest:

<CodeGroup>
  ```typescript TypeScript theme={null}
  const readDoc = service.withSpan(
    { name: "read-doc", mockOnReplay: true },
    async (id: string) => storage.read(id),
  )
  const persist = service.withSpan(
    { name: "persist", mockOnReplay: true },
    async (id: string, s: Summary) => db.summaries.insert(id, s),
  )
  ```

  ```python Python theme={null}
  @service.span(name="read-doc", mock_on_replay=True)
  def read_doc(doc_id: str) -> Doc:
      return storage.read(doc_id)

  @service.span(name="persist", mock_on_replay=True)
  def persist(doc_id: str, summary: Summary) -> None:
      db.summaries.insert(doc_id, summary)
  ```
</CodeGroup>

Now a replay of `summarize-doc` feeds each historical document straight from its recording, runs your new prompt for real, and writes nothing. Note that this only works because `read-doc` and `persist` are spans: **goal 2 is only available for steps you captured in goal 1.**

<Note>
  Replay also needs a root whose own arguments are JSON-serializable, since that is what it re-runs the trace against. Prefer an id, a request object, or a message list over a live connection, a stream, or a class instance.
</Note>

## Pitfalls

### One span around the whole workflow

```typescript theme={null}
// ❌ Every step happens inside this span, so none of them is a span
const summarizeDoc = bitfab.withSpan("summarize-doc", async (id: string) => {
  const doc = await storage.read(id)
  const summary = await generateObject({ model, schema, prompt: buildPrompt(doc) })
  await db.summaries.insert(id, summary)
  return summary
})
```

One node. No prompt recorded, no per-step timing, and replay re-reads storage and re-runs the insert every time. This is the most common instrumentation mistake by a wide margin.

### Calling the Vercel AI SDK without wrapping the model

```typescript theme={null}
// ❌ The model call is inside the span, but it is not a span
const chatTurn = bitfab.withSpan("chat-turn", () => generateText({ model, messages }))

// ✅ Wrap the model too, under the same key
const model = wrapLanguageModel({
  model: openai("gpt-4o"),
  middleware: bitfab.getVercelAiMiddleware("chat-turn"),
})
```

A `withSpan` around a function that calls a model does not create a span for the model call. See the [Vercel AI SDK integration](/frameworks/vercel-ai-sdk).

### Wrapping in an anonymous function

```typescript theme={null}
// ❌ The span records no input: the arrow function takes no arguments
const result = await service.withSpan({ name: "Summarize" }, async () => summarize(doc))()

// ✅ Pass the function directly so its arguments are captured
const result = await service.withSpan({ name: "Summarize" }, summarize)(doc)
```

An input-less root is not replayable, because replay has nothing to re-run the trace against.

### An unserializable root

```typescript theme={null}
// ❌ A live Request object cannot be recorded, so the trace cannot be replayed
export const handler = service.withSpan({ name: "handler" }, async (req: Request) => { ... })

// ✅ Move the boundary inward to the values the workflow actually needs
export const handler = async (req: Request) => summarizeDoc(await req.json())
```

The trace is still observable, but replay needs recorded inputs that round-trip.

### Mocking the thing you are changing

Mocking the model call, or running with `mock: "all"`, makes a replay return its recorded outputs and prove nothing about your change. Mock the setup around the step you are iterating on, never the step itself.

### Marking nothing, then replaying against production

With no span marked, the default strategy mocks nothing and every descendant runs for real. If the workflow writes, a replay repeats the write for every trace in the run. Mark writes before the first replay, not after.

### Hand-wrapping framework internals

If you use a [framework integration](/frameworks/overview), the handler, processor, or middleware already records the graph nodes, tools, and model calls. Wrapping them yourself produces duplicate spans. Add manual spans only for work above, alongside, or below the framework call.

## Self-check

Open a trace in the [web portal](/web-portal/overview):

* Is there more than one node?
* Is there a span for the model call, carrying the prompt and the output?
* Is every external read and write its own span, marked to mock on replay?