> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bitfab.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Building a Good Scenario

> What a scenario is for, what belongs in each part of it, and what never does, with examples to copy

This page is for whoever writes a scenario's contents: a coding agent calling `save_scenario`, your code calling `traces.saveScenario`, or a [scenario callback](https://docs.bitfab.ai/scenario-callback.md) answering Bitfab. Read it before you save one. [Scenarios](https://docs.bitfab.ai/primitives/scenarios.md) covers how scenarios are created and reviewed in the product.

Every example on this page is one you can copy. Each part of the scenario has a **Good** example and a **Not this** example, with the reason the second one is wrong.

## What a scenario is for

A scenario is one test case for a traced function. It holds two things:

1. **The situation a run starts in:** the trigger that starts it, and the data it has when the trigger arrives.
2. **What is supposed to happen in that situation.**

[Assertions](https://docs.bitfab.ai/primitives/assertions.md) are written from what is supposed to happen, and every replay of the trace is judged against them. That is the whole job of a scenario: to give assertions something exact to check, and a situation that stays the same from one replay to the next.

A scenario is not a record of what a run did. The trace already records that. The scenario records what should happen, so a replay of changed code can be checked against it.

A good scenario passes two tests:

* **Every assertion can be written from it alone.** The names, times, and values an assertion needs are in the trigger, history, or records.
* **A replay given its trigger and data faces the same situation the original run did.** Nothing in it comes from after the trigger arrived.

## The parts of a scenario

| Part | Holds | Field |
| - | - | - |
| Trigger | What starts the run, exactly as it arrives | `trigger` |
| History | Earlier messages and facts, in time order | `data.history` |
| Records | Rows your systems hold when the trigger arrives: messages, contacts, calendar events, orders | `records` or `recordsPath` |
| Missing items | What the run needs and cannot see | `data.missing` |
| Sources | How much of each kind of data was read | `sources` |
| Name and description | What the situation is and what should happen in it | `name`, `description` |
| Assertions | Each outcome a correct run produces, one checkable claim at a time | drafted with `generate_scenario_assertions`, or saved with `save_trace_assertions` |

## The example used on this page

A scheduling assistant gets a text from the executive it works for, Maya Chen, on Tuesday, October 6, 2026 at 11:42am Pacific: "Can you push my 3pm with Dana to tomorrow, same time?"

Maya's calendar has a 3pm meeting with Dana Ortiz that day, and a Board prep meeting on Wednesday from 2:30pm to 4pm. An earlier text from Dana said she is out Wednesday mornings.

What should happen: Wednesday at 3pm is taken, so the assistant leaves the meeting where it is, tells Maya about the clash, and offers her a time on Wednesday when she is free. It does not contact Dana until Maya picks a time.

## Name and description

The name says what the situation is, so someone scanning a list can tell scenarios apart. The description says the situation in a few sentences and then what should happen in it. Write what should happen as outcomes a replay can be checked for.

```json Good theme={null}
{
  "name": "Maya asks to move her 3pm with Dana to a time she already has Board prep",
  "description": "Maya texts on Tuesday, October 6 asking to push today's 3pm meeting with Dana Ortiz to the same time tomorrow. Maya has Board prep on Wednesday from 2:30pm to 4pm, and Dana said earlier that she is out Wednesday mornings. The assistant should leave the meeting on Tuesday at 3pm, text Maya that Wednesday at 3pm clashes with Board prep, and offer a Wednesday time after 4pm. It should not contact Dana until Maya picks a time."
}
```

```json Not this theme={null}
{
  "name": "Assistant moves Dana meeting to Wednesday",
  "description": "The assistant moved the Maya / Dana sync to Wednesday at 3pm and texted Maya that it was done. It should handle reschedules better."
}
```

The second one describes what one run did, which the trace already shows, and the one thing it says should happen ("handle reschedules better") is nothing a replay could pass or fail.

## Trigger

The trigger is what starts the run, exactly as it arrives. Copy the content word for word. Do not summarize, correct, or tidy it, because a replay receives it and assertions quote it.

```json Good theme={null}
{
  "inputId": null,
  "inputType": "message",
  "channel": "SMS",
  "from": "Maya Chen",
  "to": "Assistant",
  "cc": null,
  "subject": null,
  "type": "reschedule request",
  "eventTime": "2026-10-06T11:42:00-07:00",
  "eventTimeSource": "message date",
  "source": "request",
  "content": "Can you push my 3pm with Dana to tomorrow, same time?",
  "contentPassages": ["Can you push my 3pm with Dana to tomorrow, same time?"],
  "contentIsWholeMessage": true,
  "sources": []
}
```

```json Not this theme={null}
{
  "type": "reschedule request",
  "content": "Maya wants to move the Dana meeting to Wednesday 3pm.",
  "contentIsWholeMessage": true
}
```

The second one paraphrases the message. "Tomorrow" is already resolved to "Wednesday", so a replay never has to work that out, and the assertions end up checking a request nobody sent. It also claims to be the whole message when it is not the message at all.

* **`eventTime`** is when the trigger arrived, with its time zone. Everything else in the scenario is read relative to it: "tomorrow" means the day after this.
* **`type`** is a few plain words on what was asked, such as `reschedule request` or `refund request`.
* **`contentPassages`** splits the content into the parts a person would read separately, such as the paragraphs of a long email. For a one-line message, it holds the content once.
* **`sources`** lists where the content appears in the run, as `{ spanId, sourceField, text }`. Leave it empty when you do not have span ids.

## History

History is what came before the trigger, in time order: earlier messages in the thread, and facts the run has in front of it. Each entry is one fact, quoted from where it appeared.

```json Good theme={null}
[
  {
    "role": "fact",
    "kind": "earlier message",
    "quote": "Heads up, I'm out Wednesday mornings for the next month.",
    "summary": "Dana said she is out Wednesday mornings for the next month.",
    "origin": null,
    "sentAt": "2026-10-01T16:05:00-07:00",
    "weekday": "Thursday",
    "sincePrevious": null,
    "source": null
  }
]
```

```json Not this theme={null}
[
  {
    "role": "fact",
    "kind": "later message",
    "quote": "Dana: I can't do Wednesday at 3, I have a flight.",
    "summary": "Dana replied the next day that she could not make it.",
    "source": null
  }
]
```

The bad entry arrived after the trigger. A replay never sees it, so an assertion that leans on it checks something the run could not have known.

* **Quote the text as it appeared.** `quote` is the exact text. `summary` restates it in a sentence when the quote alone is hard to follow, and is `null` otherwise.
* **Set `source` when the text is in the trace.** Name the span with `{ spanId, sourceField, text }`. An entry with a source can back an assertion. An entry without one is still data the run had, but an assertion cannot cite it.
* **Use `role: "thread start"`** for the message that opened the thread, and `fact` for everything else.
* **Do not repeat the trigger.** The trigger is not history.

## Records

Records are the rows your own systems hold when the trigger arrives: messages, contacts, calendar events, memories, orders, tickets. Include every record that what should happen depends on. Here the Board prep event is the reason the meeting cannot move, so it has to be in the scenario, or no assertion can name the clash.

```json Good theme={null}
[
  {
    "kind": "calendar_event",
    "ref": "Event:88213",
    "at": "2026-09-30T10:00:00-07:00",
    "asOf": "run",
    "title": "Maya / Dana sync",
    "text": "Tuesday, October 6, 2026, 3:00pm to 3:30pm Pacific. Attendees: Maya Chen, Dana Ortiz.",
    "quote": "Maya / Dana sync"
  },
  {
    "kind": "calendar_event",
    "ref": "Event:88310",
    "at": "2026-10-02T09:15:00-07:00",
    "asOf": "run",
    "title": "Board prep",
    "text": "Wednesday, October 7, 2026, 2:30pm to 4:00pm Pacific. Attendees: Maya Chen.",
    "quote": null
  },
  {
    "kind": "contact",
    "ref": "Contact:4410",
    "at": "2026-03-12T18:00:00-07:00",
    "asOf": "now",
    "title": "Dana Ortiz",
    "text": "Dana Ortiz, VP Partnerships at Lumen. dana@lumen.example. Mobile +1 415 555 0142.",
    "quote": "Dana Ortiz"
  }
]
```

```json Not this theme={null}
[
  {
    "kind": "calendar_event",
    "ref": "Event:88213",
    "at": null,
    "asOf": "now",
    "title": "Maya / Dana sync",
    "text": "Wednesday, October 7, 2026, 3:00pm to 3:30pm Pacific.",
    "quote": null
  },
  {
    "kind": "memory",
    "ref": "note",
    "at": null,
    "asOf": "run",
    "title": "Expected behavior",
    "text": "Don't move the meeting; tell Maya about Board prep.",
    "quote": null
  }
]
```

The first bad record is the meeting as it is today, after the original run moved it, so a replay would start from that run's change instead of the real situation. The second one is what should happen, dressed up as a record. A replay reads records as data it has, so it would be handed the answer. What should happen goes in the description and the assertions.

* **Records are as of the trigger.** Set `asOf: "run"` when the record is as it was when the run started. When your system keeps no history for a kind of record and you can only read it as it is today, send it with `asOf: "now"`. Never send today's copy of a record the run itself changed.
* **Include what bears on the trigger, not everything you have.** Here that is the meeting being moved, what is on Maya's calendar at the new time, and Dana's contact, not every event on Maya's calendar this month.
* **`quote` is a short exact string** to look for in the run's spans, such as a title. Leave it `null` to look for the whole `text`. A record found in a span cites that span, and assertions can rest on it. One found in no span is kept as context the run never saw.
* **`ref` is your own pointer** to the row, such as `Event:88213`, so the record can be traced back to your system.
* **Leave out secrets.** No passwords, API keys, tokens, or full payment card numbers, even when the record holds them.

Send records with `recordsPath` rather than `records` when you can, so they never pass through the conversation. Records go in a save of their own, after the scenario has a trigger.

## Missing items

Missing items are what the run needs and cannot see. They keep assertions from expecting the run to use data it never has.

```json Good theme={null}
[
  "Dana Ortiz's calendar for Wednesday, October 7, 2026"
]
```

```json Not this theme={null}
[
  "Dana has no meetings Wednesday afternoon"
]
```

The bad one is a claim about data, not something missing, and nobody read Dana's calendar to know it. An assertion built on it would expect the assistant to know Dana is free.

Write each item as the thing that was not available, in a few words. Never write that something did not exist because it was not in the records.

## Sources

`sources` says how your records were read: when the run started, when the data was captured, and for each kind of data whether it was read in full as of the run, cut to a time window, read as of capture, or not captured at all. Send it beside the records when your capture knows these things. It stops an assertion from concluding "Maya had nothing else on Wednesday" from a calendar that was only read three days out.

Leave it out rather than guess. A kind with no sources is never treated as read in full.

## What should happen: the assertions

Each outcome in the description becomes an assertion: one claim a replay passes or fails. Draft them with `generate_scenario_assertions`, passing what should happen as the `note`, or write them yourself with `save_trace_assertions`. Either way they wait as pending until someone approves them.

```json Good theme={null}
[
  "The Maya / Dana sync stays on Tuesday, October 6, 2026 at 3pm.",
  "The assistant texts Maya that Wednesday at 3pm clashes with Board prep, which runs from 2:30pm to 4pm.",
  "The assistant offers Maya at least one time on Wednesday, October 7 after 4pm.",
  "The assistant does not email or text Dana Ortiz."
]
```

```json Not this theme={null}
[
  "The assistant moved the meeting to Wednesday at 3pm and texted Maya.",
  "The assistant handles the reschedule correctly.",
  "The assistant calls get_calendar_events before it calls send_sms_message.",
  "The assistant doesn't move the meeting and tells Maya about the conflict and suggests another time."
]
```

The first bad one restates what the original run did, which is not what a replay should do. The second has no line between pass and fail. The third checks a step on the way instead of the outcome: a replay that reads Board prep some other way and texts Maya the right thing fails it, and one that reads the calendar and still moves the meeting passes it. The fourth packs three outcomes into one, so a replay that gets two of them right still fails the whole thing.

* **One outcome per assertion**, stated as what a correct run does.
* **Take every value from the scenario**: the names, dates, times, and record ids in the trigger, history, and records. An assertion can only be checked against what the scenario holds. Never use an id the original run created, such as the id of a message it sent, because a replay creates its own.
* **Require an order only when the outcome depends on it.** "The assistant looks up Ajay Shah before sending the brief" is a good assertion, because the brief has to be written from what the lookup found.
* **Say what must not happen** when that is part of the case, such as not contacting Dana.
* **Check what the run leaves saved** when that is the outcome, not only what it says. If Maya had asked the assistant to stop booking her on Wednesday mornings, the assertion is about the rule it stores: "After the turn, Maya has an active rule saying she takes no meetings on Wednesday mornings."
* **Name a tool only when which tool is used is the outcome.** If the email has to come from the user's own mailbox and not the assistant's, the tool is what a person would check:

```json Good theme={null}
"The email to Dana is drafted in Maya's own Gmail with gmail_draft, not sent from the assistant's address with send_email."
```

When the claim has an edge case a reviewer would argue about, add `passCriteria` or `failCriteria` with the assertion, such as "A calendar read that returns an access error still counts as checking the calendar." When a program can settle the claim by reading the trace (which tools ran, the values passed to them, their order, or an exact phrase in a message), pass `kind: "code"` to `save_trace_assertions` and Bitfab writes code that decides it on every replay without an LLM judge. Leave it as a judged assertion when deciding it means reading what a message means.

## What never goes in a scenario

| Do not include | Why | Where it goes instead |
| - | - | - |
| What the original run did, its tool calls, its reply, or its reasoning | The trace already records it, and a scenario checks every replay, not that one run | The trace |
| Vague expectations ("handles it well", "responds appropriately") | Nothing a replay can pass or fail | A specific outcome in the description and an assertion |
| What should happen, written as a record or history entry | A replay reads records as data it has, so it is handed the answer | The description and the assertions |
| Anything that happened after the trigger arrived: later messages, the customer's complaint, the fix | A replay never sees it | Nowhere |
| Records as they are today when the run changed them | A replay starts from the original run's change instead of the real situation | The record as it was at the run, or leave it out |
| Guesses stated as facts ("Dana is probably free") | Assertions treat them as true | A missing item, if the fact was not available |
| "X did not exist" drawn from data that was not read in full | Absence from a partial read proves nothing | A missing item, and `sources` saying what was read |
| The agent's own prompt, model, settings, or code | They are what an experiment changes; the situation has to stay the same across them | Your code |
| Secrets: passwords, API keys, tokens, full card numbers | The scenario is shown to everyone who reviews it | Nowhere |
| Rows unrelated to the trigger | They hide the records that matter | Nowhere |

## Saving it

For a trace of any function Bitfab does not build triggers for, create the scenario with its name, description, trigger, and data in one call. Send the records in a second call once the trigger is saved, then draft the assertions. Read the scenario back with `get_scenario` before any later edit, so it starts from what is saved.

```json First call: save_scenario theme={null}
{
  "traceId": "4d3c...",
  "name": "Maya asks to move her 3pm with Dana to a time she already has Board prep",
  "description": "Maya texts on Tuesday, October 6 asking to push today's 3pm meeting with Dana Ortiz to the same time tomorrow. Maya has Board prep on Wednesday from 2:30pm to 4pm, and Dana said earlier that she is out Wednesday mornings. The assistant should leave the meeting on Tuesday at 3pm, text Maya that Wednesday at 3pm clashes with Board prep, and offer a Wednesday time after 4pm. It should not contact Dana until Maya picks a time.",
  "trigger": {
    "inputId": null,
    "inputType": "message",
    "channel": "SMS",
    "from": "Maya Chen",
    "to": "Assistant",
    "cc": null,
    "subject": null,
    "type": "reschedule request",
    "eventTime": "2026-10-06T11:42:00-07:00",
    "eventTimeSource": "message date",
    "source": "request",
    "content": "Can you push my 3pm with Dana to tomorrow, same time?",
    "contentPassages": ["Can you push my 3pm with Dana to tomorrow, same time?"],
    "contentIsWholeMessage": true,
    "sources": []
  },
  "data": {
    "hasHistory": true,
    "topic": "Moving Maya's Tuesday 3pm meeting with Dana Ortiz to a Wednesday time Maya has Board prep",
    "history": [
      {
        "role": "fact",
        "kind": "earlier message",
        "quote": "Heads up, I'm out Wednesday mornings for the next month.",
        "summary": "Dana said she is out Wednesday mornings for the next month.",
        "origin": null,
        "sentAt": "2026-10-01T16:05:00-07:00",
        "weekday": "Thursday",
        "sincePrevious": null,
        "source": null
      }
    ],
    "missing": ["Dana Ortiz's calendar for Wednesday, October 7, 2026"]
  }
}
```

```json Second call: save_scenario theme={null}
{
  "traceId": "4d3c...",
  "recordsPath": "scenario-records/4d3c.json"
}
```

The file at `recordsPath` holds the three records from [Records](#records), as an array or as `{ "records": [...] }`.

```json Third call: generate_scenario_assertions theme={null}
{
  "traceId": "4d3c...",
  "note": "The meeting should stay on Tuesday at 3pm. The assistant should text Maya that Wednesday at 3pm clashes with Board prep and offer her a Wednesday time after 4pm. It should not email or text Dana."
}
```

`topic` is one line on what the situation is about. `hasHistory` is `true` when `history` holds entries. To add history or missing items later without resending the data, use `appendData`. See the [`save_scenario` reference](https://docs.bitfab.ai/claude-plugin#save_scenario) for every argument.

## Fixing a scenario someone flagged

When someone marks a scenario **Needs updating**, their note says what is wrong with the scenario itself: a missing message, a wrong trigger, a record that should be there. Fix only what the note names, using the same rules as above, and send `needsUpdatingResolution` in the same call saying what you changed.

```text Good theme={null}
Added the Board prep event on Wednesday, October 7 from 2:30pm to 4pm to the records, from Maya's calendar as of the run.
```

```text Not this theme={null}
Fixed the scenario so the assistant now checks Maya's calendar.
```

A resolution describes the change to the scenario. Changing what the assistant does is a code change, and the scenario is what checks it.

## Before you save

* [ ] The description says what should happen as outcomes a replay can pass or fail.
* [ ] Every assertion can be written from the trigger, history, and records alone.
* [ ] Nothing restates what the original run did. The trace has that.
* [ ] The trigger content is copied word for word, and `eventTime` has its time zone.
* [ ] Every history entry and record existed before the trigger arrived, as it was then.
* [ ] Every record the expected outcome depends on is included, whether or not the run looked at it.
* [ ] What should happen appears only in the description and the assertions, never as a record or history entry.
* [ ] No secrets, and no rows unrelated to the trigger.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.