Python SDK Reference

Package: bitfab-py (imported as bitfab). Python ≥ 3.10.

Module Exports

from bitfab import (
    BITFAB_PROGRESS_PREFIX,
    AdaptContext,
    AddDatasetGradersResult,
    AddDatasetTracesResult,
    AllowedEnvVars,
    Bitfab,
    BitfabFunction,
    BitfabClaudeAgentHandler,
    BitfabLangChainCallbackHandler,
    BitfabLangGraphCallbackHandler,
    BitfabLangGraphIntegration,
    BitfabOpenAIAgentHandler,
    BitfabTracingProcessor,
    CaptureWhen,
    CapturedSpan,
    CodeChangeFile,
    ConcurrencyPrimitive,
    CurrentSpan,
    CurrentTrace,
    Dataset,
    DatasetGraderRef,
    DatasetTraceIds,
    DatasetsClient,
    DetachedTrace,
    MixedTracingError,
    DbBranchLease,
    DbBranchOptions,
    DbBranchReplayError,
    DbSnapshotRef,
    GraderRerun,
    GraderRerunProgress,
    GraderRerunResult,
    GraderRerunStatus,
    LabelAction,
    LabelConfidence,
    LabelOutcome,
    LabelUpdate,
    LabelsClient,
    MockOverride,
    MockOverrideCtx,
    MockOverrideInput,
    MockOverrideResolver,
    MockValue,
    NO_MOCK_OVERRIDE,
    NodeMatcher,
    OutputTarget,
    RemoveDatasetGradersResult,
    RemoveDatasetTracesResult,
    ReplayBranch,
    ReplayConcurrency,
    ReplayError,
    ReplayItem,
    ReplayItemFinishEvent,
    ReplayItemStartProgress,
    ReplayItemFinishProgress,
    ReplayProgress,
    ReplayRegistration,
    ReplayRegistry,
    ReplayRegistryContext,
    ReplayResult,
    RerunGradersResult,
    SaveAssertion,
    SaveDatasetResult,
    SpanNodeMeta,
    SpanOccurrence,
    SpanTarget,
    SpanType,
    TraceAssertion,
    TraceAssertionSource,
    TraceAssertionsResult,
    TraceAssertionsUpdate,
    TraceTarget,
    TraceTargetOccurrence,
    TracesClient,
    finalizers,               # finalizers.openai_chunks, finalizers.anthropic_events
    flush_traces,
    get_current_replay_branch,
    get_current_span,
    get_current_trace,
    report_replay_progress,
    seed_from_registry,
    serialize_replay_result,
)

All adapter symbols can be imported without installing their framework. The matching optional dependency is only required once the adapter enters that framework’s runtime surface. The OpenAI Agents adapters need openai-agents. The LangGraph and LangChain adapters need langchain-core or langgraph. The Claude Agent SDK adapter needs claude-agent-sdk.

Type Aliases

SpanType = Literal["llm", "agent", "function", "guardrail", "handoff", "custom"]
CaptureWhen = Literal["always", "nested"]
ConcurrencyPrimitive = Literal["async", "process"]
MockStrategy = Literal["none", "all", "marked"]

`class Bitfab`

`init`

Bitfab(
    api_key: str | Callable[[], str | None] | None = None,
    service_url: Optional[str] = None,
    env_vars: Optional[AllowedEnvVars] = None,
    capture_enabled: bool = True,
    baml_client: Any = None,
    strict: bool = False,
    trace_across_threads: bool | None = None,
    enabled: bool | None = None,
)

Param	Type	Default	Description
`api_key`	`str \| Callable[[], str \| None] \| None`	`None`	Resolved lazily at first use. Falls back to `BITFAB_API_KEY` when omitted or empty
`service_url`	`Optional[str]`	`"https://bitfab.ai"`	Base URL
`env_vars`	`AllowedEnvVars`	`{}`	LLM provider keys for `call()`
`capture_enabled`	`bool`	`True`	When `False`, decorated functions run without recording a span. Nothing is sent. Replay still records inside each item. `seed_trace` still records the one call it runs. Decorators always return a wrapper, so `replay(fn)` and the replay registry find the trace function key either way
`baml_client`	`Any`	`None`	Generated BAML client for `wrap_baml()` without explicit client arg
`strict`	`bool`	`False`	Raise on the first traced call when no API key resolves, instead of disabling tracing quietly
`enabled`	`bool \| None`	`None`	Deprecated alias for `capture_enabled`. Warns once. Combines as `capture_enabled = capture_enabled and enabled`. `enabled=False` disables capture even when `capture_enabled=True`
`trace_across_threads`	`bool \| None`	`None`	Nests spans from worker threads and thread pools under the trace that submitted the work, so replay mocks fire on them too. `True` turns this on. `False` turns it off. `None` is the default, and turns it on only when `BITFAB_TRACE_ACROSS_THREADS` is `1`, `true`, or `yes` (case-insensitive). It wraps `ThreadPoolExecutor.submit` (which covers `loop.run_in_executor`) and `threading.Thread.start`. The wrap is process-global once installed, and a later `False` never uninstalls it. It excludes pool-internal worker threads and the SDK’s own OTel transport threads, so a long-lived dispatch loop is never pinned to whichever trace spawned it. It carries only the SDK’s own context. `asyncio.to_thread` passes through unaffected, and it does not reach pre-created queue consumers or other processes. Every span records `span_data.runtime` (`thread_id`, `thread_name`, and `parent_thread_id` when the parent span’s thread is known). Spans dispatched by the wrapper also record `submit_thread_id`/`submit_thread_name`, the thread that called `submit()`/`start()`

Client properties

client.api_key: str | None
client.capture_enabled: bool
client.enabled: bool  # deprecated alias

api_key resolves the configured string or callable. It does not apply the BITFAB_API_KEY fallback, and it never warns. capture_enabled is the effective state, so it resolves the fallback key too, the same way the first traced call would. Under strict=True, reading capture_enabled without a key raises RuntimeError. enabled warns once and returns the same value as capture_enabled.

Transport environment variables

Variable	Default	Description
`BITFAB_OTEL_MAX_REQUEST_BYTES`	`3000000`	Request-size target for OTLP/JSON exports. Accepts positive integers up to `3000000`
`BITFAB_OTEL_EXPORT_CONCURRENCY`	`32`	Concurrent direct requests per export window. Accepts `1` through `64`

Commit ref environment variables

Variable	Effect
`BITFAB_DISABLE_COMMIT_REF`	Set to any value to send no `commit_ref` at all. Neither the platform variables nor `git` are consulted
`BITFAB_COMMIT_SHA`	The commit sent as `commit_ref.sha` on every root trace. Wins over every deploy platform variable and over `git`
`VERCEL_GIT_COMMIT_SHA`, `GITHUB_SHA`, `RAILWAY_GIT_COMMIT_SHA`, `RENDER_GIT_COMMIT`, `SOURCE_VERSION`, `CF_PAGES_COMMIT_SHA`, `CI_COMMIT_SHA`, `BUILD_SOURCEVERSION`, `CIRCLE_SHA1`	Read in that order when `BITFAB_COMMIT_SHA` is unset, together with the same platform’s branch and repository variables when it has them. The first one set wins, and `git` is never run

`span`

def span(
    trace_function_key: str,
    *,
    name: Optional[str] = None,
    type: SpanType = "custom",
    capture_when: CaptureWhen = "always",
    test_run_id: Optional[str] = None,
    mock_on_replay: bool = False,
    finalize: Optional[Callable[[Any], Any]] = None,
) -> Callable[[Callable[P, T]], Callable[P, T]]

Decorator. Wraps the decorated function (sync or async) with a span. Returns: a decorator that returns a function with the same signature. Semantics:

When capture_enabled=False, the wrapper runs the function without recording, except inside a replay item
name defaults to the function’s qualified name: Order.process for a method, process for a plain function or a closure (the enclosing function is dropped, since it is already the parent in the tree). If the function has no name, for example a lambda, it falls back to trace_function_key. The raw __name__ still travels separately as function_name
capture_when="nested" records the span only when another Bitfab span is active. Without a parent, the decorated function runs normally and does not create a root trace. The default is "always". An unknown value warns once and falls back to that default
Nested spans propagate via contextvars.ContextVar. This is safe across asyncio.gather, threads, and sync/async boundaries
Spans exist only where you create them. Nesting between them is automatic, but only among spans that exist. One wrapper around just the outermost function records a single-node trace. See Instrumentation
span() is the opt-in tracing surface. trace() and node() are the opt-out surface. A span() entered beneath an active trace() raises MixedTracingError, a RuntimeError subclass. Configure a function inside a subtree with node() instead
Exceptions are recorded on the span, then re-raised
test_run_id is rarely set directly. replay() injects it through a replay context
finalize records a serializable view of a streaming result as the span output. The raw result is always returned to the caller unchanged. On an async generator, finalize receives the list of yielded chunks. That list is assembled after iteration, once the caller has already streamed every chunk, so this is non-blocking and non-destructive. On a plain sync or async function, finalize receives the return value instead and runs inline before the span is recorded, so a live single-consumer stream is blocked on and consumed. Prefer an async generator for streaming to avoid that. finalize may be async on async or async-generator spans, and must be sync on sync spans. A finalize that raises records an error instead of crashing. Pair it with finalizers.openai_chunks or finalizers.anthropic_events. See Tracing streaming functions

`trace`

Experimental. New API. Behavior may change in a future release. Requires Python 3.12+ for subtree capture.

def trace(
    trace_function_key: str,
    *,
    name: Optional[str] = None,
    type: SpanType = "custom",
    mock_on_replay_default: bool = False,
    max_depth: int = 30,
    max_spans: int = 500,
    exclude: Collection[str] = (),
    include_wrappers: bool = False,
) -> Callable[[Callable[P, T]], Callable[P, T]]

Decorator. Records a span for the decorated function and for every first-party function it calls, at any depth, without those functions being decorated. Returns: a decorator that returns a function with the same signature. Semantics:

The root span behaves exactly as span(). Descendant spans are typed "function". They are named by their qualified name, for example Order.process, with any <locals>. prefix stripped
First-party means the package directory containing the decorated function. That directory is found by walking up while __init__.py exists. Standard library code, site-packages, and Bitfab’s own code never record spans
Capture is scoped to the traced call. sys.monitoring events are installed on entry and removed on exit. Code outside a traced call is unaffected
Lambdas, generator expressions, comprehensions, module-level (<module>) frames, and decorator wrappers are skipped. A decorator wrapper here means a function taking only *args, **kwargs. include_wrappers=True records wrappers instead of skipping them
Skipping is transparent. A skipped wrapper never becomes a parent. The function it wraps parents to the real caller instead
max_depth and max_spans bound a subtree. Exceeding either stops recording, without affecting the call itself. It also logs a one-time warning per traced function, so a truncated trace is not mistaken for code that never ran
Free-threaded builds warn once. Subtree capture is unverified on free-threaded builds. The root span is unaffected either way
self / cls are stripped from recorded inputs. Exceptions are recorded on the failing frame using PY_UNWIND, then re-raised
Sync, async, and async-generator roots are all supported. Suspension does not close a span. A coroutine’s span spans its full lifetime
On Python 3.11 and earlier, the root span is still recorded. A one-time warning explains that descendants were skipped
A nested trace() root starts its own independent trace. Its complete subtree also appears in every outer trace() capture. Each copy has separate span IDs, and the same structure the decorator would record alone. Two active roots double the span volume in the region they share. A node() and the LangGraph integration’s spans keep their configured name, type, test_run_id, and finalized output in every enclosing copy, with finalize running once; framework handler spans attach inside the innermost trace only. The outer trace’s span for the nested root carries nested_trace_id, nested_trace_function_key, and nested_root_span_id, and the nested root span carries enclosing_trace_id, enclosing_span_id, and enclosing_trace_function_key. Inside a replay item or seed_trace a nested root starts no trace of its own; the item’s trace records it as an ordinary descendant
trace() and node() are the opt-out tracing surface. span() is the opt-in surface. A trace() root entered beneath an active span() raises MixedTracingError. The reverse raises too. A span() entered beneath a trace() also raises. The check only runs while tracing is active, meaning capture is on or the call is inside a replay item. The root span that replay("key", fn) wraps around an undecorated callable belongs to neither surface. A trace() root called from that callable nests beneath it instead
Work dispatched to ThreadPoolExecutor / threading.Thread is not captured. An abandoned generator records no span
Async-generator capture is released between yielded items. This holds even when a caller stops consuming without calling aclose()
Automatically discovered descendants are not replay mock targets unless configured with node(mock_on_replay=True). Use node() when a descendant needs trace-owned capture policy
mock_on_replay_default=True makes replay mocking the default for configured node() descendants under mock="marked". A node with mock_on_replay=False overrides it. The trace option defaults to False. Unconfigured descendants remain non-mockable, because interpreter monitoring cannot short-circuit them

`node`

def node(
    *,
    name: Optional[str] = None,
    type: SpanType = "custom",
    capture: bool = True,
    test_run_id: Optional[str] = None,
    mock_on_replay: Optional[bool] = None,
    finalize: Optional[Callable[[Any], Any]] = None,
) -> Callable[[Callable[P, T]], Callable[P, T]]

Trace-only configuration for a function discovered beneath @client.trace. Its options intentionally mirror the applicable span() options. node() never creates a span or trace by itself. It inherits the enclosing trace function key. Semantics:

Outside an active trace() call, the function runs normally with no capture or replay behavior. This does not hold when a span() is active. node() belongs to the opt-out surface, so entering it beneath an opt-in span() raises MixedTracingError
capture=True (default) lets the enclosing trace capture the node once, with the requested name, type, test_run_id, and finalize behavior
capture=False omits the node. Captured descendants attach to its nearest captured parent
mock_on_replay=True makes a captured, configured descendant return its recorded output under the default mock="marked" strategy. None inherits the enclosing trace’s mock_on_replay_default. False overrides that default. Replay mock="all", mock="none", and mock overrides all behave exactly as they do for span()
capture=False with mock_on_replay=True raises ValueError, because an uncaptured node has no recorded output
Sync, async, and async-generator functions follow the corresponding span() behavior. This includes the existing async-generator replay-mocking limitation
Requires the Python 3.12+ subtree capture used by trace(). On earlier versions, the root still records. Configured descendants run normally and untraced

`get_function`

def get_function(trace_function_key: str) -> BitfabFunction

`wrap_baml`

Framework integration → see BAML framework guide for examples.

# Form 1: uses baml_client from constructor
def wrap_baml(
    method: Callable[..., Any],
    *,
    on_collector: Callable[[Any], None] | None = None,
) -> Callable[..., Any]

# Form 2: explicit client
def wrap_baml(
    baml_client: Any,
    method: Callable[..., Any],
    *,
    on_collector: Callable[[Any], None] | None = None,
) -> Callable[..., Any]

Returns: an async wrapper with the same signature as method, plus a .collector attribute holding the most recent call’s BAML Collector (None before the first call or if baml-py is not installed). Pass on_collector to receive the Collector after each call. Raises:

ValueError if form 1 is used without baml_client in constructor
ValueError if the method has no __name__

Semantics:

If baml-py is not installed, the method is called directly without instrumentation
Otherwise calls get_current_span().set_prompt(...) and get_current_span().add_context({...}) with extracted metadata
Must be invoked inside a @span-decorated function for the prompt/context to attach to anything

`get_trace`

def get_trace(trace_id: str) -> DetachedTrace

Returns a DetachedTrace handle for annotating a trace after its root span has closed, from any process or thread. Raises: ValueError if trace_id is not a canonical Bitfab trace ID. Semantics:

All methods on the returned handle are blocking, like get_trace_span. Each one returns only once the server has applied the change
When capture_enabled=False, methods return immediately without sending, except inside a replay item
The server returns 404 if no trace exists with that ID. The method raises rather than logging

`get_trace_span`

def get_trace_span(
    trace_id: str,
    *,
    id: Optional[str] = None,
    name: Optional[str] = None,
    occurrence: Literal["first", "last"] | int = "last",
) -> Optional[CapturedSpan]

Fetches one persisted span without loading its trace. trace_id is the canonical Bitfab trace ID. Exactly one of the span’s Bitfab id or name is required. Numeric occurrences are zero-based in start-time order. Returns None when no trace or span matches. Raises:

ValueError when id and name are both given, or both omitted
ValueError when trace_id or id is not a valid Bitfab ID
ValueError when name is empty or not a string
ValueError when occurrence is not "first", "last", or a non-negative integer. occurrence=True raises too, even though bool is a Python int subtype

`datasets`

client.datasets.save(trace_function_key: str, name: str, description: str | None = None) -> SaveDatasetResult
client.datasets.list(trace_function_key: str | None = None) -> list[Dataset]
client.datasets.get(dataset_id: str) -> Dataset
client.datasets.list_traces(dataset_id: str) -> DatasetTraceIds
client.datasets.add_traces(dataset_id: str, trace_ids: list[str]) -> AddDatasetTracesResult
client.datasets.remove_traces(dataset_id: str, trace_ids: list[str]) -> RemoveDatasetTracesResult
client.datasets.add_graders(dataset_id: str, grader_ids: list[str]) -> AddDatasetGradersResult
client.datasets.remove_graders(dataset_id: str, grader_ids: list[str]) -> RemoveDatasetGradersResult
client.datasets.rerun_graders(dataset_id: str, grader_ids: list[str] | None = None, *, wait: bool = True, timeout: float = 90.0, poll_interval: float = 1.0) -> RerunGradersResult
client.datasets.get_grader_rerun(dataset_id: str, run_id: str | None = None) -> GraderRerun | None

class DatasetGraderRef(TypedDict):
    id: str
    name: str | None

class Dataset(TypedDict):
    id: str
    traceFunctionKey: str
    name: str
    description: str | None
    traceCount: int
    graders: list[DatasetGraderRef]
    createdAt: str
    updatedAt: str

class SaveDatasetResult(TypedDict):
    dataset: Dataset
    created: bool

class DatasetTraceIds(TypedDict):
    datasetId: str
    traceIds: list[str]

class AddDatasetTracesResult(TypedDict):
    dataset: Dataset
    addedTraceIds: list[str]
    alreadyPresentTraceIds: list[str]
    skippedTraceIds: list[str]

class RemoveDatasetTracesResult(TypedDict):
    dataset: Dataset
    removedTraceIds: list[str]
    notPresentTraceIds: list[str]

class AddDatasetGradersResult(TypedDict):
    dataset: Dataset
    addedGraderIds: list[str]
    alreadyAssignedGraderIds: list[str]
    skippedGraderIds: list[str]

class RemoveDatasetGradersResult(TypedDict):
    dataset: Dataset
    removedGraderIds: list[str]
    notAssignedGraderIds: list[str]

GraderRerunStatus = Literal["pending", "running", "completed", "errored"]

class GraderRerun(TypedDict):
    id: str
    status: GraderRerunStatus
    graderIds: list[str]
    progress: GraderRerunProgress | None
    result: GraderRerunResult | None
    error: str | None
    createdAt: str
    updatedAt: str

class RerunGradersResult(TypedDict):
    run: GraderRerun
    joinedExisting: bool

Dataset operations for the authenticated organization, the same operations the Bitfab MCP tools expose to a coding agent. Results are TypedDicts whose keys match the HTTP response (traceCount, addedTraceIds, and so on).

save is an upsert keyed on (trace_function_key, name). created is True for a new dataset, and False when an existing one was updated instead. description=None leaves an existing description untouched.
list takes an optional trace function key. Without it, every dataset in the organization is returned.
Dataset carries id, traceFunctionKey, name, description, traceCount, graders ([{"id", "name"}]), createdAt, and updatedAt.
list_traces returns {"datasetId", "traceIds"}, the same membership a replay with dataset_id selects.
add_traces and add_graders each accept 1 to 100 ids. They report partial acceptance rather than rejecting the whole call. Ids outside the organization, or under another trace function, come back in skippedTraceIds or skippedGraderIds. Ids already present come back in alreadyPresentTraceIds or alreadyAssignedGraderIds.
remove_traces never deletes a trace, only its membership in the dataset. Ids that were not members come back in notPresentTraceIds. remove_graders reports notAssignedGraderIds the same way.
rerun_graders re-scores every trace in the dataset. grader_ids defaults to every assigned grader. An unassigned id rejects the call. It waits up to timeout seconds, polling every poll_interval seconds, and returns the last run seen. Pass wait=False to return as soon as the run is queued instead. A request that matches an in-flight run joins that run, and joinedExisting is True in that case.
get_grader_rerun returns the dataset’s active grader re-run, or the run named by run_id. It returns None when nothing is active, or when the named run does not belong to this dataset.
GraderRerun has status (pending | running | completed | errored), graderIds, progress ({completedTraces, totalTraces, graderCount} while running), result ({tracesGraded, gradersRun} when completed), and error.
A dataset id from another organization raises requests.HTTPError with a 404.

`traces`

client.traces.get_assertions(trace_id: str) -> TraceAssertionsResult
client.traces.save_assertions(trace_id: str, assertions: list[SaveAssertion], source: TraceAssertionSource = "agent") -> list[TraceAssertion]
client.traces.save_assertions_all(updates: list[TraceAssertionsUpdate], source: TraceAssertionSource = "agent") -> list[TraceAssertion]
client.traces.archive_assertions(trace_id: str, assertion_ids: list[str]) -> list[str]

An assertion says what should happen when a trace is replayed. Attach assertions to the ORIGINAL trace. get_assertions returns {"assertions": [...], "inheritedFrom": ...}. Reading a replay that has no assertions of its own returns the nearest ancestor’s instead. That lookup resolves through the replay lineage, and names the ancestor trace in inheritedFrom. Writing assertions onto a replay trace pins them to that one run rather than to the case, so write them on the original trace instead. targetOnEvaluatedTrace narrows what the assertion checks on the trace under evaluation. It is either {"kind": "output"} or a span identified by name and occurrence. Omit it to check the whole trace. Targets are span names, never span ids, because a span id captured on the original resolves to nothing on the replay. Saving with an entry’s id edits that existing assertion, so two callers adding different assertions to one trace never overwrite each other. save_assertions writes one trace. save_assertions_all takes one update per trace and writes them all in a single request, so a publisher covering hundreds of traces makes one call. The singular delegates to it, so both go through the same route.

saved = client.traces.save_assertions_all(
    [
        {"traceId": trace_id, "assertions": [{"assertion": "Departs before 9am"}]}
        for trace_id in trace_ids
    ]
)

The server writes the whole batch in one transaction, so a rejected batch writes nothing and there is no half-saved state to reconcile. Results come back as one flat list covering every trace, each row carrying its own traceId. A batch takes up to 500 traces, up to 50 assertions per trace, and at most 1000 assertions in total. Passing an empty list writes nothing and sends no request.

TraceTargetOccurrence = Literal["first", "last"] | int

TraceAssertionSource = Literal["human", "agent"]

class OutputTarget(TypedDict):
    kind: Literal["output"]

class SpanTarget(TypedDict):
    kind: Literal["span"]
    name: str
    occurrence: NotRequired[TraceTargetOccurrence]

TraceTarget = OutputTarget | SpanTarget

class SaveAssertion(TypedDict):
    id: NotRequired[str]
    assertion: str
    passCriteria: NotRequired[str | None]
    failCriteria: NotRequired[str | None]
    targetOnEvaluatedTrace: NotRequired[TraceTarget | None]

class TraceAssertion(TypedDict):
    id: str
    traceId: str
    assertion: str
    passCriteria: str | None
    failCriteria: str | None
    targetOnEvaluatedTrace: TraceTarget | None
    source: TraceAssertionSource
    createdAt: str
    updatedAt: str

class TraceAssertionsResult(TypedDict):
    assertions: list[TraceAssertion]
    inheritedFrom: str | None

class TraceAssertionsUpdate(TypedDict):
    traceId: str
    assertions: list[SaveAssertion]

`labels`

client.labels.save(label: bool, annotation: str, trace_id: str | None = None, original_trace_id: str | None = None, attempt: int | None = None, confidence: LabelConfidence | None = None, test_run_id: str | None = None, assertion_id: str | None = None) -> LabelOutcome
client.labels.save_all(labels: list[LabelUpdate], test_run_id: str | None = None) -> list[LabelOutcome]
client.labels.skip(trace_id: str | None = None, original_trace_id: str | None = None, attempt: int | None = None, test_run_id: str | None = None, assertion_id: str | None = None) -> LabelOutcome
client.labels.archive(trace_id: str | None = None, original_trace_id: str | None = None, attempt: int | None = None, test_run_id: str | None = None, assertion_id: str | None = None) -> LabelOutcome
client.labels.save_human(label: bool, annotation: str, trace_id: str, confidence: LabelConfidence | None = None, assertion_id: str | None = None) -> HumanLabelOutcome
client.labels.save_human_all(labels: list[HumanLabelUpdate]) -> list[HumanLabelOutcome]
client.labels.get(trace_id: str) -> TraceLabels | None
client.labels.get_all(trace_ids: list[str]) -> list[TraceLabels]

Writes the same pass/fail verdicts the save_agent_labels MCP tool writes. It runs from inside a replay process instead of a coding-agent session. Key a replay verdict by original_trace_id plus the test_run_id it ran under. Add attempt when the experiment ran each trace more than once. Use skip for an attempt that crashed, was punctured, or fell back to a schema default. Also use skip for an assertion whose target could not be resolved. Recording a FAIL in either case would read as a behavior regression rather than a check that never ran. Omit assertion_id and the verdict scores the whole trace. Pass one and the verdict scores that single assertion. A per-assertion verdict and a whole-trace verdict can both sit on the same trace. save_all carries the same field as assertionId on each entry, so one batch can mix both forms. Raises: ValueError from save, skip, and archive unless exactly one of trace_id or original_trace_id is given. save_all takes prebuilt LabelUpdate entries and does not run that check.

LabelConfidence = Literal["VeryLow", "Low", "Medium", "High", "VeryHigh"]
LabelAction = Literal["set", "archived", "no-active-label", "skipped"]

class LabelUpdate(TypedDict):
    traceId: NotRequired[str]
    originalTraceId: NotRequired[str]
    attempt: NotRequired[int]
    assertionId: NotRequired[str]
    label: NotRequired[bool]
    annotation: NotRequired[str]
    confidence: NotRequired[LabelConfidence]
    skip: NotRequired[bool]
    archive: NotRequired[bool]

class LabelOutcome(TypedDict):
    key: str
    traceId: str
    action: LabelAction

save_human and save_human_all write the verdicts save_human_labels writes over MCP. They are validated on write with no approval step, so they satisfy search_traces validated: true immediately. Use them only when a human decided the verdict, such as capturing a known production bug as a regression case. An agent’s own first-pass guesses go through save so they keep the approve-or-edit loop. Approving an existing agent verdict is not on this surface at all: that happens in Studio, by a person. Like save_all, the batch is all-or-nothing. A trace outside the organization, a repeated target, or an assertion_id that is not active on its trace rejects the whole call and writes nothing, so a raised error never leaves part of the batch committed.

class HumanLabelUpdate(TypedDict):
    traceId: str
    assertionId: NotRequired[str]
    label: bool
    annotation: str
    confidence: NotRequired[LabelConfidence]

class HumanLabelOutcome(TypedDict):
    traceId: str
    assertionId: str | None
    label: bool
    action: Literal["set"]

get and get_all read verdicts back. Each trace carries its effective verdict plus one row per scored assertion, keyed by the same assertion_id the write used, so a per-assertion verdict is verifiable per assertion rather than as a passed/failed tally. get returns None when the trace is not in this organization. One get_all call accepts up to 100 ids.

LabelStatus = Literal["labeled", "skipped", "unlabeled"]
LabelSource = Literal["human", "agent"]

class AssertionVerdict(TypedDict):
    assertionId: str
    assertion: str | None
    labelStatus: LabelStatus
    label: bool | None
    annotation: str | None
    confidence: LabelConfidence | None
    labelSource: LabelSource
    approved: bool

class TraceLabels(TypedDict):
    traceId: str
    labelStatus: LabelStatus
    label: bool | None
    annotation: str | None
    approved: bool
    passed: int
    failed: int
    assertions: list[AssertionVerdict]

`graders`

client.graders.get_labels(trace_ids: list[str] | None = None, grader_id: str | None = None, limit: int | None = None) -> list[GraderLabel]

Reads the individual verdicts each automated grader recorded, one row per grader per trace, the same breakdown the get_grader_labels MCP tool returns. Pass trace_ids to see every grader’s verdict on those traces, grader_id to see one grader’s most recent verdicts across traces, or both to narrow. This is the per-grader detail that labels.get does not carry, since that returns one grader-agnostic verdict per trace. Raises: ValueError when neither trace_ids nor grader_id is given.

GraderLabelSource = Literal["human", "live_grader"]

class GraderLabel(TypedDict):
    traceId: str
    graderId: str
    graderName: str | None
    graderStatus: GraderStatus
    label: bool | None
    labelReason: str | None
    failureDiagnostic: str | None
    labelConfidence: LabelConfidence | None
    source: GraderLabelSource
    evaluatedAt: str | None

Under bitfab-replay, this loop runs in one of two hooks. The registry entry’s on_item_finish runs in the process that owns the run, and is the default place for it. ReplayConcurrency’s on_item_finish_in_child_process runs in the child that replayed the item, and is the only option when the judge needs state that run left in memory. Both fire once the attempt’s own trace has been flushed, which is when a lineage-keyed verdict row resolves and when the replay trace ID becomes available. Reading the assertions, judging the replay, and writing the verdict back is the whole loop:

result = client.traces.get_assertions(item.original_trace_id)
verdict = judge(result["assertions"], replay_output)

if verdict is None:
    client.labels.skip(
        original_trace_id=item.original_trace_id,
        attempt=item.attempt,
        test_run_id=run.test_run_id,
    )
else:
    client.labels.save(
        label=verdict.passed,
        annotation=verdict.summary,
        original_trace_id=item.original_trace_id,
        attempt=item.attempt,
        test_run_id=run.test_run_id,
    )

Replay registry and command

registry = ReplayRegistry()
registry.register("checkout", bitfab, run_checkout)
registry.register(
    "agent",
    bitfab,
    run_agent,
    trace_function_key="support-agent",
    mock="marked",
    adapt_inputs=adapt_inputs,
    options_factory=lambda ctx: {
        "mock_override": create_support_agent_mock(ctx.params.get("scenario"))
    },
)

class ReplayRegistry:
    @property
    def names(self) -> tuple[str, ...]: ...  # registered command names, in registration order

    def register(
        self,
        name: str,
        client: Bitfab,
        fn: Callable[..., Any],
        *,
        trace_function_key: str | None = None,
        options_factory: Callable[[ReplayRegistryContext], dict[str, Any]] | None = None,
        **options: Any,
    ) -> ReplayRegistry: ...

    def get(self, name: str) -> ReplayRegistration: ...

ReplayRegistry.register stores the exact production callable, static per-function replay behavior, and an optional options_factory. It returns self, so calls chain. A @span-decorated function supplies its key automatically. A plain handler root passes trace_function_key explicitly instead. **options accepts the same keyword arguments as replay(): limit, trace_ids, name, max_concurrency, code_change_description, code_change_files, experiment_group_id, dataset_id, dataset_ids, grader_ids, only_with_assertions, mock, mock_override, adapt_inputs, db_branch, dry_run, attempts, concurrency, and on_item_finish. The child’s own grading hook is not among them. It is set on the ReplayConcurrency passed as concurrency, since only that object creates a child to run it in. The factory receives JSON values from --params and --param. It can construct executable options such as mock_override. Values passed directly with --param override values loaded from a --params file. Unknown option names are rejected when the registry loads. on_item_finish is the one lifecycle callback a registry entry can set. The command runs its own progress reporter first, then calls yours with the same finished item, which is where a replay verdict is written. It runs in the process that owns the run, so under primitive="process" it runs in the parent, once that item’s child has exited. It is also the only hook that fires for an item whose child died without producing a result. It is skipped under --dry-run, since nothing ran. A callback that raises is reported on stderr as on_item_finish failed and never fails the run. To grade inside the child instead, or in addition, see on_item_finish_in_child_process on ReplayConcurrency. ReplayRegistry.get(name) returns the stored ReplayRegistration, carrying client, fn, trace_function_key, options, and options_factory. .names lists registered command names in registration order. The package installs bitfab-replay. Run it as bitfab-replay --registry <path> <pipeline> [options]. The registry module must define registry. Raises:

ValueError for an empty or duplicate name
ValueError for a non-callable options_factory
ValueError for a non-callable on_item_finish
ValueError for a plain fn with no trace_function_key and no @span key of its own
ValueError for an option name outside the accepted set above

Flag	Value	Effect
`--limit`	`N`	Most traces to select (default 10). It bounds whatever selection is in effect, whether that selector came from the command line or the registry, truncating an ID list and pinning the first N members of the selected datasets ordered by ID
`--trace-ids`	`id1,id2`	Replay exactly these traces. Mutually exclusive with `--dataset-ids`
`--dataset-ids`	`uuid1,uuid2`	Replay the membership of one or more datasets, as their deduped union. Mutually exclusive with `--trace-ids`. `--dataset-id` is the same flag
`--name`	`NAME`	Title for the resulting experiment
`--attempts`	`N`	Replay each selected trace N times in one run (1 to 100, default 1)
`--concurrency`, `--max-concurrency`	`N`	Items in flight at once
`--dry-run`		Resolve every item’s inputs and stop without calling the function
`--experiment-group-id`	`UUID`	Add this run to an existing experiment group
`--grader-ids`	`id1,id2`	Attach graders to this run only
`--only-with-assertions`		Replay only the selected traces that carry an assertion. Narrows `--limit`, `--trace-ids` or `--dataset-ids`. Omitting it leaves a registry-set `only_with_assertions` in place
`--code-change`	`PATH`	Load a code-change description from a file
`--no-code-change`		Record no code change, overriding a registry default
`--mock`	`none\|all\|marked`	Which recorded child spans return their historical output
`--db-branch`		Turn on per-item database branching, keeping a registry-configured `db_branch` mapping (`min_cu`/`max_cu`/`warmup_sql`) if one is already set
`--no-db-branch`		Force per-item database branching off, overriding any registry default
`--seed`	`cases.jsonl`	Run the file’s cases once each and record them instead of replaying. Ignores `--params`/`--param`, because the registration’s `options_factory` never runs during seeding
`--params`	`PATH`	JSON file of values passed to `options_factory`
`--param`	`name=value`	One value passed to `options_factory`. Repeatable, and overrides `--params`
`-h`, `--help`		Print usage

A run whose selection matched no traces exits non-zero rather than reporting a clean run of zero items. --seed <cases.jsonl> runs each case once through the same registration and records the run, instead of replaying. Each line is a JSON object with an input list, plus optional kwargs, metadata, and session_id. A JSON array of those objects works too. A case’s input and kwargs are the call itself, recorded as-is. A case carrying expected is rejected, since the output is what the run produced. The registration already holds the client, the callable, and the trace function key, so the case runs against the exact function the later replay selects. The registration’s adapt_inputs is not run at seed time. It is a replay hook. A replay of the seeded trace applies it then, with the case’s metadata available on ctx.

Replay auto-capture environment variables

Variable	Effect
`BITFAB_CODE_CHANGE_PATH`	Path to a JSON file of `{"description", "files"}`, written by the Bitfab plugin. `replay()` reads it instead of running `git diff` when `code_change_files` is omitted
`BITFAB_CODE_CHANGE_BASE`	Git ref to diff against, instead of searching the default trunk candidates (`origin/HEAD`, `origin/main`, `origin/master`, `main`, `master`)
`BITFAB_DISABLE_CODE_CHANGE_CAPTURE`	Set to opt every replay in this process out of automatic code-change capture
`BITFAB_REPLAY_RESULT_PATH`	When set by the Bitfab plugin, `replay()` writes the full final `ReplayResult` JSON to this path automatically, in addition to its return value

primitive="process" re-execs bitfab-replay --registry once per work item. Each item runs in its own child interpreter. The environment for that interpreter is captured before the registry module’s own import-time side effects run. A child that runs longer than 40 minutes is killed. It is then recorded as a failed replay item. To grade an item inside the child that ran it, set on_item_finish_in_child_process on the ReplayConcurrency.

`register_mock_override`

def register_mock_override(
    match_or_override: str | NodeMatcher | MockOverride | MockOverrideResolver,
    value: Any = ...,
) -> None

Registers an instance-scoped override for every subsequent replay. Accepted forms are register_mock_override(MockOverride(...)), register_mock_override(match, value), register_mock_override(trace_function_key, resolver_or_override), and register_mock_override(resolver). A keyed resolver runs only for child spans with that trace function key. A keyed MockOverride also applies its matcher. A global resolver can route on ctx.node.trace_function_key. Return NO_MOCK_OVERRIDE to continue to lower-priority overrides and the base mock strategy. None remains a valid mocked output. Per-call overrides take precedence over registered overrides. Raises:

ValueError when a keyed call’s second argument is neither a MockOverride nor a callable resolver
ValueError when a MockOverride is passed together with a second value argument
ValueError when a bare, non-callable match_or_override is given with no value

`clear_mock_overrides`

def clear_mock_overrides() -> None

Removes every override registered on this client.

`replay`

def replay(
    fn_or_key: Callable | str | None = None,
    fn: Optional[Callable] = None,  # required when fn_or_key is a key string, or when fn_or_key is omitted (legacy replay(fn=decorated_fn))
    *,
    limit: Optional[int] = None,  # default 5; max 5,000; ignored with trace_ids/datasets
    trace_ids: Optional[list[str]] = None,  # max 100; with a dataset selector it pins which members replay
    name: str | None = None,  # display name for the resulting experiment/test run
    max_concurrency: Optional[int] = 10,
    attempts: int = 1,  # 1-100; replay each trace this many times, attempt-major
    concurrency: ReplayConcurrency | None = None,  # supersedes attempts + max_concurrency
    code_change_description: str | None = ...,
    code_change_files: list[CodeChangeFile] | None = ...,  # omit for rename-aware git diff vs trunk; None disables capture
    mock: MockStrategy = "marked",
    mock_override: MockOverrideInput | list[MockOverrideInput] | None = None,
    adapt_inputs: Callable[[list[Any], dict[str, Any], AdaptContext], tuple[list[Any], dict[str, Any]]] | None = None,
    experiment_group_id: str | None = None,
    db_branch: DbBranchOptions | bool | None = None,  # True requests a DB branch per replay item
    dry_run: bool = False,  # resolve every item's inputs and call nothing
    dataset_id: str | None = None,  # durably attributes the experiment to a dataset
    dataset_ids: list[str] | None = None,  # several datasets: replays their union, attributed to each
    grader_ids: list[str] | None = None,  # graders attached to this run, unioned with the dataset's at grading (max 100)
    only_with_assertions: bool = False,  # narrow the selection to traces carrying an assertion
    on_item_start: Callable[[ReplayItemStartProgress], None] | None = None,
    on_item_finish: Callable[[ReplayItemFinishProgress], None] | None = None,
    on_progress: Callable[[ReplayProgress], None] | None = None,  # deprecated compatibility callback
    process_launcher: ProcessLauncher | None = None,  # internal: wired by bitfab-replay --registry for primitive="process"
) -> ReplayResult

only_with_assertions narrows whatever limit, trace_ids or dataset_ids selected to the traces that carry at least one assertion, so a run measuring assertion outcomes does not pay to re-execute traces nothing can grade. The server applies it as part of selection, which is what makes it compose with limit: limit=10, only_with_assertions=True is the ten most recent traces that HAVE assertions, not the ten most recent filtered down to however few do. An archived assertion does not count, and a selection that narrows to nothing replays nothing (the CLI exits non-zero, as it does for any empty selection). mock_override accepts a MockOverride, a global resolver, or a list of either. The first matching entry whose value is not NO_MOCK_OVERRIDE wins. Per-call overrides run before registered overrides. Both take precedence over mock. on_item_start fires when a worker begins processing an item, before replay setup or customer code runs. It carries type="started", running lifecycle totals, and the historical trace and span identity. Pair it with on_item_finish. on_item_finish fires exactly once per item, as that item finishes. It carries running totals (completed, total, succeeded, errored), test_run_id, and the required finished item. It never represents whole-run completion. The finished item carries these fields:

trace_id: the server replay traces.id, read back off the ingest response and surfaced as the item finishes. Its trace is flushed on finish. trace_id is None only if that flush could not confirm delivery in time
original_trace_id: the original historical trace being replayed. source_trace_id is a deprecated alias for the same value
input, result, original_output, error, trace_error, and replay_error
duration_ms: this replay’s own duration
original_duration_ms, original_tokens, and original_model: describe the trace being replayed
tokens, model, db_snapshot_ref, and db_branch_timings
trace_outline and original_trace_outline: both None at this point, filled in at completion

Replay doesn’t know pass or fail yet. Verdicts are assigned later. The totals only split ran-ok from errored. After each item executes, the SDK flushes the shared OTel pipeline. It finishes from delivery acknowledgments once every carrier is confirmed. Only ambiguous delivery falls back to final-status and expected-span-count polling, before the test run is finalized. A raising lifecycle callback never crashes the run. The deprecated on_progress callback receives the same per-item events. It may additionally receive its legacy, item-less terminal complete event. It is ignored when on_item_finish is also provided. Pass the ready-made report_replay_progress reporter as both on_item_start and on_item_finish. The Bitfab plugin uses its stderr events to identify in-flight traces and to write finished per-item result files. There are two call forms. replay(decorated_fn) reads the trace function key from the @span decorator. It raises if decorated_fn is undecorated. replay("key", fn) takes an explicit key plus any plain callable. The SDK wraps fn in a span under that key internally. This explicit-key form is how handler-instrumented workflows replay: LangGraph, LangChain, the Claude Agent SDK, and OpenAI Agents. Those workflows have no decorated root in the app. A legacy keyword form, replay(fn=decorated_fn), is also preserved. fn_or_key used to be positionally named fn, so this spelling still works. Prefer replay(decorated_fn) in new code. When replay auto-wraps a plain callable, it passes a recorded dict root input, for example a LangGraph state, as a single positional argument. This matches the TypeScript SDK. A decorated function always gets the keyword-args splat instead, whether or not a redundant matching key is also passed. An explicit key that contradicts the decorator’s own key raises. max_concurrency=None means unlimited. max_concurrency=1 means sequential. When trace_ids is passed, limit is ignored, with a warning, because the explicit ID list already determines how many traces replay. attempts (1 to 100, default 1) replays each selected trace that many times inside the same experiment. Attempts run attempt-major. Each attempt is its own replay trace, with its own correlation id, verdict, tokens, and cost. ReplayItem.attempt reports which attempt an item is. concurrency is the options-object form of the same controls: ReplayConcurrency(attempts=..., primitive=..., max_concurrency=...). Passing concurrency alongside the attempts or max_concurrency arguments raises, rather than merging the two.

primitive="async" is the default. It runs every work item as a coroutine in one process. max_concurrency defaults to 10
primitive="process" runs each work item in its own child interpreter. max_concurrency defaults to 4, and None is refused. Only this primitive accepts on_item_finish_in_child_process

Process mode is for a replay target whose world lives in process-global state, such as a settings module, a per-item database, or a module-level registry. Two items cannot share one process in that case. The unit of concurrency is the work item, not the attempt. N traces at K attempts is N x K child processes, all drawn from one queue. Process mode re-execs the replay command once per item. It therefore only runs under bitfab-replay --registry. A bare replay() call raises instead, because it holds a live function object with no re-exec target. process_launcher is the internal seam bitfab-replay --registry uses to supply that re-exec mechanism. Its type is not exported. Calling replay() directly should never pass it. db_branch=True requests a DB branch per replay item, sized however the mirror project is sized. False, or omitting db_branch, leaves branching off. Each bounded worker resolves its own branch, including a separate branch per attempt. get_current_replay_branch() exposes that branch inside fn. db_branch also accepts a mapping to tune the branch, for example db_branch={"min_cu": 2, "max_cu": 2, "warmup_sql": "SELECT 1;"}. Every key in that mapping is optional.

min_cu and max_cu are the compute’s autoscaling floor and ceiling, in Neon Compute Units (0.25 to 56). Equal values pin a fixed size, up to 56, which keeps items comparable. An autoscaling range may not span more than 8 CU, and may not exceed a ceiling of 16 CU
warmup_sql is appended to the branch’s readiness check. It runs before fn sees the branch, so its time is not charged to the replayed call. Invalid warm-up SQL fails the branch, rather than quietly handing back a cold one

Omit db_branch’s keys entirely and the branch keeps the mirror project’s own defaults. Async-generator spans are not mockable. If mock="all", mock="marked" selects one, or a mock_override matches, the item errors before the real generator is iterated. A trace replays only when its root span has serializable inputs. It also replays when it was instrumented through a framework handler, whose recorded root input is serializable. If the original inputs were stubbed as non-serializable at capture time, the trace cannot be replayed. dry_run=True resolves every item’s inputs through selection, deserialization, and adapt_inputs. It then stops without calling fn. Each item reports the exact (args, kwargs) the function would have received. This is the cheap way to check that recorded inputs still fit the current signature. The persistence barrier is skipped, since a run that executed nothing produces no traces.

`seed_trace`

def seed_trace(
    trace_function_key: str,
    fn: Callable,
    *,
    args: Sequence[Any] = (),
    kwargs: Mapping[str, Any] | None = None,
    metadata: Mapping[str, Any] | None = None,
    session_id: str | None = None,
    name: str | None = None,
) -> str

Runs fn once and records the execution as an original trace. It returns the trace ID, for use with replay(trace_ids=[...]). Capture stays off. This records exactly one call, with the same semantics capture-on would give it: a root span, a first-party subtree under the trace decorator’s bounds, and no mocking. The recorded input is the real call. The output is what the run produced. The trace lands under trace_function_key, with ingestion_type: seeded. replay selects it like any other trace. Each replay of it links back as original_trace_id. fn resolves exactly as it does for replay. A decorated function records under its own key. That key must match trace_function_key. A plain callable is wrapped under trace_function_key instead. An exception is recorded on the root span. The trace still persists. The exception is re-raised. A call that records nothing, for example because no API key resolved or fn is a generator, raises instead of returning an ID that replay could never find. metadata is stored on the trace. It is handed to a later replay’s adapt_inputs hook as ctx["metadata"]. This lets a case’s provenance, such as its id, suite, or source row, ride with the trace instead of through the recorded inputs. If the traced function also emits its own trace through an integration, the caller’s metadata is merged onto that export and wins on any shared key, so the provenance recorded here is what replay reads back. name is the trace’s title and a searchable field. Put the case’s own label there, such as a ticket id or a dataset row name, so the seeded trace can be found by it. Call it from synchronous code. An async def fn runs to completion on a fresh event loop. Calling it from inside an already-running loop raises. With trace_across_threads=True, spans from worker threads inside the call nest under the seeded root. A seeded trace carries no database pin. db_branch refuses it as a result. It has a full recorded subtree, though. Replay mocking works on it exactly as it does for a captured trace.

bitfab = Bitfab(capture_enabled=False)

trace_id = bitfab.seed_trace(
    "agent-turn",
    run_ticket,
    kwargs={"ticket_id": "T-1"},
    metadata={"case_uid": "c-1", "suite": "smoke"},
)

bitfab.replay(run_ticket, trace_ids=[trace_id], adapt_inputs=lambda args, kwargs, ctx: (args, kwargs))

`reseed_trace`

def reseed_trace(
    trace_function_key: str,
    fn: Callable,
    *,
    trace_id: str,
) -> ReseedResult

Re-seeds one trace. Reads the trace’s recorded root inputs, name, session, and metadata from Bitfab, runs fn once exactly as seed_trace would, then asks Bitfab to adopt that run under trace_id. Returns {"trace_id", "previous_run_trace_id"}. The trace keeps its id, labels, assertions, dataset membership, name, and metadata. The previous run is kept as its own trace with reseedOfTraceId pointing back at trace_id. Nothing is mocked and no test run is created. fn resolves as it does for replay and seed_trace, and trace_function_key must match the trace’s own key. A run that raises is recorded but not adopted, and the exception is re-raised. Bitfab rejects a run from another function, one that already belongs to a dataset, or a trace that is itself a previous run. Graders on the datasets holding the trace are re-queued, and default replay selection skips previous runs.

`reseed_from_registry`

def reseed_from_registry(
    registry: ReplayRegistry,
    pipeline: str,
    trace_ids: Sequence[str],
) -> dict[str, Any]

Re-seeds each trace through an already-registered pipeline, using the registration’s client, callable, and trace function key. Returns {"pipeline", "traceFunctionKey", "reseeded": [{"traceId", "previousRunTraceId"}]}. The installed bitfab-seed --registry <path> <pipeline> --from-trace <id>[,<id>...] command calls it.

`seed_from_registry`

def seed_from_registry(
    registry: ReplayRegistry,
    pipeline: str,
    cases: Sequence[Mapping[str, Any]],
) -> dict[str, Any]

Runs each case once through an already-registered pipeline and records it. It uses the registration’s client, callable, and trace function key. Each case is a mapping with an input list, plus optional kwargs, metadata, and session_id. input and kwargs are the call itself, recorded as-is. The registration’s adapt_inputs is a replay hook. It is not run at seed time. A seeded trace is therefore never adapted twice. Returns {"pipeline", "traceFunctionKey", "traceIds"}.

`call`

def call(method_name: str, **kwargs: Any) -> Any

Executes a server-configured BAML function locally using env_vars. Raises: ValueError when the function lookup fails. An execution error re-raises whatever exception the BAML call itself raised, not necessarily ValueError.

Framework Integrations

Handlers returned by these methods plug into each framework’s callback, processor, or hook surface. They emit Bitfab spans automatically. They also reuse the owning Bitfab client’s lazy OTel worker, so Bitfab.close() releases the decorator and the framework transport together. Directly constructed LangGraph or Claude handlers are the exception. They own their own transport, and expose close(timeout=30.0) plus a context manager. For usage examples and semantics, see the per-framework guides.

`get_langgraph_callback_handler`

def get_langgraph_callback_handler(trace_function_key: str) -> BitfabLangGraphCallbackHandler

Returns a LangChain/LangGraph BaseCallbackHandler. Pass via config={"callbacks": [handler]} when invoking. The handler-created root is replayable from the framework input. A separate @span root is only needed for meaningful surrounding application work. Retriever calls are captured too, as function-type spans carrying the query and returned documents. See LangGraph framework guide. Aliased as get_langchain_callback_handler(trace_function_key) for plain LangChain projects. The returned handler and its behavior are identical. The handler class is also exported as BitfabLangChainCallbackHandler.

`get_langgraph_integration`

def get_langgraph_integration(
    trace_function_key: str,
    *,
    mock_tools_on_replay: bool | list[str] | tuple[str, ...] = True,
) -> BitfabLangGraphIntegration

integration.create_invoker(graph) -> Callable[..., Any]
integration.create_async_invoker(graph) -> Callable[..., Awaitable[Any]]

Experimental (alpha). Returns the LangGraph integration. Pass wrap_tool_call and awrap_tool_call directly to LangGraph’s native ToolNode constructor. create_invoker(graph) returns a sync callable. It adds callback_handler through LangGraph’s public with_config() API. It preserves invocation-time config and any existing callbacks. It records only the graph input as the replayable root input. Use create_async_invoker(graph) instead for async-only graphs. Use the lower-level callback_handler and wrap_invoke(fn) when meaningful application work around the graph invocation needs to live inside the trace. Under an active trace() subtree, the integration’s tool and invoke spans behave as node() calls instead of opening opt-in spans. This gives them nearest-frame parenting, span budgets, and mock_on_replay_default inheritance. A wrapped graph can therefore run inside opt-out tracing without raising MixedTracingError. Integration-managed tools are marked for replay mocking by default. Recorded ToolMessage and Command results are reconstructed with the current tool-call ID. An expected tool output that is missing during replay fails closed, instead of running the live tool. Calls are matched by tool name and occurrence order. Repeated concurrent calls to the same tool are not yet recommended, as a result. Install the langgraph extra. See the LangGraph framework guide and current limitations.

from langgraph.prebuilt import ToolNode

integration = bitfab.get_langgraph_integration("support-agent")
tools = [lookup_customer, search_docs]
tool_node = ToolNode(
    tools,
    wrap_tool_call=integration.wrap_tool_call,
    awrap_tool_call=integration.awrap_tool_call,
)
graph = build_support_graph(tools=tools, tool_node=tool_node)
run_support_agent = integration.create_invoker(graph)

result = run_support_agent(
    {"messages": [{"role": "user", "content": "Help with order 123"}]},
    config={"configurable": {"thread_id": "customer-456"}},
)

bitfab.replay("support-agent", run_support_agent, limit=10)

`get_openai_tracing_processor`

def get_openai_tracing_processor() -> BitfabOpenAITracingProcessor

Constructing the processor does not require openai-agents. Registering it with agents.add_trace_processor does require it. It captures agent internals. Pair it with get_openai_agent_handler for a replayable root. See OpenAI Agents framework guide.

`get_openai_agent_handler`

def get_openai_agent_handler(trace_function_key: str) -> BitfabOpenAIAgentHandler

Returns a handler. Its wrap_run(agent, input, **run_kwargs) is a drop-in for Runner.run. It records a keyed, replayable root span carrying the run input, and the tracing processor’s spans nest underneath it. For streamed runs, wrap_run_streamed(agent, input, **run_kwargs) is an async-generator drop-in for Runner.run_streamed. Iterate it to consume the same stream events while the run is traced. bitfab.replay() re-runs the non-streaming wrap_run, not this async generator. Both methods skip opening their own root span when already inside an enclosing Bitfab span, such as a replay auto-wrap or the caller’s own @span. In that case they run the call directly, so the tracing processor nests under the existing root instead of doubling it. See OpenAI Agents framework guide.

`get_claude_agent_handler`

def get_claude_agent_handler(trace_function_key: str) -> BitfabClaudeAgentHandler

Returns a handler exposing instrument_options(options), wrap_response(stream, input=...), and wrap_query(stream, input=...) for the Claude Agent SDK. Pass input=prompt to the wrap call to record a replayable root span. When a Bitfab span is already active, no second root opens. The handler’s spans nest under the existing one instead. See Claude Agent SDK framework guide.

`wrap_baml`

See the BAML framework guide for examples. Full signature under wrap_baml above.

`class BitfabFunction`

Returned from client.get_function(key).

def span(
    *,
    name: Optional[str] = None,
    type: SpanType = "custom",
    capture_when: CaptureWhen = "always",
    mock_on_replay: bool = False,
    finalize: Optional[Callable[[Any], Any]] = None,
) -> Callable[[Callable[P, T]], Callable[P, T]]

def get_claude_agent_handler() -> BitfabClaudeAgentHandler

def get_langgraph_callback_handler() -> BitfabLangGraphCallbackHandler

def get_langchain_callback_handler() -> BitfabLangGraphCallbackHandler

def get_langgraph_integration(
    *,
    mock_tools_on_replay: bool | list[str] | tuple[str, ...] = True,
) -> BitfabLangGraphIntegration

def wrap_baml(
    method_or_client: Any,
    method: Optional[Callable] = None,
) -> Callable[..., Any]

Delegates to the parent Bitfab instance, using the bound trace_function_key. The get_*_handler methods reuse that bound key, so a span root and the handler share it. This is the documented same-key nesting pattern, and it needs no repeated string. The experimental get_langgraph_integration() also binds the key. Pass its sync and async hooks directly to LangGraph’s native ToolNode. Then create the graph entry point with create_invoker() or create_async_invoker(). wrap_baml is the exception. It opens no span and uses no key. It enriches the current span instead. Call it inside a function that is already wrapped by this handle’s span.

Module Functions

`finalizers`

def finalizers.openai_chunks(chunks: list[Any]) -> dict[str, Any]
def finalizers.anthropic_events(events: list[Any]) -> dict[str, Any]

Prebuilt finalize= helpers for streaming spans. openai_chunks assembles OpenAI chat-completion chunks into text, finish_reason, usage, and tool_calls. usage is present only when the request set stream_options={"include_usage": True}. anthropic_events assembles Anthropic stream events into text, stop_reason, and usage. Both helpers are duck-typed. Both tolerate malformed or unfamiliar events. Neither requires the provider package at runtime.

`get_current_span()`

def get_current_span() -> CurrentSpan | _NoOpCurrentSpan

Returns a no-op object when called outside a span context. id and trace_id return "". The other methods do nothing. Never raises.

`get_current_trace()`

def get_current_trace() -> CurrentTrace | _NoOpCurrentTrace

Returns a no-op object when called outside a span context.

`get_current_replay_branch()`

def get_current_replay_branch() -> ReplayBranch | None

class ReplayBranch:
    database_url: str            # connection string for this item's branch
    neon_branch_id: str          # the provider's own id for this branch
    env_key: str                 # env var name your app reads, e.g. "DATABASE_URL"
    expires_at: str              # ISO-8601
    snapshot_timestamp: str | None  # ISO-8601, the instant the branch is pinned to
    provider_console_url: str | None
    read_only: bool | None
    region: str | None           # e.g. "aws-us-east-1"
    trace_id: str                # the historical trace this item replays

Call it inside the replayed function to get the branch resolved for the item currently running. It returns None outside a replay item. It also returns None for an item whose source trace carried no DB snapshot reference. That is the fallback path: url = branch.database_url if branch else os.environ["DATABASE_URL"]. The value object is immutable, and scoped to one item. It is built from the replay ContextVar, so concurrent items each see their own branch. Reading database_url marks the trace as having used the branch, reported as accessed. The other attributes inspect the branch without exposing the connection string, and deliberately do not mark it as accessed. repr() redacts the URL. Every field the service puts on the lease is copied onto the branch, under its snake_case name. A field added server-side is therefore readable before you upgrade the SDK. database_url is the sole exception. It is the credential, and the only member that may mark the branch as accessed.

`flush_traces(timeout: float = 30.0)`

def flush_traces(timeout: float = 30.0) -> bool

Forces the private OpenTelemetry batch processor to export pending spans. It also waits for any remaining legacy mutation requests. Together, these wait up to timeout seconds. Returns True when the queued exports completed successfully within that deadline. Returns False when delivery failed, or the flush timed out. Use it before process exit in short-lived scripts.

`Bitfab.close(timeout: float = 30.0)`

Flushes pending requests and permanently shuts down this client’s private OTel transports, within one total deadline. The method is idempotent. It returns False if delivery or shutdown misses the deadline. Bitfab also supports with Bitfab(...) as client:, which calls close() on context exit. Use either form when a long-running process creates transient clients. Shared clients may otherwise remain open until the process-wide exit hook runs.

Classes (Context Handles)

`CurrentSpan`

class CurrentSpan:
    @property
    def id(self) -> str
    @property
    def trace_id(self) -> str
    def add_context(self, context: dict[str, Any]) -> None
    def set_prompt(self, prompt: str) -> None

Method	Semantics
`id`	Canonical Bitfab span ID
`trace_id`	UUID string
`add_context`	Appends the dict as one entry to `span_data.contexts`. Non-dict input ignored. Never raises
`set_prompt`	Overwrites `span_data.prompt`. Non-string input ignored. Never raises

`CurrentTrace`

class CurrentTrace:
    def set_session_id(self, session_id: str) -> None
    def set_name(self, name: str) -> None
    def set_metadata(self, metadata: dict[str, Any]) -> None
    def add_context(self, context: dict[str, Any]) -> None
    def drop(self) -> None

set_session_id groups traces from the same user session. Unlike the other setters, it does not validate that session_id is a non-empty string. set_name sets the trace’s title in Bitfab. This is a searchable and filterable field, stored on the trace’s name column. A trace with no name set is titled by its trace function key instead. Empty strings are ignored. set_metadata shallow-merges with existing metadata, with later keys winning. add_context accumulates entries. drop flags the trace to be dropped. Once flagged, spans that complete afterward are not uploaded at all. The flag rides out on the completion payload. At completion, the server scrubs any payloads that already raced out, meaning the trace itself, its external trace, and its sibling spans. It deletes the archived S3 objects. It marks the trace dropped instead of completed, keeping only a skeleton audit row. drop is a no-op outside a span, and never raises.

`DetachedTrace`

class DetachedTrace:
    trace_id: str   # read-only property
    def add_context(self, context: dict[str, Any]) -> None
    def set_metadata(self, metadata: dict[str, Any]) -> None
    def set_session_id(self, session_id: str) -> None
    def set_name(self, name: str) -> None

Returned by client.get_trace(trace_id), where trace_id is the canonical Bitfab trace ID. Its methods send to the server immediately, and block until it responds. A later read therefore always observes the write. They raise if the server rejects the update. They are silent no-ops if the client is disabled, or if input validation fails. Validation otherwise mirrors CurrentTrace, with one exception. DetachedTrace.set_session_id does validate a non-empty string, unlike CurrentTrace.set_session_id.

TypedDicts & Dataclasses

`AllowedEnvVars`

class AllowedEnvVars(TypedDict, total=False):
    OPENAI_API_KEY: str

`CapturedSpan`

class CapturedSpan(TypedDict):
    id: str
    traceId: str
    parentSpanId: str | None
    name: str | None
    type: str
    input: Any
    output: Any
    contexts: list[dict[str, Any]]
    prompt: str | None
    metadata: dict[str, Any]
    metrics: dict[str, Any] | None
    errors: Any
    startedAt: str | None
    endedAt: str | None

Returned by get_trace_span. SpanOccurrence is Literal["first", "last"] | int.

`ReplayConcurrency`

@dataclass(frozen=True)
class ReplayConcurrency:
    attempts: int = 1
    primitive: ConcurrencyPrimitive = "async"
    max_concurrency: int | None = 10  # effective default; 4 for process mode
    on_item_finish_in_child_process: Callable[[ReplayItemFinishEvent], None] | None = None

attempts must be between 1 and 100. Async mode accepts max_concurrency=None for unlimited work. Process mode requires a positive bound instead. Passing concurrency= together with the legacy attempts= or max_concurrency= parameters raises, rather than choosing one over the other. on_item_finish_in_child_process is the child’s own item-finish hook. It runs in the child interpreter that replayed the item, which is the only place the replayed run’s in-memory state still exists, after that item’s spans are confirmed delivered so item["trace_id"] is in hand and after its result file is written, and before that child exits.

class ReplayItemFinishEvent(TypedDict):
    test_run_id: str
    item: ReplayItem

It carries no running totals, because a child ran one item and cannot count the rest of the run. Both of its keys also appear on ReplayItemFinishProgress, so one callback can serve the registry hook and this one. It does not replace the registry entry’s on_item_finish, and setting both is the expected shape.

	`on_item_finish`	`on_item_finish_in_child_process`
Set on	the registry entry, or `replay()`	`ReplayConcurrency`
Runs in	the process that owns the run	the child that replayed the item
Payload	`ReplayItemFinishProgress`	`ReplayItemFinishEvent`
Running totals	yes	no
Sees the replayed run’s memory	no, under `primitive="process"`	yes
Fires for an item whose child died	yes	no
Available under	every primitive	`primitive="process"` only

Each fires exactly once per item, and the child’s runs first. The parent only learns an item exists once its child has exited, and the child runs its hook before exiting, so a slow judge in the child delays that item’s progress event in the parent and never the reverse. Cross-item state belongs in the parent hook or in the returned ReplayResult, since each child sees only its own item. It is refused under any primitive other than process, since no child process exists to run it in. It is skipped under --dry-run. A callback that raises is reported as on_item_finish_in_child_process failed, naming the trace and attempt, instead of failing the item. The message is written to the child’s stderr and forwarded to the command’s own stderr, so a judge that throws is visible even though a successful item’s child log is never read, and the child writes its result file before running the hook. A child that dies before reaching the hook is still reported to the registry’s on_item_finish in the parent, which is where a crashed attempt is recorded as skipped.

`TokenUsage`

class TokenUsage(TypedDict):
    input: int | None
    output: int | None
    cached: int | None
    total: int | None

The shape of ReplayItem.tokens and ReplayItem.original_tokens, and of their ReplayProgressItem equivalents. Not exported from bitfab. Referenced here only for the field’s type.

`ReplayItem`

class ReplayItem(TypedDict):
    input: list[Any]
    result: Any
    original_output: Any
    error: str | None  # compatible message for either error kind
    trace_error: BaseException | None  # actual exception from the replayed trace
    replay_error: BaseException | None  # actual exception from replay setup
    duration_ms: int | None  # how long THIS replay took
    original_duration_ms: int | None  # the original trace's duration
    original_tokens: TokenUsage | None  # the original trace's token usage
    original_model: str | None  # the original trace's model
    ingestion_type: str | None  # how the source trace came to exist ("captured" or "seeded"); None on older servers, which only served captured traces
    tokens: TokenUsage | None  # the replayed run's usage (compare vs original_tokens)
    model: str | None  # deprecated alias for original_model
    trace_id: str | None
    original_trace_id: str | None
    original_span_id: str | None
    source_trace_id: str | None  # deprecated alias for original_trace_id
    source_span_id: str | None  # deprecated alias for original_span_id
    db_snapshot_ref: dict | None  # the source trace's snapshot pin, if any
    attempt: int  # 0-based attempt index within this experiment
    db_branch_timings: dict | None  # per-phase branch provisioning timings
    trace_outline: dict | None  # the replayed trace's span tree, no payloads
    original_trace_outline: dict | None  # the original trace's span tree, no payloads

trace_outline and original_trace_outline are the replayed and the original trace’s span trees, with no inputs or outputs. They are passed through from the server as-is, in camelCase keys, the same shape as the traceOutlines entries in the HTTP reference. Each trace outline carries traceId, name, status, traceFunctionKey, durationMs, spanCount, and spans. Each span in turn carries spanId, name, type, traceFunctionKey, durationMs, tokens, model, errors, mocked, and children. Both fields are None on progress items, on items whose replay produced no trace (trace_outline only), and against older servers. They exist for grading. Compare the two trees to tell whether a replay reached its output by the same path.

`ReplayResult`

class ReplayResult(TypedDict):
    items: list[ReplayItem]
    test_run_id: str
    test_run_url: str
    attempts: int

`CodeChangeFile`

class CodeChangeFile(TypedDict):
    path: str
    before: str  # "" for a newly created file
    after: str   # "" for a deleted file

One file edited as part of a code change, passed in replay(code_change_files=[...]) or read back from a BITFAB_CODE_CHANGE_PATH file. path is relative to the repo root, or any consistently used root.

`AdaptContext`

class AdaptContext(TypedDict):
    original_trace_id: str | None
    original_span_id: str
    metadata: dict[str, Any]
    source_trace_id: str | None  # deprecated alias for original_trace_id
    source_span_id: str          # deprecated alias for original_span_id

Passed as the third argument to adapt_inputs. original_trace_id is the Bitfab trace ID of the trace being replayed. It lets a table-driven adapter look up per-trace adapted inputs. original_span_id is the external span ID the recorded inputs were read from. metadata is the original trace’s stored metadata, meaning whatever seed_trace or get_current_trace().set_metadata recorded on it. It is empty when the trace carries none. When the traced function also emits its own trace through an integration that exports trace metadata, the caller’s metadata is merged with that export and wins on any shared key, so an integration no longer replaces the provenance the caller stored. Keys the integration set are kept alongside it, so this can carry keys the caller never wrote. Requires v0.52.3 or later. This lets an adapter read a seeded case’s provenance without it being smuggled through the recorded inputs.

Replay lifecycle progress

class ReplayItemStartProgress(TypedDict):
    type: Literal["started"]
    test_run_id: str
    started: int
    completed: int
    total: int
    succeeded: int
    errored: int
    item: dict[str, str | int | None]  # lineage plus 0-based attempt

class ReplayItemFinishProgress(TypedDict):
    test_run_id: str
    completed: int
    total: int
    succeeded: int
    errored: int
    item: ReplayProgressItem

ReplayProgressItem mirrors ReplayItem. It makes the fields that are unavailable during execution optional. ReplayProgress is the deprecated compatibility shape. It additionally supports a terminal type="complete" event, with an optional result.

`DbSnapshotRef`

class DbSnapshotRef(TypedDict, total=False):
    sdkWallClockBeforeFn: str

The snapshot pin attached to every root trace. This is the value ReplayItem.db_snapshot_ref carries. sdkWallClockBeforeFn is the ISO wall-clock timestamp the SDK observed immediately before invoking the wrapped function. The server-side resolver uses that timestamp as the snapshot instant. No provider is captured here. The provider is resolved later, at replay time. It uses camelCase keys, since the ref goes to the server as-is.

`CommitRef`

class CommitRef(TypedDict):
    sha: str
    branch: str | None
    dirty: bool | None
    remote: str | None
    root_sha: str | None

The commit the traced code was running at, sent as commit_ref on every root trace completion. sha is the commit. branch is the checked-out branch, or None when detached or unknown. dirty is True when the working tree had uncommitted or untracked changes, False when it was clean, and None when the SDK could not tell, which is always the case when the ref came from environment variables rather than git. remote is the origin URL reduced to host/owner/repo with any credentials removed, so a CI checkout token never reaches the trace. root_sha is the repository’s first commit, so two checkouts of the same repository match even without a remote. Resolution order is BITFAB_COMMIT_SHA, then the deploy platform’s build variables, then git in the process’s working directory (see Commit ref environment variables). Environment resolution is synchronous and free. The git path runs once per process on a background thread with a two second timeout per command, so it never sits on the thread that ran the traced function. A trace that completes before it lands ships without a commit_ref, and a process with neither variables nor a repository never sends one. The result, including a negative one, is memoized for the life of the process. Set BITFAB_DISABLE_COMMIT_REF to opt the process out entirely.

`DbBranchLease`

class DbBranchLease(TypedDict, total=False):
    neonBranchId: str
    envKey: str
    databaseUrl: str
    expiresAt: str
    snapshotTimestamp: str
    providerConsoleUrl: str
    readOnly: bool
    region: str

The per-item database branch the Bitfab service resolved from the source trace’s db_snapshot_ref. It is carried on the replay context with camelCase keys, because it arrives straight off the wire. neonBranchId is the literal Neon branch id. snapshotTimestamp is the instant the branch was pinned to. It is echoed back in db_snapshot_usage on the replayed trace’s completion. Inside a replayed function, read the branch through get_current_replay_branch() instead of this raw lease. That accessor returns a ReplayBranch.

`DbBranchOptions`

class DbBranchOptions(TypedDict, total=False):
    min_cu: float
    max_cu: float
    warmup_sql: str

Passed as client.replay(db_branch={...}), to tune how each item’s branch is sized and warmed. Every key is optional. db_branch=True branches with the mirror project’s own sizing instead. min_cu is the autoscaling floor, in Neon Compute Units (0.25 to 56). max_cu is the ceiling. Equal to min_cu, it pins a fixed size. Otherwise, a later item can run against an already-scaled-up endpoint and post a better number for the same code. warmup_sql is appended to the branch’s readiness check. It runs before fn sees the branch, so its time is not charged to the replayed call. Invalid SQL fails the branch, rather than silently leaving it cold.

`MockOverride`

@dataclass(frozen=True)
class MockOverride:
    match: NodeMatcher
    value: MockValue

NodeMatcher = Callable[[SpanNodeMeta], bool]
MockOverrideResolver = Callable[[MockOverrideCtx], Any]
MockOverrideInput = Union[MockOverride, MockOverrideResolver]

One (match, value) pair. The first override whose match returns True wins for a given span. A MockOverrideResolver is the keyless form. It runs for every child span. It returns NO_MOCK_OVERRIDE to decline a given span. mock_override accepts either form, or a list of them.

`MockOverrideCtx`

@dataclass
class MockOverrideCtx:
    node: SpanNodeMeta
    inputs: list[Any]
    kwargs: dict[str, Any]
    get_original_output: Callable[[], Any]

Passed to a MockValue callable, and to a resolver. inputs and kwargs are the live positional and keyword arguments passed to the wrapped function on this run. This lets an override compute from what the changed code actually asked for. get_original_output is synchronous. It returns this span’s original recorded output, deserialized. It is memoized for the replay item. It raises if the span has no recorded counterpart in the replayed trace.

`SpanNodeMeta`

@dataclass(frozen=True)
class SpanNodeMeta:
    trace_function_key: str
    span_name: str
    type: str
    original_span_id: str | None = None

The structural identity of a span during replay, passed to a MockOverride match predicate. It carries no output payload. Matching runs on structural metadata only. span_name is the resolved name: the name option, else the function’s qualified name (Order.process for a method), falling back to trace_function_key. original_span_id is this span’s id in the original trace. It is None when the live span has no recorded counterpart, such as a span the changed code newly introduced.

`MockValue`

MockValue = Union[Callable[[MockOverrideCtx], Any], Any]

What an override supplies for a matched span: either a flat value used as-is, or a callable that receives a MockOverrideCtx and returns the value. None is a legitimate mocked output. Returning it substitutes None, rather than declining. Return NO_MOCK_OVERRIDE to decline instead. A trace error means the replayed function started and raised. A replay error means Bitfab could not invoke it at all, for example because database warmup, input loading, or mock preparation failed. If delivery or finalization later fails, replay() raises ReplayError. Its items, test_run_id, test_run_url, and cause preserve the partial result and the original whole-run exception. If database branch resolution fails, replay_error is a DbBranchReplayError. It carries code, the server message, original_trace_id, and an optional cause. Resolver codes such as branch_create_failed, snapshot_from_replaced_origin, invalid_snapshot_ref, seeded_trace_has_no_snapshot, and internal_error remain available in memory, in progress events, in result files, and in ReplayError.items. (seeded_trace_has_no_snapshot means the source was seeded, so it pinned no database instant.) HTTP, timeout, and network failures while requesting a lease use lease_request_failed instead, with the original client exception as cause.

try:
    client.replay(process_document)
except ReplayError as error:
    print(error.cause)
    for item in error.items:
        print(item["original_trace_id"], item["trace_error"] or item["replay_error"])

`serialize_replay_result`

def serialize_replay_result(result: ReplayResult) -> str

Returns indented JSON, while preserving structured fields from trace_error and replay_error. That includes DbBranchReplayError.code, original_trace_id, and nested cause. Use it for direct-run stdout, instead of json.dumps(..., default=str). That alternative reduces exceptions to plain strings.

Error Behavior Summary

Situation	Behavior
Empty `api_key`	Resolved lazily, including a `BITFAB_API_KEY` fallback. Without `strict`, this warns and disables capture. With `strict=True`, the first traced call raises `RuntimeError` instead
Opt-in and opt-out tracing in one call stack	`MixedTracingError`. The inner function does not run
`@span` transport failure	Swallowed. User’s return value / exception passes through
Decorated function raises	Span records `error` and `error_source: "code"`, exception re-raised
`add_context` / `set_prompt` with invalid input	Silently ignored
`call()` cannot find the function	`ValueError` with URL to `/functions`
`call()` finds the function but there’s no prompt	`ValueError` with URL to `/functions/{id}`
`call()` raises during execution	Whatever exception BAML raised, re-raised as-is (not necessarily `ValueError`)
`wrap_baml` missing client	`ValueError`
`replay("key", fn)` with a plain callable (no `@span`)	Auto-wrapped under the key, so spans link to the test run (not an error)
`replay(fn)` with `fn` undecorated and no explicit key	`ValueError` (decorate with `@span`, or pass an explicit key)
`replay("key", fn)` with `fn` decorated under a different key	`ValueError` (key mismatch)
`replay()` fails to deliver or finalize after items settle	`ReplayError` with collected `items`, test-run identifiers, and original `cause`
`replay()`’s requested database branch cannot be resolved	Item `replay_error` is `DbBranchReplayError` with the resolver code, message, and original trace ID
`get_trace_span` with both or neither of `id`/`name`, an invalid ID, an empty `name`, or an invalid `occurrence`	`ValueError`
`register_mock_override` with malformed keyed, paired, or bare-value arguments	`ValueError`
`ReplayRegistry.register()` with an empty or duplicate name, a non-callable `options_factory`, no resolvable trace function key, or an unknown option name	`ValueError`
`ReplayConcurrency` with `attempts` outside 1-100, an unknown `primitive`, `primitive="process"` with `max_concurrency=None`, or a non-callable `on_item_finish_in_child_process`	`ValueError`
`ReplayConcurrency(on_item_finish_in_child_process=...)` under any primitive but `"process"`	`ValueError` (no child process exists to run it in)
`client.labels.save` / `.skip` / `.archive` with neither or both of `trace_id`/`original_trace_id`	`ValueError`
Use a framework adapter without its optional dependency	The adapter fails at the framework boundary with an installation-oriented error. Importing `bitfab` itself remains safe

​Module Exports

​Type Aliases

​class Bitfab

​__init__

​Client properties

​Transport environment variables

​Commit ref environment variables

​span

​trace

​node

​get_function

​wrap_baml

​get_trace

​get_trace_span

​datasets

​traces

​labels

​graders

​Replay registry and command

​Replay auto-capture environment variables

​register_mock_override

​clear_mock_overrides

​replay

​seed_trace

​reseed_trace

​reseed_from_registry

​seed_from_registry

​call

​Framework Integrations

​get_langgraph_callback_handler

​get_langgraph_integration

​get_openai_tracing_processor

​get_openai_agent_handler

​get_claude_agent_handler

​wrap_baml

​class BitfabFunction

​Module Functions

​finalizers

​get_current_span()

​get_current_trace()

​get_current_replay_branch()

​flush_traces(timeout: float = 30.0)

​Bitfab.close(timeout: float = 30.0)

​Classes (Context Handles)

​CurrentSpan

​CurrentTrace

​DetachedTrace

​TypedDicts & Dataclasses

​AllowedEnvVars

​CapturedSpan

​ReplayConcurrency

​TokenUsage

​ReplayItem

​ReplayResult

​CodeChangeFile

​AdaptContext

​Replay lifecycle progress

​DbSnapshotRef

​CommitRef

​DbBranchLease

​DbBranchOptions

​MockOverride

​MockOverrideCtx

​SpanNodeMeta

​MockValue

​serialize_replay_result

​Error Behavior Summary

Module Exports

Type Aliases

`class Bitfab`

`init`

Client properties

Transport environment variables

Commit ref environment variables

`span`

`trace`

`node`

`get_function`

`wrap_baml`

`get_trace`

`get_trace_span`

`datasets`

`traces`

`labels`

`graders`

Replay registry and command

Replay auto-capture environment variables

`register_mock_override`

`clear_mock_overrides`

`replay`

`seed_trace`

`reseed_trace`

`reseed_from_registry`

`seed_from_registry`

`call`

Framework Integrations

`get_langgraph_callback_handler`

`get_langgraph_integration`

`get_openai_tracing_processor`

`get_openai_agent_handler`

`get_claude_agent_handler`

`wrap_baml`

`class BitfabFunction`

Module Functions

`finalizers`

`get_current_span()`

`get_current_trace()`

`get_current_replay_branch()`

`flush_traces(timeout: float = 30.0)`

`Bitfab.close(timeout: float = 30.0)`

Classes (Context Handles)

`CurrentSpan`

`CurrentTrace`

`DetachedTrace`

TypedDicts & Dataclasses

`AllowedEnvVars`

`CapturedSpan`

`ReplayConcurrency`

`TokenUsage`

`ReplayItem`

`ReplayResult`

`CodeChangeFile`

`AdaptContext`

Replay lifecycle progress

`DbSnapshotRef`

`CommitRef`

`DbBranchLease`

`DbBranchOptions`

`MockOverride`

`MockOverrideCtx`

`SpanNodeMeta`

`MockValue`

`serialize_replay_result`

Error Behavior Summary