{serviceUrl}/api/sdk/. This page documents the wire protocol so you can integrate directly without using an SDK, or debug SDK behavior.
Base URL defaults to https://bitfab.ai.
Authentication
All endpoints require the API key in theAuthorization header:
401 Unauthorized.
Request Compression
Request bodies may be sent uncompressed or compressed withgzip (also accepted
as x-gzip) or deflate, declared with the Content-Encoding header. This is
what lets an OpenTelemetry Collector export to Bitfab with its default gzip
compression without extra configuration.
Any other Content-Encoding, including zstd, snappy, and br, returns
415 Unsupported Media Type.
A compressed body that expands beyond 8,000,000 bytes returns
413 Payload Too Large. Bitfab’s SDKs target at most 3,000,000 bytes on the
wire and never send more than 8,000,000 decompressed bytes.
Beneath that, every SDK lets a single span’s whole payload (its input, output,
contexts, prompt, and metadata together) use up to 7,800,000 carrier bytes when
its single-span request gzips below the 3,000,000-byte wire target. The carrier
is the payload re-escaped into its OTLP attribute. If the request does not
compress enough, compression is disabled, or preparing it fails, the SDK
replaces the largest fields with <unserializable: too_large_N_bytes>
placeholders until the carrier fits the 2,800,000-byte fallback budget. See
the OTel architecture for the full behavior.
The SDKs gzip their own requests once a body reaches 8,192 bytes and gzip makes
it smaller. Smaller or incompressible requests are sent uncompressed.
Set BITFAB_DISABLE_COMPRESSION to any value to send everything uncompressed.
Common Request Fields
The SDKs append these fields to most request bodies:Endpoints
POST /api/sdk/functions/lookup
Look up a function by name. Blocking.
Request:
{ "id": null } - SDKs raise a typed error pointing users to /functions.
POST /api/sdk/functions/{functionId}/traces
Record a trace produced by a server-side function execution (e.g. client.call(...)). Fire-and-forget.
Request:
200 OK.
POST /api/sdk/externalSpans
Record a single span produced by withSpan / @span / Span / bitfab_span. Fire-and-forget.
Request:
200 OK.
POST /api/sdk/otel/v1/traces
Record a batch from the TypeScript, Python, Ruby, or Go SDK, or from an OpenTelemetry
Collector, using the standard OTLP/JSON ExportTraceServiceRequest shape.
Those SDKs enrich each OTel span with two string attributes:
bitfab.operation:external_span,external_trace, orinternal_tracebitfab.payload: the JSON-encoded Bitfab replay/rendering payload
externalSpans, externalTraces, and functions/{functionId}/traces. Because
OTLP carriers have no URL to read path parameters from, an internal_trace
carrier holds that route’s body plus a functionId field. Ordinary OTel spans
without those Bitfab attributes are ignored. A fully successful OTLP/JSON
response is {}. If one or more enriched spans are malformed or unsupported,
valid carriers in the same request are still ingested and the response uses the
standard partialSuccess object with rejectedSpans and errorMessage.
Those SDKs keep each request at most around 3 MB even when OTel’s
count-based processor produces a larger candidate batch. Direct requests also
contain at most 128 carriers in TypeScript or eight in Python, Ruby, and Go. Set
BITFAB_OTEL_MAX_REQUEST_BYTES to a positive integer no greater than
3000000 to use a smaller request target. An oversized carrier gets its own
request. The SDK tries compression and, if needed, trims the largest payload
fields to placeholders before reporting an export failure if the carrier still
cannot fit. HTTP 413 responses from ingress are reported as export failures.
See Batching and payload size
for the wire limits and trimming behavior.
When configuring a Collector, set its otlphttp exporter endpoint to
https://bitfab.ai/api/sdk/otel, use JSON encoding, and supply the Bitfab API
key as an Authorization: Bearer ... exporter header. The exporter appends
/v1/traces.
See OpenTelemetry Transport Architecture
for the carrier model, direct and Collector flows, size partitioning, and
replay persistence barrier.
POST /api/sdk/externalTraces
Record the completion of a root span’s trace. Sent once per trace when the outermost span ends. Fire-and-forget.
The request’s top-level id is the canonical Bitfab trace UUID. The nested raw trace ID remains an ingestion correlation value.
Request:
name is the trace’s title: what Bitfab shows for the trace, and a field search_traces and the trace list filter match on. Unset, the trace is titled by its trace function key. workflow_name is accepted as a legacy alias of name (older SDKs sent the trace function key there, and the OpenAI Agents SDK emits it natively); when both are present, name wins.
ingestion_type records how the trace came to exist and is stored on the trace’s ingestion_type column. seeded marks a trace written by seedTrace / seed_trace from a case rather than produced by production traffic. It is read back on replay item selection, and a seeded trace is refused a database branch because it carries no snapshot pin. An absent, unknown, or malformed value resolves to captured, so a trace from an older SDK is never mistaken for a seeded one. rawTrace is accepted as an alias of externalTrace.
GET /api/sdk/sim-plan
Read the organization’s sim plan decisions about content capture, as the SDKs do in the background. Blocking, and safe to call once a minute. The SDKs give the read a five second timeout and send Connection: close.
Response:
save_sim_plan MCP tool) are listed, so every entry carries captureContent false; a node absent from the list has content capture on. A node whose content capture is locked (recorded by a framework integration, the root of its traces, imported from another platform, or named by Bitfab) is never listed, even when a stale decision for it is stored, so the SDKs never strip it. A node is matched by the trace function key of the trace’s root span plus the span’s span_data.name. When a node has captureContent false, the SDK sends the span without span_data.input, input_meta, output, output_meta (and Python’s input_serialized and output_serialized) and sets span_data.content_off_by_simulation_plan to true. The server stores that mark on the span’s content_off_by_simulation_plan column, so a span with no payload because of the sim plan is never counted as a span that recorded nothing. The span tree omits such spans unless they recorded an error; the sim plan page still counts them.
Every span from the TypeScript and Python SDKs also carries a span_origin record beside span_data, holding the SDK name (bitfab.sdk.typescript or bitfab.sdk.python), its version, and instrumentation.name, which says what produced the span: span (withSpan / @span), trace (withTrace / @trace and the subtree beneath it), openai-agents, langgraph, claude-agent-sdk, or vercel-ai (the framework integrations). The server stores instrumentation.name on the span’s instrumentation column and the SDK name and version on its span_origin_name and span_origin_version columns. Spans recorded before this change have all three empty, so they resolve their instrumentation from the trace source at read time and show no SDK name or version. Spans recorded by a framework integration always keep their content, and so does a node that is the root of its traces. The sim plan refuses to turn content off for such a node, and the SDKs never strip them.
GET /api/sdk/traces/{traceId}/span
Fetch one persisted span without loading the full trace. traceId and the optional exact id selector are canonical Bitfab UUIDs. Select exactly one of:
id={canonicalSpanId}for an exact matchname={spanName}with optionaloccurrence=first|last|{zeroBasedIndex}; the default islast
id, traceId, and parentSpanId, plus its name, type, input, output, contexts, prompt, metadata, metrics, errors, and timestamps. It never exposes ingestion source IDs.
PATCH /api/sdk/traces/{traceId}
Update a detached current trace by its canonical Bitfab ID. The body accepts appendContexts, mergeMetadata, setSessionId, and setName. Blocking: SDKs call this synchronously and surface a rejection to the caller.
POST /api/sdk/replay/start
Begin a replay session. Blocking. Timeout: 30 s on the client side.
Request:
name is an optional display name stored on the resulting experiment/test run. limit (1-5,000, default 5) caps how many recent traces are fetched. When traceIds is present (max 100 entries) the ID list determines the count and limit is ignored; the field stays accepted because older SDKs always send a defaulted value. When datasetId or datasetIds (max 50) is present, the datasets’ full trace lists determine the count and limit is ignored. datasetIds selects several datasets in one run: the run replays the union of their traces, deduped where they overlap, is graded by the union of their graders, and is attributed to every one of them, so it appears under each dataset’s experiments. datasetId is the single-dataset spelling and is folded in when both arrive. Every dataset is validated against the organization and the replay’s trace function; a foreign or mismatched ID returns a 400. graderIds (max 100) attaches graders directly to this run, independent of the dataset’s own graders; the resulting experiment is graded by the union of these and the dataset’s runnable graders at completion. Each id must be an active grader in the same organization and trace function, or the replay returns a 400. A replay with no dataset can still carry graders this way.
Response 200:
items[] entry carries reference metrics from the original trace being replayed, under originalDurationMs (end-to-end wall time in integer milliseconds), originalTokens (an object with input/output/cached/total counts), and originalModel (the model id). The unprefixed durationMs, tokens, and model are deprecated aliases carrying the same original-trace values, for SDKs that predate the rename. Any field may be null when the underlying trace didn’t capture it. The replayed run’s own token usage is returned later by /api/sdk/replay/complete (see below), once its spans are persisted.
POST /api/sdk/replay/status
Read a replay test run without finalizing it. The Python and Ruby SDKs use this as a
server-authoritative persistence barrier after flushing its OTel pipeline.
expectedSpanCounts is optional. When supplied, traceIds includes an entry
only after that trace has reached a final status and at least the expected
number of spans is persisted. When omitted, traceIds contains every replay
trace currently associated with the test run, preserving the polling contract
used by older clients.
POST /api/sdk/replay/complete
Signal that a replay session finished.
Request:
traceIds maps each SDK-generated replay trace id to the persisted server trace id. tokens is each replay trace’s token usage, keyed by server trace id and aggregated from the freshly-uploaded replay spans (the same source the experiments view reads), so it’s the replayed run’s cost rather than the original’s; a trace with no token data maps to null. traceCount is how many traces the server persisted for the run. The SDK maps tokens onto each item to populate its replay tokens.
traceOutlines is each replay trace’s outline keyed by server trace id, and originalTraceOutlines is each original trace’s outline keyed by the original trace id (the item’s originalTraceId). A trace outline is the span tree with no inputs or outputs: per span its spanId (the SDK’s span id), name, type (llm, agent, function, guardrail, handoff, or custom), traceFunctionKey, durationMs, tokens, model, errors (a list of { source, error, step? } or null), mocked (true when the span was served from the recorded output instead of re-executing), and children in start order; the trace level carries name, status, traceFunctionKey, durationMs, and spanCount. The SDK maps these onto each item’s traceOutline and originalTraceOutline so graders can compare the path a replay took against the original’s.
GET /api/sdk/externalSpans/{id}
Fetch a specific external span by ID. Used by replay internals. Returns the rawSpan object.
Datasets
Datasets are named buckets of traces scoped to one trace function. Every endpoint below is organization-scoped through the API key. A dataset id from another organization returns a 404. Add and remove calls report partial acceptance in their response instead of failing the whole request: ids the organization does not own, or that belong to a different trace function, come back under askipped list.
The dataset object returned by every endpoint:
GET /api/sdk/datasets
List datasets. Pass ?traceFunctionKey=my-function-key to scope to one function, or omit it
for every dataset in the organization.
Response 200: { "datasets": [ ...dataset ] }
POST /api/sdk/datasets
Create a dataset, or update the one that already carries this name under the same trace
function. The upsert key is (traceFunctionKey, name). On an update, an omitted
description keeps the existing one.
Request:
{ "dataset": { ... }, "created": true }
GET /api/sdk/datasets/{id}
Response 200: { "dataset": { ... } }
GET /api/sdk/datasets/{id}/traces
The ids of every trace in the dataset, which is the same membership a replay with
datasetId selects.
Response 200: { "datasetId": "uuid", "traceIds": ["uuid", "..."] }
POST /api/sdk/datasets/{id}/traces
Add traces to the dataset. Between 1 and 100 ids per call. Traces must belong to the
organization and to the dataset’s trace function; the rest are reported under
skippedTraceIds. Adding a trace that is already a member is a no-op reported under
alreadyPresentTraceIds.
Request: { "traceIds": ["uuid", "..."] }
Response 200:
POST /api/sdk/datasets/{id}/removeTraces
Remove traces from the dataset. The traces themselves are never deleted; only their
membership in this dataset is. Ids that were not members come back under
notPresentTraceIds.
Request: { "traceIds": ["uuid", "..."] }
Response 200: { "dataset": { ... }, "removedTraceIds": ["uuid"], "notPresentTraceIds": [] }
POST /api/sdk/datasets/{id}/graders
Assign graders to the dataset. Graders must belong to the organization and to the
dataset’s trace function; the rest come back under skippedGraderIds. Already-assigned
graders are reported under alreadyAssignedGraderIds.
Request: { "graderIds": ["uuid", "..."] }
Response 200:
POST /api/sdk/datasets/{id}/removeGraders
Request: { "graderIds": ["uuid", "..."] }
Response 200: { "dataset": { ... }, "removedGraderIds": ["uuid"], "notAssignedGraderIds": [] }
POST /api/sdk/datasets/{id}/rerunGraders
Re-run graders over every trace in the dataset. graderIds is optional and defaults to
every grader assigned to the dataset; a grader that is not assigned returns a 400. A dataset
with no graders assigned returns a 400. Only one re-run can be active per dataset: a request
that matches the active run’s graders joins it (joinedExisting: true), while a different
selection returns a 400 until the active run finishes. The call returns as soon as the run
is queued; poll the GET below for progress.
Request: { "graderIds": ["uuid", "..."] } (optional body)
Response 200:
GET /api/sdk/datasets/{id}/rerunGraders
The dataset’s active re-run, or the run named by ?runId=. status is one of pending,
running, completed, or errored. While running, progress carries
{ completedTraces, totalTraces, graderCount }. When completed, result carries
{ tracesGraded, gradersRun }. When errored, error carries the message.
Response 200: { "run": { ... } } or { "run": null } when nothing is active.
Assertions
An assertion says what a trace SHOULD do the next time it is replayed, as opposed to a label, which is a verdict on a run that already happened. Every endpoint is organization-scoped through the API key, and a trace from another organization returns a 404. Write assertions against the ORIGINAL trace. A replay that carries none of its own reads its original’s through the replay lineage, and the response names the original ininheritedFrom.
The assertion object returned by every endpoint:
targetOnEvaluatedTrace names what on the trace under evaluation the assertion checks. It
is null for the whole trace, { "kind": "output" } for the final output, or
{ "kind": "span", "name": "search_flights", "occurrence": "last" } for one span.
occurrence accepts "first", "last" (the default), or a 0-based index. Targets are span
names, never span ids, because an id captured on the original resolves to nothing on the
replay. An assertion whose target cannot be found on the trace being evaluated is errored,
never passed.
GET /api/sdk/traces/{traceId}/assertions
Read a trace’s assertions. Returns { "assertions": [...], "inheritedFrom": null }, or
the original trace’s id in inheritedFrom when this trace is a replay reading its
original’s.
POST /api/sdk/traces/assertions
Create or edit assertions across one or many traces. A call carries 1 to 500 traces, 1 to
50 assertions per trace, and at most 1000 assertions in total.
{ "assertions": [...] }, one flat list covering every trace in the batch, each
row carrying its own traceId.
Pass an entry’s id to edit an existing assertion and omit it to add a new one, so two
callers adding different assertions to one trace never overwrite each other. On an edit,
an omitted field is preserved and an explicit null clears it. An edit keeps the original
author and source. source defaults to "agent". The author is recorded from the API
key’s user and is required, so every assertion has one.
The whole batch is written in one transaction. A trace the organization does not own, an
id that names an assertion on a different trace, or a total over the 1000-assertion cap
rejects the request and writes nothing.
POST /api/sdk/traces/{traceId}/assertions/archive
Archive assertions by id. Archiving hides them from every read and keeps the rows for
audit; it never deletes a trace or an assertion outright.
{ "archived": ["uuid"] }.
POST /api/sdk/traces/labels
Record pass/fail verdicts, 1 to 200 per call. These are the same writes the
save_agent_labels MCP tool performs, reachable from a replay process rather than a
coding-agent session.
traceId. Key a replay verdict by originalTraceId plus the
top-level testRunId, adding attempt when the experiment ran each trace more than once;
the server resolves it to the replay trace through the lineage, so a caller never needs a
server-generated replay trace id. { "skip": true } withholds a verdict and
{ "archive": true } clears a previous one. Use skip for an attempt that crashed, was
punctured, or fell back to a schema default, and for an assertion whose target could not
be resolved, where a FAIL would read as a behavior regression rather than a check that never
ran.
Returns { "labels": [{ "key": "uuid", "traceId": "uuid", "action": "set" }] }, where key
echoes the id you addressed the verdict by and action is one of set, skipped,
archived, or no-active-label.
GET /api/sdk/traces/labels
Read verdicts back. Pass traceIds as a comma-separated list, 1 to 100 per call.
label, annotation, and approved describe the trace’s effective verdict. assertions
carries one entry per scored assertion, keyed by the same assertionId the write used, so
a verdict written per assertion is readable per assertion rather than only as the
passed / failed tally. A trace scored only as a whole returns an empty assertions.
POST /api/sdk/traces/labels/human
Record human-authored verdicts, 1 to 200 per call. These are the same writes the
save_human_labels MCP tool performs. Unlike POST /api/sdk/traces/labels, which writes
agent suggestions that start unapproved, these are validated on write and satisfy
search_traces validated: true immediately.
POST /api/sdk/traces/labels so they keep the approve-or-edit loop. Approving an existing
agent verdict is not available on any programmatic surface, by design: it happens in Studio.
The call is all-or-nothing, matching POST /api/sdk/traces/labels. A trace id outside the
organization, the same target twice in one batch, or an assertionId that is not active on
its trace fails the whole request with a 400 and writes nothing.
Returns { "labels": [{ "traceId": "uuid", "assertionId": null, "label": true, "action": "set" }] }.
GET /api/sdk/graderLabels
Read the individual verdicts each automated grader recorded, one row per grader per trace.
The same breakdown the get_grader_labels MCP tool returns.
traceIds (1 to 100, comma separated) to see every grader’s verdict on those traces,
graderId to see one grader’s most recent verdicts across traces (newest first), or both to
narrow. Passing neither is a 400. limit applies only when reading by grader alone, and
defaults to 50 with a maximum of 200.
Returns { "labels": [{ "traceId": "uuid", "graderId": "uuid", "graderName": "...", "graderStatus": "active", "label": false, "labelReason": "...", "failureDiagnostic": "...", "labelConfidence": "High", "source": "live_grader", "evaluatedAt": "..." }] }. source is human for a verdict a person recorded on the grader and live_grader for one a grader run produced; it is a different enum from the labelSource on trace labels.
Serialization
- Inputs and outputs in
span_data.input/span_data.outputare serialized using superjson by the TypeScript SDK. Theinput_meta/output_metafields are the superjson meta descriptors. - Python, Ruby, and Go SDKs use JSON-compatible fallbacks. Objects that don’t round-trip through JSON are coerced to string via the SDK’s
serializehelper.
Error Responses
SDKs swallow all non-2xx responses on fire-and-forget endpoints and log to stderr. Blocking endpoints (
lookup, replay/start) raise typed errors.