Skip to main content
The TypeScript, Python, Ruby, and Go SDKs use OpenTelemetry as their internal queueing, batching, and delivery engine. The public Bitfab API does not change: withSpan, decorators, traced methods, manual spans, Start/End, and framework handlers still create Bitfab spans with Bitfab trace IDs. The design below is shared by all four SDKs. Where a name differs: OpenTelemetry is a transport dependency here, not the source of Bitfab’s trace model. This boundary lets the SDK use standard OTel infrastructure without merging an application’s OTel traces into Bitfab traces or exposing OTel types through the Bitfab API.

​
Design invariants

  • Each Bitfab client’s HttpClient lazily owns exactly one private OTel provider and one bounded BatchSpanProcessor.
  • Framework integrations created by a Bitfab client reuse that same HttpClient; creating handlers does not create additional OTel workers.
  • Live and replay traffic use the same pipeline.
  • The private provider does not replace the application’s global OTel provider or export application spans.
  • OTel owns queueing, count-based batching, scheduling, flushing, shutdown, and its batch worker.
  • Bitfab owns logical trace identity, payload validation, persistence, idempotency, and replay readiness.
  • Each carrier is encoded exactly once; a request body is assembled from those encodings rather than re-encoded to measure its size.

​
Components and ownership

​
Carrier spans versus Bitfab spans

Every Bitfab external-span, trace-completion, or call() BAML-trace payload is placed inside an internal OTel carrier span with two attributes:
  • bitfab.operation: external_span, external_trace, or internal_trace
  • bitfab.payload: the JSON-encoded Bitfab payload
Detached trace patches (client.getTrace(id) / client.get_trace(id)) are not carried here. They are trace-API calls, not telemetry, and go straight to PATCH /api/sdk/traces/{id} like getTraceSpan / get_trace_span. The carrier’s OTel trace and span IDs are transport details. A carrier may inherit the process’s ambient OTel context because OTel context propagation is process-wide, but Bitfab ingress does not use that topology to construct the stored trace tree. It extracts bitfab.payload and uses the Bitfab sourceTraceId, span ID, and parent ID inside that payload. This is why an application can have its own OTel trace while recording one or more Bitfab traces safely: the exporter and processor are private, and Bitfab trace grouping comes from the payload rather than the carrier topology. Two Bitfab trace IDs carried inside one ambient OTel trace remain two stored Bitfab traces.

​
Delivery

The flow is:
The exporter treats each OTel export batch as a candidate window, packs its carriers into final HTTP requests bounded by both an eight-carrier limit and the exact encoded request size, and sends up to 32 complete requests concurrently by default. Set BITFAB_OTEL_EXPORT_CONCURRENCY to an integer from 1 through 64 to tune that request concurrency. Invalid values warn once and fall back to 32. Each carrier is encoded once, up front, and its byte count is carried alongside it. Packing sums those counts against the request limit and the body is built by concatenating the encodings, so the largest thing in a request, the bitfab.payload string, is escaped exactly once no matter how many carriers a window holds. This preserves OTel’s bounded queue, batch scheduling, force-flush, shutdown, and single batch worker while restoring the small independent requests that Bitfab’s serverless ingress is designed to scale. Each request retries transient network errors and the responses that mean the server is busy or unreachable rather than unhappy with the payload: 429, 502, 503, 504, and 500. (500 is not in OTLP’s retryable set. Bitfab ingestion answers every unhandled error with it, so a connection blip is indistinguishable from a permanent fault and is worth a second attempt.) Every other status is a verdict that will not change, and is not retried. Malformed carriers can be rejected through the standard OTLP partialSuccess response without affecting sibling requests. A server that sends Retry-After is asking for a specific wait, and that wait is honored exactly rather than shortened. It is refused only when it cannot be served inside the export budget the batch processor enforces, since a wait outliving that budget is killed mid-wait and loses the batch anyway. Half of the remaining budget is reserved for the request itself, so a wait is only taken when what follows it can still carry the send. Absent that instruction, attempts back off exponentially with jitter, so a struggling server is not hit on a fixed cadence and a fleet of clients does not return in lockstep. While a throttle is active it applies to the whole transport, not only the request that was refused. Retrying is safe for spans and trace completions, which ingestion keys idempotently on their source span and trace IDs. A call() BAML trace carries no such key, so a retried batch holding one can create a duplicate trace, and a request that times out client-side may still be persisted server-side. This is a known limitation in every SDK that executes BAML; resolving it needs a client-supplied idempotency key that ingestion dedupes on. Bitfab ingress validates the complete request before processing accepted carriers through a flat bounded worker pool. The pool defaults to eight concurrent carriers and can be changed with BITFAB_OTLP_INGEST_CONCURRENCY, up to a safety ceiling of 32. Carrier processing does not rely on arrival order, matching the existing ingestion contract for independently delivered SDK requests. The endpoint acknowledges only after every accepted carrier has persisted.

​
Batching and payload size

OTel’s processor batches by count and time. The current SDK defaults are a bounded queue of 8,192 carriers and a five-second scheduled export delay. OTel hands up to 512 carriers to one exporter call; the exporter immediately packs that window into count-and-byte-bounded HTTP requests and sends those complete requests concurrently. It does not own a second queue, timer, or retained batch state. Count alone is not safe for Bitfab because captured inputs and outputs vary widely in size. Requests contain at most eight carriers and are packed using the exact encoded OTLP/JSON request size under a target of approximately 3 MB. This leaves headroom beneath Bitfab’s fixed ingress request limit. Set BITFAB_OTEL_MAX_REQUEST_BYTES to a positive integer no greater than 3000000 to use a smaller request target for a proxy with a stricter limit. The value is read when the client’s lazy OTel transport is created. Invalid, non-positive, or larger values warn once and fall back to 3000000; the setting cannot raise the Bitfab-safe ceiling. A carrier above the raw packing target gets its own request. If that request compresses beneath the configured byte target and stays below the 8 MB decompressed ingress ceiling, it is sent intact. Otherwise the carrier takes the trimming fallback below. A 413 Payload Too Large response from ingress still fails the request. Tracing failures never interrupt the host application.

​
The per-span payload budget

The SDK first preserves up to 7,800,000 bytes of a span’s whole payload—its input, output, contexts, prompt, and metadata together—when the resulting single-span request compresses beneath the wire target. The ceiling leaves room under ingress’s 8 MB decompressed request limit. Both ceilings are measured on the carrier, the payload as it is re-escaped into the OTLP attribute, because that is the size the request limit applies to. Measuring the payload itself would not bound anything: the second escaping only has to escape " and \\, but it doubles each one, so backslash-dense content (escaped JSON, Windows paths, regexes) grows up to 2x on the way out. A payload sized to a 2.4 MB raw cap produced a 2.4 MB carrier as prose and a 4.8 MB carrier as backslashes, and the exporter dropped the latter. Measuring the carrier costs nothing on ordinary spans. Escaping can never shrink a body and can at most double it, so a payload under half the budget always fits and one past the budget never does; both are integer comparisons. Only a payload between those bounds is scanned exactly, and that scan counts bytes without allocating a second copy. The cap is on the payload as a whole, not on each value. Capping values independently cannot deliver this guarantee: two values that each fit can still add up to a span that has to be dropped. It also means a single legitimately large value, a document, a long message history, an agent state, may use the entire budget when the rest of the span is small. A payload past 7,800,000 carrier bytes is trimmed immediately. A smaller oversized carrier is tried intact once; if gzip is unavailable or its request still exceeds the wire target, the SDK trims it to the 2,800,000-byte fallback budget and prepares the request again. Trimming replaces the largest fields with <unserializable: too_large_N_bytes> placeholders, largest first, until the encoded payload fits, and never trims fields that identify the span. Each trim is recorded in the payload’s own errors under a payload_budget step, so the trace is flagged as incomplete and not replayed as though it were faithful, and the SDK warns once in the host application’s logs.

​
Compression

A packed request is gzipped before it is sent once its body reaches 8,192 bytes and gzip makes it smaller. Smaller or incompressible requests go uncompressed. Ordinary batches are packed under the raw byte target and compressed exactly once. Only a single carrier above that raw target checks the prepared wire size; it is compressed a second time only when the first result does not fit and the existing trimming fallback must rebuild it. A request already above the 8,000,000-byte decompressed ceiling is trimmed before gzip, because compression cannot make it admissible under that raw limit. The SDKs use their runtimes’ standard compression level; Python is explicitly pinned to level 6 instead of gzip.compress’s CPU-heavier level-9 default. Compression never runs on a thread the host application is waiting on: the TypeScript SDK uses Node’s asynchronous gzip, which runs on the libuv threadpool, and Python, Ruby, and Go release the interpreter lock or compress on their own goroutine. Compression is best-effort: if it fails, the SDK sends the original body rather than dropping the span. Set BITFAB_DISABLE_COMPRESSION to any value to send everything uncompressed. In the browser, the TypeScript SDK uses CompressionStream and falls back to uncompressed where it is unavailable.

​
Burst limits

The queue holds 8,192 carriers, and a span and its trace completion are two carriers. A burst that queues faster than the exporter drains loses the excess: OTel drops the oldest carriers to keep the queue bounded, which is what keeps a tracing backlog from growing into the host application’s memory. A measured example, against a sink that answers immediately: 20,000 spans submitted in 1.4 seconds queued 40,000 carriers, of which 8,704 were delivered. Staying inside the queue’s capacity delivers everything, exactly once. Those drops happen before delivery is attempted, so the global flush still reports success: its verdict covers what reached the exporter, not what the queue had to discard. Treat it as “everything queued was delivered,” and keep bursts under the queue size when every span matters. A second limit is server-side rather than client-side. A carrier that opens a new trace costs substantially more to ingest than one appending a span to an existing trace, so a burst of many independent traces can exceed the export deadline even when the same span count spread across one trace tree would not. Nested workloads are unaffected.

​
Behavior against a slow endpoint

Delivery runs up to 32 concurrent requests, each with a 30-second deadline, and retries a request that times out or returns a retryable status. Against an endpoint slower than that concurrency (a laptop dev server, a constrained self-hosted proxy), requests can exceed the deadline and be re-sent while the first attempt is still being processed. Bitfab ingestion is idempotent per span ID, so the retries cost bandwidth rather than duplicate rows: one measured run re-sent 46% of its carriers and still produced exactly one row per span. Lower BITFAB_OTEL_EXPORT_CONCURRENCY to match what the endpoint can absorb.

​
Replay persistence barrier

Replay is implemented in the TypeScript, Python, and Ruby SDKs; the Go SDK has no replay pipeline, so this section does not apply to it. The SDK tracks each OTel exporter’s result so the global flush reports failure when a batch fails or the deadline expires, instead of reporting only that the processor queue drained. A successful exporter response still does not prove that Bitfab has committed the trace and all of its spans. Replay therefore adds a server-authoritative barrier without adding another transport pipeline:
The submission tracker counts unique source span IDs rather than delivery attempts. This matches the server’s idempotent span key, so a duplicate submission cannot make replay wait for a duplicate database row that should never exist. Direct exporters keep each carrier’s delivery reference with that carrier’s OTel span until encoding. A successful ingestion response acknowledges those references locally, which proves the rows are durable without another server request. A dropped span drops its reference with it; references are never held or evicted independently of their spans. If delivery was not confirmed locally, the authenticated status endpoint remains the authoritative fallback. The existing status endpoint remains backward-compatible:
  • Without expectedSpanCounts, it returns every trace mapping currently associated with the test run.
  • With expectedSpanCounts, it returns a mapping only when that trace has a final status and at least the expected number of persisted spans.
If the barrier times out, replay raises and does not finalize an incomplete test run. This is stronger than relying on OTel force-flush or an exporter acknowledgment.

​
Framework integrations

withSpan, decorators, OpenAI Agents, LangGraph/LangChain, the Claude Agent SDK, and (in TypeScript) the Vercel AI SDK middleware all submit through the owning Bitfab client’s HttpClient and the same TraceTransport abstraction. Closing the client therefore flushes and shuts down the worker used by its instrumented functions and framework integrations together. Handlers instantiated directly, without a Bitfab client, own their HttpClient; call the handler’s close method (or, in Python, use it as a context manager) to release that worker. Replay still flushes all live Bitfab OTel transports in the process so multiple intentional Bitfab clients are covered before server readiness is checked. The LangGraph/LangChain handler keeps langsmith:hidden scheduler callbacks only as local parenting state. It reparents visible children to the nearest visible ancestor and does not submit hidden callbacks through OTel. The server retains its hidden-span filter for older SDK versions. Framework integrations do not create a separate replay exporter or special single-item batches. Ruby and Go ship no framework integrations; every traced method submits through its client’s HttpClient.

​
Lifecycle and failure behavior

  • Construction is lazy: a client that never sends a trace starts no OTel worker.
  • The global flush asks every active Bitfab processor in the process to flush within one deadline and returns false on exporter failure or timeout.
  • The per-client close flushes and permanently shuts down that client’s processor. In Python, with Bitfab(...) as client: closes it on context exit.
  • An exit handler gives remaining transports a bounded shutdown attempt.
  • After fork(), OTel reinitializes its batch worker, queue, and export lock; Bitfab rebuilds its own pipeline in the child, including its exporter.
  • Submissions after shutdown are rejected without rebuilding an unowned pipeline.

​
TypeScript specifics

TypeScript shares the design above, with differences that follow from the runtime:
  • The private provider is constructed with every option supplied explicitly, including its sampler and span limits. Left to its defaults it would read OTEL_* environment variables, so a host application’s global sampler or attribute-length limit could silently drop or truncate Bitfab payloads.
  • Every OTel package the SDK depends on is isomorphic, so the SDK still builds and runs in browsers.
  • A failed flush is a hint, not a verdict. The export deadline can fire while requests are still in flight and the server can still persist every one of them, so replay does not fail on the flush result alone: it polls, and fails only if the server itself never confirms. When both fail, the error names the flush failure as the likely cause. Python raises on the flush result instead.
  • No fork handling. Node.js has no fork() that duplicates the event loop, so there is no post-fork worker to rebuild.
  • Deferred spans are tracked per client. A span recorded through finalize reaches the transport only after its finalize chain settles, because the caller gets the live stream back untouched. That deferred work is tracked per client, so client.close() waits for its own spans without being failed by another client’s slow finalize; flushTraces() remains process-wide and waits for all of it. Python’s finalize runs inline, so it has nothing to track.
  • A beforeExit handler gives remaining transports a bounded shutdown attempt, which is what lets a plain script exit with its spans delivered.

​
Go specifics

Go shares the design above, with differences that follow from the runtime:
  • Adopting OTel raises the SDK’s minimum Go version to 1.25, the floor the OpenTelemetry Go modules declare. Only the OpenTelemetry trace SDK is linked, so protobuf and gRPC stay out of consumers’ builds.
  • The request envelope is derived from an encoded empty request. Go orders map keys when marshalling, so the head and tail are cut from a marshalled request with an empty span list rather than written by hand, keeping an assembled body byte-identical to marshalling the whole request.
  • No fork handling. A Go program that forks does not carry its goroutines into the child, so there is no post-fork worker to rebuild.
  • No automatic last-chance flush. Go has no atexit equivalent, so call defer client.Close(5 * time.Second) in main, or FlushTraces before exit.
  • No replay pipeline. Replay is not implemented in the Go SDK, so the persistence barrier above does not apply to it.
  • Carriers start from a background context rather than the ambient one, and a submission is a non-blocking enqueue: spans and trace completions are queued in the order they occur, and ingress processes carriers idempotently without relying on arrival order.

​
Deployment order

The replay status request is backward-compatible, but the expected-span readiness behavior lives on the server. Deploy the Bitfab web/server change before publishing an SDK version that relies on the stronger barrier. That change shipped with the Python SDK’s adoption, so the SDKs that adopted the transport afterwards needed no further server deploy.

​
Deliberate non-goals

  • The SDK does not install a Bitfab processor or exporter on the application’s global OTel provider.
  • The SDK does not infer Bitfab trace grouping from OTel carrier trace IDs.
  • The SDK does not expose transport acknowledgments as public API or treat them as the sole persistence authority.
See TypeScript SDK, Python SDK, Ruby SDK, and Go SDK for public usage, and HTTP endpoints for the ingestion and replay-status contracts.