> ## Documentation Index > Fetch the complete documentation index at: https://docs.bitfab.ai/llms.txt > Use this file to discover all available pages before exploring further. # OpenTelemetry Transport Architecture > How the TypeScript, Python, Ruby, and Go SDKs use OpenTelemetry for batching and delivery while preserving Bitfab trace identity and replay correctness The TypeScript, Python, Ruby, and Go SDKs use OpenTelemetry as their internal queueing, batching, and delivery engine. The public Bitfab API does not change: `withSpan`, decorators, traced methods, manual spans, `Start`/`End`, and framework handlers still create Bitfab spans with Bitfab trace IDs. The design below is shared by all four SDKs. Where a name differs: | Concept | TypeScript | Python | Ruby | Go | | -------------------------- | ------------------------------------------------------------------------ | ---------------------------------------------------- | ----------------------------------------------- | ---------------------------------------- | | Transport seam | `TraceTransport` | `bitfab.transport` | `Bitfab::Transport` | `traceTransport` | | Per-client owner | `HttpClient` | `HttpClient` | `Bitfab::HttpClient` | `httpClient` | | Submit entry points | `sendExternalSpan` / `sendExternalTrace` | `send_external_span` / `send_external_trace` | `send_external_span` / `send_external_trace` | `sendExternalSpan` / `sendExternalTrace` | | Global flush | `flushTraces(timeoutMs)` | `flush_traces(timeout)` | `Bitfab.flush_traces(timeout:)` | `FlushTraces(timeout)` | | Per-client close | `client.close(timeoutMs)` | `client.close()` or `with Bitfab(...) as client:` | `client.close(timeout:)` | `Close(timeout)` | | Private provider type | `BasicTracerProvider` | `TracerProvider` | `TracerProvider` | `TracerProvider` | | Collector exporter package | `@opentelemetry/exporter-trace-otlp-proto` (installed, loaded on demand) | `opentelemetry-exporter-otlp-proto-http` (installed) | `opentelemetry-exporter-otlp` (add it yourself) | `otlptracehttp` (always linked) | OpenTelemetry is a transport dependency here, not the source of Bitfab's trace model. This boundary lets the SDK use standard OTel infrastructure without merging an application's OTel traces into Bitfab traces or exposing OTel types through the Bitfab API. ## Design invariants * Each `Bitfab` client's `HttpClient` lazily owns exactly one private OTel provider and one bounded `BatchSpanProcessor`. * Framework integrations created by a `Bitfab` client reuse that same `HttpClient`; creating handlers does not create additional OTel workers. * Live and replay traffic use the same pipeline. * A configured Collector replaces direct delivery; the SDK never mirrors the same traffic through both routes. * The private provider does not replace the application's global OTel provider or export application spans. * OTel owns queueing, count-based batching, scheduling, flushing, shutdown, and its batch worker. * Bitfab owns logical trace identity, payload validation, persistence, idempotency, and replay readiness. * Collector acceptance is not treated as proof of Bitfab persistence. ## Components and ownership | Component | Owns | Does not own | | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------------- | | `withSpan`, decorators, traced methods, and framework handlers | Logical spans, trace IDs, parent relationships, replay metadata, serialized inputs and outputs | Network queues or exporter workers | | `TraceTransport` | The small boundary used by every instrumentation path: submit, flush, and shutdown | Bitfab trace semantics | | Private provider + `BatchSpanProcessor` | Bounded queue, batch worker, count threshold, schedule delay, force-flush, shutdown | Application OTel spans or the global provider | | Direct Bitfab exporter | OTLP/JSON encoding, count-and-byte-bounded request packing, retries, and bounded concurrent requests | Replay finalization | | Official OTel OTLP/HTTP exporter | Protobuf delivery from the SDK to a configured Collector | Confirmation that Bitfab persisted the data | | Bitfab OTLP ingress | Carrier extraction, schema validation, idempotent span/trace service dispatch, OTLP `partialSuccess` | SDK queueing | | Replay status service | Final trace status and expected persisted-span-count barrier | Transport scheduling | ## Carrier spans versus Bitfab spans Every Bitfab external-span, trace-completion, or `call()` BAML-trace payload is placed inside an internal OTel carrier span with two attributes: * `bitfab.operation`: `external_span`, `external_trace`, or `internal_trace` * `bitfab.payload`: the JSON-encoded Bitfab payload Detached trace patches (`client.getTrace(id)` / `client.get_trace(id)`) are not carried here. They are trace-API calls, not telemetry, and go straight to `PATCH /api/sdk/traces/{id}` like `getTraceSpan` / `get_trace_span`. The carrier's OTel trace and span IDs are transport details. A carrier may inherit the process's ambient OTel context because OTel context propagation is process-wide, but Bitfab ingress does not use that topology to construct the stored trace tree. It extracts `bitfab.payload` and uses the Bitfab `sourceTraceId`, span ID, and parent ID inside that payload. This is why an application can have its own OTel trace while recording one or more Bitfab traces safely: the exporter and processor are private, and Bitfab trace grouping comes from the payload rather than the carrier topology. Two Bitfab trace IDs carried inside one ambient OTel trace remain two stored Bitfab traces. ## Direct delivery Without `BITFAB_OTEL_EXPORTER_ENDPOINT`, the flow is: ```text theme={null} withSpan / decorator / traced method / framework handler → HttpClient.sendExternalSpan / sendExternalTrace / sendInternalTrace → private OTel BatchSpanProcessor → Bitfab OTLP/JSON exporter → POST /api/sdk/otel/v1/traces → existing external-span / external-trace / internal-trace services → Postgres ``` The exporter treats each OTel export batch as a candidate window, packs its carriers into final HTTP requests bounded by both an eight-carrier limit and the exact encoded request size, and sends up to 32 complete requests concurrently by default. Set `BITFAB_OTEL_EXPORT_CONCURRENCY` to an integer from `1` through `64` to tune that direct-request concurrency. Invalid values warn once and fall back to `32`. This preserves OTel's bounded queue, batch scheduling, force-flush, shutdown, and single batch worker while restoring the small independent requests that Bitfab's serverless ingress is designed to scale. Each request retries transient network errors and `408`, `425`, `429`, and `5xx` responses. Permanent errors are not retried. Malformed carriers can be rejected through the standard OTLP `partialSuccess` response without affecting sibling requests. Retrying is safe for spans and trace completions, which ingestion keys idempotently on their source span and trace IDs. A `call()` BAML trace carries no such key, so a retried batch holding one can create a duplicate trace, and a request that times out client-side may still be persisted server-side. This is a known limitation in every SDK that executes BAML; resolving it needs a client-supplied idempotency key that ingestion dedupes on. Bitfab ingress validates the complete request before processing accepted carriers through a flat bounded worker pool. The pool defaults to eight concurrent carriers and can be changed with `BITFAB_OTLP_INGEST_CONCURRENCY`, up to a safety ceiling of 32. Carrier processing does not rely on arrival order, matching the existing ingestion contract for independently delivered SDK requests. The endpoint acknowledges only after every accepted carrier has persisted. ## Collector delivery Set `BITFAB_OTEL_EXPORTER_ENDPOINT` to an OTLP/HTTP receiver base URL such as `http://localhost:4318`. The SDK appends `/v1/traces` and uses OTel's official OTLP/HTTP protobuf exporter. Ruby applications add the `opentelemetry-exporter-otlp` gem to enable this route; without it the SDK warns once and keeps delivering directly to Bitfab. The other SDKs install the exporter for you, so a configured Collector can never fail for want of the package: ```text theme={null} withSpan / decorator / traced method / framework handler → HttpClient.sendExternalSpan / sendExternalTrace / sendInternalTrace → private OTel BatchSpanProcessor → official OTLP/HTTP protobuf exporter → Collector receiver → Collector otlphttp/bitfab exporter → POST /api/sdk/otel/v1/traces → existing external-span / external-trace services → Postgres ``` Configure the Collector's Bitfab exporter with JSON encoding and the Bitfab API key: ```yaml theme={null} exporters: otlphttp/bitfab: endpoint: https://bitfab.ai/api/sdk/otel encoding: json headers: Authorization: Bearer ${env:BITFAB_API_KEY} ``` The Collector and direct modes are mutually exclusive. Configuring a Collector does not create a second copy of each Bitfab span. ## Batching and payload size OTel's processor batches by count and time. The current SDK defaults are a bounded queue of 8,192 carriers and a five-second scheduled export delay. In direct mode, OTel hands up to 512 carriers to one exporter call; the direct exporter immediately packs that window into count-and-byte-bounded HTTP requests and sends those complete requests concurrently. It does not own a second queue, timer, or retained batch state. In Collector mode, OTel hands at most 32 carriers to the official OTLP/HTTP protobuf exporter. A Collector can deliberately rebatch downstream when its final destination supports a different limit. Count alone is not safe for Bitfab because captured inputs and outputs vary widely in size. Collector protobuf exports are also partitioned into requests of at most approximately 3 MB. Direct requests contain at most eight carriers and are packed using the exact encoded OTLP/JSON request size under that same target. This leaves headroom beneath Bitfab's fixed ingress request limit. Set `BITFAB_OTEL_MAX_REQUEST_BYTES` to a positive integer no greater than `3000000` to use a smaller request target for a Collector or proxy with a stricter limit. The value is read when the client's lazy OTel transport is created. Invalid, non-positive, or larger values warn once and fall back to `3000000`; the setting cannot raise the Bitfab-safe ceiling. For direct delivery, a carrier that exceeds the limit by itself is rejected before sending. A `413 Payload Too Large` response from ingress also fails the request. Tracing failures never interrupt the host application. ## Burst limits The queue holds 8,192 carriers, and a span and its trace completion are two carriers. A burst that queues faster than the exporter drains loses the excess: OTel drops the oldest carriers to keep the queue bounded, which is what keeps a tracing backlog from growing into the host application's memory. A measured example, against a sink that answers immediately: 20,000 spans submitted in 1.4 seconds queued 40,000 carriers, of which 8,704 were delivered. Staying inside the queue's capacity delivers everything, exactly once. Those drops happen before delivery is attempted, so the global flush still reports success: its verdict covers what reached the exporter, not what the queue had to discard. Treat it as "everything queued was delivered," and keep bursts under the queue size when every span matters. A second limit is server-side rather than client-side. A carrier that opens a new trace costs substantially more to ingest than one appending a span to an existing trace, so a burst of many independent traces can exceed the export deadline even when the same span count spread across one trace tree would not. Nested workloads are unaffected. ## Behavior against a slow endpoint Direct delivery runs up to 32 concurrent requests, each with a 30-second deadline, and retries a request that times out or returns a retryable status. Against an endpoint slower than that concurrency (a laptop dev server, a constrained self-hosted proxy), requests can exceed the deadline and be re-sent while the first attempt is still being processed. Bitfab ingestion is idempotent per span ID, so the retries cost bandwidth rather than duplicate rows: one measured run re-sent 46% of its carriers and still produced exactly one row per span. Lower `BITFAB_OTEL_EXPORT_CONCURRENCY` to match what the endpoint can absorb. ## Replay persistence barrier Replay is implemented in the TypeScript, Python, and Ruby SDKs; the Go SDK has no replay pipeline, so this section does not apply to it. The SDK tracks each OTel exporter's result so the global flush reports failure when a batch fails or the deadline expires, instead of reporting only that the processor queue drained. In Collector mode, a successful exporter response still means the Collector accepted the carriers; it does not prove that Bitfab has committed the trace and all of its spans. Replay therefore adds a server-authoritative barrier without adding another transport pipeline: ```text theme={null} Run replay items concurrently → submit every span and trace completion through normal OTel pipelines → collect each replay trace's unique submitted span IDs → globally force-flush live Bitfab OTel transports → poll POST /api/sdk/replay/status with expectedSpanCounts → wait until every trace is final and its persisted span count is sufficient → POST /api/sdk/replay/complete ``` The submission tracker counts unique source span IDs rather than delivery attempts. This matches the server's idempotent span key, so a duplicate submission cannot make replay wait for a duplicate database row that should never exist. The existing status endpoint remains backward-compatible: * Without `expectedSpanCounts`, it returns every trace mapping currently associated with the test run. * With `expectedSpanCounts`, it returns a mapping only when that trace has a final status and at least the expected number of persisted spans. If the barrier times out, replay raises and does not finalize an incomplete test run. This is stronger than relying on OTel force-flush or a Collector acknowledgment. ## Framework integrations `withSpan`, decorators, OpenAI Agents, LangGraph/LangChain, the Claude Agent SDK, and (in TypeScript) the Vercel AI SDK middleware all submit through the owning `Bitfab` client's `HttpClient` and the same `TraceTransport` abstraction. Closing the client therefore flushes and shuts down the worker used by its instrumented functions and framework integrations together. Handlers instantiated directly, without a `Bitfab` client, own their `HttpClient`; call the handler's close method (or, in Python, use it as a context manager) to release that worker. Replay still flushes all live Bitfab OTel transports in the process so multiple intentional `Bitfab` clients are covered before server readiness is checked. The LangGraph/LangChain handler keeps `langsmith:hidden` scheduler callbacks only as local parenting state. It reparents visible children to the nearest visible ancestor and does not submit hidden callbacks through OTel. The server retains its hidden-span filter for older SDK versions. Framework integrations do not create a separate replay exporter or special single-item batches. Ruby and Go ship no framework integrations; every traced method submits through its client's `HttpClient`. ## Lifecycle and failure behavior * Construction is lazy: a client that never sends a trace starts no OTel worker. * The global flush asks every active Bitfab processor in the process to flush within one deadline and returns false on exporter failure or timeout. It does not claim Bitfab persistence when delivery goes through a Collector. * The per-client close flushes and permanently shuts down that client's processor. In Python, `with Bitfab(...) as client:` closes it on context exit. * An exit handler gives remaining transports a bounded shutdown attempt. * After `fork()`, OTel reinitializes its batch worker, queue, and export lock; Bitfab rebuilds its own pipeline in the child, including the destination exporter and its Collector HTTP session. * Submissions after shutdown are rejected without rebuilding an unowned pipeline. ## TypeScript specifics TypeScript shares the design above, with differences that follow from the runtime: * **The private provider is constructed with every option supplied explicitly**, including its sampler and span limits. Left to its defaults it would read `OTEL_*` environment variables, so a host application's global sampler or attribute-length limit could silently drop or truncate Bitfab payloads. * **The Collector exporter loads through a dynamic `import()`.** It is an ordinary runtime dependency rather than an optional peer, so a configured Collector can never fail for want of the package, but bundlers code-split it and a consumer who never sets an endpoint does not carry it in their initial bundle. Every OTel package the SDK depends on is isomorphic, so the SDK still builds and runs in browsers. * **The API key stays late-bound.** OTel's JS exporter takes static headers, so the SDK builds the Collector exporter on first export and rebuilds it if the resolved key later changes. * **A failed flush is a hint, not a verdict.** The export deadline can fire while requests are still in flight and the server can still persist every one of them, so replay does not fail on the flush result alone: it polls, and fails only if the server itself never confirms. When both fail, the error names the flush failure as the likely cause. Python raises on the flush result instead. * **No fork handling.** Node.js has no `fork()` that duplicates the event loop, so there is no post-fork worker to rebuild. * **Deferred spans are tracked per client.** A span recorded through `finalize` reaches the transport only after its finalize chain settles, because the caller gets the live stream back untouched. That deferred work is tracked per client, so `client.close()` waits for its own spans without being failed by another client's slow finalize; `flushTraces()` remains process-wide and waits for all of it. Python's finalize runs inline, so it has nothing to track. * **A `beforeExit` handler** gives remaining transports a bounded shutdown attempt, which is what lets a plain script exit with its spans delivered. ## Go specifics Go shares the design above, with differences that follow from the runtime: * **The Collector exporter is always linked.** Go has no dynamic import, so `otlptracehttp` and its transitive dependencies ship in every build rather than loading on demand. A consumer who never configures a Collector still carries it; in exchange, a configured Collector can never fail for want of a package. Adopting OTel also raises the SDK's minimum Go version to **1.25**, the floor the OpenTelemetry Go modules declare. * **The Collector endpoint is validated before the exporter is built.** Anything that does not parse as an `http`/`https` URL with a host is rejected and the client falls back to direct delivery with a one-time warning. The official exporter would otherwise log the problem and return an exporter aimed at its own default host, sending every span into the void. * **The API key stays late-bound.** OTel's Go exporter takes static headers, so the SDK supplies an `http.Client` whose round tripper stamps the resolved key on every request. * **Collector protobuf exports are partitioned by encoded JSON size.** The official exporter's protobuf transformer is internal to its module, so the JSON figure serves as a conservative bound (protobuf is strictly smaller for these payloads), with `otlptracehttp.WithMaxRequestSize` as a backstop. * **No fork handling.** A Go program that forks does not carry its goroutines into the child, so there is no post-fork worker to rebuild. * **No automatic last-chance flush.** Go has no `atexit` equivalent, so call `defer client.Close(5 * time.Second)` in `main`, or `FlushTraces` before exit. * **No replay pipeline.** Replay is not implemented in the Go SDK, so the persistence barrier above does not apply to it. * **Carriers start from a background context** rather than the ambient one, and a submission is a non-blocking enqueue: spans and trace completions are queued in the order they occur, and ingress processes carriers idempotently without relying on arrival order. ## Deployment order The replay status request is backward-compatible, but the expected-span readiness behavior lives on the server. Deploy the Bitfab web/server change before publishing an SDK version that relies on the stronger barrier. That change shipped with the Python SDK's adoption, so the SDKs that adopted the transport afterwards needed no further server deploy. ## Deliberate non-goals * The SDK does not install a Bitfab processor or exporter on the application's global OTel provider. * The SDK does not infer Bitfab trace grouping from OTel carrier trace IDs. * The SDK does not send live or replay traffic through direct and Collector routes simultaneously. * The SDK does not implement per-span transport acknowledgments on top of OTel. See [TypeScript SDK](/typescript-sdk), [Python SDK](/python-sdk), [Ruby SDK](/ruby-sdk), and [Go SDK](/go-sdk) for public usage, and [HTTP endpoints](/reference/http) for the ingestion and replay-status contracts.