> ## Documentation Index > Fetch the complete documentation index at: https://docs.bitfab.ai/llms.txt > Use this file to discover all available pages before exploring further. # Ruby SDK Reference > Pure API reference for the bitfab RubyGem. The gem is called `bitfab`. It requires Ruby 3.4 or newer. It uses a global singleton client pattern (`Bitfab.configure` / `Bitfab.client`). ## Framework Integrations No framework-native adapters are shipped for Ruby yet. Instrument Ruby code manually via `Bitfab::Traceable`, `Bitfab::Traceable.wrap`, or a bound `Bitfab::BitfabFunction`. See the [Ruby SDK guide](/ruby-sdk). Framework coverage status lives in the [Frameworks overview](/frameworks/overview). ## Module Layout ```ruby theme={null} Bitfab # top-level module & singleton facade Bitfab::Client # the underlying client class Bitfab::BitfabFunction # fluent wrapper bound to one trace_function_key Bitfab::Traceable # mixin for declarative span tracing Bitfab::Datasets # dataset namespace reached as client.datasets Bitfab::Traces # assertions and scenario namespace reached as client.traces Bitfab::Labels # label namespace reached as client.labels Bitfab::Graders # grader namespace reached as client.graders Bitfab::Subtree # TracePoint subtree capture (internal) Bitfab::CurrentSpan # span handle Bitfab::CurrentTrace # trace handle Bitfab::NoOpCurrentSpan # returned outside a span context Bitfab::NoOpCurrentTrace # returned outside a span context Bitfab::Replay # replay orchestration (internal) Bitfab::ReplayContext # thread-local replay context shared by same-thread fibers (internal) Bitfab::ReplayBranch # per-trace DB branch exposed during replay Bitfab::DbBranchReplayError # a requested database branch could not be resolved Bitfab::ReplayError # whole-run replay failure carrying partial items and the experiment ID and URL Bitfab::ReplayRegistry # project-owned replay root registry Bitfab::ReplayCommand # installed bitfab-replay command implementation Bitfab::ReplayCli # argument parsing and execution for bitfab-replay (internal) Bitfab::NO_MOCK_OVERRIDE # resolver sentinel that continues mock resolution Bitfab::SpanContext # thread-local span stack (internal) Bitfab::TraceState # trace state storage (internal) Bitfab::Transport # span/trace delivery seam (internal) Bitfab::Otel # OpenTelemetry batch transport (internal) Bitfab::MOCK_STRATEGIES # %w[none all marked] Bitfab::BITFAB_PROGRESS_PREFIX # wire prefix used by report_replay_progress ``` ## Module-Level API ### `Bitfab.configure(api_key: nil, service_url: nil, capture_enabled: nil, enabled: nil, **options)` Creates the global client. Subsequent calls replace it. | Param | Type | Default | | - | - | - | | `api_key:` | `String, Proc, nil` | `nil` | | `service_url:` | `String, nil` | `"https://bitfab.ai"` | | `capture_enabled:` | `Boolean, nil` | `nil` | | `enabled:` | `Boolean, nil` | Deprecated alias, `nil` | | `simulation_plan:` | `Boolean` | `true` | | `timeout:` | `Numeric` | HTTP seconds, `120` | | `trace_across_threads:` | `Boolean, nil` | Environment setting | | `db_snapshot:` | `Hash, nil` | Optional `{provider: "neon"}` | | `env_vars:` | `Hash, nil` | Execution credentials | | `baml_client:` | BAML client | Optional generated client | | `strict:` | `Boolean` | `false` | The key is resolved lazily, at the first traced call. If you passed a `Proc`, it is called at that point. A missing or empty configured value falls back to `ENV["BITFAB_API_KEY"]`. In normal mode, a missing key warns and disables tracing. With `strict: true`, a missing key raises instead. `capture_enabled: false` leaves ordinary calls unrecorded without resolving credentials while replay and seeding still record. The deprecated `enabled:` option has the same behavior and warns once. When neither option is passed, `ENV["BITFAB_CAPTURE_ENABLED"]` controls capture: `1`, `true`, or `yes` enables it; `0`, `false`, or `no` disables it (case-insensitive and trimmed). The environment is read at the first traced call. ### `Bitfab.client` Returns the configured `Bitfab::Client`. Raises `RuntimeError("Bitfab not configured...")` if `configure` has not been called. ### `Bitfab.client_or_nil` Returns the configured `Bitfab::Client`, or `nil` if `configure` has not been called. The traced-call path for `bitfab_span` resolves the client this way, not through `Bitfab.client`. As a result, a method invoked before `Bitfab.configure` runs untraced instead of raising. ### `Bitfab.reset!` Resets the global client to `nil`. Intended for tests. ### `Bitfab.current_span` Returns the active `CurrentSpan`, or `Bitfab::NO_OP_SPAN` outside a span context. ### `Bitfab.current_trace` Returns the active `CurrentTrace`, or `Bitfab::NO_OP_TRACE` outside a span context. ### `Bitfab.current_replay_branch` Returns the `Bitfab::ReplayBranch` resolved for the replay item currently running. Returns `nil` outside a replay item. It also returns `nil` when the item's source trace carried no DB snapshot reference. Call it inside the replayed method, then fall back to your normal connection string: ```ruby theme={null} branch = Bitfab.current_replay_branch url = branch ? branch.database_url : ENV["DATABASE_URL"] ``` The branch is a frozen, per-item value object. It exposes `database_url`, `neon_branch_id`, `env_key`, `expires_at`, `snapshot_timestamp`, `provider_console_url`, `read_only`, `region`, and `trace_id`. It is built from the thread-local replay context, so parallel items each see their own branch. Reading `database_url` marks the trace as having used the branch, reported as `accessed`. The other readers inspect the branch without exposing the connection string, and deliberately leave that mark untouched. `inspect` redacts the URL. Every field the service puts on the lease gets a reader under its snake\_case name. A field added server-side is readable before you upgrade the SDK. `database_url` is the one exception. It carries the credential. It is also the only member whose read marks the branch as accessed. `as_json` and `to_json` serialize only those exposed fields. Without this, ActiveSupport would serialize the object's instance variables by default. That would put the connection string into any Rails log line or API response that touched a branch. Because of this override, a serialized branch is safe to hand around. `database_url` still stays available to code that explicitly asks for it. ### `Bitfab.report_replay_progress(progress)` A ready-made replay lifecycle callback. Pass it into `replay` as both `on_item_start:` and `on_item_finish:`. It writes lifecycle events to stderr, prefixed with `Bitfab::BITFAB_PROGRESS_PREFIX`. The Bitfab plugin reads those events to identify in-flight traces and finished results. stdout stays free for direct-run `ReplayResult` JSON. It never raises. ### `Bitfab.serialize_replay_result(result)` Returns indented JSON. It preserves structured fields from `:trace_error` and `:replay_error`, including `Bitfab::DbBranchReplayError#code`, `original_trace_id`, and nested `cause`. Use it for direct-run stdout instead of `JSON.pretty_generate(result)`. `JSON.pretty_generate(result)` raises when a result contains an exception object. ### `Bitfab.flush_traces(timeout: 30)` Waits for queued spans and trace completions from every live client to reach the server within one total deadline. Returns `true` when everything landed and `false` on delivery failure or timeout. Records still held awaiting a simulation plan also make it return `false`. Ready records are still flushed when other records remain held. Client flush and close methods report held records the same way. Repeating `close` preserves the first shutdown result. It does not close the clients. ## `class Bitfab::Client` ### `SPAN_TYPES` ```ruby theme={null} Bitfab::Client::SPAN_TYPES # => %w[llm agent function guardrail handoff custom] ``` ### Readable client state ```ruby theme={null} client.api_key # configured String/nil, or the result of the configured Proc client.service_url client.enabled # effective state; lazily resolves the key client.datasets client.traces client.labels client.graders ``` `api_key` does not apply the `ENV["BITFAB_API_KEY"]` fallback. It also never warns. `enabled` reports the effective state. Reading it resolves the key, the same way the first traced call would. `service_url`, `datasets`, `traces`, `labels`, and `graders` are attribute readers. ### `get_trace` ```ruby theme={null} trace = client.get_trace(trace_id) trace.trace_id trace.add_context({"review" => "complete"}) trace.set_metadata({"region" => "west"}) trace.set_session_id("session-2") trace.set_name("Reviewed case") ``` Returns a handle to an existing trace by its canonical Bitfab UUID. No active span is required. The ID is validated locally. Each update waits for the server response and raises when the trace is missing or inaccessible. `add_context` appends an entry. `set_metadata` merges keys into existing metadata. `set_session_id` and `set_name` replace their respective values. Non-hash context or metadata values and empty session IDs or names are ignored. A disabled client sends no updates. ### `get_trace_span` ```ruby theme={null} get_trace_span(trace_id, id: nil, name: nil, type: nil, occurrence: "last") ``` Fetches one persisted span without loading its trace. `trace_id` is the canonical Bitfab trace ID. Pass the span's Bitfab `id` alone, or `name:` and/or `type:` (`"llm"`, `"agent"`, `"function"`, `"guardrail"`, `"handoff"`, or `"custom"`); with both, a span must match both. Any other combination raises `ArgumentError, "Provide id, or name and/or type"`, and an unknown type raises `ArgumentError` before the request is sent. `occurrence` picks among the matching spans in start order and accepts `"first"`, `"last"`, or a zero-based integer. It defaults to `"last"`. Returns `nil` when no trace or span matches. ```ruby theme={null} first_model_call = client.get_trace_span(trace_id, type: "llm", occurrence: "first") ``` ### `get_span` ```ruby theme={null} get_span(span_id) ``` Reads one span from its span ID alone, either Bitfab's span ID or the one the SDK sent when recording it, so no trace ID is needed. Returns the same span hash as `get_trace_span`, or `nil` when the span is not in the API key's organization. ### `traces` Manage the assertions that describe what a trace should do, and the scenario built from it. ```ruby theme={null} client.traces.get_assertions(trace_id) client.traces.save_assertions(trace_id, assertions, source: "agent") client.traces.save_assertions_all(updates, source: "agent") client.traces.archive_assertions(trace_id, assertion_ids) client.traces.get_scenario(trace_id) client.traces.save_scenario(trace_id, name: nil, description: nil, trigger: nil, data: nil, append_data: nil) client.traces.save_scenario(trace_id, actual_outcome: {"summary" => "...", "actions" => [...]}) client.traces.save_scenario(trace_id, category_ids: ["category-id", ...]) ``` * `get_assertions` returns `{"assertions" => [...], "inheritedFrom" => String | nil, "generation" => Hash | nil}`. `"generation"` reports the trace's assertion generation (`"status"`, `"error"`, `"assertionIds"`, `"startedAt"`, `"finishedAt"`) and is `nil` when none ran. Assertions include separate `"justification"` and `"categoryJustification"` evidence plus one read-only decision (`"approvalState"`, `"approvedBy"`, `"approvedAt"`) covering the assertion and its current category assignment. * `"assigneeEmail"` on an assertion hash names the organization member who should review it. The server resolves the address against that organization's members and refuses one belonging to nobody in it, so a typo fails the call instead of quietly leaving the assertion unrouted. Omit the key on an edit to preserve the current assignee, and pass `nil` to leave it to nobody. Reads return a nullable `"assignee"` hash with `"id"`, `"fullName"`, `"email"`, and `"imageUrl"`. Assigning routes an assertion rather than deciding it, so it never returns the assertion to `"pending"` the way an edit to the wording, criteria, target, or category does. * `save_assertions` returns the saved assertions. Each input hash requires `"assertion"`. Include `"id"` to update an existing assertion. Optional `"humanNote"`, `"passCriteria"`, and `"failCriteria"` accept strings or `nil`. `"justification"` and `"categoryJustification"` each accept an array of `{"spanId" => ..., "text" => ...}` evidence entries; omission preserves the existing value and `nil` clears it. * `"targetOnEvaluatedTrace"` accepts `nil`, `{"kind" => "output"}`, or `{"kind" => "span", "name" => "lookup", "occurrence" => "last"}`. Occurrence accepts `"first"`, `"last"`, or a zero-based integer. * `save_assertions_all` accepts `[{"traceId" => trace_id, "assertions" => assertions}]`. Empty updates return `[]` without an HTTP request. Source defaults to `"agent"`. Pass `source: "human"` for assertions supplied by a person. * `archive_assertions` returns the archived assertion IDs. * A scenario is one situation a trace function's run was in: the trigger that started the run and the data available when it arrived, with a name and a description, built from one source trace. Those fields describe the situation only, never what the run did or should do. `get_scenario` returns the scenario built from `trace_id` as a hash with camelCase keys (`"name"`, `"description"`, `"trigger"`, `"data"`, `"actualOutcome"`, `"triggerSummary"`, `"dataSummary"`, and the rest). A trace with no scenario raises `Net::HTTPError` with a 404, from either method. * `save_scenario` updates that scenario and returns the saved hash. `name:`, `description:`, `trigger:`, and `data:` each replace the saved value whole, and keywords left out are kept. `append_data:` takes `{"history" => [...], "missing" => [...]}` and adds those entries after the saved ones without resending the data; appends made at the same time by different callers all land. Pass `data:` or `append_data:`, not both. Passing both, or none of the seven, raises `ArgumentError` before any request. * `actual_outcome:` is what the run actually did in this one example, never what it should have done. Bitfab never fills it in, so it stays `nil` until you send it. It takes `{"summary" => String, "actions" => [...], "source" => Hash | nil}`, where each action is `{"name" => String, "details" => Hash, "result" => String | nil, "failed" => true | false, "at" => String | nil}` and `"source"` takes the same `{"spanId", "sourceField", "text"}` shape as a history entry's and can be left out. It replaces the saved value whole. Passing `actual_outcome: nil` clears it, and leaving the keyword out keeps it. A save that passes only `actual_outcome:`, `nil` included, is accepted. * `category_ids:` files the scenario under those categories and replaces its whole set of categories. Leave it out to keep the current set, and pass `[]` to clear it. An ID outside the organization fails the call with a 400 that names the missing IDs. Both methods return the scenario's categories under `"categories"`, each `{"id", "title", "description"}`, sorted by title. * A history entry's `"source"` names the span its text came from, on this trace or another trace linked to the same scenario. Leave it out or set it to `nil` when the text is not in such a trace. A `"source"` naming a span on any other trace is refused. The server rebuilds `"triggerSummary"` and `"dataSummary"` from what is saved. * `client.categories.save(title, id: nil, description: nil)` creates a category or updates one by ID. Omitting `description:` preserves it; pass `""` to clear it. Categories group assertions and scenarios. They are organizational metadata and do not carry approval fields. * `client.categories.get(id)`, `.list`, and `.delete(id)` read and manage the organization's categories. Deleting one detaches its assertions while preserving them and their verdicts. * `client.organization_members.list` returns the people in the organization, each a hash with `"id"`, `"fullName"`, `"email"`, and `"imageUrl"`. It takes no arguments, because an API key belongs to one organization. Read it to find the address to pass as an assertion's `"assigneeEmail"`, which is refused when it belongs to nobody here. Membership reaches Bitfab through the identity provider's mirror, so an empty list means nothing has been mirrored rather than that the organization is empty. * Assertion hashes accept `"category_id"`: omit it to preserve the assignment or pass `nil` to clear it. Reads include both that ID and the nullable `"category"` summary. ### `labels` Save and read verdicts about an individual assertion. Labels are only written on assertions, never on a whole trace, and have no approval step. ```ruby theme={null} client.labels.save(label:, annotation:, assertion_id:, trace_id: nil, original_trace_id: nil, attempt: nil, confidence: nil, experiment_id: nil, evidence: ...) client.labels.save_all(labels, experiment_id: nil) client.labels.skip(assertion_id:, trace_id: nil, original_trace_id: nil, attempt: nil, experiment_id: nil, annotation: nil) client.labels.abstain(assertion_id:, trace_id: nil, original_trace_id: nil, attempt: nil, experiment_id: nil, annotation: nil) client.labels.archive(trace_id: nil, original_trace_id: nil, attempt: nil, experiment_id: nil, assertion_id: nil) client.labels.save_human(label:, annotation:, trace_id:, assertion_id:, confidence: nil, evidence: ...) client.labels.save_human_all(labels) client.labels.get(trace_id) client.labels.get_all(trace_ids) client.labels.generate_label_evidence(trace_id, assertion_id) ``` * `save`, `skip`, and `archive` require exactly one of `trace_id:` or `original_trace_id:`. For a replay verdict, use the original trace ID with `experiment_id:`. `test_run_id:` is a deprecated alias for `experiment_id:`, and passing both with different values raises `ArgumentError`. `attempt:` selects a replay attempt. `assertion_id:` names the approved assertion the verdict scores. It is a required keyword on `save` and `skip`, and a `nil` or empty value raises `ArgumentError` before any request, since labels are only written on assertions. `archive` may omit it to clear an old whole-trace agent verdict. * `save` requires a boolean `label:` and an `annotation:` string. Optional confidence is `"VeryLow"`, `"Low"`, `"Medium"`, `"High"`, or `"VeryHigh"`. * `evidence:` accepts one `{ "spanId" => ..., "text" => ... }` entry per cited span on the evaluated trace. Omit it to preserve saved evidence. Pass `nil` to clear it. `get` and `get_all` return the evidence with each whole-trace and per-assertion verdict. * `generate_label_evidence` generates assertion label evidence. Pass the original or replay trace being judged; the assertion must belong to that trace or be inherited through its replay lineage. Each candidate comes from the selected trace and contains `spanId`, `text`, `sourceField` (the span field the quote was copied from), `spanName`, signature-only `parameters` (names and runtime types, never values), `spanType`, and `isMocked`. Save only the candidate's `{ spanId, text }` fields as label evidence; saved evidence remains on `labels.get` and `labels.get_all`. * `skip` withholds a verdict when the check did not run. `annotation:` records why, stored on the withheld verdict and read back by `get` and `get_all`. A skip sent without it clears the note the previous verdict carried. `archive` clears a previous verdict. Single writes return `{"key" => ..., "traceId" => ..., "action" => ...}`. Action is `"set"`, `"archived"`, `"no-active-label"`, `"skipped"`, or `"abstained"`. * `abstain` records an abstention, the same write as `"abstain" => true` in a `save_all` entry. An abstention is the judge's third answer: the evidence it read does not show the part of the run the assertion checks, so nothing can confirm or deny it. It counts as a fail in every pass rate, since anything short of an evidenced pass is a fail, but it is stored and read back apart from a fail, so a reader sees judge coverage rather than the agent's behavior. It is not a skip: a skip is for a check that never ran, an abstention is for a check that ran and found nothing to judge. `annotation:` says what the evidence would have had to show. It reads back as `"verdict" => "abstained"` with `"label" => false`, and the trace's `"abstained"` count says how many of its failed verdicts are abstentions. * `save_all` accepts API-shaped hashes with `"traceId"` or `"originalTraceId"`, optional `"attempt"`, `"assertionId"`, and a verdict. `"assertionId"` is required on every entry except `"archive" => true`, and a batch missing one raises `ArgumentError` before any request. A verdict uses `"label"` plus `"annotation"` and optional `"confidence"`, or `"skip" => true` or `"abstain" => true` with an optional `"annotation"`, or `"archive" => true`. It returns the outcomes array. * `save_human` records a person's verdict on the assertion `assertion_id:` names on `trace_id:`. `assertion_id:` is a required keyword. `save_human_all` accepts hashes with `"traceId"`, `"assertionId"`, `"label"`, `"annotation"`, and optional `"confidence"`, and a batch with a missing or empty `"assertionId"` raises `ArgumentError` before any request. Human outcomes carry `"traceId"`, `"assertionId"`, `"label"`, and `"action" => "set"`. * Reads use actual trace IDs, including the replay trace ID returned by a replay. `get` returns `nil` when no label is found. `get_all([])` returns `[]` without an HTTP request. * Read results include `"traceId"`, `"labelStatus"`, `"label"`, `"annotation"`, `"evidence"`, `"approved"`, `"passed"`, `"failed"`, `"abstained"`, and `"assertions"`. Each assertion verdict includes `"assertionId"`, `"assertion"`, `"labelStatus"`, `"label"`, `"verdict"` (`"pass"`, `"fail"`, or `"abstained"`, with `"label"` false for both of the last two), `"annotation"`, `"evidence"`, `"confidence"`, `"labelSource"`, and `"approved"`. ### `graders` Read the labels produced by a grader. ```ruby theme={null} client.graders.get_labels(trace_ids: nil, grader_id: nil, limit: nil) ``` Pass a nonempty `trace_ids:` list, a `grader_id:`, or both. An absent selector raises `ArgumentError`. `limit:` caps results. Each returned hash carries `"traceId"`, `"graderId"`, `"graderName"`, `"graderStatus"`, `"graderType"`, `"label"`, `"labelReason"`, `"failureDiagnostic"`, `"labelConfidence"`, `"source"`, and `"evaluatedAt"`. ```ruby theme={null} client.graders.save_label(grader_id:, trace_id:, label:, reason: nil, failure_diagnostic: nil, confidence: nil) client.graders.save_label_all(labels) ``` Posts verdicts for external graders: graders created with `external: true`, which Bitfab never runs because you evaluate traces yourself, in your own eval harness, CI, or review queue. Each verdict is stored as that grader's own, so it counts in dataset and experiment pass rates exactly like a grader Bitfab ran, and `get_labels` reads it back. The same write as the `save_grader_labels` MCP tool. `save_label_all` takes up to 200 verdicts and is all-or-nothing: a grader Bitfab runs, an archived grader, a grader or trace outside the organization, a trace from another trace function, or two verdicts for the same grader and trace rejects the call and writes nothing. Saving again for the same grader and trace replaces the earlier verdict. Each entry passed to `save_label_all` is a hash with `"graderId"`, `"traceId"`, `"label"`, and optional `"reason"`, `"failureDiagnostic"`, and `"confidence"`. Each returned hash carries `"graderId"`, `"traceId"`, `"label"`, and `"action"`. ### `experiments` Read experiments back after the fact, for a dashboard or a report outside the replay process. ```ruby theme={null} client.experiments.list(dataset_id: nil, trace_function_key: nil, git_branch: nil, experiment_group_id: nil, status: nil, metadata: nil, created_after: nil, created_before: nil, cursor: nil, limit: nil) client.experiments.get(id) client.experiments.get_rollup(id) client.experiments.get_rollup_all(ids) ``` `list` pages newest first and returns the page hash with `"experiments"`, `"nextCursor"`, and `"hasMore"`. Every filter is optional and they combine; `metadata:` is a hash matched by containment, and `created_after:` / `created_before:` take a `Time` or an ISO 8601 string. `get` returns the experiment hash with its provenance (`"gitBranch"`, `"commitSha"`, `"baseSha"`, `"experimentSha"`, `"runBy"`, `"completedAt"`, `"metadata"`). `get_rollup` returns one `{"experimentId", "totals", "rollup"}` hash and `get_rollup_all` returns an array of them for up to 100 ids in one request, or `[]` with no request for an empty list. `"totals"` counts the run's replays by state: `"total"` counts one replay per trace per attempt, `"traces"` counts distinct traces, and `"judging"` counts traces, replays or originals, with an assertion verdict still being judged. `"rollup"` carries `"traces"`, `"labels"`, `"checks"`, and `"categories"` tallies, each with `"passing"`, `"failing"`, `"abstained"` (failing verdicts the judge abstained on in every attempt), `"fixed"`, `"regressed"`, `"stillPassing"`, `"stillFailing"`, `"originalUnlabeled"`, `"unpaired"`, and `"classified"`; a trace or an assertion passes when at least 75 percent of its attempts passed, and `"passing"` / `"failing"` count every verdict, compared to the original or not, so their ratio is the pass rate. An id outside the organization raises on a 404, and a batch with one unknown id fails whole. These namespaces return hashes with the API's string keys. HTTP errors propagate to the caller. The Ruby example's `scripts/test_evaluation.rb` exercises assertions, labels, and grader reads. Set `BITFAB_API_KEY`, `BITFAB_SERVICE_URL`, and `BITFAB_TRACE_ID` to run it against an existing trace. Optional `BITFAB_EXPERIMENT_ID` and `BITFAB_ATTEMPT` demonstrate writing a replay verdict. ### `datasets` ```ruby theme={null} client.datasets.save(trace_function_key:, name:, description: nil) client.datasets.list(trace_function_key: nil) client.datasets.get(dataset_id) client.datasets.update(dataset_id, name: nil, description: nil) client.datasets.list_traces(dataset_id) client.datasets.add_traces(dataset_id, trace_ids) client.datasets.remove_traces(dataset_id, trace_ids) client.datasets.add_graders(dataset_id, grader_ids) client.datasets.remove_graders(dataset_id, grader_ids) client.datasets.rerun_graders(dataset_id, grader_ids: nil, timeout: 90) client.datasets.get_grader_rerun(dataset_id, run_id: nil) ``` Dataset operations for the authenticated organization. These are the same operations the Bitfab MCP tools expose to a coding agent. Every method returns the parsed JSON response as a `Hash` with string keys matching the HTTP API (`"traceCount"`, `"addedTraceIds"`, and so on). * `save` is an upsert keyed on `(trace_function_key, name)`. `"created"` is `true` for a new dataset. It is `false` when an existing one was updated instead. A `nil` description leaves an existing description untouched. * `list` takes an optional trace function key. Without it, every dataset in the organization is returned. * `update` renames a dataset or changes its description in place and returns the updated dataset. It keeps its id, traces, and graders. A `nil` argument leaves that field untouched, and an empty description clears it. Passing neither raises `ArgumentError` before any request. A blank name, or a name another dataset under the same trace function already uses, raises `Net::HTTPError` with a 400. * A dataset carries `"id"`, `"traceFunctionKey"`, `"name"`, `"description"`, `"traceCount"`, `"graders"` (`[{"id", "name"}]`), `"createdAt"`, and `"updatedAt"`. * `list_traces` returns `{"datasetId", "traceIds"}`. This is the same membership a replay with `dataset_id:` selects. * `add_traces` and `add_graders` accept 1 to 100 ids. They report partial acceptance instead of rejecting the whole call. Ids outside the organization, or under another trace function, come back in `"skippedTraceIds"` / `"skippedGraderIds"`. Ids already present come back in `"alreadyPresentTraceIds"` / `"alreadyAssignedGraderIds"`. * `remove_traces` never deletes a trace, only its membership in the dataset. Ids that were not members come back in `"notPresentTraceIds"`. `remove_graders` reports `"notAssignedGraderIds"` the same way. * `rerun_graders` re-scores every trace in the dataset. `grader_ids:` defaults to every assigned grader. An unassigned id rejects the call. It blocks until the run finishes and returns `{"run", "joinedExisting"}` with the completed run. A run that errors raises `Bitfab::GraderRerunError`, whose `run` holds the errored run and its `"error"`. A run still going after `timeout:` seconds (default 90) raises `Bitfab::GraderRerunTimeoutError`, a subclass of `GraderRerunError` that carries the last run seen. The timeout only stops the wait. The run keeps going on Bitfab, and `get_grader_rerun(dataset_id, run_id:)` reads it later. A request matching an in-flight run joins it instead, reporting `"joinedExisting"` as `true`. * A run has `"status"` (`pending` | `running` | `completed` | `errored`), `"graderIds"`, `"progress"` (`{completedTraces, totalTraces, graderCount}` while running), `"result"` (`{tracesGraded, gradersRun}` when completed), and `"error"`. * A dataset id from another organization raises `Net::HTTPError` with a 404. ### `traces` ```ruby theme={null} Bitfab.client.traces.search( caller_metadata: nil, name: nil, name_contains: nil, experiment_id: nil, trace_function_key: nil, cursor: nil, limit: nil ) ``` `search` finds traces by the caller metadata your code stamped with `set_metadata`, by the name it set with `set_name`, or by the experiment that produced them. It returns a page of stable trace summaries matching every filter supplied. `caller_metadata:` matches exactly and case-insensitively, with at most 10 pairs. `name:` matches the name's words in order, so it also finds a name that continues past the query, and it matches on word boundaries rather than inside a word. `name_contains:` matches any part of the name, including inside a word, and must be at least 3 characters. Both are case-insensitive and capped at 256 characters. `experiment_id:` returns only the replay traces that experiment produced. At least one of `caller_metadata:`, `name:`, `name_contains:`, or `experiment_id:` is required, so a search never returns unfiltered traces. The server defaults `limit` to 50 and caps it at 500. The result is a Hash with `"traces"`, `"nextCursor"`, and `"hasMore"`. Each entry carries `"id"`, `"traceFunctionKey"`, `"name"`, `"status"`, `"callerMetadata"`, `"createdAt"`, `"experimentId"`, `"durationMs"`, and `"tokens"`. `"experimentId"` is the experiment that produced the trace, `nil` for a captured trace. `"durationMs"` is the trace's wall-clock time, `nil` when it has no recorded end time. `"tokens"` is a Hash of `"input"`, `"output"`, `"cached"`, and `"total"` (each an Integer or `nil`) summed across the trace's spans, or `nil` when none were recorded. `"nextCursor"` is `nil` on the last page; pass it back as `cursor:` to fetch the next one. ### `spans` ```ruby theme={null} Bitfab.client.spans.list( trace_id, span_type: nil, span_name: nil, fields: nil, cursor: nil, limit: nil ) ``` `list` returns the spans of one trace in start order, as the same page hash with `"spans"`, `"nextCursor"`, and `"hasMore"`. No filter is required, so a call with only `trace_id` lists every span. `span_type:` (`"llm"`, `"agent"`, `"function"`, `"guardrail"`, `"handoff"`, or `"custom"`) and `span_name:` (exact, case-sensitive) narrow the list. `fields:` picks which of `"input"`, `"output"`, `"reasoning"`, `"content"`, `"errors"`, and `"contexts"` each span carries (default none, `nil` when that field is empty), and `limit:` is 1 to 200, or at most 50 when `fields:` is set (default 50). Pass `"nextCursor"` back as `cursor:` for the next page. An unknown trace raises on a 404. Unset arguments are left out of the request. Each span hash has `"id"`, `"traceId"`, `"parentSpanId"`, `"name"`, `"type"`, `"startedAt"`, and `"endedAt"`, plus only the fields you asked for. ```ruby theme={null} page = client.spans.list(trace_id, fields: ["input"]) page["spans"].each { |span| puts "#{span["name"]}: #{span["input"]}" } ``` ### `initialize(api_key: nil, service_url: nil, enabled: nil, strict: false)` Same semantics as `Bitfab.configure`. ### `flush(timeout: 30)` Waits for the spans and traces this client queued to be delivered, without closing it. Returns `true` when everything landed inside the deadline. ### `close(timeout: 30)` Flushes, then permanently shuts down this client's OpenTelemetry batch worker. Returns `true` when everything queued was delivered inside the deadline. Later submissions on the closed client are dropped with a one-time warning. A shared client does not need this. `at_exit` already shuts remaining transports down. ### Replay registry and command Experimental: Add `--cloud` to any replay command to run the same replay on your own GitHub Actions. Every replay option works as it does locally, your registry is loaded only on the runner, and the command prints the replay's progress and a short summary of its result and exits with its exit code. Every trace's input and output are in the experiment, where the Bitfab MCP tools read them; add `--cloud-full-output` to print the whole result instead. It runs the Python helper that ships with the SDK, using the one workflow `bitfab-replay --cloud-init` writes. See [Cloud replay](/cloud-replay) for setup, snapshot review, and recovery commands. ```ruby theme={null} REGISTRY = Bitfab::ReplayRegistry.new REGISTRY.register("checkout", CheckoutService.new, :run) REGISTRY.register( "agent", AgentService.new, :run, trace_function_key: "support-agent", mock: "marked", adapt_inputs:, options_factory: ->(ctx) { {mock_override: SupportAgentMocks.build(ctx.params["scenario"])} } ) ``` `Bitfab::ReplayRegistry#register` stores the exact production receiver method. It also stores static per-function replay behavior and an optional `options_factory:`. A `bitfab_span` method supplies its trace function key automatically. A plain handler method must pass `trace_function_key:` explicitly. Pass `client:` to route this entry's replay through a specific client. It defaults to `Bitfab.client`. Unlike `Traceable.wrap`'s `client:`, which resolves on every call, this one resolves once, when `register` is called. The factory receives JSON values from `--params` and `--param`. It can use them to construct executable options such as `mock_override:`. A direct `--param` value overrides one loaded from a `--params` file. Registered and factory-produced options are limited to these names: * `limit`, `trace_ids`, `name`, `notes`, `concurrency`, `max_concurrency`, `attempts`, `only_with_assertions`, `skip_assertion_judging`, `dry_run` * `code_change_description`, `code_change_files` * `experiment_group_id`, `experiment_group_name`, `dataset_id`, `dataset_ids`, `grader_ids` * `mock`, `mock_override`, `adapt_inputs`, `db_branch`, `on_item_finish` An unknown option name is rejected, either when the registry loads or when the factory runs. `on_item_finish` is chained after the command's progress reporter. Other lifecycle callbacks remain owned by the command. The gem installs the `bitfab-replay` executable. Run it as `bundle exec bitfab-replay --registry [options]`. The registry module must define `REGISTRY`. When the registry cannot load one of its requires, the command raises `ArgumentError` instead of `LoadError`. Its message keeps the original error and adds a note that a registry belongs in the app that builds the traced method's call. The command prints that message and exits 1. When assertion-only selection and a limit narrow an explicit ID list or dataset, the command reads approved assertions in selection order until the bound is filled. An assertion still awaiting review does not make a trace eligible. Inherited assertions do not make a replay trace eligible. A failed assertion read aborts selection. The server revalidates the pinned IDs, so assertions changed concurrently can still reduce the final count. | Flag | Value | Effect | | - | - | - | | `--limit` | `N` | Most traces to select (default 10). It bounds whatever selection is in effect, whether that selector came from the command line or the registry, keeping explicit IDs in caller order and dataset members in ID order. With assertion-only selection, it counts eligible traces before applying the bound | | `--trace-ids` | `id1,id2` | Replay exactly these traces. Mutually exclusive with `--dataset-ids` | | `--dataset-ids` | `uuid1,uuid2` | Replay the membership of one or more datasets, as their deduped union. Mutually exclusive with `--trace-ids`. `--dataset-id` is the same flag | | `--name` | `NAME` | What this run is testing, such as `baseline`. Bitfab already records the commit, branch, and datasets | | `--notes` | `TEXT` | Run conditions Bitfab cannot see, such as a forced feature flag | | `--concurrency`, `--max-concurrency` | `N` | Items in flight at once | | `--attempts` | `N` | Attempts per source, from 1 to 100 | | `--primitive` | `async` or `process` | Execution mode | | `--memory-throttle`, `--no-memory-throttle` | | Configure process memory admission | | `--only-with-assertions` | | Select sources with approved own assertions | | `--skip-assertion-judging` | | Don't judge approved assertions on each replay. Judging is on by default and costs model calls. Omitting it leaves a registry-set `skip_assertion_judging` in place | | `--dry-run` | | Resolve inputs without executing customer code | | `--fail-on-error` | | Exit 1 after printing the result when any replayed item errored, printing `[replay] N of M replayed items errored` to stderr. Without it, errored items are reported in the summary and the command still exits 0. Under `--dry-run`, items whose inputs failed to resolve count | | `--experiment-group-id` | `UUID` | Add this run to an experiment group. A new UUID creates the group | | `--experiment-group-name` | `NAME` | Name the experiment group when this run creates it or it has no name yet. A group that already has a name keeps it. Requires `--experiment-group-id` | | `--grader-ids` | `id1,id2` | Attach graders to this run only | | `--code-change` | `PATH` | Load a code-change description from a file | | `--no-code-change` | | Record no code change, overriding a registry default | | `--mock` | `none\|all\|marked` | Which recorded child spans return their historical output | | `--db-branch`, `--no-db-branch` | | Turn per-item database branching on or off | | `--params` | `PATH` | JSON file of values passed to `options_factory:` | | `--param` | `name=value` | One value passed to `options_factory:`. Repeatable, and overrides `--params` | | `-h`, `--help` | | Print usage | As soon as the server creates the experiment, the command prints `[replay] Experiment : ` to stderr. If the run later fails with `Bitfab::ReplayError`, it prints `Experiment : ` again before the error message, so the experiment ID is never lost. ### `register_mock_override` ```ruby theme={null} register_mock_override(override) register_mock_override(match, value) register_mock_override(trace_function_key, resolver_or_override) register_mock_override(match: matcher, value: value) ``` Registers an instance-scoped override for every subsequent replay. A keyed resolver runs only for child spans with that trace function key. A keyed `{ match:, value: }` hash also applies its own matcher. A global resolver can route on `ctx[:node][:trace_function_key]`. Return `Bitfab::NO_MOCK_OVERRIDE` to continue to lower-priority overrides and the base mock strategy. `nil` remains a valid mocked output. Per-call overrides take precedence over registered overrides. ### `clear_mock_overrides` Removes every override registered on this client. ### `replay(receiver, method_name, attempts: 1, only_with_assertions: false, skip_assertion_judging: false, dry_run: false, trace_function_key:, limit: nil, trace_ids: nil, name: nil, metadata: nil, max_concurrency: 10, code_change_description: Replay::CODE_CHANGE_UNSET, code_change_files: Replay::CODE_CHANGE_UNSET, mock: "marked", mock_override: nil, adapt_inputs: nil, db_branch: nil, experiment_group_id: nil, experiment_group_name: nil, dataset_id: nil, dataset_ids: nil, grader_ids: nil, on_item_start: nil, on_item_finish: nil, on_progress: nil, on_experiment_start: nil)` | Param | Type | Description | | - | - | - | | `receiver` | `Object` or `Class` | Instance for instance methods. Class for class methods | | `method_name` | `Symbol` | Method to replay | | `trace_function_key:` | `String` | Trace function key for this method. It must match the key the method is traced under, via `bitfab_span` or `Bitfab::Traceable.wrap`. A contradicting key raises `ArgumentError` | | `limit:` | `Integer, nil` | Max recent traces to replay (default `5`, maximum 5,000). Ignored when `trace_ids:` or a dataset selector is passed | | `trace_ids:` | `Array, nil` | Optional explicit trace IDs. Max 100. Passed alongside `dataset_id:` or `dataset_ids:` it pins which members of that selection replay, and the server rejects any ID none of those datasets contains. The ID count determines how many traces replay | | `name:` | `String, nil` | Optional display name for the resulting experiment | | `notes:` | `String, nil` | Optional run conditions Bitfab cannot see on its own, such as an environment override or a forced feature flag. Kept on the experiment next to its name (max 10,000 characters) | | `metadata:` | `Hash, nil` | Your own string tags on the experiment, such as which schedule launched it (`--metadata KEY=VALUE`, repeatable, on the replay command). Keys match `^[A-Za-z0-9_.:-]{1,40}$`, values are at most 500 characters, at most 50 keys. Read back on every `experiments` read and filtered on by `experiments.list` | | `max_concurrency:` | `Integer, nil` | Positive maximum parallel threads (default `10`, `nil` means unlimited). Other values raise before an experiment starts | | `code_change_description:` | `String, nil` | Optional rationale for the code change being tested. It is stored on the experiment. When supplied by itself, it is preserved. Files are still captured automatically in that case | | `code_change_files:` | `Array, nil` | Optional list of edited files, each `{ path:, before:, after: }`. Omit it for a rename-aware diff of uncommitted working-tree changes against HEAD. Pass `nil` instead to suppress capture entirely | | `mock:` | `String` | Mock strategy for child spans. One of `"marked"` (default), `"none"`, or `"all"`. Raises `ArgumentError` on any other value | | `experimental_selective_replay:` | `Hash, nil` | Experimental `{must_run: [{trace_function_key:, span_name:}]}` manifest. Requires marked mocks and rejects overrides. See [selective replay](/replay-mocking#experimental-selective-replay-all-sdks) for reuse declarations and coverage reports | | `mock_override:` | `Hash, Proc, Array, nil` | Per-call `{ match:, value: }` overrides or global resolvers. The first matching result other than `Bitfab::NO_MOCK_OVERRIDE` wins. These run before registered overrides and the base `mock:` strategy | | `adapt_inputs:` | `#call, nil` | Reshapes recorded positional and keyword arguments for the method's current signature | | `experiment_group_id:` | `String, nil` | Optional UUID that groups multiple replay runs into a single experiment batch. An ID no group has yet creates the group | | `experiment_group_name:` | `String, nil` | Optional display name for the experiment group, applied when this replay creates the group or the group has no name yet. A group that already has a name keeps it. Requires `experiment_group_id:`; passing it alone raises `ArgumentError` | | `dataset_id:` | `String, nil` | Optional UUID of the dataset this replay runs against. Mutually exclusive with `dataset_ids:`. Alone it replays the dataset's full membership, and with `trace_ids:` only those members. Durably attributes the experiment to that dataset | | `dataset_ids:` | `Array, nil` | Optional UUIDs of the datasets this replay runs against. Mutually exclusive with `dataset_id:`. Replays the union of their traces, or only the selected members when combined with `trace_ids:`. Uses the union of their graders and attributes the experiment to every selected dataset | | `grader_ids:` | `Array, nil` | Optional UUIDs of graders attached directly to this run, up to 100. They are unioned with the dataset's runnable graders when the experiment is graded. Each grader must be active, in the same organization and trace function. Otherwise the replay is rejected with a 400 | | `db_branch:` | `Boolean, Hash, nil` | Optional. `db_branch: true` requests a DB branch for each replay item, using the mirror's own sizing. `false` or `nil` leaves branching off. Each bounded worker resolves its own branch. As a result, `max_concurrency` also bounds how many branches are live at once. `Bitfab.current_replay_branch` exposes the branch inside the replayed method. Its keys tune the branch, for example `db_branch: {min_cu: 2, max_cu: 2, warmup_sql: "SELECT 1;"}`. All of these keys are optional. Symbol or string keys both work. This matters because a hash built from JSON or YAML arrives string-keyed. `min_cu` and `max_cu` set the compute's autoscaling floor and ceiling, from 0.25 to 56. Equal `min_cu`/`max_cu` values pin a fixed size, up to 56. This keeps items comparable to each other. An autoscaling range may not span more than 8 CU. It may also not exceed 16 CU overall. `warmup_sql` is appended to the branch's readiness check, warming the cache before the replayed method sees the branch. That way, warm-up time is not charged to the call itself. Invalid warm-up SQL fails the branch, rather than quietly handing back a cold one. Omitted keys leave the mirror's own defaults in place | | `on_item_start:` | `#call, nil` | Optional callback fired when a worker begins each item, before replay setup or customer code runs. It carries `type: "started"`, running lifecycle totals, and the historical trace/span identity. Pair it with `on_item_finish:` to distinguish queued work from in-flight work. A raising callback never crashes the run | | `on_item_finish:` | `#call, nil` | Optional callback fired exactly once per item, as it finishes. It receives a running-totals hash: `{ experiment_id:, completed:, total:, succeeded:, errored:, item: }` (plus the deprecated alias `test_run_id:`). Every invocation carries `:item`. The callback itself never represents whole-run completion. `item[:trace_id]` is the server replay `traces.id`. It is read back off the ingest response. It is surfaced once the item finishes. Its trace is flushed on finish. `item[:trace_id]` is `nil` only if that flush could not confirm delivery in time. `item[:original_trace_id]` is the original historical trace being replayed. `:source_trace_id` is a deprecated alias for it. The item also carries `input`, `result`, `original_output`, `error`, `trace_error`, `replay_error`, `duration_ms`, `tokens`, `model`, and `db_snapshot_ref`. Use this callback to render live progress while the replay runs. Its running totals only split ran-ok from errored. Pass or fail isn't known yet at this point. A raising callback never crashes the run. Pass `Bitfab.method(:report_replay_progress)` as both lifecycle callbacks. The Bitfab plugin uses its stderr events to identify in-flight traces. It also uses them to write finished per-item result files | | `on_experiment_start:` | `#call, nil` | Optional callback fired once, right after the server creates the experiment and before any item runs, with a hash `{ experiment_id:, experiment_url: }`. It does not fire on a dry run, which creates no experiment. Use it to hold the experiment ID even if the run later fails. A raising callback never crashes the run | | `on_progress:` | `#call, nil` | Deprecated compatibility callback. It receives the same per-item events, plus its own legacy item-less terminal `"complete"` event. It is ignored when both `on_progress:` and `on_item_finish:` are provided | `attempts:` accepts integers from 1 through 100. Work is queued attempt-major. Each attempt gets its own replay trace and historical database branch. Start and finish callbacks carry the zero-based `:attempt` on their item. By default every approved assertion is judged on each replay as it finishes, and the verdict is saved as that assertion's agent label with the evidence it cited. An assertion that already carries a label is left alone. Pass `skip_assertion_judging: true` to turn judging off, since each judgement costs model calls. Replays from SDK versions released before judging became the default never ask for it, so they stay unjudged. `only_with_assertions:` selects traces carrying an approved assertion. An assertion still awaiting review does not qualify a trace, because only an approved one is checked on a replay. An input adapter receives the original trace's stored metadata as `ctx[:metadata]`. The SDK requests metadata only when an adapter is configured. `dry_run:` resolves and adapts inputs without invoking the method or provisioning database branches. It creates no experiment, so the result's `experiment_id` and `experiment_url` are `nil`, and it does not require any persisted replay traces. The registry CLI exposes `--attempts N`, `--only-with-assertions`, `--skip-assertion-judging`, and `--dry-run`. The item's `:input` contains the resolved positional arguments followed by a keyword-argument hash when keywords are present. A keyword-only call therefore reports `[{key: value}]`. Item-finish callbacks use the same shape. **Returns:** a `Hash` with the keys `:items`, `:experiment_id`, `:experiment_url`, and `:attempts`. `:test_run_id` and `:test_run_url` are deprecated aliases carrying the same values. A root returning an `Enumerator` is consumed inside its replay context before the item finishes. The item's `:result` is the collected array. Iteration failures are reported as `:trace_error`. The duration includes iteration, and a database branch stays leased until iteration finishes. Each item in `:items` carries these fields: * `:attempt`, the zero-based attempt index * `:ingestion_type`, `"captured"` or `"seeded"`, or `nil` against older servers * `:input`, `:result`, `:original_output`, `:error`, `:trace_error`, `:replay_error` * `:duration_ms` (`Integer, nil`), this replay's own duration * `:original_duration_ms`, `:original_tokens`, `:original_model` * `:db_branch_timings` (`Hash, nil`) * `:tokens` (`{ input:, output:, cached:, total: }` or `nil`) * `:model` (`String, nil`) * `:trace_id`, `:original_trace_id`, `:original_span_id` (each `String, nil`) * the deprecated aliases `:source_trace_id` and `:source_span_id` * `:db_snapshot_ref` (`Hash, nil`), the source trace's snapshot pin, if any `:trace_error` retains the actual exception raised while executing the replayed trace. `:replay_error` retains the actual exception raised before the method could even run, such as a database warmup or input-loading failure. `:error` holds the compatible message for either one. A database branch resolver failure surfaces as a `Bitfab::DbBranchReplayError`, carrying `code`, the server message, and `original_trace_id`. This is true both in progress JSON and in `Bitfab::ReplayError#items`. HTTP, timeout, and network failures use the code `lease_request_failed`, with the original client exception as `cause`. Malformed refs use `invalid_snapshot_ref`. Unexpected resolver failures use `internal_error`. `:duration_ms` is how long this replay took. `:tokens` is the replayed run's own token usage. The trace being replayed is described separately, by `:original_duration_ms`, `:original_tokens`, and `:original_model`. `:model` is a deprecated alias for `:original_model`. Compare `:tokens` against `:original_tokens` to see the cost delta. `:trace_id` is the new replay trace's server id. It becomes available to `on_item_finish` once that item's trace has been flushed. It stays `nil` if delivery could not be confirmed, or against an older server. `:original_trace_id` and `:original_span_id` identify the original trace and root span being replayed. A replay whose delivery Bitfab cannot confirm in time is a warning on stderr, `[bitfab] replay # could not confirm all spans reached Bitfab within s. The run continues; Bitfab will show this replay as capture incomplete.`, never a failure, in thread, sequential, and process concurrency alike: the run completes and the item is not marked errored. A process child forwards it to the command's stderr as a `[replay] span delivery problem for ...` line. If no completed item's trace persisted at all, or finalization fails after items settle, `replay` raises `Bitfab::ReplayError`. Its `items`, `experiment_id`, and `experiment_url` retain the partial result. `test_run_id` and `test_run_url` are deprecated aliases for the last two. Its `cause` retains the original whole-run exception. **Mock strategies:** * `"marked"`: only child spans declared with `mock_on_replay: true` return historical output. If a selected occurrence is unavailable, the item errors without executing the real child. Default. * `"none"`: every child span runs real code. * `"all"`: every matched recorded child span returns its historical output. Only the root runs real code. A missing or exhausted occurrence fails the item closed. Child calls are matched by `trace_function_key` and span name. Repeated calls with the same key and name are distinguished by call order. Calls that share a trace function key but have different span names are tracked independently. ### Replay auto-capture environment variables | Variable | Effect | | - | - | | `BITFAB_CODE_CHANGE_PATH` | Path to a JSON file of `{ "description", "files" }`, written by the Bitfab plugin. `replay` reads it instead of running `git diff` when `code_change_files:` is omitted | | `BITFAB_CODE_CHANGE_BASE` | Git ref whose merge-base to diff against instead of `HEAD` (for example `origin/main`), so committed branch changes are included | | `BITFAB_DISABLE_CODE_CHANGE_CAPTURE` | Set to opt every replay in this process out of automatic code-change capture | | `BITFAB_REPLAY_RESULT_PATH` | When set by the Bitfab plugin, `replay` writes the full final `ReplayResult` JSON to this path automatically, in addition to its return value | ### `get_function(trace_function_key)` Returns a `Bitfab::BitfabFunction` bound to `trace_function_key`. Mirrors `client.get_function` in the Python SDK and `client.getFunction` in TypeScript. ### `node(klass, method_name, name: nil, type: "custom", capture: true, experiment_id: nil, mock_on_replay: nil, finalize: nil)` Client-bound external-class form of `bitfab_node`. The method is configured only beneath an active `bitfab_trace` owned by this client and otherwise runs unchanged. ### `execute_span(...)` (internal) Called by `Traceable`. Not intended for direct use. ### Client environment variables | Variable | Default | Description | | - | - | - | | `BITFAB_API_KEY` | unset | The API key used when `api_key:` is omitted or resolves empty. Read at the first traced call, not at construction | | `BITFAB_CAPTURE_ENABLED` | unset | The tracing switch when `enabled:` is left at `nil`. `1`, `true`, or `yes` traces; `0`, `false`, or `no` does not (case-insensitive, trimmed). Unset traces. An unrecognized value warns once and is ignored. Read at the first traced call, so env the app loads after the client is built still counts, and `enabled:` in code always wins | ### Transport environment variables | Variable | Default | Description | | - | - | - | | `BITFAB_OTEL_MAX_REQUEST_BYTES` | `3000000` | Request-size target for OTLP/JSON exports. Accepts positive integers up to `3000000` | | `BITFAB_OTEL_EXPORT_CONCURRENCY` | `32` | Concurrent direct requests per export window. Accepts `1` through `64` | | `BITFAB_DISABLE_COMPRESSION` | unset | Set to send every request body uncompressed instead of gzip-encoding bodies of 8,192 bytes or more that compress smaller | ### Commit ref environment variables | Variable | Effect | | - | - | | `BITFAB_DISABLE_COMMIT_REF` | Set to any value to send no `commit_ref` at all. Neither the platform variables nor `git` are consulted | | `BITFAB_COMMIT_SHA` | The commit sent as `commit_ref.sha` on every root trace. Wins over every deploy platform variable and over `git` | | `VERCEL_GIT_COMMIT_SHA`, `GITHUB_SHA`, `RAILWAY_GIT_COMMIT_SHA`, `RENDER_GIT_COMMIT`, `SOURCE_VERSION`, `CF_PAGES_COMMIT_SHA`, `CI_COMMIT_SHA`, `BUILD_SOURCEVERSION`, `CIRCLE_SHA1` | Read in that order when `BITFAB_COMMIT_SHA` is unset, together with the same platform's branch and repository variables when it has them. The first one set wins, and `git` is never run | ### `CommitRef` ```ruby theme={null} { "sha" => String, "branch" => String | nil, "dirty" => true | false | nil, "remote" => String | nil, "root_sha" => String | nil } ``` The commit the traced code was running at, sent as `commit_ref` on every root trace completion. `sha` is the commit. `branch` is the checked-out branch, or `nil` when detached or unknown. `dirty` is `true` when the working tree had uncommitted or untracked changes, `false` when it was clean, and `nil` when the SDK could not tell, which is always the case when the ref came from environment variables rather than `git`. `remote` is the `origin` URL reduced to `host/owner/repo` with any credentials removed, so a CI checkout token never reaches the trace. `root_sha` is the repository's first commit, so two checkouts of the same repository match even without a remote. Resolution order is `BITFAB_COMMIT_SHA`, then the deploy platform's build variables, then `git` in the process's working directory (see [Commit ref environment variables](#commit-ref-environment-variables)). Environment resolution is synchronous and free. The `git` path runs once per process on a background thread, with a two second timeout per command, so it never sits on the code path that ran the traced function. A trace that completes before it lands ships without a `commit_ref`, and a process with neither variables nor a repository never sends one. The result, including a negative one, is memoized for the life of the process. Set `BITFAB_DISABLE_COMMIT_REF` to opt the process out entirely. `Bitfab::CommitRef.current` returns the hash, or `nil` while unresolved. ## `class Bitfab::BitfabFunction` Fluent wrapper bound to a single `trace_function_key`, returned by `Bitfab::Client#get_function`. ### `attr_reader :trace_function_key` ### `wrap(klass, method_name, name: nil, type: "custom", capture_when: "always", mock_on_replay: false, finalize: nil, experiment_id: nil)` Wraps `method_name` on `klass` with span tracing under the bound `trace_function_key`. Delegates to `Bitfab::Traceable.wrap`. ### `trace(klass, method_name, name: nil, type: "custom", max_depth: 30, max_captured_subtree_spans: 500, exclude: [], include_wrappers: false, mock_on_replay_default: false)` Experimental. Wraps `method_name` as the root and captures its first-party Ruby call subtree via `TracePoint`. Delegates to `Bitfab::Traceable.trace`; the bound key is used for the root and every captured call. ## `module Bitfab::Traceable` Mixin for declarative instance-method tracing. ### `include Bitfab::Traceable` Extends the including class with `ClassMethods` below. ### Class Methods (after `include`) #### `bitfab_function(key)` Sets the default `trace_function_key` for all `bitfab_span` declarations in this class. #### `bitfab_span(method_name, trace_function_key: nil, name: nil, type: "custom", capture_when: "always", mock_on_replay: false, finalize: nil, experiment_id: nil)` Wraps `method_name` with span tracing. Spans exist only where you create them. Nesting is automatic, but only between spans that already exist. One wrapper around the outermost function records a single-node trace. See [Instrumentation](/instrumentation). | Param | Type | Default | | - | - | - | | `method_name` | `Symbol` | required | | `trace_function_key:` | `String, nil` | falls back to the class-level `bitfab_function` key | | `name:` | `String, nil` | defaults to the qualified class and method name when the class is named | | `type:` | `String` | `"custom"` (one of `SPAN_TYPES`) | | `capture_when:` | `String, Symbol` | `"always"` / `:always` (default). `"nested"` / `:nested` records only when there's an active parent span. Otherwise, it runs the method untraced. Unknown values warn once and default to `"always"` | | `mock_on_replay:` | `Boolean` | `false`. When `true`, replay returns this span's historical output, under the default `"marked"` mock strategy | `finalize:` receives the completed result and changes only recorded output. For an Enumerator it receives collected values after iteration finishes. A finalizer failure records `finalize failed: ...` without changing the caller's return value. `experiment_id:` attributes the span to an existing experiment. `test_run_id:` is a deprecated alias for it. Supports three call styles: 1. **Before `def`**: registered via `method_added` hook, wrapped when method is defined 2. **Inline**: `bitfab_span def foo ... end, type: "function"` (Ruby's `def` returns `:foo`) 3. **After `def`**: wraps immediately if the method already exists Raises `RuntimeError` if no `trace_function_key` is provided and no class-level `bitfab_function` was set. #### `bitfab_trace(method_name, trace_function_key: nil, name: nil, type: "custom", max_depth: 30, max_captured_subtree_spans: 500, exclude: [], include_wrappers: false, mock_on_replay_default: false)` Experimental. Records the root method and every first-party Ruby method it calls, at any depth, without declaring those methods separately. The root uses `name` and `type`; descendants use their method names and type `"function"`. Capture is active only while the root runs, including while a returned `Enumerator` is consumed. It follows recursive, module, inherited, `define_method`, singleton-class, cross-file, and same-thread child Fiber calls. An alias is named and matched by `exclude` using the invoked alias rather than its original definition name. Ruby standard-library, dependency-gem, Bitfab SDK, synthetic dynamic-evaluation, and child-thread methods are excluded. An installed gem containing the traced root is treated as first-party, without admitting neighboring gems or SDK internals. Blocks and lambdas are not method-call events and are not recorded. Methods whose only parameters are `*args`, `**kwargs`, and `&block` are treated as forwarding decorator wrappers and skipped unless `include_wrappers: true`. Child Fibers created during capture inherit node policies, trace identity, parenting, and depth limits through Ruby Fiber storage, with separate call stacks. Pre-existing Fibers and `Fiber.new(storage: nil)` do not inherit this context. After an inner capture ends, child Fibers retain any still-active outer capture, with their existing parenting and depth limits. Inherited context expires once all enclosing captures end. While a root remains active, hitting a span limit stops new capture but allows already-open spans in every Fiber to finish. Independently traced Fibers remain isolated during interleaving, including when one root finishes before another. When capture ends, already-open automatic calls in suspended Fibers are emitted once with a nil output and `Subtree capture ended before the call returned` as the error, unless a real error was already observed. Their inputs and parenting are retained. The SDK does not resume Fibers, and later resumption cannot add spans to the ended capture. If hot reload moves the root to an unresolved source, the last valid first-party boundary and its exclusions remain in use. A returned `Enumerator` defers capture completion until iteration finishes or raises. Calls already paused in child Fibers retain their inputs, parenting, node context, depth and span-budget state across that handoff. Between the initial return and consumption, the session is inactive and its deferred frames are detached from the thread dispatcher. Calls still unfinished when iteration ends receive the incomplete-capture error described above. An Enumerator that is never consumed remains pending; the SDK does not force iteration. Automatically captured methods that return Enumerators record a `` output placeholder without consuming them; the traced root records the values yielded during consumption. Each nested `bitfab_trace` starts an independent trace during ordinary capture, including recursion and calls using the same key. Every enclosing trace retains its own recording of the shared subtree. The enclosing span and nested root carry reciprocal trace and span links. Each trace applies its own limits and exclusions. During replay and seed execution, nested annotations remain in the managed trace. `max_depth` limits how deep capture goes. There is no cap on the total number of spans one trace root sends. `max_spans` was removed, and passing it raises `ArgumentError`. `max_captured_subtree_spans` (default `500`) caps discovered calls sent with their inputs and outputs. It does not count configured nodes, nested trace roots, or spans the loaded sim plan turned capture off for, so those keep recording past it. Spans sent without inputs and outputs only because the sim plan could not be read still count toward it. Past it, further discovered calls with capture on record nothing. Hitting any limit warns once per trace function key, and when the root finishes its trace carries the flat metadata keys `bitfab.truncated_by` (the limits that fired, in the order `max_depth`, `max_captured_subtree_spans`) and `bitfab.dropped_spans` (how many calls were not recorded). `exclude` contains unqualified method names as strings or symbols. Automatically captured spans are observations, not replay mock targets. Use `bitfab_node` to make a descendant replay-mockable. A configured node records once in each enclosing trace and parents its automatically captured descendants. Mixing `bitfab_span` or `Traceable.wrap` with subtree tracing raises `Bitfab::MixedTracingError`. The SDK prepares the project boundary and exclusions when the method is declared, refreshes the boundary if hot reload moves the root definition, caches canonical source paths, and shares one `TracePoint` dispatcher across active roots on a calling thread. After a span limit is reached, it stops reading inputs and outputs immediately, then stops processing events for that session once its captured calls finish. Each active root still captures and serializes its own copy of every overlapping call. Keep this experimental API for discovery; prefer explicit `bitfab_span` declarations on production hot paths. Supports the same before-`def`, inline, and after-`def` declaration styles as `bitfab_span`: ```ruby theme={null} class OrderService include Bitfab::Traceable bitfab_function "order-processing" bitfab_trace :process_order, type: "agent", exclude: [:healthcheck] def process_order(order_id) order = load_order(order_id) summarize(order) end end ``` The wrapper preserves the root method's public, protected, or private visibility. #### `bitfab_node(method_name, name: nil, type: "custom", capture: true, experiment_id: nil, mock_on_replay: nil, finalize: nil)` Experimental. Configures a method only when it is discovered beneath an active `bitfab_trace`. It never creates a standalone span or trace and inherits the active root's trace function key, limits, parent, and lifecycle. | Param | Type | Default | | - | - | - | | `method_name` | `Symbol` | required | | `name:` | `String, nil` | defaults to the qualified class and method name when the class is named | | `type:` | `String` | `"custom"` (one of `SPAN_TYPES`) | | `capture:` | `Boolean` | `true`; `false` omits the node while captured descendants attach to its nearest captured parent | | `experiment_id:` | `String, nil` | optional experiment attribution (`test_run_id:` is a deprecated alias) | | `mock_on_replay:` | `Boolean, nil` | Inherits the trace's `mock_on_replay_default:` when omitted. Explicit `false` overrides the marked-replay default. All-mocking and overrides retain precedence. `true` reuses recorded output under marked replay | | `finalize:` | `#call, nil` | transforms the recorded output without changing the method's return value; failures are recorded on the node and do not escape | Captured nodes count toward the enclosing root's `max_depth`, but not `max_captured_subtree_spans`. A root-level `exclude` wins over the node configuration for that root. Overlapping roots each record the configured name, type, experiment attribution, and finalized output. Finalization runs once per invocation. An Enumerator is finalized with its collected values after iteration finishes. `capture: false` applies to every active root because it is an unconditional property of the method. `capture: false, mock_on_replay: true` raises `ArgumentError` because an omitted node has no recorded output. Supports the same before-`def`, inline, and after-`def` declaration styles as `bitfab_span` and preserves public, protected, or private visibility. ### Module-Level Wrapping #### `Bitfab::Traceable.wrap(klass, method_name, trace_function_key:, name: nil, type: "custom", capture_when: "always", mock_on_replay: false, finalize: nil, experiment_id: nil, client: nil)` Wraps a method on a class you do not own. Uses `Module#prepend`. | Param | Type | | - | - | | `klass` | `Class` or `Module` | | `method_name` | `Symbol` | | `trace_function_key:` | `String` | | `name:` | `String, nil` | | `type:` | `String` | | `capture_when:` | `String, Symbol` (`"always"` / `:always` or `"nested"` / `:nested`). Unknown values warn once and default to `"always"` | | `mock_on_replay:` | `Boolean` | | `client:` | `Bitfab::Client, nil`. Route through a specific client instead of resolving the global client on every call | #### `Bitfab::Traceable.trace_function_key_for(receiver, method_name)` Returns the trace function key carried by a method wrapped with `bitfab_span` / `Traceable.wrap`, or `nil` when the receiver method has no Bitfab wrapper. Replay and replay registries use it to reject key mismatches. #### `Bitfab::Traceable.trace(klass, method_name, trace_function_key:, name: nil, type: "custom", max_depth: 30, max_captured_subtree_spans: 500, exclude: [], include_wrappers: false, mock_on_replay_default: false)` Experimental external-class form of `bitfab_trace`. Uses `Module#prepend`; its options have the same semantics as the class macro. The guides lead with `client.get_function(key).trace(klass, method_name, ...)`, which binds the key once, or `bitfab_function` inside a class you own. Passing `trace_function_key:` here is equally supported and is what those forms do underneath. #### `Bitfab::Traceable.node(klass, method_name, name: nil, type: "custom", capture: true, experiment_id: nil, mock_on_replay: nil, finalize: nil)` Experimental external-class form of `bitfab_node`. Uses `Module#prepend`; pass `client:` internally or prefer `client.node(...)` when configuration must belong to a specific client. ## `class Bitfab::CurrentSpan` ```ruby theme={null} def id # => canonical Bitfab span ID def trace_id # => String def add_context(context) # Hash → appends one entry; ignores non-Hash; never raises def set_prompt(prompt) # String → overwrites; ignores non-String; never raises ``` ## `class Bitfab::CurrentTrace` ```ruby theme={null} def set_session_id(session_id) # String def set_name(name) # String → the trace's title, searchable; empty strings ignored def set_metadata(metadata) # Hash → shallow-merge, later keys win def add_context(context) # Hash → appends entry def drop # flags the trace to be dropped at completion ``` Caller metadata is filterable by exact key/value pairs case-insensitively. It accepts up to 50 keys; keys use 1-40 letters, numbers, `_`, `.`, `:`, or `-`, and scalar values are at most 500 characters. All methods swallow exceptions internally. They never raise. `drop` flags the trace to be dropped. Once flagged, spans that complete afterward are not uploaded at all. The flag itself still rides out on the completion payload. At completion, the server scrubs any payloads that already raced out, meaning the trace itself, its external trace, and any sibling spans. It deletes the archived S3 objects. It marks the trace `dropped` instead of `completed`, keeping only a skeleton audit row. Calling `drop` outside a span, on `NO_OP_TRACE`, is a no-op. ## `class Bitfab::NoOpCurrentSpan` / `NoOpCurrentTrace` Singleton instances exposed as `Bitfab::NO_OP_SPAN` and `Bitfab::NO_OP_TRACE`. All methods are no-ops. `id` and `trace_id` return `""`. ## Thread & Concurrency Model * Span stack is stored in fiber-local `Thread.current[:__bitfab_span_stack]`. Enable `trace_across_threads: true` to propagate it to submitted threads and thread-pool jobs. Traced methods that return an `Enumerator` bridge their parent span into the same-thread `Enumerator.new` / `enum_for` source fiber. This keeps nested spans and replay mocks as descendants while the stream is consumed. * Replay context lives in true thread-variables instead (`Thread.current.thread_variable_get` / `thread_variable_set`). This context includes the mock tree, the DB branch lease, and the experiment ID. Unlike the fiber-local span stack, it persists across every fiber on the same thread. The thread propagation option copies it for each submitted job. Process replay creates a fresh context from its assigned historical item. * Trace state is stored in a mutex-protected module-level hash (`Bitfab::TraceState`), keyed by `trace_id`. * Spans and trace completions are queued on the client's private OpenTelemetry `BatchSpanProcessor`. They are delivered by its batch worker. Every other API call is blocking. * `Bitfab.flush_traces(timeout:)` waits for every live client's queued spans. It returns `false` when delivery fails or the deadline expires. `client.close(timeout:)` shuts a single client's worker down permanently. * Subtree capture reuses one `TracePoint` dispatcher per calling thread and keeps a separate method stack per fiber and active root. Nested roots share the dispatcher, but every root still filters and serializes its own copy of overlapping calls, so capture work grows with the number of overlapping roots. Same-key nested annotations and recursion also start independent roots during ordinary capture. Replay and seed execution keep nested annotations in the managed trace. Automatic `TracePoint` discovery stays on the root thread. The thread propagation option carries explicit spans and replay context into worker jobs. ## Error Behavior Summary | Situation | Behavior | | - | - | | Empty `api_key` | Resolved lazily, including an `ENV["BITFAB_API_KEY"]` fallback. Without `strict`, it warns and disables tracing. With `strict: true`, the first traced call raises `RuntimeError` instead | | `bitfab_span`-wrapped method called before `Bitfab.configure` | Runs untraced via `Bitfab.client_or_nil`. No error | | `Traceable.wrap` / `BitfabFunction#wrap`-wrapped method called before `Bitfab.configure`, with no `client:` bound | `RuntimeError("Bitfab not configured...")`. This path resolves `Bitfab.client`, not `client_or_nil` | | `bitfab_span` method raises | Span records `error` and `error_source: "code"`. The exception is re-raised | | `bitfab_trace` descendant raises | Descendant records the exception; root behavior and exception propagation are unchanged | | `bitfab_node` finalizer raises | Node records `finalize failed: ...`; the original method result is still returned | | `add_context` / `set_prompt` with invalid input | Silently ignored. Never raises | | Span transport failure | Swallowed, though it warns once. The host app never crashes. `flush_traces` returns `false` | | Span submitted after `client.close` | Warned once and dropped | | `Bitfab.client` before `configure` | `RuntimeError("Bitfab not configured...")` | | `bitfab_span` without `trace_function_key` | `RuntimeError` | | `replay`: `trace_function_key:` differs from the method's declared key | `ArgumentError` (key mismatch) | | `replay`: Bitfab never confirms a replay's spans in time | One warning per replay on stderr; the run completes and no item is marked errored | | `replay`: no completed item's trace persisted, or finalization fails, after items settle | `Bitfab::ReplayError` with collected `items`, the experiment ID and URL, and original `cause` | | `replay`: requested database branch cannot be resolved | Item `:replay_error` is `Bitfab::DbBranchReplayError` with the resolver code, message, and original trace ID | ## Capture policy and simulation plans `Bitfab::Client.new(capture_enabled: false)` turns off ordinary recording while keeping traced methods replayable. Replay and seed execution still record when credentials are available. `enabled:` remains a deprecated alias; use `capture_enabled:` for new code. The same options work with `Bitfab.configure`. The SDK reads the simulation plan in the background and applies capture-off settings using the owning trace function key and span name. Once the plan has loaded, a span whose capture is off is not sent, and its children carry the nearest captured ancestor the SDK resolved while the trace ran, so Bitfab nests them under the closest call above them whose capture is on. The root span of each trace, framework-origin spans, and spans that recorded an error are still sent without content. Configured nodes and nested trace roots ignore the plan and always keep their content, and a discovered call that shares a configured node's name is still dropped. The client starts reading the sim plan when it is built, or at first use when its API key is resolved lazily (a proc, or read from `BITFAB_API_KEY`). A `bitfab_trace` root that starts while that first read is still running waits for it, for at most five seconds after that first read began. Once the first read has finished, whether it succeeded or failed, no later root waits. Nested roots and the calls beneath a root never wait, and nothing waits when the plan is disabled. While the first read is still running, it holds records off the calling thread. When that read succeeds, held spans are sent with the plan applied. When it fails (a network error, a timeout, an HTTP error other than 404, or a response it cannot understand), held spans and every later span are sent without inputs and outputs until a read succeeds, and the SDK warns once. Those spans carry `content_off_by_simulation_plan` and still count toward `max_captured_subtree_spans`. The root span of each trace, framework-origin spans, configured nodes, and nested trace roots keep their content. Sending a span without content removes inputs, outputs, their serialized forms, and the prompt while keeping timing, errors, context, and links. A 404 means the server has no plan feature, so records are sent unchanged. A failed refresh after a successful read keeps the last plan. At most 1,000 records are held, and any beyond that are sent without inputs and outputs instead of being dropped. `flush` and `close` wait within their timeout for held records and in-progress submission, preserving span-before-completion ordering. `close` and process exit never drop held records. Records still held because the plan has not loaded are sent without inputs and outputs. Use `simulation_plan: false` or a nonblank `BITFAB_DISABLE_SIM_PLAN` to disable plan reads. `timeout:` configures the default HTTP timeout in seconds; plan reads have a separate five-second bound. Every emitted span carries `span_origin` with the Ruby SDK version and its instrumentation source (`span`, `trace`, or its framework adapter). ## Seed and reseed `seed_trace` records a supplied case or runs a callable once. Both forms work with `capture_enabled: false`, require an API key, mark the trace as `seeded`, and omit database snapshot pins. ```ruby theme={null} double = ->(value) { value * 2 } id = client.seed_trace("double", input: [3], expected: 6, fn: double, metadata: {row: "example"}) run_id = client.seed_trace("double", double, args: [3], name: "fresh case") adopted = client.reseed_trace("double", double, trace_id: id) ``` Case inputs are positional argument arrays. Optional `fn:` validates the callable's signature without executing it. Run-based seeds also accept `kwargs:`. Both forms accept `metadata:`, `session_id:`, and `name:`. Reseeding loads the stored inputs and metadata and adopts the successful run under the existing trace ID. Labels, assertions, and dataset memberships stay on that trace; `:previous_run_trace_id` identifies the previous run. A failed execution is recorded and raised without adopting it. The registry accepts plain callables with an explicit key: ```ruby theme={null} REGISTRY = Bitfab::ReplayRegistry.new.register( "double", double, client:, trace_function_key: "double" ) ``` Use `bitfab-seed --registry ./registry.rb double --cases cases.jsonl` to record cases without execution, add `--run` to execute each case once, or use `--from-trace id1,id2` to reseed existing traces. The replay command also accepts `--seed` as an alias for `--cases`. JSON or JSONL case rows require an `input` array and may include `expected`, `metadata`, `sessionId`, and `name`. Run-based cases must omit `expected`. The CLI rejects empty replay selections and emits the final result as JSON. Registry option `on_item_finish:` receives each item completion after the progress reporter. Use it to save grading labels while items finish. Callback errors are reported without aborting replay; dry runs skip the project callback. `Bitfab::Client.new(db_snapshot: {provider: "neon"})` explicitly records Neon as the database snapshot provider on ordinary trace roots. Unsupported providers are rejected at construction. Without this option, roots carry only their timestamp and the server resolves the provider at replay time. This configuration performs no database I/O; seeded traces still carry no snapshot pin. Replay summaries follow TypeScript's case semantics: seeded rows show “Matched expected” or “Missed expected” against their stored output, while captured rows show “Same” or “Changed.” For a case seed, that stored output is `expected`; for a run-based seed, it is the previously recorded execution result. Python currently seeds by running code and uses “Same/Changed” summaries for all rows. Errors are counted separately and are never counted as matches. ## BAML execution `client.call(name, **inputs)` executes a stored BAML prompt locally and records its result and native collector. Configure `env_vars:` to supply provider credentials. Only `OPENAI_API_KEY` is forwarded to the native runtime. Install the optional `baml` and `sorbet-runtime` gems. `client.wrap_baml(client_or_method, method = nil, on_collector: nil)` enriches the active span with the rendered prompt, model, provider, token counts, and duration. A method can be a bound Method, symbol, or string. Set `baml_client:` in the constructor or pass the generated client explicitly. The returned callable exposes `collector`. The fluent function handle forwards `wrap_baml` to the owning client. Collector callbacks cannot change the wrapped method's result. ## Worker threads Set `trace_across_threads: true` on the client or `BITFAB_TRACE_ACROSS_THREADS=1` to carry span, replay, and seed context into newly submitted work. This installs process-wide hooks for `Thread.new`, `Thread.start`, and `Thread.fork`. If your application includes `concurrent-ruby`, it also hooks thread-pool `post` calls. SDK transport threads and pool implementation threads are excluded. A worker's spans nest under the submitting span. Replay mocks and the item's database lease remain available there. Each job gets its own span stack. Reused workers restore their previous context after the job finishes or raises. This option does not install automatic subtree discovery on worker threads. Every span's runtime metadata identifies the executing thread. Propagated jobs also record the submitting thread's ID and name. Join application work before closing the client so its spans can finish. ## Process replay Use `Bitfab::ReplayConcurrency` on a registry entry when each replay item needs a fresh interpreter. ```ruby theme={null} REGISTRY = Bitfab::ReplayRegistry.new.register( "workflow", workflow, client:, trace_function_key: "workflow", concurrency: Bitfab::ReplayConcurrency.new( primitive: "process", attempts: 2, max_concurrency: 4, on_item_finish_in_child_process: ->(event) { puts event[:item][:trace_id] } ) ) ``` Run `bitfab-replay --registry ./registry.rb workflow`. Process mode reloads that registry for every item and attempt. It requires this installed registry command. Calling `client.replay` directly with process mode raises before creating an experiment. The parent starts and completes one experiment. Each child resolves its own inputs and database lease. Its result is written after trace delivery is confirmed. The child-process hook runs afterward and receives `experiment_id` (plus the deprecated alias `test_run_id`) and the finished `item`. Hook errors are reported without discarding the result. Registry `on_item_finish:` still runs in the parent. Dry runs skip customer execution, leases, and grading hooks. `primitive: "async"` uses the existing thread pool and defaults to ten workers. `primitive: "process"` defaults to four children. `attempts` defaults to one and accepts 1–100. Process concurrency must be bounded. Passing the `concurrency:` object together with scalar `attempts:` or `max_concurrency:` is rejected. CLI `--primitive`, `--attempts`, and `--max-concurrency` override registry defaults. Process mode enables memory admission by default. The next launch waits when free memory cannot cover active children's expected growth, another child, and a reserve. Critical memory pressure also delays launches, meaning the critical `kern.memorystatus_vm_pressure_level` on macOS or every task stalled on memory for 10% or more of the last ten seconds on Linux. Swap at 85% used delays launches only where pressure cannot be read. At least one child is always permitted to make progress. Linux and macOS use local memory readings. Other platforms continue without a memory estimate. Set `memory_throttle: false` or CLI `--no-memory-throttle` to disable it. `BITFAB_REPLAY_CHILD_MEMORY_MB` and `BITFAB_REPLAY_MEMORY_FLOOR_MB` both default to 2048. `BITFAB_REPLAY_MEMORY_THROTTLE=0` disables admission for the process. `child_timeout:` defaults to 2400 seconds. Timed-out children are terminated and produce an item error when they have not written a result. A completed result survives a timeout in the later grading hook. Child processes receive the environment captured when the SDK loaded. Mutations made while loading the parent registry do not leak into later children. `child_delivery_timeout:` is how many seconds a child waits for its spans to reach Bitfab. It bounds two waits. The first runs before the result is written: the child flushes the item's spans to read back `trace_id`, and if that flush does not confirm delivery, it asks Bitfab whether the trace is stored for up to the same number of seconds again. The second runs after `on_item_finish_in_child_process`: the child closes its client, which sends whatever is still queued, including spans the hook itself recorded, before it exits. Raise it when a large trace uploads slowly and items come back with an error or with spans missing. It defaults to `30.0` under `primitive: "process"`, must be a positive number, and is refused under `primitive: "async"`, since no child process exists there. Keep it well below `child_timeout:`. A child reports span delivery failures, such as a rejected or failed upload or delivery not confirmed within `child_delivery_timeout:`, as `[replay] span delivery problem for #: ...` on the command's own stderr. Repeated messages are counted instead of repeated. ```ruby theme={null} Bitfab::ReplayConcurrency.new( primitive: "process", on_item_finish_in_child_process: grade, child_delivery_timeout: 180 ) ``` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.