Tracing works with proxies and computed getters
Automatic tracing now handles code that runs a function on property access, such as a database client wrapped in a proxy or an object with computed getters. Previously the SDK read those properties while it was still recording, which ran your code again and could exhaust the call stack, leaving the run with no trace at all. Update to TypeScript SDK v0.53.4.DashboardPluginsTypeScript SDKPython SDKRuby SDKGo SDKCLI
Plugins v0.11.5, TypeScript SDK v0.53.3, Go SDK v0.51.2, CLI v0.2.330
One reference for SDK setup
SDK setup now uses the public documentation for language and framework details. The docs also clarify Go replay support, filtering dataset replays, and tracing shared packages in TypeScript monorepos.Install the TypeScript transform under the Bitfab scope
The TypeScript build adapters are now available as@bitfab/transform. If you used bitfab-transform, replace the dependency and update your imports and build configuration to the scoped package. The Next.js adapter now resolves its loader from @bitfab/transform/loader, and plugin setup instructions use the new package name.Keep partial output when model streams stop
Vercel AI SDK tracing now records partial output and the error or cancellation when a model stream stops early. Once the stream ends or is cancelled,flushTraces() can finish delivering its span without waiting on an unfinished stream finalizer.Capture shared monorepo packages
The Node transform now resolves its injected runtime from the SDK installation, so shared packages can be captured even when only the application depends on the SDK. UseBITFAB_SDK_RESOLVE_FROM to select the application’s package location when needed; ESM and CommonJS are both supported.Turning a node off turns off the nodes beneath it
When you turn content capture off for a node on the Sim plan page, every node beneath it in the map turns off too, the same way untracing a span in a trace plan drops the spans under it. Nodes that always keep their content (recorded by a framework integration, the root of the workflow, imported from another platform, or named by Bitfab) stay on. Turning a node back on leaves the nodes beneath it as they are, so switch those on individually.Thesave_sim_plan tool in the Claude Code, Cursor, Codex, and Amp plugins follows the same rule, and a node you name in the same call keeps the value you give it.DashboardPluginsTypeScript SDKPython SDKCLI
Plugins v0.11.1, TypeScript SDK v0.53.1, Python SDK v0.53.1, CLI v0.2.326
Turn content off per node from the Sim plan
The TypeScript and Python SDKs now read your Sim plan and stop sending inputs and outputs for any node whose content you turned off. The span still arrives with its name, type, timing, errors, and links, marked as content off, so the Sim plan page keeps counting it while the payload never leaves your process. Turn a node off on the Sim plan page or with thesave_sim_plan tool, and new traces pick the change up within a minute. Spans are held until the plan has loaded, so short-lived processes honor it too.See which SDK and integration recorded a span
Every span now carries its origin: the SDK name and version, and whether it came fromwithSpan or @span, withTrace or @trace, or a framework integration (Vercel AI SDK, Claude Agent SDK, OpenAI Agents, LangGraph). The trace header shows it as “Recorded by”. Spans recorded before this release resolve their instrumentation from the trace source instead, and show no SDK name or version.Roots and framework spans always keep their content
The Sim plan refuses to turn content off for a node that is the root of its traces, a node recorded by a framework integration, an imported node, or a node Bitfab named from its model or type. The switch shows on and disabled with the reason, and thesave_sim_plan tool returns an error that says why.Browse a node’s recent occurrences from the Sim plan
Click any row on the Sim plan page to open that node’s recent occurrences, newest first, each with its size and whether content was captured. Open one to see the span inside its full trace.DashboardPluginsTypeScript SDKPython SDKCLI
Plugins v0.11.0, TypeScript SDK v0.53.0, Python SDK v0.53.0, CLI v0.2.325
Decide per call what a Sim plan records
Each call on a trace function’s Sim plan now has a Capture switch. Turning it off marks that call as one whose inputs and outputs should stop being recorded, while the call itself, with its name, type, timing, and errors, keeps being captured so the map stays complete. The decision is stored with the plan and shows who made it. The SDKs will start honoring it in an upcoming release. Sim plan is an alpha switched on per organization.See what each call costs before you decide
Every call on the Sim plan shows its average recorded payload and an estimate of what it costs per month at $2 per GB, based on how often the function ran in the last 30 days. Click the payload to open recent examples of that call in the standard trace view, inputs and outputs included, and step through them. The plan also flags a call reached from more than one parent, since a switch there affects every one of them.Edit the Sim plan from your coding agent
Two new MCP tools let a coding agent read and change a plan.get_sim_plan lists a trace function’s calls with their type, call counts, payload size, monthly estimate, and capture state, and save_sim_plan turns capture off or on for the calls you name. The Sim plan page’s Edit plan button shows the prompt to use, and a trace function’s traces page now links to its plan.Method spans are named after their class
A span on a class method now defaults toClass.method in both SDKs, so two methods that share a name are two distinct spans rather than one. Plain functions, closures, and explicit names are unchanged, and the raw function name still travels separately as function_name.Dashboard
Jump between a nested trace and the trace that started it
When a trace function runs inside another traced function, the span tree now shows the connection. In the outer trace, the span for the nested call carries a small link naming the nested trace function, and clicking it opens that trace at its root span. In the nested trace, the root span carries a link back to the outer trace and opens it at the span that made the call. Traces recorded before this release show no link.TypeScript SDKPython SDKPluginsCLI
TypeScript SDK v0.52.6, Python SDK v0.52.7, Plugins v0.10.12, CLI v0.2.324
Nested trace roots record both traces
When atrace() root runs beneath another one, both functions now get a complete trace: the outer keeps recording every call beneath it, and the nested root records its own trace under its own key, so you can label, collect, and replay one part of an agent on its own. This is new in TypeScript, where a nested withTrace() or trace() root used to take over the outer’s capture; Python already recorded both traces and now links them. Each root applies its own limits, exclusions, and capture policy to its copy, and a node() inside the nested region keeps its name, type, test run, and finalized output in every copy.Nested traces link to each other
The outer trace’s span for the nested root now carries the nested trace’s id, trace function key, and root span id, and the nested root’s span carries the enclosing trace’s id and span, so the two traces can be followed from either side. Inside a replay or while seeding a trace, a nested root is absorbed into the item’s trace instead of starting a second one, so an experiment never gains traces under another key.TypeScript SDKPython SDKRuby SDKGo SDKDashboard
TypeScript SDK v0.52.5, Python SDK v0.52.6, Ruby SDK v0.51.2, Go SDK v0.51.1
Every trace records its commit
Every trace now carries the commit its code was running at, in all four SDKs, so you can tie a captured run to a point in your repository’s history rather than only to a timestamp. The SDK reads it from your deploy platform’s build variables on Vercel, GitHub Actions, Railway, Render, Heroku, Cloudflare Pages, GitLab CI, Azure Pipelines, and CircleCI, falls back togit in a plain checkout, and never slows the traced call. In a container built without either, set BITFAB_COMMIT_SHA at build time, and set BITFAB_DISABLE_COMMIT_REF to turn the capture off. The ref records the commit, the branch, whether the tree had uncommitted changes, and the repository as host/owner/repo with any credentials removed.Dashboard
An error partway through no longer erases a trace’s verdict
A trace that hit an error somewhere in its span tree but still ran to completion now keeps its verdict and counts toward your experiment pass rate. Until now a single error anywhere in the tree pulled the whole trace out of the results, out of the pass rate, and out of the before and after comparison, even when graders had already scored it. Only a run that crashed outright, where the traced function threw all the way out and produced no result, is still set aside as ungradable.Errors read as their own signal, beside the verdict
Trace rows now carry an error badge showing how many spans threw, next to the pass rate rather than in place of it. A run that crashed at the root reads Crashed and keeps its red marker, while a run that recovered shows its error count alongside its normal verdict, so you can tell a broken run from a noisy one at a glance.Experiment breakdowns keep a label’s baseline
The Labels and Attempts rows of an experiment breakdown no longer report work as having no baseline when the Traces row already had something to compare it against. An original labelled as a whole, or scored by different graders than its replay, carried a baseline at the trace level but not at the label level, so its labels fell into the no-baseline segment of the bar and sat outside the change in points. All three rows now read the original’s verdict the same way, so that segment is left for originals that really were never labelled.Sim plan, an alpha code map of each trace function
A trace function now has a Sim plan: a code map of every call its traces have recorded, built from its 20 most recent runs and shown in the sidebar below Graders. Each call appears once, with its span type, how many times it ran, the share of recent runs it appeared in, and when it was last seen, so a branch that stopped running fades while the rest of the map stays put. Each call also carries the judgments the plan will drive, whether its inputs and outputs are captured and whether a reviewer needs to see it, and the root of every span tree links to its function’s plan. Sim plan is an alpha and is switched on per organization, so ask us if you would like it enabled.Dark mode
Bitfab now has a dark theme. Flip it with the sun and moon toggle in the header, or choose Light, Dark, or System from the account menu, where System follows your operating system. Your choice is remembered, and it carries across the trace, dataset, experiment, and grader views as well as Studio and the plugin pages.Dashboard
See how an experiment moved, at three zoom levels
Opening an experiment now shows a breakdown beneath it that reads the same run three ways: by trace (a trace passes when at least 75% of its labels pass), by label (each assertion or grader rolled up across its attempts, with at least 75% of attempts passing counting as a pass), and by every label on every attempt. Each row shows the pass rate, the change against the original traces in points, and a bar splitting the run into held, fixed, regressed, and still failing, so a run that looks unchanged by trace can show where it improved underneath. Runs with several attempts per trace get a Jitter row counting the attempts that disagreed with their trace’s verdict. When there is nothing to compare against, the rows show plain passing and failing instead of an empty change, and a run that mixes compared traces with ones whose originals were never labelled counts both, drawing the uncompared ones in a lighter shade.Filter an experiment’s traces by how they changed
The trace list under an experiment now filters to fixed, regressed, still passing, or still failing traces, with an Unstable filter on multi-attempt runs for traces whose attempts disagreed. The list scrolls on its own, so the breakdown above it stays in view.Grader labels count the same as your labels
An experiment’s pass rate, raw count, change buckets, and filters now read one verdict per trace that includes grader labels alongside labels written by people or agents. A grader’s label counts whenever it is on the trace, whether or not that grader is attached to the experiment. Two things follow. A grader attached to an experiment that has not yet labelled a trace no longer holds that trace’s verdict back, and a run’s headline pass rate counts traces rather than individual grader checks, so grader-scored experiments will read differently than before. The graders panel keeps its own per-grader counts.Experiment rows show the count and the change
Each experiment’s row now carries the passed-over-scored trace count and the change against the originals in points beside its pass rate, and its bar is a single fill at the pass rate. All four read the trace rollup, so a five-attempt run of 50 traces reads 36/50 rather than 180/250. The four-way split moved into the breakdown, where it has the room to be read.Dashboard
Label badges count every check a trace is held to
A trace’s label badge now counts what the trace is judged against, which is its assertions when it has any and the trace as a whole when it has none. Nothing counts as passed except a label you marked pass, so a trace whose assertions nobody has labelled reads as unlabelled instead of showing a full pass. Anything still unlabelled counts on neither side of the badge, and a trace where one assertion passed and another is still open now shows the pass instead of reading as unlabelled.Deselect a label by clicking it again
Clicking the Pass, Fail, or Skip you already chose now clears it and returns the trace or assertion to having no label. This works the same way on trace labels and on grader labels. Clearing your own label leaves any agent or automated suggestion untouched, so the suggestion is still there when you come back to it.TypeScript SDKPython SDKPluginsCLI
TypeScript SDK v0.52.4, Python SDK v0.52.4, Plugins v0.10.9, CLI v0.2.321
Re-seed a trace in place
When a captured trace errored, or its recorded run is stale, you can now re-seed it: run the function once more on the trace’s recorded inputs and record the result under the same trace id. The trace keeps its id, labels, assertions, dataset membership, name, and metadata, so anything you stored against it still points at the same case. The previous run is kept as its own trace, linked back withreseedOfTraceId, so nothing is lost. Nothing is mocked and no experiment is created; a re-seed is a seed, not a replay.bitfab-seed is the new home of seeding: --from-trace re-seeds, and --cases cases.jsonl does what bitfab-replay --seed did (that spelling still works). From code, bitfab.reseedTrace(key, fn, { traceId }) / reseed_trace(key, fn, trace_id=) does the same. The Bitfab assistant’s replay mode takes this path when you ask for a re-seed or the trace errored, and warns first when the function has side effects, since a re-seed runs them for real. Graders on the trace’s datasets re-run afterwards, and default replay selection skips previous runs.Dashboard
Empty spans hidden in the trace view
The span tree now hides spans that recorded no input and no output, lifting their children up a level, so every row you click has something to show. A toggle at the top of the span panel says how many are hidden and brings them back, and your choice is remembered. Spans that recorded an error, and spans that differ in a replay comparison, always stay visible.TypeScript SDKPython SDKPluginsCLI
TypeScript SDK v0.52.3, Python SDK v0.52.3, Plugins v0.10.8, CLI v0.2.320
Seeded metadata survives a streamed agent run
Metadata you record withseedTrace / seed_trace or setMetadata / set_metadata now stays on the trace even when an agent framework exports its own trace after your traced function has returned. A streamed OpenAI Agents run ends its trace when the stream drains, which happens after the function returns, so its export used to take the trace back and a later replay read the wrong values in adaptInputs / adapt_inputs. Requires TypeScript SDK v0.52.3 or Python SDK v0.52.3.Clashing metadata keys warn once per trace
When you and an integration both set the same metadata key to different values, your value is the one kept on the trace and the SDK warns. That warning now fires once per trace and names the trace id instead of once per process, so a large replay run shows every affected trace rather than only the first.TypeScript SDKPython SDKRuby SDKPluginsCLI
TypeScript SDK v0.52.2, Python SDK v0.52.2, Ruby SDK v0.51.1, Plugins v0.10.7, CLI v0.2.319
Replay part of a dataset with --limit
--limit now narrows a dataset or trace ID selection instead of being ignored. bitfab-replay --dataset-ids <id> --limit 10 replays ten of that dataset’s traces rather than all of them, so you can sample an expensive corpus while iterating and pay for the full run only when you want it. The same bound applies to an explicit --trace-ids list, and to a dataset your registry entry already declares.A bounded dataset run selects the same traces every time, so two runs at the same limit stay comparable. Available in the TypeScript, Python, and Ruby SDKs.TypeScript SDKPython SDKPluginsCLI
TypeScript SDK v0.52.1, Python SDK v0.52.1, Plugins v0.10.6, CLI v0.2.318
Seeded trace metadata survives an agent framework’s own trace
Metadata you pass toseed_trace / seedTrace, or set with set_metadata / setMetadata, now always reaches a replay’s adapt_inputs / adaptInputs hook. Tracing a function that also opened an OpenAI Agents trace previously replaced your metadata with the framework’s own, so a seeded case’s provenance went missing exactly where the replay needed it. Both sets of keys are kept now, and yours win on a conflict.Save assertions across many traces in one call
Both SDKs can now write assertions for many traces in a single request instead of one request per trace.saveAssertionsAll in TypeScript and save_assertions_all in Python take one entry per trace, so a script attaching assertions to hundreds of traces makes one call rather than hundreds.Dashboard
Traces are identified by name
The trace list now shows the name your code gave each trace instead of an eight-character id. Names come fromsetName in the SDK, from a seeded trace, or from the trace’s name on a platform you imported from, so a run reads as something you recognize rather than a fragment of a uuid. A trace whose name only repeats its function name keeps showing the id, so no row gains a label that tells you nothing new.Assertions show on dataset and experiment lists
Dataset and experiment trace lists now name who authored a trace’s assertions, which until now only the main traces list did. Hovering says how many assertions the trace carries, and on an experiment row it adds that they were written on the original trace rather than on the replay in front of you.Your coding agent can name the trace to open
Trace reads through the Bitfab MCP server now include the trace name, so an agent can tell you which trace to look at by name instead of quoting a uuid. Reading traces, searching them, and listing an experiment’s traces all report it.DashboardPluginsTypeScript SDKPython SDKRuby SDKGo SDKCLI
TypeScript SDK v0.51.0, Python SDK v0.51.0, Ruby SDK v0.51.0, Go SDK v0.51.0, Plugins v0.10.4, CLI v0.2.316
Replay several datasets in one experiment
A replay could only point at one dataset, so measuring a function against several corpora meant running each of them separately and comparing the experiments by hand. You can now name as many datasets as you like. The run replays their combined traces once, scores them with the graders from every dataset you named, and files the experiment under each one, so it shows up on all of their experiments pages.bitfab-replay <pipeline> --dataset-ids ds-a,ds-b. The existing --dataset-id flag keeps working and now takes a list too, so the replay scripts you already have need no changes. All four SDKs have it, as datasetIds, dataset_ids, and DatasetIDs.DashboardPluginsTypeScript SDKPython SDKCLI
TypeScript SDK v0.50.0, Python SDK v0.50.0, Plugins v0.10.3, CLI v0.2.315
Read verdicts back per assertion
A verdict written against one assertion could only be read back as a passed and failed count, so an agent that scored six assertions on a trace could not say which one failed without opening Studio.get_trace_labels now lists each scored assertion on its own line under the trace, with that assertion’s verdict, annotation, confidence, and author, keyed by the same assertion id the write used.Labels and grader verdicts from the SDKs
The TypeScript and Python SDKs can now read verdicts, write human-validated verdicts, and read the individual verdicts each grader recorded, the same operations available through the Bitfab MCP tools. Reading returns each trace’s effective verdict plus one row per scored assertion, so a judge running inside a replay can verify what it wrote without leaving the process.labels.saveHuman writes a verdict that is validated on write, for cases a person has already decided. Human and agent batches are all-or-nothing. A trace outside your organization or an assertion that is no longer active rejects the whole call and writes nothing. Python exposes the same methods as labels.get_all, labels.save_human, and graders.get_labels.Retire an assertion from your coding agent
archive_trace_assertions retires one or more assertions on a trace so later replays stop checking them. Archiving keeps the row for audit and leaves any verdicts already recorded against it in place. The call is all-or-nothing, so one unknown or already-archived id fails it and archives nothing.DashboardPluginsTypeScript SDKPython SDKCLI
TypeScript SDK v0.48.0, Python SDK v0.49.0, Plugins v0.10.2, CLI v0.2.314
Replay only the traces you have assertions for
Replaying a dataset re-ran every trace in it, including the ones nobody had written an expectation for. Those runs cost a full re-execution and come back with nothing to grade. PassonlyWithAssertions to narrow a replay down to the traces carrying at least one assertion.bitfab-replay the flag is --only-with-assertions, and it narrows --limit, --trace-ids, and --dataset-id alike. Paired with --limit it picks the N most recent traces that have assertions, instead of taking the N most recent and leaving you with however many of those turn out to be gradeable.Available in the TypeScript and Python SDKs.Dashboard
Experiment totals now match the verdicts on your traces
When a trace carried a label on each of its assertions, its experiment totals could count it as passed even though the trace list showed it as failed. Totals and trace verdicts now follow the same rule, so any failing assertion fails the trace. Grader results count toward that verdict as well, so an experiment’s pass rate reflects every label written against a trace.Dashboard
Replay diffs stop flagging repeated calls that just ran in a different order
When a replay called the same function many times, the diff matched those calls by position, so a set of identical calls that came back in a different order was reported as a wall of changes. Calls are now matched by what each one was given, so only real differences surface. On one replay with 31 repeated calls, the reported changes dropped from 30 spans to 2, and the change that mattered (a verdict moving from approve to request changes) went from buried to second in the tree.The Diff view has a new Diff algorithm control. Smart is the default and matches calls by content. In order keeps the old positional matching, for when the sequence itself is what you are testing.Threads in the span tree
A trace that ran work on more than one thread now marks each span with the thread it ran on, as a smallT1 or T2 chip on the row with the full thread name in the row tooltip. Traces that used a single thread are unchanged.Grade a replay item inside the process that ran it
ReplayConcurrency now takes on_item_finish_in_child_process, a hook that runs in the child interpreter that replayed the item rather than in the process that owns the run. Under primitive="process" that child is the only place the replayed run’s own state still exists, so a judge can read what the run left in memory before the process exits. The item arrives with its replay trace ID already resolved, so a verdict can be keyed by trace_id directly instead of by lineage.on_item_finish is unchanged and still runs in the process that owns the run, so both can be set at once. The child grades what only the child can see, and the parent keeps the running totals and still reports an item whose child died before finishing.DashboardTypeScript SDKPython SDKPluginsCLI
TypeScript SDK v0.47.0, Python SDK v0.47.0, Plugins v0.10.0, CLI v0.2.312
One verdict per assertion
A trace’s verdict now comes from its assertions instead of a single pass or fail. Each assertion carries its own verdict and its own reasoning, so a run that meets four of six expectations reads as 4/6 rather than one failure with the detail buried in a note. The trace verdict is derived from those labels, and any failing assertion fails the trace.Traces with no assertions are unchanged. They keep their single whole-trace verdict, and every verdict recorded before this release reads exactly as it did.Labels and assertions are separate panels
The trace drawer now has a Labels panel and an Assertions panel behind a toggle, and either one collapses to give the full width back to the trace. Labels is where you record verdicts, one row per assertion with its own pass, fail, and note. Assertions stays read only and shows what the trace is expected to do when it is replayed.Labels are counted rather than named. A badge reads passing over total, so a single verdict shows as 1/1 and six assertions with one passing show as 1/6. Traces with nothing recorded keep the dashed unlabeled pill.Score one assertion from the SDKs
The labels namespace takes an optional assertion id, so a replay can grade one expectation at a time. Omit it to write the trace’s whole-trace verdict, which behaves exactly as before.assertion_id on save, save_all, skip, and archive.Dashboard
Live in-progress trace indicator
Trace detail pages now make it clear when a trace or replay is still running, with a banner that appears while it’s in flight and disappears once results are in. The page keeps checking for updates in the background and jumps to the latest span when you open a trace that’s still going.Diff view always highlights a span
The experiment diff view now always opens with a span selected, even when every difference between the original and replayed run is a span that simply didn’t run this time. Previously the view could open with nothing highlighted.Python SDKTypeScript SDKPluginsCLI
TypeScript SDK v0.46.0, Python SDK v0.46.0, Plugins v0.9.105, CLI v0.2.311
Grade each replay attempt as it finishes
A replay registry entry can now carry anon_item_finish callback, and bitfab-replay runs it after its own progress reporter instead of replacing it. That is the earliest point a verdict can be written, because a replay verdict is keyed by the original trace plus the attempt plus the test run, and that row only exists once the attempt’s own trace has been flushed.--dry-run, since nothing ran, and one that raises is reported on stderr with the trace and attempt instead of failing the run. Python for now, with Ruby and TypeScript to follow.One verdict with save, many with save_all
save_one is gone. save is now the single-verdict call and takes the same target arguments skip and archive take, and save_all takes a list. TypeScript is save and saveAll.This is a breaking change. Update any call that used save_one or saveOne, and any call that passed a list to save.Dashboard
Faster loading for large traces
Traces with large payloads now open in about a second instead of the fifteen to twenty seconds they could take before. The trace page used to download the full input and output of every span before it could show you anything, which made big agent traces slow to open. Span content now loads when you select a span, so the span tree and timings appear right away.Dashboard
Automatic capture policy is in Alpha
The capture control on a TypeScript SDK function trace is now labeled Policy with an Alpha badge. It still opens the Studio capture policy for that function, but the page is read-only for now: you can see which functions include content on future traces, and you cannot change or save the selection while the feature is in Alpha.TypeScript SDKPython SDKPluginsDashboardCLI
TypeScript SDK v0.45.0, Python SDK v0.45.0, Plugins v0.9.104, CLI v0.2.310
Say what a trace should do the next time it runs
A grader checks something that must hold for every trace of a function. An assertion is the other half: what the right answer is for one specific input. Write it against the trace you are looking at, and it is checked against the replay of that trace rather than against the run you wrote it on.Your coding agent writes them through two new MCP tools,save_trace_assertions and get_trace_assertions, and both SDKs read and write them directly:client.labels is new too. Verdicts from a replay process used to be reachable only through MCP, which meant a judge running inside your own replay script had nowhere to write. It now writes the same rows, keyed the same way, and skip withholds a verdict on an attempt that crashed or an assertion whose target could not be found, so neither one is recorded as a behavior regression.Assertions show up read-only on the trace page and in the labeling panel. Fill them in from your agent or the SDK for now.An assertion takes the same assertion, passCriteria, and failCriteria a grader takes, so one that proves out across many traces is promoted into a grader by copying its fields.TypeScript SDKPython SDKRuby SDKGo SDKPluginsDashboardCLI
TypeScript SDK v0.44.1, Python SDK v0.44.1, Ruby SDK v0.43.1, Go SDK v0.41.1, Plugins v0.9.103, CLI v0.2.309
Trace outlines on every replay item
Every replay result item now carries two trace outlines:originalTraceOutline for the trace that was replayed and traceOutline for the trace the replay produced. An outline is the span tree without any inputs or outputs, so it stays small: each span’s name, type, order, duration, tokens, model, errors, and whether it was served from a recording. Use them to grade how a replay reached its output and not only what it produced, such as whether it called the same tools in the same order or leaned on a mocked span that used to run real code.trace_outline and original_trace_outline, and Go as TraceOutline and OriginalTraceOutline. Both are filled in when the replay completes, so they are null on the per-item finish callback and against older servers.The assistant grades replays by their path
The Bitfab plugin now writes both outlines into each replay run’s per-item files and compares the two span trees when it judges a replay, so a replay that produced the right text by skipping a required tool call, erroring in a child span, or leaning on a mocked span no longer passes on output alone. Items judged while the replay is still running are re-checked against their outlines once the run completes.Replay each trace in its own process
Python replay can now give every work item its own process, which makes it possible to replay code that keeps its world in process-global state: a settings module read once at import, a database provisioned per item, or a module-level registry. Previously all items shared one process, so anything that could not be set up twice in the same interpreter only ever replayed its first trace.ReplayConcurrency also carries attempts and max_concurrency, so a run’s repeat count and its parallelism are set in one place and cannot disagree. The fan-out unit is the work item rather than the attempt, so 40 traces at 3 attempts is 120 processes drawn from one queue, bounded by max_concurrency (4 by default in process mode). The default primitive stays "async", so existing replays are unchanged. Process mode runs through the bitfab-replay --registry command, which is what re-runs your pipeline once per item.The Amp plugin, at parity
Amp support graduates from the Alpha skill pack to a native plugin. It registers the same three skills the other hosts get,bitfab:setup, bitfab:assistant, and bitfab:update, and every Bitfab tool, so Amp runs the full workflow itself instead of handing off to a second agent in your terminal.bitfab:update sdk, and Bitfab tools you can call directly. Ask Amp to search your traces and it can. The tools are registered by the plugin under their bare names (search_traces, get_traces, save_trace_plan) rather than through an MCP server.bitfab-cli installs the plugin into ~/.config/amp/plugins/bitfab and confirms Amp loaded it. analyze-repo --editor amp runs headless through amp -x.Still to come on Amp: session log capture and the auth and update banner at session start. Both ship through Amp’s agent.start and agent.end events in a follow-up. Details are in the Amp plugin docs.Bitfab in Amp, in Alpha
Amp is now a supported host, in Alpha.bitfab init installs Bitfab for Amp the same way it does for the other editors:--editor amp works for plugin-install, setup, and analyze-repo too, and the CLI offers Amp automatically when it is the editor on your PATH.Three skills. bitfab-setup instruments workflows and covers modify, inspect, replay, database snapshots, and templates. bitfab-analyze-repo scans the repository and uploads draft trace plans without prompts or code edits. bitfab-account handles sign-in, organization switching, and health checks.Alpha means the improvement loop is not there yet. Datasets, graders, experiments, trace labeling, and Studio review all need the native Amp plugin, which is planned. Amp also registers no Bitfab MCP tools of its own, so you cannot ask Amp to query your traces directly, though the setup flow itself has the full tool set. Run the improvement loop in Claude Code, Cursor, or Codex for now. Full limits are in the Amp plugin docs.Mixing the two tracing styles now fails fast
withSpan records exactly the functions you wrap. withTrace, trace, and node record a root plus the first-party calls beneath it. Mixing them in one call stack used to record both surfaces and log a warning. It now throws MixedTracingError, matching the Python SDK. A blended stack produced a trace whose shape misrepresented how your code was instrumented and left replay boundaries unpredictable, so it is better caught at the call that made the mistake.node() or withNode() when a discovered call needs its own name, type, or replay-mocking policy. Two things are deliberately exempt: framework integrations open their spans on whichever surface surrounds them, and the root that replay() and seedTrace() wrap around an undecorated function still accepts a subtree trace inside it.Dashboard
A clearer span tree in the trace view
Span icons are now filled in their type color instead of drawn as outlines, so the kind of each span reads at a glance. Rows are tighter and every row is the same height, which fits more of a trace on screen before you scroll. The detail pane sits on a light grey ground against the white span list, so the two halves of the page no longer blend into one sheet.The collapsed sidebar opens when you click a function
Clicking a function name or All Functions while the sidebar is collapsed now expands it, instead of only navigating. The section links under a function name still navigate without expanding, so you can jump straight to Traces or Datasets and keep the sidebar narrow.Seed a replayable trace by running your code once
seedTrace can now run a function once and record that execution as a replayable original. Pass the function instead of a case, and the SDK records the real input, the real output, and the first-party subtree beneath it, while capture stays off for everything else. A corpus you hold outside Bitfab becomes traces that replay can select, without waiting for production to produce them.adaptInputs hook now receives the seeded trace’s stored metadata on ctx.metadata, so a case keeps its provenance instead of smuggling it through the recorded inputs. The replay CLI gained --run for seeding a whole cases file through the registered function.captureEnabled replaces enabled
The client option that turns capture off is nowcaptureEnabled. The old enabled still works and warns once. Turning capture off no longer removes the tracing wiring, so wrapped functions keep their trace function key and both replay and seedTrace still record against a capture-off client. If you relied on enabled: false handing back your original function, you now get a wrapper that runs it untraced.A warning when both tracing styles meet
withSpan records exactly the functions you wrap. withTrace, trace, and node record a root plus the calls beneath it. Both still work when they meet in one call stack, and the spans nest where they were called, but the SDK now warns once per trace function key so a mixed setup is visible rather than silent.TypeScript subtree tracing is ready to install
Marking one function withtrace() or withTrace() records every first-party call beneath it, at any depth, with no wrapper on each one. The build adapters that make those nested calls visible now ship as bitfab-transform. Install it next to the SDK and add one adapter where your server code is compiled.tsx. The August 25 entry called this package @bitfab/transform. It is published as bitfab-transform, and the docs now match.Dashboard
The Diff view collapses to the spans that changed
A replay’s Diff view now folds the spans that match the original trace out of the span tree. Any run of two or more neighboring spans with no change, their children included, collapses into one row reading how many spans it hides, which opens and closes on click. A span whose child changed stays on screen, so nothing that moved is ever hidden behind a fold.Because the fold follows the diff, loops no longer group the tree here: a loop with one changed iteration reads as that iteration between two folded runs rather than as a single loop row. Each changed span’s+N −M line counts now sit at the right edge of its row, so they line up down the tree.Archive trace functions you are done with
You can now archive a trace function from the sidebar, so a workflow whose code is gone stops crowding the function list. Archiving deletes nothing. Its traces, datasets, experiments, and graders all stay, and the function comes back on its own the next time a trace arrives for it. Archived functions collect in an Archived section at the foot of the sidebar, where you can restore one at any time.See whether a traced function still exists in your code
Clicking a function in the sidebar now shows when it last traced, when an agent last confirmed its trace plan, and how likely it is that the code producing it still exists. That likelihood reads as a plain label rather than a number, because it is inferred from those two signals rather than measured.LangGraph tracing works inside trace() subtrees
A LangGraph graph wrapped with get_langgraph_integration() can now run inside a Python trace() subtree. Previously the integration’s tool and invoke spans raised MixedTracingError and stopped the run, so opt-out tracing and the LangGraph tool replay hooks could not be used together. Put @trace on the function that calls the graph and everything beneath it is recorded, with tool spans attaching under the call that ran them, counting toward the trace’s span budget, and following its mock_on_replay_default policy. Outside a trace() subtree the integration behaves exactly as before.Framework spans nest where they were called
Inside a Pythontrace() subtree, spans from the OpenAI Agents tracing processor, the Claude Agent SDK handler, and the LangGraph callback handler now attach to the function that invoked them instead of collapsing onto the trace root. An agent run started deep inside a workflow now appears at that depth in the span tree, so you can see which step made the call. Span types are unchanged.DashboardTypeScript SDKPython SDKRuby SDKPluginsCLI
TypeScript SDK v0.42.0, Python SDK v0.42.0, Ruby SDK v0.42.0, Plugins v0.9.92, CLI v0.2.299
Replay every trace K times in one experiment
Replays can now run each trace several times inside a single experiment, so a flaky result reads as flaky instead of as a regression. Passattempts to replay in the TypeScript or Python SDK, or --attempts N to bitfab-replay, and every trace replays that many times, each attempt with its own verdict, tokens, and database branch.attempt alongside originalTraceId in save_agent_labels.Dashboard
Loops fold up in the span tree
When an agent or batch run repeats the same call, the trace viewer now folds those repeats into one row showing the iteration count, how many iterations failed, and the loop’s total duration. Expand it to see the first, last, and failed iterations, and reveal the rest ten at a time. Repeating cycles fold too, such as an LLM call followed by a tool call, and arrowing into a folded loop opens it on the span you land on.A cleaner span tree
Each span type now has its own icon color, so LLM calls, agents, and tool calls read at a glance, and connector lines fold into each span instead of running as bare borders. The span panel in the trace view can be dragged wider or narrower, and folds away behind a “Spans” button in the header when you want the full width for the span you are reading.TypeScript SDKPython SDKRuby SDKGo SDKDashboardPluginsCLI
TypeScript SDK v0.41.0, Python SDK v0.41.0, Ruby SDK v0.41.0, Go SDK v0.41.0, Plugins v0.9.91, CLI v0.2.298
Name a trace by the record it is about
Every trace now has aname: the title Bitfab shows for it, and a field you can search and filter on. Set it from inside any traced function with getCurrentTrace().setName(...) (set_name in Python and Ruby, SetName in Go), pass name when seeding a trace, or set it later through the detached trace handle. Give it the ticket, order, or dataset row the run handled, and the trace list filter and search_traces will find that run by its label.workflow_name field the SDKs used to send is now name; older SDKs and direct HTTP integrations that still send workflow_name keep working, and an explicit name always wins over it. The CSV export’s workflowName column is now name.Seed originals by running a function once with capture off
seed_trace in the Python SDK now runs your function once and records that run as an original trace, with capture still off. The recorded trace carries the root span, the full first-party subtree, and the real inputs and output, so replay selects it like any captured trace and every replay of it links back as original_trace_id. Pass metadata to store a case’s provenance on the trace.seed_trace from synchronous code. With trace_across_threads=True, spans from worker threads nest under the seeded root, and an exception is recorded on the root span and re-raised. bitfab-replay --seed cases.jsonl runs each case the same way: a case’s input and kwargs are the call itself, and expected is no longer accepted because the output is what the run produced. This replaces the earlier Python seed_trace, which wrote a root-only trace from a case without running it. The TypeScript seedTrace keeps that behavior for now.Original trace metadata in the replay input adapter
Theadapt_inputs hook’s context now includes metadata, the original trace’s stored metadata, so a table-driven adapter can key off a case id or suite without smuggling it through the recorded inputs. It is fetched only when an adapter is registered. In the trace list, a seeded trace that is still running or threw now shows its running or error state instead of the seeded icon.TypeScript SDKPython SDKRuby SDKGo SDKDashboardPluginsCLI
TypeScript SDK v0.40.0, Python SDK v0.40.0, Ruby SDK v0.40.0, Go SDK v0.40.0, Plugins v0.9.89, CLI v0.2.296
Manage datasets from the SDKs
You can now create, read, and modify datasets programmatically, without a coding agent in the loop. Every SDK exposes a datasets namespace (client.datasets in TypeScript, Python, and Ruby, client.Datasets in Go) with the same operations the Bitfab agent tools offer: save a dataset, list datasets, fetch one, list its trace ids, add or remove traces, assign or unassign graders, and re-run graders over the dataset./api/sdk/datasets for anything not using an SDK.Opt-in and opt-out tracing never mix
The Python SDK has two tracing surfaces.@span is opt-in: only the functions you decorate are recorded. @trace with @node is opt-out: one root records every first-party function beneath it, and @node configures a discovered call without creating a boundary. They were never meant to share a call stack, and a @trace root called beneath a @span used to detach into a trace of its own, or, inside a replay item, into a second parentless root. The SDK now raises MixedTracingError at the boundary in either direction, naming which decorator was entered inside which.@trace subtree, configure a step with @node. The root span that replay("key", fn) wraps around an undecorated callable belongs to neither surface, so an entrypoint that calls a @trace root now nests that root beneath the item root instead of leaving it parentless. Nested @trace roots still record independent traces.Replay with capture off in the Python SDK
Bitfab(enabled=False) used to hand your functions back undecorated, so a client with tracing off could not replay them: replay(fn) and the replay registry had no trace function key to find. The flag is now capture_enabled, and it only controls capture. Decorated functions keep their wrapper, a replay item always records its trace, and seed_trace always writes, so one client serves both the environments where you do not want capture and the replay runs you start on purpose.enabled still works as a deprecated alias and logs a one-time warning; move to capture_enabled when you upgrade to Python SDK v0.39.1. TypeScript, Ruby, and Go are unchanged for now.DashboardPluginsTypeScript SDKPython SDKCLI
TypeScript SDK v0.39.0, Python SDK v0.39.0, Plugins v0.9.86, CLI v0.2.293
Seed replayable traces from cases you already have
Replay used to require a captured trace, so a corpus of test cases could not be replayed until the code had run against every one of them.seedTrace / seed_trace writes a replayable trace directly from a case, recording its arguments as the input and the value you expect as the output, with no execution at all.fn checks the case against the function’s real signature, so a case that could never run is rejected while you seed rather than failing later during replay. The replay registry seeds too, with bitfab-replay --registry <path> <pipeline> --seed cases.jsonl, which binds every case to the exact function that replay will select.A seeded trace records no inner calls, so replay mocking has nothing to substitute and database snapshots are refused rather than silently pinned to the wrong moment. Replays of seeded traces report each item as matched or missed against its expected value instead of same or changed against a previous run, and the trace list marks them with their own icon.Keyword arguments now replay as they were called
In the Python SDK, a traced function called with keyword arguments recorded them correctly but replayed them as a single trailing positional argument, so a keyword-only signature raised aTypeError during replay. Whether it broke depended on the values, not the signature: a datetime or UUID anywhere in the call happened to preserve the shape, and plain JSON did not.Recorded inputs now keep the positional and keyword split whenever keyword arguments are present. If you worked around this with an adapt_inputs hook that re-splits a trailing dict, remove it when upgrading to Python SDK v0.39.0, as replay now hands your function the arguments it was originally called with.Preview what a replay will run
--dry-run resolves every selected trace’s inputs, applies any input adapter, and prints the exact arguments your function would receive without calling it. It is the quick way to confirm that recorded inputs still fit a signature you have changed since capture, and no database branches are provisioned for a run that executes nothing.A replay whose selection matched no traces now exits non-zero instead of reporting a clean run of zero items.Setup now surfaces subtree tracing
Setup treated spans as the only instrumentation primitive, so it reached forwithSpan and @span and never surfaced the newer subtree API. It now names all three primitives when it reads the SDK reference, so trace (a root plus every first-party call beneath it, with no decorators on those calls) and node (naming, typing, capture, and replay-mock policy for a single call inside that subtree) are both on the table while your code is instrumented. The Reference overview in the docs gained a table showing which primitives each SDK supports.Setup entirely from the CLI
Bitfab’s experimental setup can now plan, instrument, and verify AI workflows without leaving your terminal. Runbitfab init --v2 for onboarding or bitfab setup --v2 [mode] for a specific setup workflow; repository edits, commands, and setup decisions stay behind terminal approval. Use --diagram to print the flow’s state diagram without starting the agent.DashboardPluginsTypeScript SDKPython SDKRuby SDKGo SDKCLI
Plugins v0.9.83, TypeScript SDK v0.38.10, Python SDK v0.38.7, Ruby SDK v0.38.3, Go SDK v0.37.0, CLI v0.2.290
Replay production traces from Go
Go SDK v0.37.0 can now run a trace’s historical inputs through your current Go code with typed arguments and results.GetFunction(...).Replay(...) supports replay lifecycle callbacks, replay mocking, mock overrides, input adapters, code-change capture, and database snapshots, while waiting for each replayed trace to reach Bitfab before finishing the experiment.Stable call ordering across every SDK
TypeScript, Python, Ruby, and Go now record strictly increasing microsecond timestamps, preserving call order even when several calls start within the same millisecond. Go replay also uses delivery acknowledgments from the SDK transport before finalizing an experiment, matching the other SDKs; server polling remains a fallback when delivery is ambiguous.Opt a traced subtree into default replay mocking
@trace(..., { mockOnReplayDefault: true }) / @trace(..., mock_on_replay_default=True) now establishes replay mocking as the default for nodes under that trace. Use @node({ mockOnReplay: false }) / @node(mock_on_replay=False) to override the default for code that should keep running live.This behavior is opt-in per trace. Existing traces and replay strategies are unchanged, including mock: "all", which still mocks every matched recorded descendant. TypeScript transformed descendants inherit the trace policy automatically; Python requires @node() on descendants that need replay boundaries.More reliable large trace uploads
Large span payloads now go directly to durable storage as they arrive, reducing duplicate work during trace ingestion. If that upload fails, Bitfab temporarily retains the payload and retries automatically so the span is not lost. TypeScript SDK v0.38.8 also sends OpenAI Agents span data only when the span is complete, giving Bitfab one authoritative snapshot with its output, error, and timing.DashboardPluginsTypeScript SDKPython SDKCLI
Plugins v0.9.80, TypeScript SDK v0.38.7, Python SDK v0.38.5, CLI v0.2.287
Replay LangGraph tools without running them again
LangGraphToolNode calls can now use output mocks during replay. Selected tools return their recorded or overridden output without executing, so unsafe side-effects such as sending an email or charging a card are not triggered again.getLangGraphIntegration() in TypeScript wraps the tool array passed to ToolNode. get_langgraph_integration() in Python exposes the native wrap_tool_call and awrap_tool_call hooks accepted by ToolNode. createInvoker() / create_invoker() then returns the normal graph entry point with the callback handler and replayable root already connected. Both integrations preserve native ToolMessage and Command results and refuse to execute a live tool when replay expected a recorded result but none exists.This API is experimental (alpha). Calls are currently matched by tool name and occurrence order, so start with deterministic tool flows and test replay against non-production dependencies before relying on it for unsafe side-effects.Dashboard
Live experiment grading stays responsive
Experiments now apply grader results smoothly as they arrive, without freezing the page or repeatedly refreshing summaries and statistics. Run headers and grader statistics reconcile once grading finishes, so final results remain accurate even when checks complete together.Dashboard
Faster, more reliable trace ingestion
Trace ingestion now completes without waiting for search indexing, reducing timeouts when applications upload large or highly concurrent traces. Newly completed traces may take a brief moment to appear in dashboard search while their lexical and semantic indexes finish in the background.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.78, TypeScript SDK v0.38.5, Python SDK v0.38.4, Ruby SDK v0.38.2, CLI v0.2.285
Route replay mocks by traced function
You can now register replay mock overrides once on a TypeScript, Python, or Ruby client and scope them to a trace function key, so large replay suites can share mocks without re-declaring them on every call. Resolvers can inspect each span, return a replacement, or returnNO_MOCK_OVERRIDE (Bitfab::NO_MOCK_OVERRIDE in Ruby) to fall through to lower-priority overrides and the replay’s base mock strategy; null and undefined remain valid mocked outputs.Capture complete TypeScript trace subtrees
@bitfab.trace and withTrace now capture every discovered call’s full inputs, output, and thrown error by default. No node() annotation or Configure capture step is required. Source exclusions and capture: false still omit calls, and an optional Studio policy can narrow later traces to selected function IDs.Use @bitfab.node(options) on transformed methods or bitfab.withNode(options, fn) around named standalone functions only when one call needs an explicit name, type, finalizer, omission, or replay policy. Outside an enclosing trace they execute normally without emitting a span. Omitted nodes keep captured descendants attached to the nearest captured parent, and replay-mocked nodes return their recorded outputs.Dashboard
Experiment updates keep your place
Experiment pages now keep open modals and expanded run details in place while new experiment results arrive. Live updates refresh the list and its statistics without flashing the page or closing what you had open.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.76, TypeScript SDK v0.38.3, Python SDK v0.38.3, Ruby SDK v0.38.1, CLI v0.2.283
Replay with one SDK-owned command
TypeScript SDK v0.38.3, Python SDK v0.38.3, and Ruby SDK v0.38.1 now installbitfab-replay, so your project owns only a replay registry mapping short names to the exact traced functions production calls. Pass the registry path and registered name to the command; common flags, progress reporting, and result output stay current when you update the SDK.Parameterize custom replay behavior
Registry entries can combine defaults with an options factory that receives values from--param or --params, so a replay can configure mock overrides and input adapters without editing the registry. Command-line values override overlapping defaults, while source-conflict validation catches incompatible trace and dataset selections before running.Dashboard
Know which function every row came from
The All Functions views for traces, datasets, graders, and experiments pool work from every traced function, and until now nothing on a row said which one produced it. Every row now names its function. Traces, datasets, and graders give it a column of its own, so the names line up and you can scan down them to see where one function ends and the next begins. Lists already scoped to a single function are unchanged, since repeating the same name on every row tells you nothing. Dataset names also stopped truncating early, because the pass rate column was holding on to width it never used.Trace complete TypeScript call paths from one root
Mark one TypeScript workflow with@bitfab.trace(key, options) or bitfab.withTrace(key, options, fn) to record its nested repository calls as lightweight spans. Add bitfab-transform through your build tool, then use Configure capture in the trace view to choose which functions include inputs, outputs, and errors on future traces. This experimental release supports Babel, the TypeScript compiler, SWC, common bundlers, direct Node execution, Next.js, and NestJS.Dashboard
Labels update everywhere without a refresh
Labeling a trace now updates its trace list row, open trace, dataset bucket, and related experiments together without refreshing the page or refetching whole lists. Trace-function counts also update immediately, while older sessions and uncertain filter matches still reconcile safely.Dashboard
An interactive tour of how Bitfab works
There is a new tutorial page at/tutorial that walks through Bitfab in two steps: a small traced workflow you can read in TypeScript, Python, or Ruby, and a live replay of it. Every call in the replay plan has a Re-run or Mock toggle, and flipping one restarts the animation, so you can watch what a replay actually does: which calls execute against your current code, which answer from the recording, and which are skipped because they sit inside a mocked call. Hovering a call links it to the lines of code that created it, and back.Dashboard
Runs nobody labels no longer look unfinished
An experiment run with no graders attached now reads as finished once its replays land. Until now the pass rate kept circling and Done never showed the run as complete, because a replay waiting on a person or an agent to label it counted as work still in flight. Runs that do have graders attached now circle until the last grader check reports, instead of reading as done the moment their replays finish.Dashboard
See how much each span changed
Comparing a replay against its original trace now shows a +N −M line count on every span in the list, the way a diff counts changed lines. The Diff view already marked which spans differed, but working out which one moved the most meant opening each span in turn. Spans whose content is identical, and spans that ran on only one side of the comparison, show no count.Dashboard
Filters stay put when you open a trace
Opening a trace from the traces, experiments, or dataset lists now slides the detail panel in directly beneath the page header, with the filter bar behind it left where it is. You can change Group by, switch between original and replay traces, or switch organizations without first closing the trace you are reading. Regrouping experiments keeps the open trace open and leaves expanded runs expanded, instead of resetting the list underneath you.Links to an experiment trace open that trace
A link to a trace inside an experiment now opens straight to it, and reloading the page keeps it open. The run holding that trace opens with it, so the arrows in the panel still step to the next and previous traces in the run.Configure traced Python calls without another span
Python SDK v0.38.2 adds@client.node to configure a function only when it runs inside an enclosing @client.trace, without turning that function into a standalone span. Set capture=False to omit the node while keeping its captured descendants connected to the nearest captured parent, or set mock_on_replay=True on a captured node to return its recorded output during replay.Name what to instrument instead of picking from a list
/bitfab:setup now takes a file, function, or directory directly when you already know what you want traced. Type it at the prompt and setup reads only that location, instead of scanning your codebase and handing you a list of candidates. If you would rather it find the workflows for you, that option is still there and still the recommended default.Setup stops asking questions you already answered
When your project already has instrumentation, setup asks once what you want to do and carries that answer forward. Naming a workflow at the first prompt survives the SDK install, so you are not asked the same thing again a few steps later, and naming an existing trace function key takes you straight to changing what it captures. Setup now also tells you before it installs the SDK and writes your API key, rather than doing it silently.Dashboard
Reliable experiment startup
New experiment pages now open into a preparation state while the first replay starts, rather than failing because the experiment group has not been created yet. If setup takes longer, the page switches to a neutral waiting message; genuine request and authorization failures remain distinct and provide diagnostics the Bitfab team can investigate.Dashboard
Experiment history appears sooner
Experiment history now shows its first page with the initial page load, so organization-wide and function-specific lists no longer wait for an extra browser request before appearing. Statistics still fill in asynchronously, and pagination and live updates continue working as before.Dashboard
Faster, more reliable live updates
Live trace, label, dataset, and experiment updates now arrive with less duplicate work across open Bitfab dashboard tabs. Experiment summaries update immediately when Bitfab has the complete result, while targeted refreshes keep moved runs and labels on replayed or original traces accurate.Dashboard
Closely timed calls keep their captured order
Trace views and replays now preserve the original order of calls captured within the same millisecond. Bitfab keeps the full timestamp precision from supported SDKs and trace imports, so closely timed calls no longer swap positions between rendering and replay.Dashboard
Faster experiment history
Experiment history now shows each run as soon as its metadata arrives, then fills in verdicts, trace counts, and token costs without blocking the page. Loading rows keep their final size while those statistics arrive, so the list stays stable as it becomes interactive.Setup restores a disabled Bitfab MCP server
Runningbitfab init now re-enables the Bitfab MCP server if it had been switched off for that project. Coding agents remember that setting per project and it survives reinstalling the plugin, so the Bitfab skills would come back while every Bitfab tool stayed missing.Clearer recovery when the Bitfab tools are missing
When setup cannot reach the Bitfab tools, it now points you at your editor’s MCP settings and stops, rather than continuing into steps that depend on those tools. In Claude Code, enable Bitfab from/mcp and re-run setup: the tools load into the running session, and restarting does not help because the setting is stored per project.Nested Python traces capture complete trees
Nestedtrace() roots in Python now each record a complete independent trace, matching the result you would get from running either root alone. Shared work appears in both traces with distinct span IDs, while span()-decorated functions remain in their active trace and record only once.Async-generator traces also release capture between pulls and still emit their span if closing raises, preventing unrelated work from leaking into a suspended generator’s trace. Nested trace regions intentionally record each session separately, so their shared region produces twice the span volume.Dashboard
Re-run graders on a single trace
You can now re-run graders on one trace instead of re-grading a whole dataset. Open a trace and use the Graders panel: each grader row has its own re-run control, and the panel header opens a picker for running several at once. A re-run also takes over a grader that never finished, so a trace left mid-grade can be restarted rather than waited out.Grading progress reads as a live pass rate
While a trace is being graded, the score counts passes over the graders that have answered so far, so “3/4” means three of the four back so far passed. A trace whose graders were all just queued shows a grading indicator instead of a zero score.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
TypeScript SDK v0.38.0, Python SDK v0.38.0, Ruby SDK v0.38.0, Plugins v0.9.70, CLI v0.2.277
Replay items report their own duration, and name the original’s fields
Each replay result item now reports how long that replay took underdurationMs / duration_ms, measured around the replayed call. It previously carried the duration of the original trace, which meant a live progress line could show a number from a run that happened weeks earlier.Everything describing the trace being replayed now carries an original prefix: originalDurationMs, originalModel, and the new originalTokens, which was already computed server-side but never surfaced. With tokens reporting the replayed run, one item now holds both halves of a cost or latency comparison. model / :model stays as a deprecated alias for the original’s model, since there is no replay equivalent.Replay branch provisioning is also measured now: dbBranchTimings / db_branch_timings breaks down how long resolving the project, creating the branch, connecting the compute, proving it serves, and running your warm-up SQL each took, on success and on failure alike.If you read durationMs today, it now means the replay’s duration. Switch to originalDurationMs to keep the previous value.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.69, TypeScript SDK v0.36.13, Python SDK v0.37.12, Ruby SDK v0.36.11, CLI v0.2.276
Replays stop before an unsafe mocked call can run
Selected replay mocks now fail closed across TypeScript, Python, and Ruby. If a replay cannot load the historical span tree, match the recorded occurrence, or retrieve its output, the item returns an error instead of falling through to the real child call. The Claude, Cursor, and Codex plugins also audit unsafe actions and execution-context limitations before running a replay, and database-snapshot checks target the exact captured trace.Trace a whole call tree from one decorator
The Python SDK can now record an entire call tree from a single annotation.trace() captures the function you decorate plus every function in your own code that it calls, at any depth, with none of them decorated.max_depth and max_spans bound each subtree and warn once if they truncate it.Nested trace() roots produce complete independent traces, each with its own span IDs; two roots double span volume in their shared region. A span()-decorated call behaves differently: it stays in the active trace, records once, and parents the automatically captured calls beneath it. Async-generator capture is released between yielded items, even when the caller stops consuming without closing the generator.Experimental, and requires Python 3.12 or newer. On older versions the decorated function still records its own span exactly as span() does.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.67, TypeScript SDK v0.36.12, Python SDK v0.37.10, Ruby SDK v0.36.10, CLI v0.2.274
Replay diffs show every line
Comparing a replay against its original trace now shows the input and output in full. Unchanged lines that used to sit behind an “N unchanged lines” toggle are always visible, so you read a change in the context it happened in without expanding anything.See how each replayed span was produced
The diff view now carries the same header as the span view, so a span keeps its name and any error while you are reading its diff, alongside how the replay produced it. A mocked span says whether it returned the output recorded in the original trace or a value your replay code supplied, and a replay that ran against a database snapshot says which database it branched from and when the snapshot was pinned.Reporting the kind of mock needs the TypeScript, Python, or Ruby SDK version above. Spans captured by earlier versions keep the general “Mocked” label.Reach every organization in the switcher
The organization switcher scrolls now. Long organization lists were cut off at the bottom with no way to scroll, which put the organizations latest in the alphabet out of reach.Dashboard
More resilient concurrent Studio sessions
Studio is more resilient when multiple trace sessions are active at once. Live event polling now uses backend capacity more efficiently, reducing interruptions while traces are updating.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.66, TypeScript SDK v0.36.11, Python SDK v0.37.9, Ruby SDK v0.36.9, CLI v0.2.273
Replay diffs recognize renamed files
Replay code diffs now recognize files you renamed, so experiments show the edits inside a moved file instead of counting the whole file as deleted and added. Automatic code-change capture applies this behavior across the TypeScript, Python, and Ruby SDKs and ignores moves with no content changes.Replay surfaces each trace’s server ID as it finishes
Replay now gives you the servertraceId for each item the moment that item finishes, instead of only after the whole run completes. The id arrives on both the returned ReplayItem and the onItemFinish callback, so you can link straight to the trace in Bitfab or start per-item work while the rest of the run is still going. Available across the TypeScript, Python, and Ruby SDKs.Dashboard
Function counts arrive with the sidebar
Function names and their exact trace counts now appear together when the Bitfab dashboard loads. The function sidebar no longer fills in totals after navigation is already available, while counts still stay current as traces arrive.Trace ingestion survives count update failures
Trace ingestion now completes even if Bitfab encounters a temporary problem updating dashboard function counts. Any resulting count drift is repaired automatically, so newly ingested traces remain available without sacrificing accurate sidebar totals.Graders stay on the model you chose
Graders using Gemini on Vertex now retry temporary connection failures without switching away from the model you selected. If a run still fails, Bitfab records safer diagnostic context for faster investigation while keeping credentials out of error reports.TypeScript replays finish reliably
TypeScript replays now keep the process alive until their final traces are safely stored or the persistence deadline is reached. This prevents a locally completed replay from leaving its experiment unfinished while the SDK is still waiting for confirmation.Dashboard
Faster trace function counts
Trace counts in the function sidebar now load with the page instead of waiting for Bitfab to scan your trace history. Counts stay current as new traces arrive and when traces move between functions, so navigating large projects remains fast without showing stale totals.Dashboard
Grader results stay current during re-runs
Re-running graders now shows grading as soon as work is queued, then shows the latest verdict when each check finishes. Past automated results remain available in history, while counts and verdict pills use only the newest result and any human label stays authoritative. Dataset and experiment views update across open browser windows throughout the re-run.Dashboard
Live traces keep up with busy runs
The Traces page now stays responsive and up to date when many traces arrive at once. High-volume arrivals refresh in batches, and newly created traces remain visible instead of being lost among follow-up updates.One trace plan per function, kept up to date
Bitfab’s coding plugins now revise the trace plan you are already looking at instead of posting a new one beside it. Asking for a change while the plan is open in Studio updates that page in place, so the plan you are reviewing is the plan that gets saved, and the window stays where you left it.The same rule now covers the plans an earlier repo analysis drafted. A draft whose code has moved or been renamed since it was written is rebuilt against your current code and saved back onto the same plan, its trace boundary included, rather than being set aside for a fresh scan to duplicate. A draft for work that is already instrumented, or for a workflow that no longer exists, is retired so it stops being offered every time. Re-analyzing a repository refreshes the drafts it made before instead of adding a second draft per function, and leaves alone any function whose plan you are in the middle of working on.Every path that would post a plan now checks whether the function already has one, so instrumenting or modifying a function picks up the plan waiting for it instead of starting a rival. Viewing a function’s plan shows an unconfirmed one where before it reported nothing to view.Dashboard
Trace function pages open instead of erroring
Opening a trace function’s traces, or one of its versions, now loads rather than showing an error page. Those two pages worked out which function you meant differently from every other page in that group; they all share one answer now.A color means one thing
Language and agent chips, status badges and cautions are now easy to tell apart at a glance. A Python chip used to render in exactly the color that means “passed” and a Go chip in the color that means “mocked”, so a language could read as a result. Cautions shared the brand color, so a warning looked like an ordinary highlight. Each of these now has a color of its own, and hover labels that had gone missing from the session timeline are back.Dashboard
Graders that error or are still running are now visible
When a grader throws while scoring a trace, or is still working on it, that state now shows on the trace verdict pill, the experiment bars, and the labeling panel. Previously a broken grader looked the same as one that had never run. Neither state counts toward the pass rate, so a grader that failed reads as failed rather than as a lower score, and you can still label a trace by hand while its graders are mid-run.One failing grader no longer stops the others
If a grader errors while scoring a trace, the remaining graders now finish instead of being skipped. Re-running graders also picks up the ones that errored, rather than treating them as already scored.Dashboard
Database snapshots replay from their original source
Replays now restore database snapshots from the exact source recorded when each trace was captured, even when both a direct Neon connection and a mirrored connection are enabled. If that source has since been replaced or disconnected, replay stops safely instead of restoring from a different database.Replay completion stays fast under heavy span traffic
Python and Ruby replays now keep their fast completion path even when applications submit spans faster than the local export queue can drain. A dropped span no longer leaves acknowledgment state behind, and sustained traffic no longer displaces acknowledgments for spans still waiting to be sent, so completed replays avoid falling back to slower server polling.Dashboard
Faster trace function navigation
The trace functions sidebar now loads function names before waiting for exact trace counts, so you can navigate as soon as the sidebar appears. Trace totals fill in shortly afterward and remain scoped to your selected environment, without showing placeholder zeros while data is still loading.TypeScript SDKPython SDKRuby SDKGo SDKPluginsCLI
TypeScript SDK v0.36.8, Python SDK v0.37.6, Ruby SDK v0.36.6, Go SDK v0.36.2, Plugins v0.9.61, CLI v0.2.268
Replays finish as soon as their traces are safely stored
Replays no longer wait on a fixed delay before checking whether their traces were saved. Each SDK now tracks the acknowledgment the server already returns when it accepts a batch, so a replay finishes the moment its traces are durable instead of pausing and then polling. Runs that used to sit waiting now complete as quickly as the data lands.Steadier span delivery when ingestion is busy
All four SDKs now retry span delivery the way the OpenTelemetry protocol prescribes. When the server asks for a pause, that pause is respected exactly rather than cut short, and the whole client holds off rather than only the one request that was turned away. Without such a request, retries back off with jitter so many clients do not return in lockstep. Spans are also retried in more of the cases worth retrying, so a brief hiccup during ingestion is less likely to cost you data.Dashboard
More reliable live trace updates
Live trace updates in the Bitfab dashboard now stay connected when trace details cannot be loaded, so a temporary database problem no longer interrupts the rest of the stream. The dashboard also stops pending trace loads when you disconnect and fails stalled database connections promptly instead of leaving the stream hanging.DashboardPluginsTypeScript SDKPython SDKRuby SDKGo SDKCLI
Plugins v0.9.60, TypeScript SDK v0.36.7, Python SDK v0.37.5, Ruby SDK v0.36.5, Go SDK v0.36.1, CLI v0.2.267
Large spans stay complete
Bitfab SDKs now preserve highly compressible span payloads up to 7.8 MB instead of trimming them at the normal 2.8 MB request budget, keeping large documents, message histories, and agent state available for replay and evaluation. The TypeScript, Python, Ruby, and Go SDKs send an oversized single span intact when it fits within the 3 MB wire target; payloads above the 8 MB ceiling or those that cannot fit still use the visible trimming fallback.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.59, TypeScript SDK v0.36.6, Python SDK v0.37.4, Ruby SDK v0.36.4, CLI v0.2.266
Know when every replay item starts and finishes
Replay lifecycle callbacks now identify exactly when each item starts and finishes, so integrations can distinguish queued work from in-flight work and handle each result as it arrives. UseonItemStart and onItemFinish in TypeScript, or on_item_start and on_item_finish in Python and Ruby; every callback includes the replay item, including finishes with an error. The deprecated onProgress and on_progress callbacks remain compatible for existing integrations.Trace class methods with TypeScript decorators
TypeScript 5+ projects can now trace class methods with standard ECMAScript decorators. Decorators are an optional shorthand;withSpan remains the recommended default for TypeScript 4.x, standalone functions, class fields, accessors, and legacy decorator projects.Dashboard
Replay counts on the usage page
The usage page now reports replays alongside the traces you send. A Replays card sits next to Total traces, the chart adds a Replays metric that works in both the total and by-function views, and the usage-by-period table and CSV export each carry a Replays column. Replays are counted within your total traces, not on top of them.Dashboard
Move experiments between groups
You can now move an existing experiment into another experiment group, or remove it from its group, from your coding agent. Usesave_experiment with a group ID to move it or null to ungroup it; open experiment views update automatically.Dashboard
Experiment history your agent can organize
Coding agents can now name and annotate individual experiments and experiment groups, then list recent groups or retrieve one group with its member experiments. This keeps related replay runs discoverable and gives your agent the context behind each iteration when it returns to an experiment later.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.56, TypeScript SDK v0.36.3, Python SDK v0.37.2, Ruby SDK v0.36.3, CLI v0.2.263
Every replay failure stays visible
Replay now returns one item for every attempted trace, even when setup fails before your function runs or prevents a replay trace from being created. The TypeScript, Python, and Ruby SDKs distinguish trace errors from replay errors, preserve the original exceptions and database snapshot failure details, and retain collected items if the whole run throws. New replay-result serializers keep those structured errors intact in direct stdout and SDK-managed result files.Dashboard
More reliable background jobs
Scheduled jobs and other durable background work now keep reaching Inngest when optional authentication or request-logging services are unavailable. This prevents unrelated dependency disruptions from interrupting grading, imports, enrichment, and other workflows that run in the background.Trace plans stay current with your code
When you modify tracing for an existing function, Bitfab’s coding plugins now reread the current instrumentation before revising its trace plan. The stored plan preserves your earlier capture and replay choices, while newly added, removed, moved, renamed, or unwrapped calls are reconciled against the code so the plan does not silently omit current work. If the trace boundary itself changed, the existing plan moves to the current root instead of preserving an obsolete one. Newly discovered calls also receive samples and replay analysis before the updated plan is saved.Dashboard
Grading progress across a whole dataset
Re-running graders on a dataset or experiment now shows every trace in it as grading right away, instead of lighting up a handful of rows at a time as the work moves through them. Each row settles back to its pass rate as its own graders report, and a re-run that fails part-way no longer leaves rows stuck looking like they are still being graded.Trace rows say what they are waiting on
A trace whose graders are still scoring now counts up from zero, so a re-grade no longer shows the previous run’s pass rate as though it were current. Rows still waiting on the trace itself read “Trace” rather than “Label”, and anything in flight carries a moving marker around its pill, so you can tell at a glance which rows are still working.Dashboard
Live grading recovers from stalled checks
Live grading now stops individual grader checks that stay stuck too long, then retries a timed-out check once without consuming the retry budget for other temporary failures. The deadline covers the full check, including its own model retries and failure explanation, so one stalled grader no longer keeps an entire live-grading run open indefinitely.All Functions keeps the section you are in
Switching from a single function to All Functions in the sidebar now keeps you in the section you were already viewing. Previously it always returned you to Traces, so going from one function’s Experiments to every function’s Experiments took an extra click. This applies to Traces, Datasets, Experiments, and Graders.Simpler trace plan saves
Bitfab’s coding plugins now use onesave_trace_plan tool to create new plans and revise existing ones. Older plugin installations that call create_trace_plan or update_trace_plan remain compatible, so trace setup and re-instrumentation continue working during upgrades.Dashboard
More reliable live grading
Live grading now preserves completed grader checks when a later check needs to retry, avoiding repeated evaluations on traces with multiple graders. Longer grading runs also get more time to finish, while successful results remain authoritative if overlapping runs settle in a different order.Trace plans stay within the workflow you are tracing
When the Bitfab plugin drafts a trace plan, it no longer attaches callers from above your workflow’s root function. Those nodes counted toward the plan’s node total but were never drawn, so the count could be higher than the tree you actually saw. Plans now cover the root and the code beneath it, and the node count matches what is in front of you.DashboardPluginsTypeScript SDKPython SDKRuby SDKCLI
Plugins v0.9.51, TypeScript SDK v0.36.2, Python SDK v0.37.1, Ruby SDK v0.36.2, CLI v0.2.258
Faster replay mocking
Replays now transfer only the recorded input and output data needed to reconstruct calls, reducing payload size for traces with large captured context. Eagermock: "all" runs also avoid fetching the root output twice, while marked and selective mocking continue to load recorded outputs only when they are used. This improvement is available in the TypeScript, Python, and Ruby SDKs.Dashboard
Neon databases connect on every plan
Connecting your own Neon database now works whichever Neon plan you are on. Bitfab used to ask Neon for a custom scale-to-zero delay on the branches it creates, which Neon accepts only on its Scale plan and rejects outright on Free and Launch, so the connection check failed at the create-branch step. Bitfab now leaves that setting alone on databases you own, and the branches it creates for replay follow your own project’s scale-to-zero.Dashboard
Neon connection errors say what actually went wrong
When connecting a Neon database on the Integrations page fails, the error now quotes Neon’s own explanation instead of a bare HTTP status. A refusal that used to read “Request failed with status code 412” now names the cause, such as reaching your project’s branch limit, alongside Neon’s error code and request id so their support can trace it. Each failed step is still listed separately, so you can see exactly how far the check got.Trace plans update in place
Your coding agent now revises an existing trace plan instead of creating a new one each time it changes what gets captured. Each traced function keeps a single authoritative plan, so coming back to it later shows the current capture and replay decisions rather than a stack of competing drafts. Every revision is recorded, so the earlier version is still there to compare against.The trace plan page updates while you watch
The trace plan page in Studio now reflects your agent’s changes as they happen, with no reload. Spans it captures, uncaptures, or switches between replay and mock update in place, keeping the groups you expanded and your place on the page. The primary button reads Save once the plan differs from the one you were first shown.Codex setup launches autonomously again
Codex users can once again runbitfab init without permission prompts. The CLI’s --skip-permissions option now uses Codex’s supported autonomous mode instead of passing a removed flag that stopped the agent before setup began.Dashboard
Replay traces always open in the comparison view
Opening a replay trace now shows the same comparison view experiments use: a verdict chip and a Diff | Original | Replayed toggle that flips between a per-span diff against the original trace, the original itself, and the replayed run. This applies everywhere a replay opens, including the traces list and direct links, so replays no longer fall back to the plain trace view. Labels and a code change are optional; the view shows whatever the replay has.Fixed invisible tooltip text
Fixed an issue where tooltips across the dashboard rendered their text in a dark color on the tooltip’s dark background, making them unreadable. Tooltips on the sidebar navigation, trace status icons, and elsewhere now show their labels clearly.Span-by-span diffs in the replay comparison view
The Diff view now compares a replay to its original span by span instead of only the trace’s overall input and output. Spans whose content changed open a line diff, spans the replay never ran appear as faded red rows in the span tree, and new spans the change introduced are tinted green with their full content one click away. Spans that match the original stay selectable and render their content as unchanged lines, so you can still inspect what didn’t move.Dashboard
Span timing at a glance in the trace view
Every span row in the trace tree now carries a timing bar showing how long that span took as a share of the whole trace, offset by the time that had already elapsed when it started. Durations sit inline next to each span name, index badges are gone, and nesting is tighter, so deep traces fit in the sidebar without truncating span names as early.The trace and span views also move to a calmer, more consistent color treatment: the selected trace and span read as one highlight color everywhere, span types are identified by their icons, and error, replay, and mock states keep their own distinct colors so problems still stand out.Trace across threads in the Python SDK
Functions dispatched to worker threads (ThreadPoolExecutor, loop.run_in_executor, or threading.Thread) previously recorded their spans as separate single-span traces, and replay mocking could not serve them. Construct the client with trace_across_threads=True (or set BITFAB_TRACE_ACROSS_THREADS=1) and those spans now nest under the trace that submitted the work, with mock="marked" replay serving their recorded outputs on the worker thread. Plain asyncio code needs no flag.See where every span ran
Every Python SDK span now records aruntime block in its raw data: the thread it executed on, the thread its parent span ran on, and, for work dispatched across threads, the thread that submitted it. Open any span’s raw view in the dashboard to inspect it.Replay warns when a mock cannot be served
During replay, a span markedmock_on_replay=True with nothing to serve now logs a warning saying it ran live, and whether that is because the replayed trace recorded no matching span or because all recorded occurrences were already consumed by earlier calls.Dashboard
Faster experiment history
Experiment history now stays responsive in organization, dataset, function, and multi-run views by loading trace details only when you expand an experiment. Single experiments and experiment groups still preload their traces, so focused drill-down remains immediate.More reliable Neon database snapshot setup
Connecting a Neon project for database snapshots no longer depends on a restore point from the last minute, so valid projects are not rejected when restore history is still catching up. The Integrations flow now validates the connection against the project’s current state and clearly tells you when point-in-time restore is disabled and a restore window needs to be enabled.Cancel database snapshot setup while it runs
Setting up per-trace database snapshots can take a while, and you can now stop it partway. The Database section of Integrations shows which step is running (checking your database, preparing branching, validating branches) instead of a single spinner, and offers Cancel setup, which stops the work and removes whatever had been built.Snapshot setup failures say what actually went wrong
A failed setup no longer always points at your connection string. Bitfab now tells apart a problem with your own database, a setup it could not finish on its side, and a database that is already connected elsewhere in Bitfab, and gives each its own advice. A status Bitfab could not read is reported as unverified and keeps re-checking itself instead of being shown as a failure.Disconnecting a database removes its snapshot copy
Deactivating snapshots now deletes the snapshot copy Bitfab built for that database, so nothing of yours keeps being replicated afterwards. The confirmation spells out what that means first: existing snapshots go with it, traces captured earlier will no longer replay against a snapshot, and connecting again rebuilds from scratch.Connecting a Neon project one step at a time
The Neon connection flow now asks one question at a time and collapses each answer into a line you can change, so the key, project, and database you picked stay visible as you go. Start over is always available, and leaving the page and coming back mid-setup no longer clears what you had entered.TypeScript SDKPython SDKRuby SDKPluginsCLIDashboard
TypeScript SDK v0.36.1, Python SDK v0.36.1, Ruby SDK v0.36.1, Plugins v0.9.47, CLI v0.2.253
Replay branches tell you what they are pinned to
The database branch handed to your code during a replay now carries the moment it was pinned to, so you can confirm a replay is really reading your data as it stood when the original trace ran. It also carries the branch’s own id and the name of the environment variable your app reads. Reading the connection string is still the only thing that counts as using the branch, so inspecting these costs nothing.Traces record the region a replay branch ran in
A replayed trace now records the region its database branch lived in, and the snapshot badge on the trace names it. When one replay looks slower than another, that tells you whether the difference is your code or a database in another region paying a round trip on every query.Ruby replay branches keep the database URL out of your logs
In the Ruby SDK, turning a replay branch into JSON no longer includes its database connection string or the internal replay state. Under Rails, serializing one of these objects into a log line or an API response previously carried the connection string with it. It now serializes only the descriptive fields, while code that asks for the connection string directly still gets it.Dashboard
Experiment results count every graded replay
Experiments now show a grader verdict on every replay the graders scored, including replays that hit an error partway through. Those rows used to show no result at all and their checks were left out of the run’s totals, so the pass rate could disagree with the traces listed under it. Grader progress now reads passed, failed, and grading, and in label mode the counts add up to the number of traces in the run.Labels show what is still waiting on you
A verdict written by an agent now stays marked as unreviewed until someone approves it, while keeping its pass or fail color, so you can scan a list and see which traces still need a look. Once approved it reads as settled while still showing that a machine wrote it.TypeScript SDKPython SDKRuby SDKGo SDKPluginsCLIDashboard
TypeScript SDK v0.36.0, Python SDK v0.36.0, Ruby SDK v0.36.0, Go SDK v0.36.0, Plugins v0.9.46, CLI v0.2.252
Spans capture much more of your data
Each span now records up to 2.8 MB of its inputs and outputs, up from 512 KB. A large document, a long message history, or an agent state that used to be replaced with a placeholder is captured in full, so traces stay complete enough to read and replay. This applies to the TypeScript, Python, Ruby, and Go SDKs.Large spans arrive instead of going missing
A span carrying more than the limit now ships with its largest fields replaced by placeholders, keeping its name, timing, and remaining fields intact. Previously a span that big could be discarded on the way out, leaving a hole in the trace with nothing to explain it. When a span is trimmed the SDK warns once in your logs and marks the trace as incomplete, so a partial capture is never mistaken for a faithful one.Dashboard
Steadier trace list when opening a trace
Opening a trace from the trace list no longer shifts the list sideways. The list used to slide left and drift back as the detail panel slid in, which made it easy to lose your place while working through a run of traces. The list now stays put while the panel opens, and the span you select still scrolls into view inside the panel.Dashboard
Long code change descriptions no longer cut off the window
Opening the code change behind an experiment whose description ran several lines used to push the bottom of the window off screen, with no way to scroll back to it, leaving the file diffs underneath unreachable. The window now stays on screen, with the description pinned at the top and the diffs scrolling beneath it.Confirm your database snapshot connection
Your coding agent can now tell whether database snapshots use a direct Neon project or a Bitfab-managed Postgres mirror. Theget_database_connection_status tool also reports the pinned Neon project name and ID for direct connections, making it easy to confirm which project replay uses.TypeScript SDKPython SDKRuby SDKGo SDKPluginsCLIDashboard
TypeScript SDK v0.35.0, Python SDK v0.35.0, Ruby SDK v0.35.0, Go SDK v0.35.0, Plugins v0.9.44, CLI v0.2.250
SDKs gzip the traces they send
Every SDK now compresses request bodies of 8,192 bytes or more with gzip, which is where most of a trace’s bandwidth goes. Smaller requests are unchanged, because compressing them would cost more than the bytes it saves. Nothing to configure, and no new dependencies: each SDK uses its language’s standard library.Compression is best-effort. If it ever fails, the SDK sends the original body rather than dropping the recorded call. SetBITFAB_DISABLE_COMPRESSION to turn it off, and note that browsers without CompressionStream fall back to uncompressed automatically.Shareable links to any trace
Opening a trace now puts it in the page address everywhere you can open one: the trace list, inside a dataset, and inside an experiment. Copy that link to send someone the exact trace you are looking at, and it opens straight to it. Back and Forward now step through the traces you opened instead of leaving the view.A clearer sidebar
The sidebar runs the full height of the window and lists Traces, Datasets, Experiments and Graders under whichever function you are working on, so you can move between them directly instead of through a menu. Each function shows how many traces it has. Collapsing it leaves a strip of icons rather than hiding it, so it is always one click back.Dashboard
Reliable trace delivery
SDK trace uploads now complete reliably during concurrent ingestion instead of timing out. This keeps traces flowing into the dashboard even when many spans for the same trace arrive together.Traces with self-referencing data are readable again
Opening a trace whose recorded input or output contained a self-referencing value used to fail. Because traces are read in batches, a single affected trace could also stop the others in the same request from loading. These traces now open normally, with the repeating value shown as a placeholder, and one unreadable trace no longer affects the others in the same request. This applies to traces you have already recorded, so nothing needs to be re-run.Ruby spans containing NaN or Infinity now send
The Ruby SDK now records non-finite numbers such asNaN and Infinity as a placeholder instead of failing to encode them. Previously a single one of these values could stop an entire batch of spans from reaching Bitfab, so unrelated spans were lost alongside it. This matches how the Python SDK already handles the same values.Dashboard
Sharper cutoff for snapshots from a replaced database connection
The date Bitfab uses to decide which traces predate a replaced database connection now comes from the connection that actually holds your snapshots, rather than the oldest database connection on your account. If you connect a second snapshot provider after running on a first one, traces captured before that switch now correctly show “Snapshot on old branch” instead of replaying against a project that never held their data.Dashboard
Grader reruns recover from interrupted jobs
Dataset and experiment grader reruns no longer stay blocked forever when an earlier run is interrupted. After a stale run expires, you can start the graders again instead of remaining stuck on an in-progress state.More reliable database snapshot replay
Database snapshot metadata now fills in reliably across large trace histories, avoiding stalls that could leave older traces incomplete for replay. This improves replay consistency for organizations with extensive trace histories.Re-run graders from your coding agent
Your coding agent can now re-score existing traces with thererun_graders_on_dataset and rerun_graders_on_experiment tools. Reach for them when traces were added to a dataset before a grader existed, or after you change a grader’s criteria and want everything scored again, since attaching a grader records the assignment without grading anything on its own. Both default to every grader attached to the dataset or experiment, and report how many traces were graded once the run finishes.Grading progress on dataset and experiment rows
Traces now show that grading is under way while a re-run is in flight, instead of holding the previous run’s scores until new results arrive. Each row’s pass-rate pill switches to a pending state when its graders start and settles on the new score when they finish.Dashboard
Open Studio resources across organizations
You can now open Studio datasets, experiments, traces, graders, functions, and trace plans from any organization you belong to without switching your active organization first. When a resource belongs to a different organization, Studio shows its organization in the header so you can see which workspace you are working in.Leaner plugin skill discovery
Bitfab’s Claude, Cursor, and Codex plugins now load leaner skill descriptions, leaving more of the agent’s startup context available while preserving natural-language routing. Command usage hints are generated from the same declared modes the flows execute, so setup, update, and assistant guidance stays aligned as those modes evolve.Setup reuses trace plans you already drafted
When you instrument your app with the Bitfab plugin, setup now picks up a trace plan you drafted earlier while analyzing your repository instead of rebuilding one from scratch. If a saved draft already covers the workflow you’re instrumenting, the plugin reuses it, so the analysis work carries straight into instrumentation.Discover existing trace plans from your coding agent
A newlist_trace_plans tool lets your coding agent see the trace plans already in your workspace, filtered by status or how they were created. This is what lets setup find and reuse an earlier draft rather than starting over.Snapshots from a replaced database connection
Traces whose database snapshot was captured against a connection you have since replaced now show “Snapshot on old branch” instead of “Snapshot captured”, and replaying one is refused with an explanation. Those replays previously ran against your current database without saying so, returning results that looked valid but restored the wrong data. The check applies whether Bitfab maintains the snapshot copy for you or you connect your own Neon project.TypeScript SDKPython SDKRuby SDKGo SDK
TypeScript SDK v0.34.2, Python SDK v0.34.0, Ruby SDK v0.34.0, Go SDK v0.34.0
Faster trace export in every SDK
Each carrier is now encoded exactly once and the request body is assembled from those encodings. Packing a batch used to re-encode the whole request for every span it considered, which re-escaped the captured inputs and outputs over and over. A full export window is now roughly 10x cheaper to prepare, which in Node.js is time given back to your event loop.OpenTelemetry Collector delivery has been removed
BITFAB_OTEL_EXPORTER_ENDPOINT no longer routes SDK traces through your own OpenTelemetry Collector; setting it has no effect and SDKs deliver to Bitfab directly. Removing it drops @opentelemetry/exporter-trace-otlp-proto from the TypeScript SDK, opentelemetry-exporter-otlp-proto-http (and protobuf) from the Python SDK, and gRPC, protobuf, and grpc-gateway from the Go SDK’s dependency graph entirely.If you route telemetry through a Collector today, point its otlphttp exporter at Bitfab instead. Bitfab’s ingress accepts standard OTLP/JSON, so a Collector can still forward to us:More reliable TypeScript trace delivery
TypeScript SDK traces now use OpenTelemetry’s batching and lifecycle machinery behind the existingwithSpan wrapper and framework integrations, so no instrumentation changes are required. Large captured inputs and outputs are packed beneath request-size limits, completed requests are delivered concurrently, and flushTraces() now reports whether delivery actually succeeded instead of only that the queue drained.Route TypeScript traces through an OpenTelemetry Collector
TypeScript applications can send Bitfab traces through an OpenTelemetry Collector by settingBITFAB_OTEL_EXPORTER_ENDPOINT to the Collector’s OTLP/HTTP base URL. Collector delivery replaces direct delivery, so enabling it does not duplicate traces. The protobuf exporter ships with the SDK and is code-split, so it loads only when you set an endpoint.Replay waits for every TypeScript span
TypeScript replay now verifies that each trace is complete and every expected span has persisted before finalizing the run. A successful queue flush or Collector acknowledgment can no longer make an incomplete replay appear ready.BAML call() traces ride the same pipeline
Traces from locally executed BAML functions now travel through the same batching transport as spans and trace completions, instead of posting one request each. Nothing changes in how you call call().Detached trace updates now confirm they applied
client.getTrace(id).addContext(), .setMetadata(), and .setSessionId() now wait for the server and reject if the update is refused. Previously they resolved regardless and a rejected update, such as one naming a trace ID that does not exist, was only written to the log. If you were ignoring the returned promise, await it.Close a client when you are done with it
client.close() flushes and releases the trace transport shared by a client’s wrapped functions and framework handlers, which matters for long-running processes that create short-lived clients.See OpenTelemetry Transport Architecture for the full design.More reliable Ruby trace delivery
Ruby SDK traces now use OpenTelemetry’s batching and lifecycle machinery behind the existingbitfab_span and Bitfab::Traceable APIs, so no instrumentation changes are required. Large captured inputs and outputs are packed beneath request-size limits, completed requests are delivered concurrently, and Bitfab.flush_traces now reports whether queued traces actually landed instead of only that the queue drained. A new client.close releases a transient client’s batch worker when a long-running process builds clients it later discards.Route Ruby traces through an OpenTelemetry Collector
Ruby applications can now send Bitfab traces through an OpenTelemetry Collector by adding theopentelemetry-exporter-otlp gem and setting BITFAB_OTEL_EXPORTER_ENDPOINT to the Collector’s OTLP/HTTP base URL. Collector delivery uses the official OpenTelemetry exporter and replaces direct delivery, so enabling it does not duplicate traces. If the gem is missing, the SDK warns once and keeps delivering directly rather than dropping spans.Bring your own OpenTelemetry version
The gem acceptsopentelemetry-sdk from 1.2 up to but not including 2.0, verified across that range, so an application already pinning its own OpenTelemetry version does not have to move it to upgrade Bitfab.Replay waits for every Ruby span
Ruby replay now verifies that each trace is complete and every expected span has persisted before finalizing the run, instead of waiting on per-item upload threads. A successful queue flush or Collector acknowledgment can no longer make an incomplete replay appear ready.Faster trace export in the Python SDK
Preparing a batch of spans to send is now about 2.5 times faster, continuing the encoding work in v0.33.8. Sending 64 spans spends 4.9ms on preparation instead of 12.8ms, and the saving grows with batch size, so the busiest traced functions benefit most. Nothing changes in how you instrument your code.Go SDK spans now ship over OpenTelemetry
The Go SDK now batches and delivers spans through the OpenTelemetry trace SDK instead of one HTTP request per span. Nothing changes in how you instrument:Span, Start/End, and GetFunction still produce Bitfab spans with Bitfab trace IDs, and no OpenTelemetry type appears in the Bitfab API. Under load this means far fewer, larger requests, and a bounded queue that will not pile up unbounded work behind a slow network.Each client lazily starts one background worker on its first span, so a client that never traces starts nothing. The SDK keeps its own private tracer provider and never touches your application’s global OpenTelemetry provider, so your traces and Bitfab’s stay separate.Close a client when you are done with it
Close flushes pending spans and shuts the worker down for good. FlushTraces now also reports whether delivery actually succeeded.Send Go spans through your own collector
If you already run an OpenTelemetry collector, setBITFAB_OTEL_EXPORTER_ENDPOINT to it (for example http://localhost:4318) and Bitfab spans flow through your existing telemetry pipeline instead of going straight to us. Configure that collector to forward to Bitfab. Leave the variable unset and the SDK talks to Bitfab directly, which is the default.Go 1.25 is now required
The OpenTelemetry Go modules require Go 1.25, so the Bitfab Go SDK does too. The official OTLP/HTTP exporter is linked into every build, because Go has no dynamic import to defer it behind the collector opt-in.One note on v0.12.7’s encoding change: spans are still encoded only once, but that encode now happens on the thread that ran your traced function rather than in a per-span background sender, because queueing a span no longer starts a goroutine. Queueing is non-blocking, so the delivery itself never sits on your thread.See OpenTelemetry Transport Architecture for the full design.Lower tracing overhead in the Python and Go SDKs
Both SDKs were encoding each span twice on its way out: once to check the data could be serialized, then again to build the request that ships it. They now reuse the first encode, which roughly halves the encoding work a span costs in Python and cuts it by about a third in Go. The saving scales with how much your spans capture, so the largest inputs and outputs benefit most.The Go SDK also moves the span’s final encode into its background sender, so less work happens on the thread that ran your traced function.Plugin updates work wherever the plugin is installed
Claude Code can install the Bitfab plugin for a single project instead of your whole account, and updating a project-scoped install now works. Previouslynpx bitfab-cli init and /bitfab:update assumed the account-wide install, so they failed with a “not installed at scope user” error and quietly left you on the old version. Both now update the install that belongs to the project you are in, and if you have the plugin installed in more than one place, a failure on one of them is reported instead of being hidden behind a success message.Dashboard
Database connection changes now take effect
Updating the connection string for database snapshots now applies to the connection you already have, instead of being saved without changing anything. Rotating a password or username updates in place, with no interruption to snapshots and no loss of history.Confirmation before replacing a connected database
Pointing snapshots at a different database now asks you to confirm first. The existing snapshot connection cannot be moved across, so it has to be torn down and rebuilt, and snapshots captured before the switch are lost. Bitfab checks the new database is set up for replication before anything is torn down, so a database it cannot mirror is refused while your existing connection stays untouched.More accurate snapshot status
The database snapshot status now reflects the live state of your connection. A temporary outage on our side no longer shows a working connection as failed, and a connection that stalled partway through setup now reports a failure you can act on instead of showing as still activating.Diff view opens on the first real change
The experiments Diff view now lands directly on the first span whose input or output actually differs from the original run, instead of opening on the trace root. Spans with no differences (and the trace root itself) are dimmed and unselectable, and when a replay matches the original everywhere, the view says “No diff available” instead of showing an empty comparison.Detached trace updates now confirm they applied
In the Python SDK,client.get_trace(id).add_context(), .set_metadata(), and .set_session_id() now wait for the server and raise if the update is rejected. Previously they returned a threading.Thread you could join, and a rejected update, such as one naming a trace ID that does not exist, was only written to the log. If you were joining the returned thread you can drop that call, because the update has already been applied by the time the method returns.Dashboard
Compressed trace ingestion
Bitfab’s ingestion endpoints now accept gzip-compressed request bodies. If you send traces through an OpenTelemetry Collector, its default compression now works as-is, so there is no need to turn compression off on the exporter. Requests compressed with deflate work too, and the accepted encodings are listed in the HTTP endpoints reference.A connected Neon database now counts everywhere
If you connect a Neon project directly, Bitfab treats it as your database connection across the product: traces show the “Snapshot captured” badge, and the connection status shown on the Integrations page and reported to your coding agent reflects the Neon project you pinned. Those signals previously only recognized a connection made through the Database section, so a Neon-connected workspace could look unconnected while replay was already branching its database.A brief Neon outage no longer looks like a broken connection
When Neon cannot be reached for a moment, your connection is now marked unverified and re-checked on its own instead of being reported as broken. You are only asked to reconnect when the API key or the project has actually changed.Setup teaches the primitives before it touches your code
Setup now opens by explaining the two primitives you instrument with,withSpan and replay, including the five ways replay can change a method when you re-run a captured trace: run it normally, feed it the recorded inputs, feed it modified inputs, or skip it and return its recorded or modified outputs. Instrumenting asks you to make that choice per method, so the explanation now comes first, right after you sign in and before any part of your repository is read. Running /bitfab:setup explain on its own shows the same thing without signing in.Choose guided setup, or instrument it yourself
After signing in, setup asks whether you want to be walked through instrumenting or would rather do it yourself. Guided is the default and drives the whole thing, checking with you at each decision. Choosing to do it yourself hands over the SDK guide for your language and where to fetch your API key, then stops without scanning your codebase or changing a line.Know what setup will do before it starts
Once you pick the guided path, setup says what is about to happen before it reads anything: it analyzes your repository, instruments the AI features you choose, and writes a replay script. It also notes that it will prompt you whenever it needs input, and that the whole thing usually takes 10 to 17 minutes depending on how many features you instrument.Close or update a trace plan and keep going
The trace plan page in Studio now has a single button. It reads Close when you have not changed anything, and Update once you have toggled which spans get captured or how they replay. Either one saves the plan and hands straight back to your coding agent, which carries on writing the instrumentation. Closing the window does the same thing, so opening a plan to look at it is never a dead end.Skip the browser without losing your edits
While the plan is open, your agent asks in the terminal whether to continue instrumenting, so you can skip the review entirely and it will close the plan page for you. Answer it whenever you like: a capture set you saved in Studio always takes precedence over that prompt, so the spans you chose are the spans that get traced.Dashboard
Connect a Neon database directly
If your production database runs on Neon, you can now connect it directly from the Integrations page instead of provisioning a replicated copy. Bitfab branches your own project at each trace’s timestamp, so replay reads the database as it was when the trace was captured. Paste a Neon API key and Bitfab handles the rest, asking which project or database only when there is more than one to choose from.Before anything is saved, the key is checked against the real workflow: Bitfab creates a branch at a past point in time, reads its connection string, and deletes it again. A key that cannot do all three is rejected with the step that failed, so a broken connection surfaces during setup instead of during your first replay.DashboardPluginsTypeScript SDKPython SDKCLIVS Code
Plugins v0.9.28, TypeScript SDK v0.33.7, Python SDK v0.33.6, CLI v0.2.234
More reliable Python trace delivery
Python SDK traces now use OpenTelemetry’s batching and lifecycle machinery behind the existing Bitfab decorators and framework integrations, so no instrumentation changes are required. Large captured inputs and outputs are packed beneath request-size limits, completed requests can be delivered concurrently, and shutdown reports when queued traces could not finish instead of silently claiming success.Route Python traces through an OpenTelemetry Collector
Python applications can now send Bitfab traces through an OpenTelemetry Collector by settingBITFAB_OTEL_EXPORTER_ENDPOINT to the Collector’s OTLP/HTTP base URL. Collector delivery uses the official OpenTelemetry exporter and replaces direct delivery, so enabling it does not duplicate traces.Replay waits for every Python span
Python replay now verifies that each trace is complete and every expected span has persisted before finalizing the run. A successful queue flush or Collector acknowledgment can no longer make an incomplete replay appear ready.Dashboard
Clearer errors when replay warm-up SQL fails
When a replay provisions a database branch and your warm-up SQL fails, the error now names the actual failure instead of reporting it as a four-minute timeout. A statement the database rejects, such as one referencing a table or index that is not in the snapshot, comes back immediately asbranch_warmup_invalid, while a warm-up that genuinely runs out of time still reports as branch_warmup_failed.Faster failure when a database branch cannot start
A replay database branch that never accepts a connection now fails in about ten seconds instead of holding the request for up to four minutes.Dashboard
Collapse the labeling panel on any trace
The labeling panel can now be collapsed on every trace you open, including the traces list and dataset review, where it was previously pinned open and always held a column of the drawer. Collapsing hands that space back to the trace itself, and your choice carries as you step between traces and reopen the drawer. The toggle sits in the trace header beside the previous and next controls, and names what it opens: Graders on grader-scored traces, Label everywhere else.Two ways into every empty rail section
A dataset’s Graders and Experiments sections now stay on the page even when they’re empty, and each teaches both ways to fill it: ask your coding agent (a card with a copyable prompt and the MCP endpoints it uses) or do it yourself with the Manage modal. Editing a dataset moved into an Edit popover in the header, and Manage and Re-run live in a compact menu on the Graders section.Bulk approval back in the grader rail
When your coding agent has labeled traces that await your review, the Graders section shows “Approve all pending” with a live count, opening the same confirmation flow as before. This also fixes the Re-run action in the section menu, which previously closed without opening the grader picker.Studio keeps experiments in reach
Finishing a labeling session in Studio brings back the list of experiments run against that dataset, so the natural next step after labeling stays one click away.Clearing a filtered experiments view
Opening experiments scoped to a dataset or a single run now shows that scope as a pill in the list header, with an x to clear it. Clearing drops you back to every run for the function, so leaving a filtered view no longer means editing the URL or navigating away.Dashboard
Settings pages scroll again
Content that ran past the bottom of the window on the Integrations, API Keys, and Export Traces pages was being cut off with no way to scroll down to it. These pages scroll normally now, so the connection string form on Integrations and the full list on API Keys stay reachable whatever the height of your window.Dashboard
The MCP endpoints behind graders, datasets, and experiments
Each of these pages now lists the MCP endpoints your coding agent calls to read and manage that resource, so the surface is visible from the page itself instead of living only in your agent’s tool list. The list sits under the empty state on a function with nothing to show yet, and the “New” button on each of these pages carries the same list in a section you can expand, next to a copyable prompt scoped to the function you’re on.Graders explain how they get made
A trace function with no graders now shows what a grader is and a prompt you can copy into your coding agent to create one, matching what datasets and experiments already did. A grader’s own page adds an “Edit” button that hands you a prompt naming that grader, for when you want to tighten its criteria or rename it.Dashboard
Experiments list, grouped and one line per run
The experiments list now gives each run a single line, gathered under the experiment group that produced it, so pass rates, before and after deltas, and code changes line up in columns you can read straight down. A “Group by” control in the header switches between grouping by experiment group and a flat list, and your choice travels in the URL, so a link you share opens on the same view you were looking at.Opening a run is quicker as well. Its traces show a placeholder the moment you click, then fill in, and they stay put when you collapse the run, so reopening it is immediate.Dashboard
Framework-observed spans can’t be mocked in replay plans
Replay plans no longer let you mock spans that a framework’s instrumentation only observes, such as LangChain or LangGraph calls. Because the SDK can’t intercept these spans, a mock would silently have no effect and the call would re-run anyway, so the trace plan now locks them with a clear explanation instead of showing a warning you could override. Spans that genuinely support mocking, including Vercel AI SDK model calls, are unaffected.Turn on replay database branching with a boolean
Replaying against a database branch restored to each trace’s original state now reads as the switch it is. PassdbBranch: true in TypeScript, db_branch=True in Python, or db_branch: true in Ruby, and every replay item runs against a branch sized by your mirror project’s own defaults. Passing an options object still works and is now only for tuning the branch’s compute and warm-up, and false turns branching off explicitly.Trace helpers only when nested
Reusable helper functions can now appear as child spans inside an existing Bitfab trace without creating standalone root traces when called on their own. SetcaptureWhen: "nested" in TypeScript, capture_when="nested" in Python or Ruby, or WithCaptureWhen(CaptureWhenNested) in Go. Your helper still runs normally without a parent span, and unknown values warn once before falling back to the existing always-capture behavior.Control replay code-change capture
Replay scripts now distinguish omitted code-change files from an explicit null value, so you can keep automatic git-diff capture by default or turn it off for a specific run. Supplying only a custom code-change description preserves that description while Bitfab still captures the changed files automatically. PasscodeChangeFiles: null in TypeScript, code_change_files=None in Python, or code_change_files: nil in Ruby to opt out.Dashboard
Clearer experiment completion and grader progress
Completed experiment replays without labels now appear as unlabeled and count as complete, instead of carrying a separate awaiting-label status. While graders are still running, experiment and trace progress bars now separate finished pass/fail results from pending grader checks in purple, so replay completion and grader completion remain distinct.The usage page opens right away
The usage page now renders as soon as you open it, instead of waiting on a scan of your whole trace history before showing anything. For organizations with a lot of history that wait ran past ten seconds, long enough that the Usage link looked like it did nothing at all. The heading, controls, and layout appear immediately now, and the numbers fill in as they arrive.Loading placeholders instead of misleading empty states
While usage data is still loading, the stat cards, chart, and period table now show placeholders rather than dashes and a spinner. The period table also no longer says “No usage in this period yet” before its data has arrived, and the CSV download stays disabled until there is something to export.Consistent names across every MCP tool
Bitfab’s MCP tools now follow one naming rule:list_ for collections, get_ when you already have the id, create_ for things that are new every time, save_ for create-or-update, and add_/remove_ for attaching and detaching. Ten tools were renamed under it, including read_traces to get_traces and update_agent_labels to save_agent_labels. Nothing breaks if you are on an older plugin version: the previous names are still accepted, so your coding agent keeps working until you update.Datasets stop duplicating when a flow re-runs
save_dataset (previously create_dataset) now updates the existing dataset when you save one whose name already exists on the same traced function, instead of adding a second copy. Re-running a labeling flow, or picking one back up after your coding agent loses context, no longer leaves duplicates behind to clean up. The same name under a different traced function still creates its own dataset.Grader changes across a whole experiment group
add_graders_to_experiment_group now takes several graders in one call, and the new remove_graders_from_experiment_group detaches them again from every experiment currently in the group. A grader archived after it was assigned can still be removed, and ids that were never assigned are reported back instead of failing the call.Replay concurrency now bounds database branches
Replay now creates a historical database branch only when an item enters a worker, somaxConcurrency and max_concurrency limit both active work and live branches. Larger runs avoid provisioning every branch at startup, and older SDKs remain compatible through the eager server path. If a branch cannot be created, the item reports the resolver’s specific reason instead of silently running without historical data.Warm-up SQL gets 240 seconds
Warm-up SQL passed with a replay’sdbBranch settings now gets 240 seconds to run, rather than sharing the 10 seconds the branch readiness check allows. Warming a real working set takes far longer than proving a branch answers a query, so warm-ups that used to fail the lease now finish. If warm-up SQL does fail or outrun its budget, the error says so specifically instead of reporting the branch as never having come up.Starting a replay with database branching on now also waits up to 300 seconds rather than 180. A warm-up longer than about three minutes needs an SDK carrying that longer wait, so upgrade the TypeScript, Python, or Ruby SDK before relying on one.Dashboard
Keyboard navigation in the trace list
Arrow keys now move through traces one row at a time instead of jumping to the end of the list. Up and down (orj and k) walk the list, left and right (or h and l) step to the previous or next trace, and Enter opens the one under the cursor. With a trace open, up and down move between its spans while left and right keep moving between traces.Save & Next keeps you moving through a dataset
Saving a label from a dataset’s trace panel now opens the next trace in the list instead of closing the panel. This previously only worked for traces waiting on your approval, so labeled and unlabeled traces dropped you back to the list and made you pick the next one by hand.Database-snapshot setup wires the new replay branch
/bitfab:setup db-snapshot in Claude Code, Cursor, and Codex now wires replay using the dbBranch option and the replay-branch accessor introduced in SDK v0.33.0, in place of the removed ReplayEnvironment.Breaking: ReplayEnvironment is replaced by a replay-branch accessor
Database branching for replay is now turned on by the dbBranch replay option, and the resolved branch is read through an accessor rather than an object you construct and pass in. ReplayEnvironment is removed.The old shape asked two unrelated questions with one signal: constructing an environment object was both “give me a database branch” and “here is the thing I will read inside my function”. Since the branch was always resolved through the replay context, never from the object you passed, two different instances behaved identically and the object could not actually be used for dependency injection. Making dbBranch’s presence the switch and the branch an accessor states what was already true.Before:db_branch={"min_cu": 2} with get_current_replay_branch()) and Ruby (db_branch: {min_cu: 2} with Bitfab.current_replay_branch). Passing dbBranch at all is what enables branching, so dbBranch: {} turns it on with your mirror’s own sizing. The accessor returns null / None / nil outside a replay item and for items with no branch, which replaces the old active flag: gate on it and fall back to your normal connection string. Branch sizing and warm-up, and every size the server accepts, are unchanged.Breaking: database branch settings move onto ReplayEnvironment
The branch sizing and warm-up settings introduced in v0.31.0 are now constructor arguments on ReplayEnvironment, not a separate replay option. As a sibling of environment they could be passed on their own, where they were silently discarded: the settings only ever applied to a replay that had an environment. On the constructor that state cannot be expressed at all.Before:ReplayEnvironment(min_cu=2, max_cu=2, warmup_sql="SELECT 1;")) and Ruby (Bitfab::ReplayEnvironment.new(min_cu: 2, max_cu: 2, warmup_sql: "SELECT 1;")). The dbBranch / db_branch replay option is removed rather than deprecated. All three fields stay optional, so an environment you construct with no arguments keeps working and leaves the mirror’s own defaults in place. Everything the settings do, and the sizes the server accepts, are unchanged.Superseded by v0.33.0 above:
ReplayEnvironment is gone, and these settings are back on the dbBranch replay option.Clear replay outcomes when local capture is unavailable
When a replay command finishes successfully but the plugin cannot read its local result, Claude Code, Cursor, and Codex now report the run as unverified instead of showing a misleading background-command failure. The assistant checks the server for the actual experiment outcome before evaluating results, while genuine replay-script crashes still appear as failures.Quoted numbers no longer break read_span_field over MCP
Some MCP clients send numeric arguments as strings. When a client connected directly tohttps://bitfab.ai/mcp called read_span_field with a quoted maxChars (for example "200000"), the request failed with an invalid-arguments error, even though the identical call succeeded through the Bitfab editor plugins. The endpoint now accepts either form, so reading a full span field works the same way from any client.Size and warm the database branch your replay runs against
When you replay traces against a historical database branch, that branch used to start at whatever compute the snapshot mirror happened to be provisioned with, and it served its first query from a cold cache. Both made replay latency reflect the branch instead of your code. PassdbBranch to size the compute and warm it before your function runs:minCu equal to maxCu pins the size, so every item in a run is measured against identical compute rather than one that warms up as the run proceeds. warmupSql runs inside the branch’s readiness check, before your function sees the lease, so warm-up time is never charged to the replayed call. The same options are available as db_branch in the Python and Ruby SDKs.Replay tells you when it could not reach a database branch
When a database branch could not be provisioned, replay used to run your function against your live database and hand back a result that looked normal. It now fails that item and reports why, so a run that could not use the historical data you asked for never reports a passing result. Traces captured before snapshot support still replay against your live database as before, since no branch was requested for those.See which grader failed a trace, and why
Your coding agent can now read the individual verdict each automated grader recorded on a trace, instead of only the overall pass or fail. In Claude Code, Cursor, or Codex,read_grader_labels returns each grader’s name, its verdict, the reasoning behind it, and its failure diagnostic, so you can ask which specific check a trace failed. Query it by trace to see every grader’s verdict on those traces, or by grader to review that grader’s most recent verdicts across traces when you are working out why a grader keeps misfiring.Name experiment groups from your coding agent
You can now give experiment groups a name and rename them later, making related runs easier to identify across replay and grading workflows. In Claude Code, Cursor, or Codex,save_experiment_group creates a group from existing experiments or updates an existing group name without changing its membership.Browse workflows across every trace function
All Traces now includes Datasets, Experiments, and Graders in the same workflow dropdown used inside each trace function. The organization-wide dataset and experiment views let you browse and open work across functions from one place, with function context on dataset rows, pagination for larger histories, and live experiment progress and results.Read a single experiment’s grader results from your coding agent
The Bitfab MCP tools now includeget_experiment, which returns one experiment (replay test run) by id: its status, pass/fail totals, replay-versus-original delta, experiment group, and grader results, including a per-grader passing and failing breakdown. Use it when you already have an experiment id and want just that run’s results, instead of listing a function’s whole experiment history. The list_experiments results now also include each run’s experiment group id.Dashboard
Consistent MCP tools across local and remote connections
Coding agents now receive the same 32 Bitfab tools, descriptions, validation schemas, and metadata whether they connect through the local plugin or the remote MCP server. Trace searches now validate real calendar dates and integer limits consistently, and trace-plan creation derives initial replay mocking decisions server-side so agents cannot accidentally override them before confirmation.Attach the first grader from an experiment
You can now add or remove graders from an experiment even when none are currently attached, as long as its trace function has graders available. The Graders section remains visible in this state, so you can open Manage without needing an existing assignment first.Choose the model that judges your graders
You can now pick which model runs each LLM-as-judge grader instead of every grader using the same one. On a grader’s detail page, the new Model dropdown lets you choose Gemini 2.5 Flash (the default), GPT-5.6 Sol, GPT-5.6 Terra, Claude Opus 4.8, or Gemini 3.1 Pro, and the choice takes effect the next time that grader evaluates a trace. You can also set it from your coding agent: thesave_grader plugin tool now takes a model option, so a request like “have this grader use Opus 4.8” sticks. Graders you do not change keep running on the default.Dashboard
Arrow keys move between spans inside an open trace
With a trace open on the experiments, dataset, or traces pages, up and down (or j and k) now step through that trace’s spans instead of jumping you to a different trace. Left and right (or h and l) still move between traces, and when no trace is open, up and down keep moving the cursor through the list.Incremental summaries for resumed coding sessions
Coding sessions resumed under the same session now produce a summary of only the new work instead of repeating earlier turns. Sessions that never include both a user request and an assistant response no longer generate a summary, reducing noise from open-and-close sessions.Dashboard
Grader results in the experiment comparison view
Opening a trace in a grader-scored experiment now shows its grader pass and fail results in the before-and-after comparison, matching what the trace list already shows, instead of an “Awaiting label” placeholder. You see the grader pass rate move from the original run to the replay, or a live grading indicator while scoring is still in progress.Filter traces by original vs. replay
The traces list now has an Original / Replays / All filter so you can separate live production traces from replay traces generated by experiments and test runs. It defaults to Original, so you see real production activity first, and you can switch to Replays or All whenever you need to inspect experiment re-runs.Dashboard
Complete grader set on experiment panels
The graders panel on an experiment now lists every grader scoring that run, including the ones inherited from its dataset, so it matches the pass and fail results shown on each trace. Graders that come from the dataset are marked with a small icon you can hover to confirm, and re-runs stay limited to the experiment’s own graders.Dashboard
Reliable grader reruns
You can now see live progress when re-running graders on a dataset or experiment, and that progress resumes if you refresh or return while the run is still active. Starting the same rerun again reconnects to the existing work instead of creating a duplicate, while a different selection is blocked until the current run finishes. Completion updates recover even if a live update is missed, so results do not stay stuck in a running state.Live grading progress on trace rows
When graders re-score a dataset or experiment, each trace row now shows its grading as it happens instead of going blank. A loading ring appears the moment scoring starts and the pass rate fills in live as each grader reports, so you can watch results arrive. Finished rows keep their green or red verdict, with the pill’s border showing the pass/fail split.Dashboard
Per-grader labeling on the trace list
When you open a trace that has graders from the Traces page, the labeling panel now shows each grader’s verdict so you can review and label them one by one, the same per-grader view you get inside a dataset. Traces without graders keep the standard pass/fail labeling panel.Dashboard
Related experiments on the dataset page
Each dataset now shows the experiments run against it, right in the dataset view. See each run’s pass rate at a glance and jump straight to a single experiment, or open the full list filtered to that dataset, without leaving your workflow.Dashboard
Graders now have their own section
Every trace function has a Graders section alongside Traces, Datasets, and Experiments. It lists your graders with their pass rate and the number of datasets and experiments each one is attached to, and you can sort by status, pass rate, evaluation volume, or recency, and archive or unarchive a grader without leaving the list.Grader overview with dataset and experiment footprint
Opening a grader now shows an overview of what it does and where it runs: its overall pass rate, the criteria it evaluates, and every dataset and experiment it is attached to with that grader’s pass rate on each. Pass rates combine the grader’s automated results with your own human labels.Automatic code-change capture on replay
When you replay traces after editing your code, the SDK now attaches the diff to the resulting experiment automatically, so you can see exactly what changed alongside the results with no extra arguments. If you don’t passcodeChangeFiles to replay(), it captures your working-tree changes against your trunk and includes them; passing an explicit code change still takes precedence when you want a precise per-edit before/after.Available in the TypeScript, Python, and Ruby SDKs. Opt out with BITFAB_DISABLE_CODE_CHANGE_CAPTURE.TypeScript SDKPython SDKRuby SDKDashboard
TypeScript SDK v0.30.0, Python SDK v0.30.0, Ruby SDK v0.30.0
Attach graders to a single replay
replay() now takes a graderIds / grader_ids option so you can grade one experiment with specific graders without permanently adding them to the dataset. The run is graded by the union of the graders you pass and the dataset’s own graders, so a one-off check runs alongside your standard ones.grader_ids), and Ruby (grader_ids) SDKs.Dashboard
Accurate experiment pass rates
Completed experiments without assigned graders now show a pass percentage from their trace labels instead of--%, so the summary reflects the results already available below it. Experiment rows use grader results when graders are assigned and label results otherwise, while runs with labels still outstanding remain pending.Honest experiment comparisons
Experiment trace rows now omit the before-to-after grader badge when the original trace has no compatible grader results. The current grader score remains visible, so you can see the replay outcome without a misleading missing baseline.Graders on the experiment page
Experiments now show the graders attached to each run right beside the traces, with a per-grader pass/fail breakdown for the run. Open a grader to read its criteria, use Manage to attach or detach graders, and Re-run to re-grade the experiment’s traces. The pass and fail bars update live as grading runs.Group and grade experiments from your coding agent
You can now organize existing experiment runs into a shared group and attach a grader across every current experiment using the newcreate_experiment_group and add_grader_to_experiment_group MCP tools. Completed experiments queue only missing grader results immediately, while pending experiments use the grader when they finish. Grader assignments stay on the experiments already in the group, so experiments added later do not inherit them automatically.Dashboard
Replays for up to 5,000 recent traces
Replay runs can now select up to 5,000 recent traces at once, up from 100, so larger evaluations no longer need to be split into manual batches. Starting a replay now avoids loading every child span up front, while dataset-backed replays continue to use the dataset’s full trace list regardless of the recent-trace limit.Dashboard
Guidance for creating datasets and experiments
Datasets and experiments are created by asking your coding agent, and the dashboard now shows you how. When a datasets or experiments page has none yet, it explains what the primitive is and gives you a ready-to-copy prompt to paste into your coding agent. The “New” button on those pages opens the same guidance with an example, so you are never left on a blank screen wondering how to start.Paginated grader lists in coding agents
list_graders now returns manageable pages for functions with large grader collections, with name search and a cursor for fetching the next page. It returns active graders by default, can include archived definitions when requested, and leaves grader-training pipeline entries out of the results.Assistant runs your plan without pausing to ask
The Bitfab assistant no longer stops after each experiment to ask whether to keep going or to revert a fix. It now reports the results and automatically continues through the experiments you already approved, wrapping up on its own once the plan is done. A multi-experiment run finishes in one pass instead of prompting between rounds.Attach graders to experiments from your coding agent
You can now attach graders directly to an experiment so they run against its replay traces, using two new Bitfab MCP tools,add_graders_to_experiment and remove_graders_from_experiment. Your coding agent can manage an experiment’s grading scope without leaving its workflow, the same way it already manages dataset graders. At completion an experiment is scored against the union of these direct attachments and its dataset’s current graders.Dashboard
Experiments grouped by run
Your experiments page now groups a trace function’s runs into groups, so each iteration reads as one unit rather than a flat list of runs. Groups are ordered by their most recent run, each header links back to the dataset it was run against, and you can collapse a group to focus. Runs launched without a group are gathered under an “Ungrouped” heading.Dashboard
Live dataset grader pass rates
Dataset grader pass-rate pills now stay current as new grader results arrive or grader assignments change. The datasets list refreshes automatically, so you can monitor evaluation progress without reloading the page.Dashboard
Experiment labeling shows only the run’s graders
When you open a trace inside an experiment, the labeling panel now shows only the graders that experiment was scored against: its dataset graders plus any graders attached to the run, rather than every grader defined for the function. Experiments with no graders open straight to the pass/fail labeling panel.Grader definitions after saves
After you create or update an automated grader through Bitfab MCP tools, your coding agent now presents the full saved definition instead of a generic success message. The response includes the grader function, status, evaluation focus, and any passing or failing criteria so you can immediately verify what will be evaluated.Dashboard
Reliable Studio links
Links that open Studio now preserve all handoff parameters, including repeated values, across sign-in, datasets, experiments, trace plans, and template previews. Dataset-linked experiments also switch to the dataset organization automatically, so shared links open in the correct context.Dashboard
Manage graders from a dataset
You can now attach and remove graders directly from a dataset. Open Manage in the Graders panel on a dataset page to move graders between Available and Attached, with search and sorting to find the right one. Attached graders score every trace in the dataset, and changes take effect immediately.Dashboard
Experiment graders run across every replay
You can now attach graders directly to an experiment, and Bitfab runs them together with the dataset’s assigned graders across every trace in the completed replay. The finalized grader set stays with the experiment, so late-arriving replay traces are evaluated consistently and completed experiment results retain the exact grader coverage that ran.Dashboard
Framework labels on trace plans
Trace plan headers now show the instrumentation framework Bitfab detected for your workflow, next to the function name and language. Plans that span multiple frameworks list each one, so you can see at a glance how a workflow is instrumented.Dashboard
Consistent labels across trace lists
Workflow, dataset, and experiment trace lists now use the same current label state, so pass, fail, and skipped results stay consistent across views. List loading also avoids fetching unused trace metadata, making these pages more efficient without changing how traces are managed.Dashboard
Keep your place when switching functions
Switching trace functions from the sidebar now keeps you in the current section, such as Traces, Datasets, Graders, or Experiments, instead of sending you back to Traces. When you switch from a specific trace, dataset, or grader, Bitfab opens the matching section list for the new function so resource IDs are not carried across functions.Dashboard
Live grader results across review views
Dataset reviews, experiment cards, experiment trace rows, and trace lists now update immediately as grader results and dataset grader assignments change. Pass rates and per-grader verdicts stay current during grading and review without a manual refresh.Dashboard
More reliable dataset grader labeling
Dataset grader labels now stay consistent when automated evaluations and human reviews overlap, so one source no longer overwrites the other. Newly added completed traces are automatically graded by their assigned dataset graders, and the labeling panel preserves in-progress selections when a save or refresh fails.Dashboard
Grader pass rates on trace lists
When a dataset has graders assigned, trace rows now show how many graders passed as a pass-rate bar (for example, 3 of 4) instead of a single Pass or Fail. It appears on the dataset review page, on experiment rows, and in the experiment header, so you can see at a glance how each trace and each run scored across all of its graders. The main traces list shows the same pass rate for grader-evaluated traces.Dashboard
Run graders without tuning
Active graders now evaluate traces directly from their saved criteria, so dataset grader runs work immediately without tuning. If a tuned prompt exists, Bitfab still uses it; empty or stale prompts fall back to the grader criteria.Dashboard
Live dataset review updates
Dataset pages, trace lists, and labeling panels now stay in sync as graders are assigned or removed and labels are added. Reviewers see the current dataset grader order immediately, while removed graders disappear from active labeling workflows without deleting their historical labels.Dashboard
Label traces while reviewing experiments
You can now label traces without leaving the experiments view. Open the before/after comparison for any trace and use the new Label toggle, next to the Diff / Original / Replayed switch, to score it. When the dataset has graders, the panel shows each grader for per-grader approval or override; otherwise it is a simple pass/fail verdict with notes.Dashboard
Smoother dataset grader reruns
Dataset grader reruns now keep traces in a grading state through transient evaluation failures, so temporary issues no longer appear as permanent errors. If every retry fails, the trace still moves to an error state instead of remaining stuck in grading.Dashboard
Compare individual spans in the Diff view
When you compare a replayed trace against its original, the experiment Diff view now lets you open any span that changed, not just the whole trace. The span tree greys out spans whose input and output stayed the same and keeps the changed ones selectable, so you can jump straight to what your code change actually affected.Clearer replay diffs
The Diff view now shows the input side fully expanded, so you can read exactly what the model received, and hovering a highlighted line tells you whether the replayed run excluded or included it.Dashboard
Trace details open in a side panel
Clicking a trace on the traces page now opens a slide-in detail panel next to the list, matching how dataset and experiment review work, instead of navigating to a separate page. You can label the trace Pass, Fail, or Skip right from the panel and move through traces with Save & Next. Links to individual traces still work: sharing or reloading a trace URL opens the same list with the panel already open, and your active filters stay in the URL.Redesigned trace list rows
Trace rows across the traces, dataset, and experiment lists now share one design that separates how the run went from how it was judged. A leading icon shows the run state (running, completed, errored, or a replay), while the verdict pill shows Pass, Fail, Skip, or an agent suggestion awaiting review, with a robot glyph marking machine verdicts. Replayed traces are tinted indigo so re-runs stand out, unlabeled rows show an input and output preview, and reviewer notes appear inline on labeled rows.Re-run graders on a dataset
When a dataset has graders attached, you can now re-run them across every trace in the dataset directly from the dataset page. Open the Graders panel, choose which graders to run, and follow the run from running to finished. You can also click any grader to view its evaluation criteria and prompt.Dashboard
Label datasets grader-by-grader
When a dataset has graders assigned, reviewing a trace now shows a labeling panel with one row per grader instead of a single pass/fail. Approve an automated grader’s suggested verdict in one click or override it, add a note, and move through the dataset trace by trace. Human labels always take precedence over the automated suggestions.Dashboard
Reliable experiment history pagination
Experiment histories now load every run when several experiments start at nearly the same time. Infinite scrolling no longer skips or repeats experiments that share the same timestamp.Dashboard
Scroll through complete experiment histories
Experiment histories now keep loading as you scroll, so older runs remain available instead of stopping after the newest 50. This works across function, dataset, and experiment-group views. The loading footer stays visible while more runs remain, with a spinner while the next page arrives.Dashboard
Dataset graders now grade automatically
The graders you assign to a dataset now run automatically on that dataset’s traces, and on the replayed traces when you run an experiment against the dataset. You get grades on your dataset and experiment results without kicking off anything by hand.Create and edit graders from your coding assistant
Claude Code, Cursor, and Codex can now create and edit automated graders for a traced function with the newsave_grader and list_graders tools. Ask for a check like “the reply never invents order numbers” and your coding agent defines the grader, then renames, updates, archives, or restores it on request without leaving your workflow. Graders are saved as definitions for now; nothing runs them against new traces automatically yet.Dashboard
Diff view for replay comparisons
When you replay a trace to test a code change, the trace comparison drawer now opens on a new Diff view that shows the replayed run against the original side by side, input on the left and output on the right, with changed lines highlighted. You can tell whether the change helped without flipping between the Original and Replayed panes and holding both in your head. Use the Diff, Original, and Replayed toggle at the top of the drawer to switch views; your choice sticks as you step through the run.Assign graders to datasets from your coding assistant
Claude Code, Cursor, and Codex can now assign graders to datasets withadd_graders_to_dataset and remove_graders_from_dataset. list_datasets now includes assigned graders, so you can inspect evaluation coverage and update it without leaving your coding workflow.Dashboard
See experiment annotations in trace comparisons
Experiment trace comparisons now show the replay annotation beside the pass/fail transition, so you can see the labeler’s reasoning without returning to the trace list. Hover over a truncated annotation to read its full text in a tooltip.Keyboard navigation in dataset review and experiments
You can now move through traces and spans with the keyboard while labeling a dataset or comparing experiment results. Use the arrow keys or Vim keys (h, j, k, l) to step between traces and their spans, and press Esc to close a trace’s detail view.Start setup with a specific request
You can now pass--prompt (or -p) to bitfab init and bitfab setup to tell the setup agent what you want instrumented from the start. The prompt is forwarded into setup in Claude Code, Codex, and Cursor, so onboarding can begin with the workflow you already have in mind.Lower tracing overhead in the Python SDK
Traced functions in the Python SDK now return as soon as their trace data is captured, instead of waiting for spans to finish uploading to Bitfab. For latency-sensitive code, this takes a network round-trip out of your own request path while your traces keep uploading in the background.Dashboard
Browse past experiments from the dashboard
Every trace function now has an Experiments tab next to Traces and Datasets, listing the experiments that have run against it, newest first. Each run shows its pass rate and how many traces were fixed, regressed, still passing, or still failing versus the original, so you can tell at a glance whether a change helped. Expand a run to inspect its traces, compare the original and updated output, and view the code change that produced it.TypeScript SDKPython SDKRuby SDKDashboardPluginsCLI
TypeScript SDK v0.29.1, Python SDK v0.29.1, Ruby SDK v0.29.1, Plugins v0.8.158, CLI v0.2.203
Replay verdicts persist by the original trace
When your coding agent evaluates a replay run, its pass/fail verdicts now persist against the trace each item was replayed from: passtestRunId plus the item’s originalTraceId to update_agent_labels and Bitfab resolves them onto that run’s replay traces. Verdicts reliably reach the experiments page without the agent ever needing a server-generated replay trace id.Replay items rename source to original
Replay items, progress events, and adapt-inputs context now name the replayed-from traceoriginalTraceId and originalSpanId (snake_case in Python and Ruby); the previous sourceTraceId/sourceSpanId names keep working everywhere as deprecated aliases, so existing scripts are unaffected. An item’s traceId is now null while the run streams and is filled in with the server replay id when the run completes.Replay verdicts are saved automatically
When you replay a single trace to check whether a fix worked, the assistant now saves its pass/fail verdict onto that replay trace instead of only showing it in chat, so your conclusion sticks and appears alongside the trace. The replay path stays lightweight, with no Studio, dataset, or experiment setup. If your SDK is too old to return a replay trace ID, the verdict stays in chat with a prompt to upgrade the SDK.TypeScript SDKPython SDKRuby SDKDashboardPluginsCLI
TypeScript SDK v0.29.0, Python SDK v0.29.0, Ruby SDK v0.29.0, Plugins v0.8.156, CLI v0.2.201
Inject custom values into specific spans during replay
When you replay a trace, you can now override the output of a chosen span instead of running its real code or replaying its recorded output. Match a span by its name, function key, or type, then return a fixed value or one computed from the span’s live inputs and its original recorded output. Available in the TypeScript, Python, and Ruby SDKs.Dashboard
Long trace results stay visible
Large arrays in trace Input and Output views now open automatically, so you can see long result lists without an extra click. When those lists contain nested objects or arrays, each entry stays collapsed to keep the trace readable; compact scalar values remain visible.DashboardPluginsTypeScript SDKPython SDKRuby SDKGo SDKCLI
Plugins v0.8.155, TypeScript SDK v0.28.11, Python SDK v0.27.8, Ruby SDK v0.23.8, Go SDK v0.12.5, CLI v0.2.200
Read one span without loading the full trace
Fetch a single persisted span directly from a trace in the TypeScript, Python, Ruby, and Go SDKs. Select it by canonicalid or by name; repeated names return the last occurrence by default, with options for the first or a zero-based occurrence.Reliable nested traces from the first Node.js call
The TypeScript SDK now preserves nested span context from the first traced call in Node.js, including ESM and CommonJS applications. For async-generator streams, wrap the controller that owns the iteration in an outerwithSpan call so service-side production and consumer-side work appear under one trace.Trace plan warnings for spans that won’t replay
Trace plans now flag spans that won’t replay cleanly before you confirm. If the entry point’s input can’t be serialized, the plan shows a “Root not replayable” warning; spans mocked despite output that can’t be serialized are called out the same way. When a plan carries any of these warnings, Confirm opens a confirmation step so you accept them deliberately rather than by accident. The warnings show on the trace plan review page, and your coding agent applies the same rules when it drafts a plan.Dashboard
Structured Map and Set trace output
Trace outputs containing JavaScript Maps and Sets now render as navigable structured data instead of collapsed string representations. Maps preserve distinct keys even when their string forms collide, and nested Maps and Sets remain expandable in the trace viewer.Dashboard
More reliable dashboard startup
The Bitfab dashboard now avoids analytics initialization crashes in browsers where cookie access is unavailable or blocked. Analytics stays inactive until it is ready, so affected sessions can load the dashboard normally.Edit span templates without leaving the trace you’re viewing
When you ask your coding agent to change how spans render, Bitfab no longer pulls you onto a separate template-preview page. If you’re already looking at a trace of that function, it now offers to edit the templates in place, and your open trace re-renders live with each change you make. The dedicated preview page, with click-to-target editing on the function’s most recent trace, is still one option away when you want it. Available in the Claude, Cursor, and Codex plugins.Fix flow reverts changes that cause regressions
When you use the Bitfab assistant’s fix flow and re-run your full dataset to lock in a fix, it now checks whether your change broke traces that were passing before. If it finds real regressions, the assistant recommends reverting the fix and starting a fresh attempt, so you never ship a change that trades one fixed trace for several broken ones. The trace you were fixing stays saved in your dataset as a regression test to revisit. Available in the Claude, Cursor, and Codex plugins.Choose where fixed traces get saved
When you fix a failing trace with the Bitfab assistant (/bitfab:assistant fix), it now asks which dataset to save the fixed scenario to, instead of silently adding it to whichever dataset already existed. Pick an existing dataset, create a new one, or continue without saving. When you have several datasets it recommends the most recently used one and keeps the list short, so the choice stays quick.Dashboard
Trace plans stay available longer
Trace plans now remain available for 365 days, giving you much more time to return to an instrumentation plan before confirming it. The longer review window applies to newly created trace plans.Session length in chat summaries
Chat session summaries now show how long each coding session lasted, so you can put the product feedback in context at a glance. Slack notifications show the readable duration, while generic webhooks include the exact start time and duration in milliseconds.Dashboard
Redesigned trace input and output view
Trace spans now show Input, Output, Context, and Error as distinct, color-coded zones with sticky headers, so you always know which part of a span you’re reading as you scroll. Input and Output sit side by side and stack automatically when the panel is narrow. When a span has an error, the Output header shows a control that jumps straight to the error details.Collapsible JSON for trace payloads
Trace input and output now render as an interactive tree you can expand and collapse, instead of a flat text dump. Deeply nested objects, large arrays, and long embedding vectors stay collapsed by default, so you can drill into just the parts you care about.Dashboard
A clearer trace plan review
The trace plan review page now speaks the same visual language as its “What is a replay?” explainer. Each span shows a type-colored dot with an icon for what happens on replay (re-runs live, mocked from the recording, or skipped inside a mock), its classification written out beside the name, and a Re-run | Mock toggle on the right. Every control has a tooltip, and the tree navigates with the arrow keys.The replay entry point always re-runs
A replay starts by re-running the top traced span, so its Mock control is now disabled with an explanation instead of silently having no effect. If you untrace spans above a mocked one, the newly promoted entry point switches to re-run and its children are no longer marked as skipped. Run/mock choices also survive untracing and re-tracing a span, and closing the replay explainer with Escape no longer cancels the whole plan.DashboardTypeScript SDKPython SDKRuby SDK
TypeScript SDK v0.28.9, Python SDK v0.27.7, Ruby SDK v0.23.7
See which spans were mocked on a replay
When you replay a trace, spans set to mock on replay are served from the original trace’s recorded output instead of re-executing. The trace view now marks those spans with a badge, both in the span tree and on the span header, so you can tell at a glance which nodes were replayed from history and which re-ran live. Recording that disposition requires the latest TypeScript, Python, or Ruby SDK.A replayability check before you accept a trace plan
When you instrument an AI workflow, Bitfab now verifies the traced function’s root can be replayed before it proposes the trace plan, instead of letting you accept a plan and only then discover the root cannot be replayed. If the root’s inputs are not serializable, setup resolves it up front by moving the trace boundary inward, using a framework handler, or refactoring, so the plan you confirm is one you can actually replay against later. Available in Claude Code, Cursor, and Codex.A faster path through instrumentation setup
Bitfab setup now moves directly from finishing one instrumented workflow to choosing the next workflow, selecting another target, or finishing setup. It shows how to exercise the workflow and run its generated replay command, while replay coverage checks remain available as an explicit action. When you choose the next workflow, Bitfab refreshes its workflow scan so targeted setup runs do not miss other candidates. Available in Claude Code, Cursor, and Codex.Studio opens in your normal browser
Bitfab Studio now opens in your usual browser as a regular tab, so you keep the address bar, tab controls, and the rest of your browser workflow. When a Studio session ends on macOS, the plugin closes only its matching tab and leaves your other browser tabs untouched.Honest Studio launch reporting
When a plugin command opens Bitfab Studio, it now always surfaces a clickable link in chat, and the message no longer claims a window opened before one actually did. If a browser could not be launched at all (common on remote or SSH sessions, or when no supported browser is available), the command reports why and the surfaced link still connects the session when you click it.Automatic recovery when a Studio window never appears
Previously, if a Studio window failed to surface, commands could wait indefinitely and later opens kept pointing at the dead session until it was cleared by hand. The plugin now detects a window that never connected, ends the wait with a clear reason so your coding agent offers a retry instead of treating it as a cancel, and clears the session automatically so the next open starts a fresh window. Stale or unresponsive Studio background processes are also detected and restarted on their own, so Studio commands always run the version of the plugin you have installed.Dashboard
Animated replay explainer on the trace plan page
The trace plan’s “What is a replay?” modal now teaches by showing. A side-by-side animation plays the original run recording each call’s input and output, then a replay re-running it: injecting the recorded input, answering mocked calls straight from the recording, and skipping everything nested under a mock. Each call is annotated with its replay plan (runs live, from recording, or skipped) so you can see why the replay behaves the way it does before confirming your plan. With reduced motion enabled, the modal shows the final annotated diagram as a static picture instead.Recover a stuck Studio session with bitfab login --force
If bitfab login reported that a Studio window was recorded as open but was not responding, there was previously no way to clear it from the command line. Running bitfab login --force now clears the stale Studio session before opening a fresh window, so you can get straight back to signing in. The error message also points you to the flag whenever you hit that state.Dashboard
More reliable trace ingestion
Bitfab now handles traces and spans that contain Postgres-incompatible text in their raw payloads, so ingestion can continue instead of failing the write. This improves reliability for SDK uploads that include null bytes or malformed Unicode from upstream tools.Readable analyze-repo summaries
bitfab analyze-repo now prints a compact terminal report after uploading draft trace plans, so you can see the selected workflows, frameworks, instrumentation effort, suggested capture methods, replay mocks, and real-data value without opening plan links. The same summary output works across Claude Code, Codex, and Cursor runs, with long lines wrapped for terminal readability and skipped candidates kept in the report.More reliable trace plan setup
Asking Bitfab to create a trace plan now reliably opens it in Bitfab Studio for review, instead of occasionally rendering the plan inline in the chat. Requests like “create a trace plan” or “instrument the next function” route directly into the setup flow, and following up to instrument another function reopens the Studio confirmation UI automatically.Dashboard
More reliable trace search indexing
Bitfab now recovers from temporary upstream interruptions while preparing trace search summaries, reducing cases where newly ingested traces fail to become searchable. The retry behavior covers rate limits, provider-side failures, and network transport failures while still stopping on deterministic request errors.Steer analyze-repo with a prompt
bitfab analyze-repo now takes free-text guidance so you can point it at the parts of your codebase you care about. Pass --prompt (short form -p, or just a trailing quoted argument) with something like “focus on the billing and checkout flows” and the scan biases toward those areas when picking which AI workflows to draft trace plans for, topping up any remaining slots from the rest of the repo. Combine it with --limit to cap how many plans it uploads.Go SDK: drop() is safe to call across goroutines
Calling drop() on a trace from one goroutine while another goroutine finishes a span on the same trace no longer races on the trace’s dropped state. If your Go service drops traces from a different goroutine than the one running the traced work, that path is now safe.Dashboard
More reliable CLI sign-in handoff
Bitfab now waits for the browser sign-in handoff to confirm that the CLI received its credentials before showing success. The Studio close page stays in a finishing state while sign-in completes, and if delivery fails it leaves the page open with a retry option instead of making the terminal wait silently.TypeScript SDKPython SDKRuby SDKGo SDK
TypeScript SDK v0.28.8, Python SDK v0.27.6, Ruby SDK v0.23.6, Go SDK v0.12.3
drop() now stops later spans from being sent
Calling drop() on the current trace now prevents any span that finishes afterward from being uploaded at all, so dropping a run that carries sensitive data keeps that data local instead of sending it and clearing it server-side. Spans already sent before the drop() call are still cleared, and the trace is still marked dropped. Available in the TypeScript, Python, Ruby, and Go SDKs.TypeScript SDKPython SDKRuby SDKGo SDK
TypeScript SDK v0.28.7, Python SDK v0.27.5, Ruby SDK v0.23.5, Go SDK v0.12.2
Discard an in-flight trace with drop()
You can now discard the current trace at runtime from your own code, when you decide it shouldn’t be recorded (a health check, a cache hit, any path with no useful signal). Call drop() on the current trace and Bitfab skips it: its inputs, outputs, and spans are never stored, and anything already uploaded for that trace is cleared.getCurrentTrace().drop(), get_current_trace().drop(), and GetCurrentTrace(ctx).Drop() in Go). The call is always safe: if there’s no active trace it does nothing, and it never throws or interrupts your code.More reliable analyze-repo from the CLI
bitfab analyze-repo now runs non-interactively through Claude Code, Codex, and Cursor Agent, so you can scan a repository and upload draft trace plans without opening an editor UI. Use --editor to choose the agent and --limit to cap how many plans are drafted.The CLI also gives these headless runs clearer outcomes: uploaded plans share a Bitfab run identity, logs redact API keys safely, and terminated agent processes fail with a clear error instead of looking successful.Analyze-repo runs link plans and session logs
Thebitfab analyze-repo command now gives each scan a shared run identity, so the draft trace plans it uploads and the optional captured Claude Code session are tied together in Bitfab. This makes it easier to audit what the agent found, which plans it created, and the conversation that produced them.Codex can run analyze-repo headlessly
bitfab analyze-repo --editor codex now runs through codex exec without opening the Codex TUI, using the same non-interactive scan and draft trace-plan upload flow that was already available through Claude Code. Cursor Agent is now supported too through its --print headless mode.Analyze-repo can sign in before scanning
Thebitfab analyze-repo command now opens the Bitfab sign-in flow when an interactive run is not authenticated, then continues the scan after login succeeds. Non-interactive runs still stop with clear instructions, and if a stale environment API key blocks verification the CLI explains how to fix it.Cleaner plugin login and logout in local projects
Plugin login now keeps the Studio sign-in URL visible and reports success as its own status line, which makes local and dev sign-ins easier to follow. Logout now clears project-local credentials before falling back to global credentials, so worktree-specific logins can be reset without affecting other projects.Editor sign-in checks before setup
Thebitfab CLI now checks whether Claude Code, Codex, or Cursor is signed in before it launches setup, assistant, SDK update, or analyze-repo. If the editor agent is logged out or the editor CLI is missing, Bitfab stops early with the login command to run instead of opening an agent session that fails later.Command-specific help in the Bitfab CLI
Thebitfab CLI now shows help for individual commands, so you can check the right flags and usage without triggering the command itself. Use bitfab help <command>, <command> --help, or <command> -h to inspect commands before running onboarding, login, install, or other workflows.More accurate natural-language routing in the plugin skills
The setup and assistant skills now route free-form requests to the right mode more reliably. Phrases like “trace a new workflow,” “why aren’t my traces showing up,” or “did my fix work on this trace” land in the correct mode without you having to name it. Each skill also lists its modes and what they do up front, so the full set of things it can do is visible at a glance.Clearer plugin setup status
Bitfab plugin setup now makes local and dev authentication easier to verify. Login success messages include the non-production endpoint when the plugin is pointed away from production, and plugin update checks show the version that is already installed or was just updated.Trace plans isolate external calls by default
Setup-generated trace plans now mark mockable external reads and side effects, such as database queries, HTTP calls, and writes, to return their recorded output during replay. LLM calls and local code stay live by default, so replay still tests the model behavior you are trying to improve. If an external parent span would skip live child spans, the server now keeps that parent live and expects the smaller external boundary to be mocked instead.Replay scripts stay aligned with production roots
Bitfab setup and assistant workflows now require generated replay scripts to call the same production root wrapper for traced functions, instead of introducing a replay-only helper that can drift from runtime behavior. For handler-based integrations, replay guidance now points back to the same production framework entrypoint and includes a root-parity checklist before setup finishes.PluginsTypeScript SDKPython SDKRuby SDK
Plugins v0.8.122, TypeScript SDK v0.28.6, Python SDK v0.27.4, Ruby SDK v0.23.4
More reliable replay results
Replays run through the Bitfab plugin no longer fail to report their result when the replay script prints extra output (framework logs, env-loader noise). The TypeScript, Python, and Ruby SDKs now write the full replay result to a file the plugin reads directly, so a passing replay is never mistaken for a failed one.Scan a repo for AI workflows from the terminal
Runnpx bitfab-cli analyze-repo to headlessly scan a repository for its AI workflows and upload a draft trace plan for each of the top candidates, with no prompts and no code changes. Cap how many plans it uploads with --limit (default 5), then review and confirm the drafts in Studio. Available for Claude Code.Setup asks before rewriting your code
When/bitfab:setup instruments a function, it now pauses and asks for approval before restructuring an existing framework or SDK call to attach a trace, instead of rewriting it silently. Purely additive instrumentation (wrapping an unchanged call) proceeds as before; only a change that would modify existing code stops for your confirmation, and any rewrite you approve preserves the original behavior exactly.Dashboard
Trace plans stay valid longer
Trace plans no longer expire after 30 minutes. When you set up tracing and step away before confirming a plan, it now stays valid for 7 days, so you can pick up where you left off instead of recreating it.More reliable Studio sessions
Studio now stays put across restarts. If Studio’s background process reloads, your active session is restored and reopens on the page you were last viewing instead of being lost or opening a duplicate window. Switching between pages (a dataset, a trace plan, the experiments view) now reuses the open Studio tab instead of closing and reopening it.Verified replay labels in assistant runs
Bitfab plugins now verify replay labels immediately after persisting them, so benchmark scorecards only continue once the server reports the expected effective PASS, FAIL, or skipped state. During assistant and replay workflows,persistReplayLabels parses the update_agent_labels response and stops with verification-failed if any trace label is missing or mismatched, giving the agent a clear retry path instead of reporting partial results.Framework-aware replay mocking
The trace planner now decides which spans can be mocked on replay based on how your framework captures them. Spans your code wraps directly can be mocked; spans a framework observes from the outside (LangChain, LangGraph, and similar callback-based integrations) re-run live on replay instead, while Vercel AI SDK model calls stay mockable. This keeps model calls from being dropped or wrongly mocked when you set up tracing on a framework app with/bitfab:setup.Mock a span the planner flagged
When the trace planner marks a span as not mockable, you can now override it and mock it anyway from the trace plan page in Studio. The choice is kept as a warning rather than blocked, and the plan footer shows a warning count so you can review these before confirming.Batch-analyze a repo for what to trace
The Bitfab setup plugin can now scan a whole codebase and draft trace plans in one non-interactive pass. Run/bitfab:setup analyze-repo and it finds your AI workflows, picks the top few worth tracing, and uploads a draft trace plan for each, without prompting or changing any code. Review the drafts in Studio, then run /bitfab:setup instrument on the ones you want to wire up.Replay status is available as an MCP tool
The Claude, Cursor, and Codex plugins now exposeget_replay_status directly through the local Bitfab MCP server. During replay-based assistant runs, agents can map local replay trace IDs to server trace IDs while the run is still in progress, so they can persist per-trace verdicts incrementally without relying on a separate command wrapper.The plugin MCP tool list has also dropped the deprecated grader tools, keeping trace inspection, datasets, labeling, experiments, setup, and replay status aligned across the editor plugins and Bitfab MCP endpoint.Clearer replay label errors
Replay label persistence now fails loudly when a verdict batch includes trace IDs that do not exist in the active organization, instead of saving only part of the batch and treating the rest as skipped. The Claude, Cursor, and Codex plugins now guide agents to remap replay results to server replay trace IDs and retry, so experiment labels are less likely to disappear behind a misleading success message.Trace plans analyze context nodes before review
Bitfab setup now asks Claude, Cursor, and Codex to classify replay behavior for every trace-plan node, including surrounding context nodes that are not initially captured. When you toggle those context nodes into capture from the trace-plan review, they already have a replay decision instead of needing a second classification pass.Modify flows now backfill missing analysis on older trace plans before opening review, so expanding an existing plan uses the same per-node replay guidance.Clearer dataset wording in assistant fixes
The Bitfab assistant fix flow now describes the final step as adding the trace to a dataset with a validated failing label, instead of using capture language for dataset membership. In Claude, Cursor, and Codex, the flow still replays the target trace first, then saves it to a dataset only after the replay passes and lets you choose Studio, a full dataset rerun, another iteration, or stop.Assistant fixes ask before guessing
The Bitfab assistant fix flow now confirms why a trace is wrong before changing code when the trace or conversation does not already provide a clear failure reason. In Claude, Cursor, and Codex, targeted trace fixes reuse an existing failing label, a user-stated defect, or obvious trace evidence; otherwise the agent asks what correct behavior should be before replaying or saving the trace as a regression test.Dashboard
See when experiment results are still settling
When you open the code-change summary for an experiment that is still running, it now shows how many replays are still running or awaiting agent labels, with a note that the breakdown is provisional and will keep updating as they finish. Before, the summary presented partial results as if they were final.More accurate unpaired counts
Replays that are still running, awaiting labels, or errored are no longer counted as “unpaired” in the experiment breakdown, and each now appears as its own segment in the run progress bar. The unpaired count now reflects only replays that genuinely could not be matched to an original.Dashboard
Experiment replays update as traces finish
Experiment pages now refresh replay progress as each trace finishes, so results appear while the replay is still running instead of waiting for later labeling work. Runs outside an experiment group no longer trigger experiment updates, keeping grouped experiment views focused on the runs they are showing.Trace descriptions route to assistant fixes
The Bitfab assistant now treats requests like “find and fix the trace where…” as targeted fixes, even when you describe the bad output instead of pasting a trace ID. In Claude, Cursor, and Codex, the fix flow first verifies that the request is trace-backed, then uses local instrumentation and trace search to find the matching failing trace before making changes.Dashboard
Expired trace plans show their final state
Trace plan review now shows when an awaiting plan has expired instead of letting you try to confirm it and then showing an error. The review bar switches to a disabledPlan expired action as soon as the plan is stale, including when an already-open review page crosses its expiry time.More reliable BAML client execution
BAML client execution is more reliable across the Bitfab dashboard and TypeScript SDK. Generated OpenAI clients no longer send unsupported temperature options to GPT-5 and o-series models, while supported OpenAI, Claude, and Gemini clients keep deterministic sampling where the provider accepts it.Shared Bitfab BAML fallbacks now use current tested Gemini and Claude models, so duplicate detection, entity extraction, trace summaries, and custom BAML prompts avoid stale model endpoints that could fail before the prompt ran.Trace plans show when spans cannot be mocked
Trace plan review now distinguishes spans that can return recorded output during replay from spans that must re-run live. When Bitfab cannot mock a span because the recorded output is not serializable or the span comes from library instrumentation, the plan keeps it on re-run and shows the reason in the replay control tooltip, so reviewers know why the mock toggle is unavailable.The Claude, Cursor, and Codex plugins now pass that mockability metadata throughcreate_trace_plan, so generated plans can mark those spans before you confirm instrumentation.PluginsTypeScript SDKPython SDKRuby SDK
Plugins v0.8.105, TypeScript SDK v0.28.4, Python SDK v0.27.3, Ruby SDK v0.23.3
Replay mocks are marked by default
Replay now usesmock: "marked" by default in the TypeScript, Python, and Ruby SDKs. A span tagged with mockOnReplay: true / mock_on_replay: true now returns its recorded output during replay without also passing a mock option. Pass mock: "none" when you explicitly want every child span to run real code, or mock: "all" when you want every child span to return historical output.The docs now treat replay mocking as a first-class workflow, with a dedicated Replay Mocking guide linked from the introduction and SDK pages.Redesigned trace plan review
The trace plan you confirm when setting up tracing has a cleaner, more informative review screen. Each captured span is now labeled with what it does,code, llm call, read, or write, and whether replay re-runs it live or serves its recorded output, so you can see at a glance how your workflow will replay. The span tree is easier to scan: you can toggle spans in or out of the capture set and mark which ones to mock on replay, and a “How to review” guide walks you through it the first time.Manual Studio links in plugin flows
Bitfab plugins now print a copyable Studio link whenever they open a fresh Studio page, so you can still get to login, setup, datasets, experiments, and trace-plan flows if your editor hides the browser launch. The agent instructions for Claude, Cursor, and Codex now tell the agent to surface that link in chat asStudio opened: <url>, while the command output also includes Studio opened at: <url> for terminal visibility.Setup signs you in before instrumenting
Running/bitfab:setup instrument now signs you into Bitfab before it starts analyzing your code, so instrumenting a new AI workflow no longer stalls partway through when you aren’t logged in. Previously the flow could begin and then fail later at the trace-plan step; it now authenticates up front, matching how the other setup modes already work.Safer failed-fix regression capture
The Bitfab assistant now only marks an unresolved fix as saved after the failing trace is actually attached to the selected dataset. If a dataset attach is skipped, the Claude, Cursor, and Codex plugins keep the fix unsaved and guide the agent to choose a function-scoped dataset instead, so a failed fix cannot disappear without a real dataset entry.Codex Studio navigation also returns control to the conversation once the Studio page is open and ready to report events, instead of waiting in the foreground while you inspect the page.More reliable Codex session capture
The Codex plugin now captures active Bitfab sessions more reliably while you work, so chat-session history is less likely to miss turns during tool-heavy workflows. Final session uploads still complete when Codex stops, and late background capture work can no longer revive stale local progress after a session has closed.Dashboard
Coding-agent session summaries
Bitfab now turns ended coding-agent chat sessions into structured product-feedback summaries, so teams can see what the agent tried, where Bitfab helped, where the plugin or service got in the way, and what should be improved next. Idle sessions are closed automatically, reopened sessions are summarized again after new work, and configured webhooks can send the summary to Slack with a link back to the chat session.Trace environment in read_traces
read_traces now shows each trace’s environment alongside the trace id, function, status, and timing details, so agents can distinguish production, staging, local, and unset traces during investigations. When a trace has no stored environment tag, the tool prints unset instead of hiding the field.Test-first fixes with the assistant’s fix command
The assistant’s fix command now proves a fix before it saves anything. It diagnoses the failing trace, makes the change, and replays just that one trace; only once the replay passes does it add the trace to a dataset with a validated failing label, then offers to re-run the whole dataset (in Studio or your terminal). If the fix does not land, you can save the trace as a failing test to revisit later. When you open the fixed trace in Studio, the experiments page now lands directly on that trace’s before/after comparison instead of a list you have to click into. Available in the Claude, Cursor, and Codex plugins.Replay your first captured trace during setup
When you instrument a workflow with/bitfab:setup, Bitfab now waits for your first trace to land and then offers to replay that exact trace right away, so you can confirm your replay script works before moving on. Available in the Claude, Cursor, and Codex plugins.Codex uses project-local Bitfab credentials from cached launches
The Codex plugin now resolves its active worktree even when the MCP server starts from Codex’s plugin cache without a recorded session id. That keeps setup, login, and plugin actions pointed at the project’s local Bitfab connection instead of falling back to stale global credentials.More dependable coding-agent session capture
Bitfab now keeps coding-agent chat sessions in order across transcript compaction and retry recovery, so older turns stay attached to the same session instead of shifting sequence positions. Sessions that stop without new turns also preserve their real activity and end times, and the dashboard now recognizes Bitfab slash-command activity in the same session activity view.More reliable replay mocks
Replay mocking in the TypeScript SDK now matches older historical spans even when the recorded span tree does not include a span name, falling back to the trace function key for the mock lookup. This keepsmock: "all" and mock: "marked" replay runs working on older traces instead of rerunning child spans that were meant to use their historical outputs.The TypeScript and Python replay examples now show both mock modes with mockOnReplay and mock_on_replay, so you can verify that expensive child steps are skipped before running a full experiment.Cleaner final replay results
The Bitfab assistant now keeps the final replay event log focused on one server-backed item reference per replayed trace, even when progress events already wrote per-trace payload files. This prevents assistant evaluations from scoring the same replayed trace twice in mixed-payload runs, while live progress rows still support incremental evaluation as traces finish.PluginsTypeScript SDKPython SDKRuby SDK
Plugins v0.8.89, TypeScript SDK v0.28.1, Python SDK v0.27.1, Ruby SDK v0.23.1
Live per-item replay evaluation
The Bitfab assistant now scores and records each replayed trace as soon as it finishes, instead of waiting for the entire replay run to complete. On long replays, Studio’s experiment view fills in pass/fail verdicts trace by trace while the run is still going, so you see results as they land instead of all at once at the end.The replayonProgress callback in the TypeScript, Python, and Ruby SDKs now reports per-item detail for each trace as it settles, including the deserialized inputs, the replayed output, the original output, and token usage, so tools watching a running replay can evaluate results mid-run rather than waiting for the final ReplayResult.TypeScript SDKPython SDKRuby SDKGo SDK
TypeScript SDK v0.28.0, Python SDK v0.27.0, Ruby SDK v0.23.0, Go SDK v0.12.0
Reliable tracing regardless of API key load order
The Bitfab SDKs now resolve your API key the first time a traced function runs, instead of when the client is constructed. If your key (or.env file) loads after the client is created, tracing still activates, so a client built before your environment is ready no longer silently drops every trace.All four SDKs (TypeScript, Python, Ruby, Go) also gain a BITFAB_API_KEY environment fallback when you don’t pass a key explicitly, a callable key form for deferred resolution (e.g. new Bitfab({ apiKey: () => process.env.BITFAB_API_KEY })), and an opt-in strict mode that fails loud on a missing key instead of disabling tracing quietly.Trace-first assistant investigations
The Bitfab assistant now grounds traced-function investigations in actual trace evidence before recommending a fix or revert. When it investigates an AI workflow failure, it first derives the function key from local instrumentation, searches the matching traces, and separates what the trace proves from code-based inference, so debugging starts from the run that failed instead of only from static code.Targeted trace fixes before dataset runs
The assistant fix flow now replays only the target failing trace first, then asks whether to inspect the before/after in Studio, run the full dataset, keep iterating, or stop. This makes/bitfab:assistant fix <trace-id>, /bitfab-assistant fix <trace-id>, and $bitfab:assistant fix <trace-id> faster to validate and prevents a dataset run from hiding whether the original bug is green. If you choose the full dataset run, Bitfab opens Studio as an experiment so you can catch regressions after the target fix is proven.Clearer LangChain setup guidance
Bitfab setup guidance now recognizes LangGraph and LangChain projects earlier and recommends the callback handler instead of manually wrapping graph nodes, tools, retrievers, or model calls. The docs and setup prompts also clarify that the handler already records a replayable framework root, so you only need a same-key outer span when there is meaningful application work around the graph or chain invocation.DashboardPluginsTypeScript SDKPython SDK
Plugins v0.8.80, TypeScript SDK v0.27.2, Python SDK v0.26.2
In-progress LangGraph and LangChain traces
Bitfab now shows LangGraph and plain LangChain runs as in progress as soon as the framework callback root starts, so long-running agents appear in the dashboard before their final output is available. The TypeScript and Python callback handlers also keep configured LangChain run names on chain, model, tool, and retriever spans, making trace trees easier to line up with your graph or chain config.Cleaner framework replay inputs
Python tool spans now preserve empty structured tool inputs as{} instead of falling back to raw text, and TypeScript chain callbacks handle both LangChain callback argument orders for run names and parent IDs. This keeps handler-captured traces more accurate for replay and avoids nested framework callbacks marking the outer trace complete too early.Project-local plugin credentials from cached launches
Bitfab plugins now keep using your project-local connection settings when Claude Code, Cursor, or Codex launches the plugin from a cache or nested workspace. Localconfig.local.json and credentials.local.json files are resolved together from the same project search path, so agents are less likely to fall back to the wrong workspace or global credentials.Consistent Studio navigation across plugins
Studio navigation commands now work consistently in the Claude Code, Cursor, and Codex plugins when a Bitfab skill opens a specific Studio page. The plugins now ship matching command wrappers, so agent workflows can rely on the same Studio actions regardless of which editor plugin you use.DashboardPluginsTypeScript SDKPython SDKRuby SDK
TypeScript SDK v0.27.1, Python SDK v0.26.1, Ruby SDK v0.22.1, Plugins v0.8.77
Name replay experiments
You can now give Bitfab experiments a readable name when you create a dashboard test run or start a replay. Experiment cards show the name next to the short run ID, making it easier to compare baselines, prompt edits, and code-change runs without opening each result.Replay names in SDKs and plugins
TypeScript, Python, and Ruby replay calls now accept aname option, and replay scripts can forward it with --name. The Bitfab plugins detect whether the installed SDK and script support experiment names before using the flag, and recommend an upgrade when a project is too old.TypeScript SDKPython SDKRuby SDKPluginsDashboard
TypeScript SDK v0.27.0, Python SDK v0.26.0, Ruby SDK v0.22.0, Plugins v0.8.76
Live replay progress in setup and assistant runs
Bitfab replay runs can now stream per-trace progress in Claude, Cursor, and Codex while the replay keeps running in the background. The plugins show each trace as it settles, write a per-run event log under.bitfab/replays/<run-id>/events.jsonl, and store full per-item outputs under .bitfab/replays/<run-id>/items/ so agents can review complete outputs without mixing progress logs into the result JSON.Replay progress reporters for SDK scripts
TypeScript, Python, and Ruby SDKs now include ready-made replay progress callbacks:reportReplayProgress, report_replay_progress, and Bitfab.report_replay_progress. Pass them into replay scripts to emit plugin-readable progress events while stdout stays reserved for the final ReplayResult JSON.TypeScript SDKPython SDKRuby SDKGo SDKPluginsDashboard
TypeScript SDK v0.26.2, Python SDK v0.25.2, Ruby SDK v0.21.2, Go SDK v0.11.2, Plugins v0.8.75
Capture warnings stay out of execution errors
SDK spans that have to degrade serialization now mark those payload warnings as SDK-sourced capture notices. The dashboard shows them as a bordered “Capture incomplete” tag on the span instead of mixing them into execution errors, so real runtime failures stay distinct from lossy capture metadata.Studio opens in an app-style window from Codex
The Bitfab Codex plugin now opens Studio in the same focused app-style browser window used by the other editor plugins when Chromium supports it. If the app-window launch cannot start, the plugin falls back to your normal browser, andBITFAB_DISABLE_CHROME_APP_WINDOWS=1 still forces the tabbed fallback.Dashboard
Chat-session capture handles unsupported text characters
Bitfab now keeps coding-agent chat-session capture working when terminal output includes characters that databases cannot store in JSON fields. Unsupported characters are safely replaced so the rest of the transcript, tool calls, and usage details remain available in the dashboard.Dashboard
More reliable chat session capture
Bitfab now captures coding-agent chat sessions more reliably when turns include token usage details. This keeps session histories and usage metadata flowing into the dashboard instead of dropping affected turns during ingestion.Fix a failing trace end to end
The Bitfab assistant has a newfix mode that takes one failing trace and drives it to green. Point it at a trace (/bitfab:assistant fix <trace-id> in Claude Code, /bitfab-assistant fix <trace-id> in Cursor) and it adds the trace to a dataset with a validated failing label, edits the code, then replays until the trace passes, flagging any regressions and offering to fix the other failing traces too. It only engages when you point it at a real Bitfab trace, so an ordinary “fix this bug” on untraced code still goes through normal coding. Works across the Claude, Cursor, and Codex plugins.Plugin login reuses its sign-in window
Signing in to the Bitfab plugin no longer opens duplicate Studio windows when a login is retried before it finishes. The plugin now reattaches to the window that’s already waiting for you, so a sign-in that looks slow won’t stack up extra browser windows.Filter trace search by database snapshot
Thesearch_traces tool now accepts a hasDbSnapshot filter, so you can scope a search to traces that captured a database snapshot, or only those that didn’t. Use it to quickly find the traces that can be replayed against their historical database state. Works across the Claude, Cursor, and Codex plugins.Dashboard
Plugin sign-in with Google no longer hangs
Signing in to the Bitfab plugin with Google now completes reliably and connects your coding agent. Previously a Google sign-in could leave the plugin waiting on a login that never registered.Dashboard
Faster usage page
The usage page loads noticeably faster, especially for organizations with high trace volumes, and switching the function filter now applies instantly instead of pausing while the page catches up.Dashboard
Shareable links for usage filters
The usage page now keeps your function filter in the page URL, so a filtered view can be bookmarked or shared as a link and it stays put as you move around. Your browser’s back and forward buttons also step through filter changes.Dashboard
Accurate “Snapshot captured” badge on traces
The “Snapshot captured” badge now appears only on traces that actually captured a database snapshot, instead of on every trace once your organization connected a source database. The trace list and trace view now give an accurate at-a-glance signal of which traces can be replayed against their database state from when they ran.Dashboard
Break down usage by function
The usage page can now be scoped to a single traced function: pick a function and the totals, chart, periods table, and CSV export all narrow to just that one. A new “By function” chart mode plots one line per function so you can compare volume across functions over time. The function picker and chart mode appear once your organization has more than one traced function.Bind framework handlers to your trace key once
ThegetFunction() handle now hands you framework handlers and middleware already bound to its trace key, so an outer span and the handler share one key without repeating the string. Available for the Claude Agent SDK, LangGraph/LangChain, and (TypeScript only) the Vercel AI SDK.get_function(): pipeline.get_claude_agent_handler() and pipeline.get_langgraph_callback_handler().Live progress callbacks for replay
Replay now takes an optional progress callback, so you can show live progress while a run is in flight instead of waiting for it to finish. It fires once per trace as each one settles, with running totals you can render however you like.onProgress in the TypeScript SDK and on_progress in Python and Ruby. The totals report how many traces have finished, how many ran without error, and how many threw; pass/fail verdicts are assigned after the run, so the live totals split ran-ok versus errored. Upgrade to @bitfab/sdk v0.25.0, the bitfab Python package v0.24.0, or the Ruby gem v0.21.0.Dashboard
Clearer experiment run summaries
Experiment run rows now report accurate counts: a replay that finished with an error shows as “errored” instead of “awaiting labels”, and a trace still awaiting an agent label is no longer also counted as “unpaired”. Hover any status pill to see the full breakdown of passing, failing, awaiting, and errored traces. The row also stays readable at any panel width, collapsing to the essentials when space is tight.Replay results report the replayed run’s token usage
When you replay traces, each result item’stokens now reflects the token usage of the replayed run instead of the original trace. Compare it against the original trace’s recorded usage to see how a code change moved cost. durationMs and model stay as the original trace’s reference values, and the token counts match what the experiments view shows. Upgrade to @bitfab/sdk v0.24.1, the bitfab Python package v0.23.2, or the Ruby gem v0.20.2.Dashboard
See database snapshot status on your traces
Traces now show whether they can be replayed against the database state from when they ran, and whether a replay actually used that historical state. A “Snapshot captured” badge marks an original trace you can replay against its past database; on replays, “Snapshot replayed” means your code read the historical database branch and “Snapshot unused” means it fell back to the live database. The indicator appears on the trace list, the trace and span detail headers, and in dataset and experiment rows, and only shows for organizations with a connected source database.Watch experiment results appear live during replay
When you start a replay, its experiment now appears right away and fills in trace by trace as results come back, instead of staying on “Waiting for experiments to start…” until the whole replay finished. You can watch the pass and fail counts climb in real time as each trace completes.The TypeScript SDK never crashes or hangs your app
Tracing is now fully fail-open. If anything in the SDK’s instrumentation goes wrong (a runtime without a usablecrypto, an oversized payload, a serialization edge), your traced function still runs and returns its real value, and your Node process still exits cleanly instead of being held open by a pending timer. When tracing has to degrade, the SDK logs a one-time [bitfab] warning so a dropped span is visible rather than silent. Upgrade to @bitfab/sdk v0.24.0.Type-checking no longer needs the OpenAI Agents package
The SDK’s published type definitions no longer reference@openai/agents, so projects that don’t use the OpenAI Agents integration type-check cleanly without installing that package (previously tsc could fail with “Cannot find module ‘@openai/agents’” under skipLibCheck: false). If you use getOpenAiAgentHandler(key).wrapRun(...), its result is now typed structurally: read finalOutput and cast it to your agent’s output type.Dashboard
More reliable trace ingestion under load
Traces now ingest reliably during high-volume bursts. We fixed a case where ingesting a trace could time out and fail when its search index was being built inside the same database operation that saved the trace. Search indexing now runs after the trace is safely stored, so a slow index can never block or fail ingestion.TypeScript SDK: builds no longer fail on unused optional integrations
If you used the TypeScript SDK with one integration (like the Vercel AI SDK) but not others, your bundler could fail at build time withModule not found: Can't resolve '@openai/agents', even though you never used the OpenAI Agents integration. The SDK no longer references its optional peer dependencies (@openai/agents, @boundaryml/baml) in any way a bundler tries to resolve up front, so your app only needs to install the integrations it actually uses. Upgrade to @bitfab/sdk v0.23.3.Vercel AI SDK tracing
Bitfab now traces the Vercel AI SDK out of the box. Wrap your model withgetVercelAiMiddleware and every generateText, streamText, generateObject, and streamObject call is captured as a traced LLM span, including streaming responses and which provider answered each call (handy when you fall back between models). The Bitfab plugin also detects and instruments Vercel AI SDK projects automatically during setup.Ruby SDK guards against replay key mismatches
replay() in the Ruby SDK now raises an ArgumentError when the trace_function_key: you pass does not match the key the method is actually traced under, instead of silently fetching one function’s history and recording the run under another. This brings Ruby in line with the TypeScript and Python SDKs.Replayed traces match the original trace structure
Replaying a handler-instrumented run (OpenAI Agents, Claude Agent SDK) now produces a root span named after your trace function key, matching the original production trace instead of the replayed function’s name. For OpenAI Agents, a replayed run also nests under a single root span instead of an extra duplicated one, so a replay’s span tree lines up with the trace it replays.Closed Studio windows stay closed
When you close a Studio window, it now stays closed. Previously the plugin could reopen a Studio window on its own shortly after you dismissed it; that no longer happens. Studio windows open only when you or your coding agent explicitly ask for one.Recovering a stalled Studio session
If a Studio window crashed, was closed, or your machine went to sleep, reopening Studio could stay stuck reporting that the existing session was unreachable, even after you cleared it. Clearing a stalled session now reliably reopens a fresh Studio window.More reliable Studio sessions
Fixed an issue where leftover Studio background helpers from earlier plugin versions could pile up and cause a Studio session to hang or fail to open. The plugin now clears out stale helpers before starting a session, so opening Studio stays reliable across plugin updates.LangGraph and LangChain retriever steps are now traced
Retriever calls in your LangGraph or LangChain graphs now appear as spans, capturing the query and the documents returned, so retrieval shows up alongside the rest of your agent’s work. No code change is needed beyond the callback handler you already pass.Trace streamed OpenAI Agents runs
You can now trace streamed agent runs. In TypeScript, pass{ stream: true } to wrapRun; in Python, use the new wrap_run_streamed async generator as a drop-in for Runner.run_streamed. The run’s input and final output are recorded on the root span once the stream finishes, so streaming runs are traced without changing how you consume the events.Read the BAML Collector after a call
wrapBAML / wrap_baml now expose the BAML Collector from the most recent call through a .collector attribute, plus an onCollector / on_collector callback that runs after each invocation. Use it to inspect the prompt, model, and token usage yourself, in addition to the metadata Bitfab already captures.Dashboard
Faster trace lists for high-volume functions
The traces list now loads quickly for functions with very large trace volumes, where it could previously take several seconds or fail to load. The speedup is automatic, with nothing to configure.Replayable OpenAI Agents runs with one line
If you trace the OpenAI Agents SDK with Bitfab, you can now make agent runs replayable with a drop-in change. The new run wrapper records the run’s input as a replayable root, with the tracing processor’s spans nested underneath, so the trace replays by key with no hand-written wrapper. Keep registering the processor for the internals and swap your run call for the wrapper.get_openai_agent_handler("my-agent").wrap_run(agent, "user input").Dashboard
More reliable organization switching
Switching between organizations now reliably loads the selected organization’s data. Previously, changing orgs (including opening a Studio page for a resource that belongs to another organization you’re a member of) could leave the view stuck on your previous organization and show missing or “not found” data until you refreshed. The switch now finishes applying before the page reloads, so the correct organization’s data loads the first time.Replayable instrumentation for OpenAI Agents
When you set up Bitfab tracing, it now makes sure each traced workflow can be replayed, not just observed. For OpenAI Agents, that means wrapping the agent run in a root span that captures its input, so you can re-run the trace against your current code. The setup health check also flags any instrumented function whose root can’t be replayed and points you to the fix.Full Claude Agent SDK tracing in TypeScript
Tracing for the Claude Agent SDK now captures complete agent runs in TypeScript: every LLM turn, tool call, and subagent becomes a span, with token usage included. Get a handler withgetClaudeAgentHandler, inject the hooks with instrumentOptions, and wrap the query() stream:Replay handler-traced agent runs
Agent runs captured by the Claude Agent SDK handler are now replayable without wrapping your code in an extra span. Pass the prompt asinput to the wrap call and the handler records it as the run’s root, so replay() can re-run each historical prompt against your current code. Works in both the TypeScript and Python SDKs.Dashboard
Trace and dataset labeling panels adapt to narrow widths
The dataset review and trace detail views now reflow to fit tight spaces, so labeling traces in the Studio side panel stays comfortable even when the panel is narrow. The labeling toolbar collapses onto a compact top row, and a trace’s input, output, and context sections stack cleanly instead of overflowing or leaving uneven gaps between them.Compact span tree with hover details
When the trace panel is narrow, the span tree collapses to an icon rail that keeps the full call hierarchy while handing the freed space to the span content. Hover any span to see its name, position, duration, and token count, and the token figure follows your selected token view (all or uncached).Cost-optimize mode for the assistant
The Bitfab assistant has a dedicated cost-optimize mode for cutting token spend. Run/bitfab:assistant cost-optimize <function-key> and it first profiles where a dataset’s tokens go (prompt size, redundant context, output shape, model choice), then edits and replays against your labeled dataset to lower cost while holding the pass rate. The token-cost view turns on automatically, reporting token deltas next to pass/fail. Available across the Claude, Cursor, and Codex plugins.Before and after on every experiment trace
In the experiments view, any replay trace paired with an original now always shows the before and after comparison: the verdict change, the Original/Updated toggle, and the token trend. Values that aren’t known render as blanks instead of hiding the comparison, so still-passing and still-failing traces are as easy to inspect as fixed and regressed ones.More accurate OpenAI Agents trace capture
Traces from the OpenAI Agents SDK integration now capture tool-call and generation inputs correctly. Previously the TypeScript SDK could drop these inputs and record them as empty, and both SDKs added empty input and response fields to agent and other non-LLM spans. Tool calls, agent steps, and other spans now show exactly the data the run produced. Update to TypeScript SDK 0.21.1 or Python SDK 0.21.2 to pick up the fix.Faster labeling and evaluation on large datasets
When the assistant labels a fresh batch of traces or evaluates an experiment, it now judges the traces in parallel on larger datasets, so big batches finish noticeably faster. The speed-up kicks in automatically above roughly 20 traces; smaller batches are unchanged.Read a span field in full when a trace is truncated
When Bitfab shows you a trace, large span fields (a long input or a big tool output) are truncated so the response stays readable. The newread_span_field tool fetches the complete, untruncated value of a single field (input, output, reasoning, content, errors, or context) for one span, so your coding agent can pull the full text only when it actually needs it. read_traces now points you to it whenever a field is truncated. Available across the Claude, Cursor, and Codex plugins.Dashboard
Experiments show when a replay is awaiting labels
In the experiments view, a replay that has finished running but hasn’t been scored yet is now marked “awaiting agent labels” instead of an indefinite loading spinner. The state appears on the run’s pass-rate pill, its trace rows, and the progress bar, and stays out of the pass rate until a verdict lands, so a run that is done replaying reads as settled rather than stuck.Studio windows close more reliably at the end of a flow
Studio windows now close themselves when a flow finishes, with a backup that helps close the window if the browser does not. You will see fewer stray Studio windows left open after setup, assistant, and other flows wrap up.Reliable sign-in from the Bitfab CLI
Setting up Bitfab withnpx bitfab-cli init (or signing in with bitfab login) now completes reliably. A recent CLI version could fail to open the sign-in window and report that you were not authenticated; the CLI now opens Studio and finishes login as expected.Faster trace loading in the assistant
When the Bitfab assistant builds or labels a dataset, it now loads the dataset’s traces in parallel batches instead of one group at a time. Large datasets come into context faster and more reliably, with no change to how you run the assistant. This applies across the Claude, Cursor, and Codex plugins.Studio session daemon for persistent browser management
Bitfab plugins now include a session daemon that manages Studio browser windows as a persistent background process. The daemon keeps Studio alive across coding agent restarts: if a browser window crashes, it re-spawns automatically; if you navigate to a new page, it reuses the existing window instead of opening a second one; and if a page refresh fires a transient close event, the daemon waits briefly before treating it as a real close.Built-in help on every command
Every Bitfab plugin and CLI command now responds to-h / --help with a usage line, a short description, and per-argument details, then exits without running. You (and your coding agent) can discover how to call a command without leaving the terminal, and bitfab init -h now prints help instead of starting the full onboarding flow.Dashboard
Studio sessions reconnect after expiring
Returning to Bitfab Studio after a session has been idle now reconnects automatically instead of showing a “Could not connect to this session” error. Previously an expired session left you stuck until you signed in again; the session is now re-established transparently when you come back.No duplicate Studio window after signing in
Signing in to Bitfab from your coding agent no longer leaves a leftover Studio window or opens a duplicate. The next action reuses the window you signed in with instead of spawning a second one.More reliable tracing for Python agent frameworks
The Python SDK no longer drops a span or trace when a value that can’t be JSON-serialized (such as a Pydantic model) appears outside a span’s input or response, like a tool’s output or a trace’s metadata. Spans and traces from the OpenAI Agents, LangGraph, and Claude Agent SDK integrations now always reach Bitfab, and any value that couldn’t be captured faithfully is flagged as non-replayable instead of silently lost.Compact trace reads by default
Theread_traces tool now returns bounded, truncated span details by default (scope: "summary"), so the Bitfab assistant can scan many traces in one pass without overflowing its context. Each span keeps the head and tail of its largest fields, so big inputs and outputs stay legible at a glance; pass scope: "full" when you need complete, untruncated detail on a handful of traces.Trace streaming functions without a refactor
You can now instrument a streaming function (one that returns a live response stream) by adding a singlefinalize option, with no need to restructure your code. Your function still streams to users exactly as before, while Bitfab records a clean, replayable summary of the turn (text, token usage, tool calls) as the trace output.In the TypeScript SDK, pass finalize to withSpan and use the built-in finalizers.aiSdk helper for the Vercel AI SDK:finalizers.openai_chunks or finalizers.anthropic_events. And when you run setup on a streaming endpoint, Bitfab now guides you straight to this instead of asking you to refactor.Experiments stay linked to their dataset
When you benchmark or run an experiment against a dataset, the run is now durably attached to that dataset, so it always appears on the dataset’s experiments page, even when the underlying traces can’t be matched back by lineage. Before, the link was inferred from shared traces and could be missed.datasetId option when replaying
The TypeScript, Python, and Ruby SDKs’ replay() now takes a datasetId / dataset_id option that attributes the run to a dataset. Passed on its own, it replays exactly that dataset’s traces, so you no longer need to look up and pass the trace IDs yourself.Point Bitfab setup at a specific workflow
When you set up Bitfab tracing, the plugin now asks how you want to find what to instrument: let it scan your codebase for AI workflows, or point it straight at a specific file, function, or directory. Choose the targeted option when you already know what you want traced and want to skip the full scan. Available in the Claude, Cursor, and Codex plugins.Faster Bitfab assistant runs
The Bitfab assistant no longer pauses between steps while it reports progress to the Studio sidebar. Those activity updates are now sent in the background, so the assistant keeps moving instead of waiting on the network. The difference is most noticeable during longer flows like dataset building and experiments.Dashboard
Studio follows your work across organizations
Opening a Studio link for a dataset, trace plan, trace, test run, or experiment that lives in a different organization than your session started in now switches you into the right organization and loads the page, instead of showing a not-found error. If you belong to only one organization, nothing changes. When you are viewing a page in an organization other than the one your coding agent is connected to, a small indicator in the Studio header makes that clear.Sign-in returns you to where you were headed
Following a link to a specific trace, trace plan, or other page while signed out now takes you to that exact page after you sign in or sign up, instead of dropping you on your default traces view. This works for both email and Google sign-in.DashboardTypeScript SDKPython SDKPlugins
TypeScript SDK v0.19.1, Python SDK v0.19.1, Plugins v0.8.10
Uncached token basis in the experiments cost view
The experiments cost lens can now show “uncached” tokens (input minus cached reads, plus output), so you can see the tokens you actually pay full price for instead of the cheap cache reads. With the token lens on, switch between All and Uncached from the “Token count” control in the experiments header, and each trace’s cost trend recolors so the two views read apart at a glance. This surfaces cost regressions that prompt caching would otherwise hide behind a flat total.Claude Agent SDK reports the full prompt size in inputTokens
The Claude Agent SDK handler (TypeScript and Python) now folds cache reads and cache creation intoinputTokens, so it reports the full prompt size, consistent with the LangGraph integration. The cached portion is still reported separately as cacheReadTokens. If you read inputTokens directly for cache-heavy calls, expect a larger value than before.TypeScript SDKPython SDKRuby SDKGo SDKPlugins
TypeScript SDK v0.19.0, Python SDK v0.19.0, Ruby SDK v0.18.0, Go SDK v0.11.0, Plugins v0.8.9
Spans never silently drop on non-serializable inputs
All four SDKs now keep a span even when an input or output can’t be JSON-encoded: the offending value is replaced with a placeholder and a warning flags that the trace may not be replayable, instead of the whole span being dropped. The LangGraph, OpenAI Agents, and Claude Agent SDK handlers, along with the span decorator, also capture nested values more completely, so handler-instrumented traces stay replayable.Dashboard
Cleared the phantom “Grading” status on trace rows
Trace rows no longer show a spinning “Grading…” status when your project has no graders configured. Previously every incoming trace briefly displayed a grading indicator that never resolved. Traces that do have graders are unaffected and still show their pass and fail scores.Cached tokens in the per-span token breakdown
When the experiments token view is on, each span’s token breakdown now shows cached input tokens alongside input and output, so you can see how much of a span’s prompt was read from cache rather than processed fresh.Token usage in experiment trace results
Theget_experiment_traces tool now reports token usage (input, output, cached, and total) for each replay trace and the original it is compared against. Your coding agent can use this to reason about cost and cache-read changes between runs, not just the pass or fail verdict.Dashboard
Before/after comparison for replays with an unlabeled original
In the experiments view, a replay whose original trace was never labeled now shows a neutral “Unlabeled” badge alongside its own Pass or Fail result and opens the same before/after comparison as a scored trace, instead of hiding the comparison entirely. Open the replay to toggle between the original and updated runs and see the change in context, even while the replay is still being evaluated.DashboardTypeScript SDKPython SDKRuby SDK
TypeScript SDK v0.18.2, Python SDK v0.18.2, Ruby SDK v0.17.1
Replay tracks whether the database snapshot was actually used
When you replay traces against a historical database branch, each replayed trace now records whether a branch was provisioned for it and whether your code actually read the branch connection URL. A replay that checksenv.active but never reads env.databaseUrl (env.database_url in Python and Ruby) is recorded as not having used the branch, which catches the common silent failure where a connection pool created at module import sends the replay to your live database instead. Tracking is automatic in the TypeScript (0.18.2), Python (0.18.2), and Ruby (0.17.1) SDKs with no code changes; the Database Branching docs describe the three states a replayed trace can record.Dashboard
See your organization’s usage
A new Usage page (in the user menu) shows how many traces your organization has ingested and how much data they carry over time. Switch between daily, weekly, and monthly views, pick a date range, and read totals with trend indicators at the top; a usage-by-period table breaks it down further and exports to CSV. Dates and time buckets follow your local time zone.Ask Bitfab to cut token usage
The Bitfab assistant now responds to token and cost reduction requests. Tell your coding agent something like “use Bitfab to reduce token usage on one of my datasets” and it enters the experiment flow: it changes prompts or code, replays your labeled dataset, and checks that token usage drops without hurting the pass rate. The experiments view shows original vs. replay token totals next to each verdict, so you can see the savings as results stream in.Dashboard
Skipped replay traces are marked done, not left spinning
When you skip a replay trace during review, the experiments view now shows it with a “Skipped” badge instead of leaving the row spinning as if it were still being evaluated. The run summary reads “9/11 passing · 1 skipped” so a skipped trace is clearly counted as intentionally unscored, rather than dropped from the totals or mistaken for a result that is still loading.Dashboard
Replay environments branch your organization’s own database copy
When replaying with aReplayEnvironment, each replayed item now branches your organization’s own managed database copy (the one provisioned when you connect a database in the dashboard) at the trace’s capture time, so replays read your data as it was, never anyone else’s. Replay branches are cleaned up automatically after each run, with a background sweep catching anything an interrupted run leaves behind.Token cost in experiments and the trace viewer
When an experiment is about reducing token usage, you can now see per-trace and per-span token counts alongside pass/fail, so you can tell whether a change actually cut cost as the results stream in. It shows up in the experiments comparison and the trace viewer, with the token figures styled distinctly from the green/red verdict so cost reads as a measurement, not another pass/fail signal. The Bitfab assistant turns it on automatically when an iteration is framed around cutting tokens or cost.Guided setup for database branching in replay
A new/bitfab:setup db-branching flow walks you through replaying traces against your database as it was at trace time, instead of today’s data (TypeScript, Python, and Ruby). Connect a Postgres database once in the dashboard, and the flow polls until the branchable copy is ready, then wires a ReplayEnvironment into your replay scripts so each replayed item reads from a per-trace branch. A new connection-status check lets the flow tell you exactly when your database is connected and provisioned.Dashboard
Reliable database connection setup
Connecting a source database for trace replay now completes reliably regardless of database size. Setup runs in the background in resumable stages and the dashboard’s connection status tracks it as it progresses, instead of timing out partway through on large databases. Failures (an unreachable database, an invalid connection string) now surface promptly as a failed connection with the underlying reason, rather than leaving the status stuck on checking.Full token-usage capture for LangChain and LangGraph
LLM spans traced through the LangChain/LangGraph callback handler now record token usage from modern LangChain releases, which put counts on each message’susage_metadata rather than the legacy llm_output.token_usage location. Streaming agents are covered too: usage is read from the final aggregated chunk. Provider-native shapes from OpenAI, Anthropic, and Google are recognized as fallbacks, and the legacy location keeps working.Cached prompt tokens are now reported as cachedInputTokens on the span, and Anthropic input counts include cache reads so they reflect the true prompt size (heavily cached agents will see input counts go up; that is the previous under-count being fixed). Only provider-reported numbers are recorded, nothing is estimated. For OpenAI streaming, enable stream_usage=True (Python) or stream_options: {"include_usage": true} (TypeScript) so usage arrives on the stream.Replay for handler-instrumented workflows
Workflows traced through a framework handler (LangGraph, LangChain, Claude Agent SDK, OpenAI Agents) can now be replayed, even though they have no decorated root function in your code. Pass the handler’s trace function key plus any plain callable toreplay(), and the SDK wraps it internally so every replayed run records a trace tied to the experiment:bitfab.replay("my-agent", fn, options), where plain callables are wrapped automatically. The setup and assistant plugin skills now offer handler instrumentation as a first-class option for workflows whose entry points take live objects (database handles, billing callbacks), and write the matching replay script for you. See “Replaying handler-instrumented functions” in the Python and TypeScript SDK docs.Dashboard
Fixed database connector provisioning in production
Fixed a bug where connecting a database for trace snapshots failed in production before provisioning could start. Database connector setup from the dashboard now completes as expected.More reliable Studio sessions
Studio now keeps a single, durable connection open for the whole assistant session. Moving between pages (dataset review, experiments, trace plans) reuses the open Studio window instead of opening extra background connections, so signals like marking a dataset done or ending a session are no longer missed. The live Studio updates in the assistant flow are dependable from start to finish, even when a session is reused from an earlier run.Replay a trace against its historical database state
The Python and Ruby SDKs can now replay a recorded trace against the database as it was when the trace ran. Pass aReplayEnvironment to replay() and read its database URL inside your function; Bitfab resolves a per-trace database branch for each item and releases it when the item finishes. This brings Python and Ruby to parity with the TypeScript SDK.Trace LangChain by its own name
The Bitfab callback handler has always traced plain LangChain chains as well as LangGraph graphs (both share the same callback system), but every entry point was named after LangGraph. The SDKs now expose LangChain-named aliases that return the identical handler:getLangChainCallbackHandler() in TypeScript and get_langchain_callback_handler() in Python, with the class also exported as BitfabLangChainCallbackHandler.Keep your app’s API key in sync when you switch organizations
When you switch organizations with the Bitfab plugin, it now offers to update your project’s localBITFAB_API_KEY as well. Previously the switch only repointed the plugin, so traces your own code sent kept landing in the old organization until you updated the key by hand. After you run /bitfab:setup switch-org (or /bitfab-setup switch-org in Cursor, $bitfab:setup switch-org in Codex), the agent finds every .env file that defines the key and, with your go-ahead, updates them in place.Dashboard
Connection fields on the Integrations page no longer clear while you type
Entering a database connection string or integration secret on the Integrations page now stays put as you type. Browser password managers were treating these masked fields as login passwords and overwriting them mid-entry; they’re now opted out, so your input holds.Switch organizations from your coding agent
You can now switch which Bitfab organization a plugin reads and writes without leaving your editor. Run/bitfab:setup switch-org (or /bitfab-setup switch-org in Cursor, $bitfab:setup switch-org in Codex): the agent lists the organizations you belong to, switches to the one you pick, and swaps the plugin’s API key to the new org. The agent can also call the new list_organizations tool on its own to check which org it is currently pointed at. Your already-open browser tabs keep showing their current org until the next time the plugin opens Studio.Dashboard
Faster traces list
The traces list now loads significantly faster for functions with a large number of traces. Opening a function’s traces no longer waits on a full count of every matching trace before showing results, so the page appears as soon as the traces are ready.Sign in without a duplicate Studio window
When you run a Bitfab plugin command while signed out and a Studio window is already open, the plugin now reuses that window to sign you in instead of opening a second one. There’s only ever one Studio window now, where before a fresh window could appear and leave the original orphaned. Running login while you’re already signed in is also a no-op, and you can re-authenticate on demand by running login with--force.See a dataset’s experiment history in Studio
You can now open the experiments page scoped to a single dataset. Ask the Bitfab assistant to “show experiments for a dataset” and Studio lists every experiment that replayed one of the dataset’s traces, giving you that dataset’s full run history in one place. Previously the experiments page opened only for specific test runs or an experiment group.More reliable Studio session cleanup
Studio sessions now close themselves cleanly when an assistant or setup run finishes: the browser tab closes, background processes stop, and nothing is left running even if the window refuses to close. Recovering from an unreachable Studio window (“Open a new Studio”) also cleans up the old window and its background process instead of leaving them behind.Dashboard
Accurate error indicators in the trace viewer
Spans with no recorded errors no longer show a false “Error detected” badge or an “Execution Error” section in the trace viewer. Error detection now correctly ignores empty error data, and a related fix ensures a real execution error can no longer be hidden by an empty error list.Live updates arrive reliably on the hosted dashboard
Pages that update in real time, such as dataset review, trace lists, and experiment results, could miss updates on the hosted dashboard and only show new data after a manual refresh. Event delivery now completes reliably, so new traces, labels, template changes, and experiment results appear the moment they happen.Replay a single trace to check your fix
The assistant has a newreplay mode for the quickest version of the improvement loop: you already made a fix and just want to know whether one specific trace passes now. Run /bitfab:assistant replay <function-key> <trace-id> (or simply ask “did my fix work on <trace-id>?”) and the agent finds your replay script, re-runs that one trace through your current code, and reports a pass/fail verdict in chat. It skips everything heavier: no browser, no dataset, no labeling, and nothing is persisted, so it’s safe to run as often as you like while iterating.TypeScript SDKPython SDKRuby SDKPlugins
TypeScript SDK v0.16.1, Python SDK v0.16.1, Ruby SDK v0.16.1, Plugins v0.7.9
Replay accepts limit and trace IDs together
Passing bothlimit and traceIds (trace_ids in Python and Ruby) to replay() no longer throws. The SDK now logs a warning and ignores limit, since an explicit trace ID list already determines how many traces replay. This applies to the TypeScript, Python, and Ruby SDKs, so replay scripts that forward both flags keep working instead of crashing.TypeScript SDKPython SDKRuby SDKPlugins
TypeScript SDK v0.16.0, Python SDK v0.16.0, Ruby SDK v0.16.0, Plugins v0.7.7
Replay traces whose function signature changed
When you rename, reorder, or restructure a traced function’s arguments, replay can no longer feed it the inputs recorded against the old shape.replay() in the TypeScript, Python, and Ruby SDKs now takes an adaptInputs (adapt_inputs in Python and Ruby) hook that reshapes each trace’s recorded inputs onto the current signature, so older traces keep running.The assistant recovers replays broken by signature changes
When you iterate on a traced function in your coding agent and a replay fails because the signature drifted since the traces were captured, the Bitfab assistant now recognizes the mismatch instead of treating it as an environment error, and helps you write a small committed input adapter so those traces rejoin the run.Spot traces that can’t be replayed
The trace viewer now flags traces that won’t replay against your current code, either because they captured no top-level span or because their recorded inputs no longer fit the function’s current signature. A “Can’t replay” badge appears on dataset rows and in the trace detail view, so you can see at a glance which traces a replay will actually cover before you run it.When your coding agent opens a dataset, the Bitfab plugins pass along your function’s current input shape, and the check runs live against it. Nothing is stored, so the badge can’t go stale.TypeScript SDKPython SDKRuby SDKDashboardPlugins
TypeScript SDK v0.15.0, Python SDK v0.15.0, Ruby SDK v0.15.0, Plugins v0.7.5
Replay results now persist reliably
replay() in the TypeScript, Python, and Ruby SDKs now waits for each replayed item’s trace to be fully persisted before completing the test run. item.traceId (trace_id in Python and Ruby) is a real server trace ID you can use immediately. Previously, a race could leave every trace ID null and the experiments page empty, even though the replay appeared to succeed.Failures are no longer silent. If none of the replayed items’ traces reached the server (for example, the replayed function isn’t instrumented), replay() raises an error explaining why. If only some items fail to persist, those items return a null trace ID with a logged error and the rest of the run comes back intact, so one bad trace costs you one data point instead of the whole run.Clearer skill routing in plugin flows
The setup, assistant, and update skills now state exactly where each choice leads (“Update all → step 7”, “Skip → stop”), so coding agents follow multi-step flows more reliably instead of inferring the wiring from prose. The update skill on Claude Code also runs as chained sub-skills: each phase hands off directly to the next with the invocation mode attached, removing a class of lost-context routing mistakes in long sessions.Live dataset review in Studio
Dataset review now always happens on the dataset’s own page in Studio, which updates in real time as your coding agent adds traces and applies labels. Previously the agent could leave you on a function-level review page that only showed new activity after a manual refresh.That older function-level page now redirects to the function’s most recent dataset, so existing links and older plugin versions keep working.TypeScript SDKPython SDKRuby SDKDashboardPlugins
TypeScript SDK v0.14.0, Python SDK v0.14.0, Ruby SDK v0.14.0, Plugins v0.7.2
Replay by trace IDs no longer truncates
Replaying specific traces by ID previously capped the list at the defaultlimit, silently dropping the rest of your selection: 12 IDs in could mean only 5 replayed, skewing experiment results without warning. An explicit ID list now always replays every trace in it (up to 100).limit and traceIds are now mutually exclusive: limit means “replay my last N traces”, and an ID list speaks for itself. Passing both raises a clear error instead of guessing.trace_ids semantics in Python and Ruby. Older SDK versions get the core fix server-side: explicit ID lists are no longer truncated by a default limit.No more duplicate Studio windows
Your coding agent now keeps exact track of its Studio window. Ending a session no longer makes the next Studio command open a second window while the old one lingers: the existing window is reused, and the agent only forgets a window once the browser confirms it actually closed. Refreshing the Studio page mid-session is also safe; the connection re-establishes itself instead of being mistaken for a close.Commands that reconnect to an already-open Studio window now react only to what happens after they connect, so a previously ended session or an earlier run’s activity can no longer end a new command prematurely. If a Studio window disappears without a trace (for example the browser quit entirely), the agent detects that it is unreachable and offers to reopen instead of guessing.Watch benchmark runs live in Studio
The assistant skill’sbenchmark mode is terminal-only by default, but you can now add the studio keyword (for example, benchmark <function> studio, or just ask in natural language to “open studio”) to open Studio’s experiments page and watch each trace’s pass/fail verdict stream in as the replay runs. The default stays terminal-only, so existing benchmark runs are unchanged unless you opt in.Dashboard
Traces are flagged errored only when your code fails
A trace is now marked as errored only when your traced code throws, not when the Bitfab SDK hits a serialization or ingestion error while recording the trace. The error indicator in the trace list now reflects failures in your own functions, so SDK-side noise no longer surfaces as a failed trace.Dashboard
Errored traces open on the failing span
When you open a trace that recorded an error, the trace viewer now jumps straight to the first span that failed instead of starting on the trace root. You land on the error and its message right away, without scanning the span tree for the red marker. Traces without errors open exactly as before, and any span you deep-link to still takes precedence.Dashboard
Errored spans highlighted in the trace viewer
Spans that recorded an error are now flagged directly in the trace viewer. The span tree marks failed spans in red, and the span header shows an Error tag with the error message on hover, so you can spot failures in a trace without opening each span.Replay keeps going when an individual trace fails
The Python and Ruby SDKs now isolate per-trace errors duringreplay(): if one historical trace fails to load or its function raises, that result is marked with an error and the rest of the run still completes, instead of the whole replay aborting. This matches the TypeScript SDK’s behavior, so a single bad trace no longer costs you the entire run.Dashboard
Copy buttons in custom span templates
Custom span templates can now drop in a clipboard icon button next to any field value with{{ value | copyButton | safe }}. Pass a string for the tooltip and accessible label, e.g. {{ span.id | copyButton("Copy span id") | safe }}. The button copies the value (objects and arrays are pretty-printed as JSON), flashes “Copied”, and works inside the template’s isolated shadow DOM without any extra JavaScript or CSS in the template.More reliable Studio sign-in
When your coding agent opens Bitfab Studio and you sign in, the Studio tab now connects reliably on the first try. Previously a timing issue could leave a freshly opened session showing “Could not connect to this session” until you re-ran the command; signing in now hands off cleanly to the live session every time.TypeScript SDKPython SDKRuby SDKGo SDKPluginsDashboard
TypeScript SDK v0.13.8, Python SDK v0.13.4, Ruby SDK v0.12.4, Go SDK v0.10.2, Plugins v0.6.74
Error source classification on spans
Span errors now carry an explicit source tag so you can tell whether an error came from your code or from the SDK itself. When your traced function throws, the error is recorded withsource: "code". SDK-internal failures (like serialization errors) are tagged source: "sdk". Both types appear in the unified errors field on the span, replacing the previous split between span_data.error and the errors column.All four SDKs support this: set error_source: "code" automatically when a traced function fails.Dashboard
Connect a database for per-trace snapshots
You can now connect your Postgres database from the new Database page so Bitfab can take an isolated snapshot per trace at replay time. Paste a connection string and activate: replays run against a fresh branch of your database, so they never touch or slow down production. The page shows live status (activating, active, or failed, with a support contact if it can’t reach your database), and you can deactivate at any time from a confirmation dialog.More reliable trace reads under heavy use
Reading several traces at once through the Bitfab plugin (theread_traces tool) no longer returns intermittent errors when many requests are in flight at the same time. Trace reads and agent-label updates are now more efficient, so larger reads stay fast and reliable.Quickly add a trace to a dataset
The/bitfab:assistant command has a new lightweight add-trace mode that attaches one or more existing traces to a dataset and stops, without the full label-and-iterate flow. Run /bitfab:assistant add-trace <trace-id> (the function key is inferred from the trace) or just ask your coding agent to “add this trace to a dataset”. It picks or creates the right dataset for you, and if you point it at several traces it makes sure they all belong to the same function before attaching.Diagnose your tracing setup with /bitfab:setup inspect
A new inspect mode checks whether your Bitfab tracing is healthy: whether you’re authenticated, what’s instrumented in this repo, whether the plugin and SDK are up to date, whether your replay scripts cover every trace function, and whether traces are actually arriving. It then walks the available fixes one at a time, asking before each change. Run /bitfab:setup inspect, or just ask your coding agent something like “why aren’t my traces showing up?”.Get oriented with /bitfab:setup explain
A new read-only explain mode prints a quick overview of what Bitfab is and what each setup mode does, without authenticating or scanning your code. Run /bitfab:setup explain or ask “what is Bitfab?”.Smarter Studio window reuse
The Bitfab plugin now reuses your existing Studio window whenever it’s still open, instead of risking a duplicate window or a stale “not responding” prompt. Close the Studio tab and the next action opens a fresh window right away; only a window that’s genuinely unreachable, like after a crash or your machine sleeping, will ask whether to retry or open a new one.Clearer Studio connection status
Studio’s connection indicator now tells you exactly what’s happening: “Studio connected” when your agent is live, “Awaiting agent” while it’s away, and “Studio disconnected” if the browser loses its live connection to the session (it reconnects automatically). Reconnecting to a session you already have open now reuses that Studio tab instead of opening a second window.Benchmark a dataset against your current code
Run/bitfab:assistant benchmark <key> to replay a labeled dataset against your current code without changing anything, then read a pass/fail scorecard that shows which traces still pass, still fail, regressed, or were fixed. Use it to measure where your function stands right now, as a regression baseline or a quick check after unrelated work, instead of starting an experiment loop. You can also just say “benchmark my dataset” in plain language and the assistant routes there.More reliable Studio sessions
Studio no longer shows a stray “agent disconnected” popup on the session-complete page. When your agent disconnects mid-session, the reconnect prompt now rejoins your existing Studio session instead of starting a new one. And if you run more than one coding agent in the same project, each keeps its own Studio session instead of overwriting the other’s.Sign-in URL shown when the browser doesn’t open
When you sign in to Bitfab from your coding agent and the browser doesn’t open automatically, the login flow now prints the sign-in URL so you can open it manually. Previously it told you to visit the URL without showing one.Simplified plugin login
The setup login flow now uses a single authentication method that works everywhere, including SSH sessions, containers, and cloud IDEs. The separatelogin headless mode has been removed since the standard login already handles these environments automatically via its server-polled channel.Seamless login in the assistant flow
Running/bitfab:assistant without being authenticated now logs you in inline through Studio instead of stopping the flow. Previously, unauthenticated users were told to run a separate login command first, breaking the workflow. The assistant now opens Studio’s sign-in page directly and continues automatically once you’ve signed in.Plugin login no longer hangs for CLI users
Thebitfab init login flow now completes reliably instead of hanging after sign-in. Previously, the published CLI used a query parameter the close page didn’t recognize, so the authentication callback never fired and the CLI waited indefinitely. The close page also now displays “Login complete” instead of the generic “Session Complete” message.Experiment results show label annotations
The experiments page now displays the label annotation for each trace instead of the raw function output. This matches how the dataset page already renders traces and makes it easier to scan experiment results for what passed, what failed, and why. Annotations from both human reviewers and the agent’s automated labeling are shown.Replay script upgrades no longer interrupt the assistant flow
When the assistant detects that your replay script needs an upgrade (missing code-change or experiment-group support), it now edits the script directly instead of launching a separate setup flow. Previously, this would break the assistant’s continuity and drop you to an empty prompt. The upgrade happens inline and the experiment flow continues automatically.Dashboard
Faster disconnect detection in Studio
When an agent closes its Studio session, the browser now shows “disconnected” within milliseconds instead of up to 60 seconds. Previously, the connection status indicator could display “Agent connected” long after the agent process had exited because the server-side heartbeat lingered in cache.Studio redirects to sign-in instead of showing a session error
Opening a Studio page without being signed in now redirects you to the sign-in page instead of showing a “Could not connect to this session” error. After signing in, you’re returned to the page you originally requested with the session intact.Studio commands now open login instead of erroring
Plugin commands that open Studio pages (experiments, trace plans, datasets, template previews) no longer fail with “Not authenticated” when you haven’t logged in yet. Instead, they open Studio directly and redirect you to the sign-in page. After you sign in, you land on the page the command originally requested. For interactive commands like trace plan confirmation and dataset review, the plugin saves your credentials automatically so the bidirectional event channel works normally after login.Automatic replay script capability detection
The assistant now checks whether your replay script supports the latest experiment features before running experiments. If your script is missing support for code diffs, experiment groups, or trace ID tracking, the assistant offers to upgrade your SDK and regenerate the script in place. You can also choose to continue without the missing features. This replaces the previous behavior where outdated scripts would silently skip features or produce incomplete experiment results.Dataset mode now continues through failure diagnosis and experiments
Fixed a bug where the assistant’s dataset mode could stop after building the dataset instead of continuing to diagnose failures and run experiments. The flow’s internal instructions contradicted its routing in three places, which could cause the agent to exit early. Dataset mode now reliably progresses through the full pipeline: build dataset, diagnose failures, iterate with experiments, and wrap up.Dashboard
Clear error messages when Studio can’t connect to a session
When Studio fails to connect to an agent session or switch to the correct organization, it now shows a clear error message instead of silently loading in the wrong context. This prevents the confusing state where traces appear missing because Studio was looking in a different organization than the one your plugin authenticated against.DashboardPluginsTypeScript SDKPython SDKRuby SDK
Plugins v0.6.50, TypeScript SDK v0.13.6, Python SDK v0.13.3, Ruby SDK v0.12.3
Live experiment streaming
The experiments page now streams results in real time as replays complete. When the assistant runs experiments, it opens the experiments viewer before the first replay starts and new results appear automatically via server-sent events as each test run finishes. Previously, the experiments page only opened after all replays completed.All three SDKs now accept anexperimentGroupId parameter on replay() that groups multiple test runs into a single experiment batch:?experimentGroupId= as a query parameter, and falls back to ?testRunIds= for replay scripts that haven’t been updated yet.Inline template editing during labeling
You can now edit trace view templates in chat while labeling traces, without leaving the dataset review page. When you ask the assistant to change how a span type renders (e.g. “edit the LLM template”), it reads and updates the template inline using MCP tools, and the dataset page re-renders automatically. Previously, template editing required invoking the setup flow, which navigated Studio away from the dataset and broke the labeling session.Dashboard
Auto-update trace plan in Studio
When you modify your span capture setup and create a new trace plan, Studio now automatically navigates to the updated plan. Previously, you had to manually refresh or re-navigate to see the latest version after making changes to the capture configuration.Investigate mode continues through the full pipeline
When you run the assistant in investigate mode, the flow now continues through diagnosis and experiments after building a dataset, matching how dataset mode already works. Previously, investigate mode stopped after dataset building, requiring you to restart in experiment mode to iterate on fixes. All assistant modes now serve as entry points into the same pipeline, converging at wrap-up regardless of where they start.Graceful Studio close on flow exit
When the assistant finishes a flow (wrap-up, early stop, or sub-mode completion like dataset-only), the Studio tab now closes gracefully instead of lingering in a disconnected state. The agent navigates to a close route that lets the Studio clean up before the background process is terminated.Better error messages when agents pass wrong CLI arguments
Plugin commands now validate arguments against a declarative schema before executing, catching common agent mistakes like invented--flag syntax, missing arguments, or invalid UUIDs. When a command receives bad input, it prints a usage string showing the expected arguments and a clear error message, so the agent can self-correct on the next attempt.Dashboard
Reconnect guidance when Studio loses agent connection
When your coding agent disconnects from Studio, a popup now appears after 30 seconds with a copiable prompt you can paste into your agent to reconnect. The popup includes your session ID so the agent can rejoin the same session. If you dismiss it, the popup reappears with increasing intervals, and it auto-dismisses if the agent reconnects on its own.Dataset mode continues to diagnosis and experiments
When you run/bitfab:assistant dataset, the flow now continues past labeling into Phase 4 (diagnosis and experiments), matching the behavior of the full /bitfab:assistant flow. Previously, dataset mode stopped after labeling, requiring you to restart in the default mode to run experiments on the same dataset.Live experiment verdicts in Studio
When your replay script returns trace IDs (requires SDK v0.13.5+), the assistant now opens the experiments page in Studio before running evaluations, so you can watch pass/fail verdicts populate in real time. If your SDK predates trace ID support, the assistant prompts you to update and falls back to showing evaluation results as text in the agent.Code-change diffs in the experiment viewer
When the assistant runs experiments, it now captures before-and-after file snapshots for every edit and attaches them to the replay. The experiment viewer can then display the literal code change alongside pass/fail results, so you can see exactly what was tried in each iteration. Existing replay scripts that predate this feature continue to work; the assistant detects whether the script supports the new--code-change flag and gracefully skips the metadata if it doesn’t.TypeScript SDKPython SDKRuby SDKDashboard
TypeScript SDK v0.13.5, Python SDK v0.13.2, Ruby SDK v0.12.2, Plugins v0.6.42
Replay results include trace IDs
Each replay result item now includes atraceId (or trace_id in Python/Ruby) that links directly to the server-side trace created during replay. Previously, matching a replay result back to its trace in the dashboard required heuristics based on input similarity. Now you can navigate straight to the trace.Experiments distinguish errored traces from pending
When a replay trace fails during execution (before it can be graded), the experiments page now shows it as “errored” instead of lumping it in with pending traces. Errored traces appear with a red “Error” badge, a red row tint, and a dedicated segment in the progress bar. The pass-rate pill also now shows context-appropriate states: “N errored” when all traces errored, “N pending” with a spinner when grading is in progress, and “X/Y so far” for partially graded runs.Assistant reviews existing dataset traces before searching for new ones
When you pick a dataset that already has traces, the assistant now goes straight to the review page instead of asking how to source new candidates. Previously, datasets with unlabeled (but present) traces were treated like empty datasets, which skipped past the traces you already had. Empty datasets still get the “what kind of traces should I find?” prompt as before.Dashboard
Experiments page updates after replay completion
The experiments page now refreshes automatically when a replay finishes and as labels are applied. Previously, completing a replay didn’t trigger an update, so the page could appear empty until manually refreshed. Summary counts and individual trace labels now appear as soon as they’re available.Studio session recovery after agent interruptions
The plugin assistant now automatically reconnects to an existing Studio browser session when the background polling process is interrupted (for example, by a long conversation triggering context compaction). Previously, losing the background process meant the assistant would open a duplicate Studio window. Now it resumes the existing session seamlessly, keeping the same browser tab and session state.Experiment and dataset tools for the assistant flow
The plugin assistant can now work with experiments and datasets directly. Two new MCP tools,list_experiments and get_experiment_traces, let the assistant list recent experiments for a function and drill into individual trace verdicts (fixed, regressed, still-passing, still-failing). The search_traces tool also now accepts testRunId and datasetId parameters, so you can scope trace searches to a specific experiment run or dataset without manual filtering.Fixed investigate mode opening a broken Studio page
Running/bitfab:assistant investigate <key> no longer opens Studio to a non-existent page. The investigate mode now lands on the Studio root, which loads correctly. The investigation itself (trace reading, code exploration, findings summary) was unaffected since it runs via tool calls, not Studio navigation.Trace plan links now open correctly in Studio
Clicking a trace plan link from the plugin assistant now opens the trace plan inside the Studio session. Previously, the link pointed to a standalone route outside of Studio, which bypassed the authenticated Studio context.Dashboard
Experiment results stream in real time
Running experiments on a dataset no longer blocks until every test finishes. The experiments page opens immediately and results stream in trace by trace: the progress bar fills, pass/fail counts update, and trace rows appear as each test completes. If background execution fails, the run is marked as failed and the page updates accordingly instead of getting stuck on “pending.”Dashboard
Trace list loads reliably for high-volume functions
The traces page now loads correctly on the first visit for functions with very high trace creation rates. Previously, real-time update events could interfere with the initial page load, causing the trace list to appear empty until you navigated away and back.Reliable Studio navigation during assistant sessions
Studio navigations that include query parameters (such as opening the experiments page with specific test run IDs) no longer time out with “not responding.” The same fix also ensures that navigating to the same page with different parameters is recognized correctly, so the assistant flow proceeds without interruption.Dashboard
Dataset traces now stream in real-time in Studio
Traces added to a dataset while viewing it in Studio now appear immediately without a page refresh. A recent migration to Studio’s route tree accidentally dropped the real-time event connection, so newly added traces were invisible until you reloaded.Fixed experiment mode navigation
The plugin assistant’s experiment mode now correctly opens the experiments page. Previously, it attempted to navigate to a non-existent per-function experiments route.Dashboard
Studio reconnects automatically after sleep
Studio connections now recover automatically when your laptop wakes from sleep. Previously, closing your lid and reopening could leave the session unresponsive until you manually refreshed the page.Dashboard
Dataset page shows errors instead of misleading empty state
The dataset review page now displays a clear error message when traces fail to load, instead of incorrectly showing “No traces yet.” This helps you quickly identify loading failures, such as viewing a dataset while signed into the wrong organization.Per-trace DB branching for replay (alpha)
Replay can now run against the database state at the moment a trace was recorded, not your current production database. This raises fidelity for agents whose behavior depends on stored state, like a refund decision that read a since-cancelled order or a retrieval agent that saw last week’s index. Available in the TypeScript SDK; backed by Neon preview branches on the Bitfab service.Wire it up:Dashboard
Studio sessions survive page refreshes
Refreshing the Studio tab no longer kills your coding agent’s session. Previously, a browser refresh was indistinguishable from closing the tab, so the plugin would immediately tear down the connection. Now the plugin waits up to 10 seconds for the page to reload before ending the session, so you can refresh freely without interrupting your workflow.Plugin login respects project-local credential isolation
When your project has a.bitfab/credentials.local.json file (used to isolate credentials per project), logging in now writes the new API key to that file instead of the global credentials store. Previously, login always wrote to ~/.config/bitfab/credentials.json, which meant the project-local file stayed empty and the plugin fell back to the global key, defeating the isolation.Studio auto-switches org to match the plugin session
When your browser’s active org differs from the org bound to the plugin’s API key, Studio now detects the mismatch and automatically switches to the correct org on load. This fixes the “Awaiting agent” stuck state that could occur when you belong to multiple organizations and your browser happened to be on a different one than your plugin.If you’re not a member of the session’s org, the plugin now cleanly aborts and tells you why, instead of hanging indefinitely.Dashboard
Studio agent connection recovers after laptop sleep
Studio now correctly restores the agent connection indicator after your laptop sleeps and wakes. Previously, the “agent disconnected” banner could get stuck even though the agent had successfully reconnected. The fix ensures heartbeat tracking refreshes on every poll and that the browser verifies the actual connection state when resuming after a gap.TypeScript SDK is now @bitfab/sdk
The TypeScript SDK package has moved from bitfab to @bitfab/sdk. New installs should use the scoped name:bitfab package continues to work but now prints a deprecation warning on import. Running the plugin’s update command (/bitfab:update) detects the legacy package and warns you to switch, even if the version number is current.Studio is now the single browser surface for all plugin flows
Every plugin CLI flow (login, trace plan confirmation, dataset review, template preview) now opens inside Studio instead of launching a separate browser window. If Studio is already open, the plugin navigates it in place rather than opening a new window. This means fewer browser tabs, a consistent UI, and the ability to stay in one window while working with the assistant.Headless login is now available for environments where a browser can’t reach your terminal (SSH, cloud IDEs, CI). Visit/studio/auth/claude in any browser, sign in, copy the token, and paste it back into your coding agent.Closing Studio during a trace plan confirmation now cleanly cancels the operation instead of leaving the CLI in an error state.Automatic package rename in update flow
Running/bitfab:update now detects the legacy bitfab npm package and offers to switch it to @bitfab/sdk. The update flow removes the old package, installs the new one, and rewrites imports in your source files. If you’re already on @bitfab/sdk, nothing changes; the flow works as before.Studio URL guard and auth verification
Studio now always opens at the correct/studio path. Previously, certain launch conditions could cause the Studio window to open at the site root instead of the Studio interface. A path guard now normalizes the URL before the browser window opens.The plugin’s auth status check now verifies your API key against the current server. If you switch between servers (e.g., local development to production), the status command correctly reports that re-authentication is needed instead of showing a stale “authenticated” state.Live-streaming dataset pages in Studio
When the Studio assistant creates or picks a dataset, the dataset review page now opens immediately instead of waiting for all traces to be labeled and attached first. Traces appear on the page in real time as the agent finds, labels, and attaches them. Label changes on traces already in a dataset also update live, so you can watch rows move between the “Agent labeled,” “Labeled,” and “Unlabeled” sections without refreshing.If the page is empty while the agent is still working, a “Building your dataset” indicator shows that traces are on the way.Dashboard
Trace plan review page scrolls
The trace plan review page now scrolls when a plan has more captured nodes than fit on screen. Before, long plans clipped the Advanced selection toggle and any inline error messages below the fold; opening a plan with around 40 captured nodes now scrolls cleanly through the full call tree.TypeScript SDK available as @bitfab/sdk
The TypeScript SDK is now published under the scoped package name@bitfab/sdk in addition to the existing bitfab package. Both names resolve to the same code and will stay in sync on every release. If you prefer scoped package names for clarity in your package.json, you can switch your import at any time:import { Bitfab } from "bitfab" and import { Bitfab } from "@bitfab/sdk" work identically.TypeScript SDKPython SDKRuby SDKPlugins
TypeScript SDK v0.13.1, Python SDK v0.13.1, Ruby SDK v0.12.1, Plugins v0.6.14
SDK serialization hardening
Trace spans now ship reliably even when function inputs or outputs are difficult to serialize. Objects with circular references, oversized payloads (over 512 KB), or classes that throw during serialization no longer cause lost spans. Instead, the SDK replaces the problematic value with a descriptive<unserializable: ClassName (reason)> stub so the span still appears in your traces with full timing and metadata.This fix applies to the TypeScript, Python, and Ruby SDKs. No code changes are needed on your side; update to the latest SDK version to get the improvement automatically.Plugin can query Studio browser state
Plugins can now check whether the Studio browser tab is connected and which page is currently active via the newgetStudioState function. This lets the plugin make smarter decisions before navigating, for example skipping a navigation command when Studio is already on the target page, or surfacing a connection warning when the browser tab has been closed.Studio crash recovery and agent navigation guardrails
Studio now shows a recoverable error screen when a runtime error or unexpected crash occurs during a session. Instead of a blank page or a full 404, you see a “Try again” button that retries without losing your session context.Agent-initiated navigation is now validated against a known route whitelist. When an agent tries to navigate to an invalid or out-of-scope path, it receives an immediatenavigation-blocked event with a reason string instead of waiting for a 12-second timeout. This helps agents self-correct faster when a requested page doesn’t exist.Studio stays connected through long conversations
The Bitfab plugin now persists the link between your coding agent conversation and your Studio session. Previously, when a long conversation triggered context compaction, the agent lost track of which Studio window it had opened, requiring you to reopen Studio manually. Now the mapping is written to disk and recovered automatically after compaction, so Studio commands continue working seamlessly in extended sessions.Dashboard
Code change diffs in experiments
When an experiment replays traces against a code change, you can now view the exact diff that was tested. Click the file stats on any experiment card or the code-change pill in the trace detail header to open a side-by-side diff modal. The modal also shows how the dataset reacted overall (fixed, regressed, still passing, still failing) or, when opened from a single trace, whether that specific trace flipped.Dashboard
Fixed plugin login falling through to manual paste flow
Signing in to a Bitfab plugin (Claude Code, Cursor, or Codex) via the browser now reliably completes the automatic handoff back to your terminal. Previously, the login page could lose the callback parameters during a redirect, causing every login to fall through to the manual “copy and paste this token” flow even when the browser and terminal were on the same machine.Investigate a trace function with /bitfab:assistant
Run/bitfab:assistant investigate [<key>] to characterize an issue in a trace function without going through the full assistant flow. The agent reads recent traces and your code based on what you describe, then offers three follow-ups: stop with an in-chat summary, save a written report under .bitfab/analysis/, or hand off to dataset building when the findings include reproducible failures worth labeling. The function key is optional; when omitted, the agent picks it from your description or asks.Agent-initiated Studio session close
Agents can now programmatically end a Studio session when their work is complete. ThecloseStudio() helper sends a completion event with an optional message, and the browser automatically closes or shows a “Session Complete” screen with the agent’s message. This replaces the need for users to manually click “End session” when the agent is done.Trace plan confirmation lands inside Studio
The Bitfab plugin’s trace-plan confirmation page (where you review which spans your function will capture) now renders inside your existing Studio tab during/bitfab:setup instead of spawning a second browser window. Studio’s header and agent indicator stay visible while you decide; Confirm or Cancel keeps the tab open for the rest of the flow, no more orphan windows.If no Studio is running (you invoked /bitfab:setup outside an /bitfab:assistant session), the confirmation falls back to the standalone chromeless window as before.Annotate a closed trace from any process
The TypeScript and Python SDKs now expose a detachedclient.getTrace(id) handle that lets you add context, merge metadata, or set the session id on a trace after its root span has closed. The handle works from any process, thread, or agent that knows the trace id, with no shared in-memory state. Useful when a downstream worker or a forked AI agent needs to attach information to the original conversation’s trace.bitfab v0.13.0 for TypeScript and Python.Smarter Studio session management
The assistant flow now reuses an existing Studio session instead of opening a new browser window each time. If the Studio becomes unresponsive (tab closed, page crashed), the agent detects this within 12 seconds and offers options to refresh the tab or open a fresh session.Live activity progress in Studio
Studio now shows which phase the assistant is working on in real time. As the skill progresses through steps like identifying the trace function, building a dataset, or running experiments, the header displays the active phase name with a live elapsed timer. When one phase completes and the next begins, you see the previous phase’s duration before it transitions.If the agent disconnects or crashes mid-phase, the activity indicator automatically resets within 30 seconds instead of showing stale state indefinitely.Open a trace plan from inside /bitfab:assistant
Ask the assistant to “open the trace plan for X” (or “show me what’s captured”) and it now routes your open Studio tab to that function’s most recent trace plan in place. The Studio shell stays mounted around the plan, so your agent session, header, and connection indicator persist across the navigation, and no new browser tab pops up. The canonical /trace-plan/[id] URL still works as a standalone shareable link outside Studio.Dashboard
Click any trace while reviewing a dataset
Fixed a bug that blocked clicks while a trace detail was open. You can now switch traces or press Done without closing the open one first.Redesigned trace planner
The trace planner now leads with what you actually need to know: a validation summary at the top that calls out anything blocking replay (live writes inside captured spans, missing samples, disconnected roots), then a flow diagram of the captured spans and a sample-trace preview of how the recorded trace will look in the viewer. The legacy two-pane tree picker is still there, tucked behind an Advanced selection toggle for power-users. Confirm and Cancel still flow through the same Cmd+Enter / Esc handoff, so muscle memory carries over.Live agent connection indicator in Studio
Studio now shows a real-time connection status in the header. A green dot with “Agent connected” appears when the coding agent is actively polling, and transitions to a gray dot with “Awaiting agent” if the agent disconnects. The indicator updates instantly when the agent reconnects, with no page refresh needed.Mutual presence detection for plugins
The agent plugin now receivesbrowserConnected in its poll response, indicating whether a user has Studio open in the browser. This enables plugins to adapt their behavior based on whether someone is actively watching the session.Studio is now the default assistant mode
The/assistant skill now opens Studio automatically on every invocation. You no longer need to pass a studio argument to get the companion browser surface. Studio is always there, from start to finish.Studio opens directly at the relevant page
When you start in dataset or experiment mode (/assistant dataset <key> or /assistant experiment <key>), Studio now opens directly at that function’s datasets or experiments page instead of opening at the root and navigating after. This shaves a few seconds off each focused session and puts you in context immediately.Dashboard
Logout redirects to sign-in page
Signing out no longer lands on a blank page. You’re now redirected to the sign-in page, where you can immediately log back in or close the tab.Session log capture fix and standalone opt-in
Session log capture now works correctly after opting in during setup. A configuration mismatch previously caused the plugin to silently skip session capture even when you’d consented, so no session data was being collected. You can also now toggle session log capture on or off by running/bitfab:setup session-logs, a standalone mode that doesn’t require authentication.Dashboard
Trace viewer skips empty spans on open
Opening any trace now lands on the first span that has data instead of a blank trace root or an empty span. This applies across the dashboard: trace detail pages, the labeling panel, the experiments comparison view, the dataset detail panel, and the template preview studio. The hard template filter still hides non-matching spans; you just no longer have to scroll past empty ones to see meaningful content.Dashboard
Studio navigation events for coding agents
When you navigate between pages in Studio, the coding agent now receives real-time navigation events with the current path. This gives the agent immediate awareness of where you are in Studio, so it can tailor its responses and actions to the page you’re viewing without needing to ask.Studio sign-in stays within the Studio shell
When your coding agent opens Studio and you’re not signed in, you now see a branded sign-in page inside the Studio window instead of being redirected to the main Bitfab login. The session context persists across the sign-in flow, so the agent picks up exactly where it left off once you authenticate. The CLI receives real-timeauth-required and authenticated events, letting it wait for sign-in without polling.Accurate offline SDK update checks
The plugin’s session-start update check now always reports the correct latest SDK versions. Previously the baked version snapshot could lag behind by one release, causing the plugin to miss update notifications or report you were up to date when a newer SDK was available.Hill-climb from existing labels in /bitfab:assistant
When you start a new dataset for a function that already has validated labels, the assistant now offers a Reuse option that seeds the dataset with those labels instead of starting from scratch. Pick Reuse when you’re spinning up a different cut for experimentation but want to keep the labeling work you already trust. Define and Open are still there for the from-scratch and broad-sample cases.Replay verdicts persist with a coverage gate
After a replay in Phase 5, the assistant writes its pass/fail verdicts on the replay traces through a bundled script that verifies every replay trace got a verdict before moving on. Previously a verdict could die mid-session if the agent forgot to persist it; now the script enforces full coverage before continuing. If a trace is genuinely ambiguous, you can record it as an explicit skip rather than leaving it silently unverdicted.Plugins surface which Bitfab org they’re writing to
The plugin MCP now flags which Bitfab org it reads and writes from, so you’ll catch mismatches between your project’sBITFAB_API_KEY and the org open in your Studio tab before traces land somewhere unexpected. Coding agents now call get_api_key_context at the start of a plugin MCP session, and again whenever you mention data you just wrote isn’t visible in Studio. The same tightening applies to the remote MCP server in the Dashboard for direct (non-plugin) callers.Ruby SDK: skip child spans during replay
When you replay historical traces throughclient.replay(...), you can now have child spans return their recorded outputs instead of running real code. Three strategies control which children get short-circuited:mock: "none"(default) reruns every child span as before.mock: "all"returns historical output for every child.mock: "marked"returns historical output only for spans declared withmock_on_replay: true, and runs everything else real.
mock: "marked" to iterate on agent logic without paying for the marked child calls on each replay. Use mock: "all" for the cheapest possible replay (only the root function runs real code). Brings the Ruby SDK to parity with the existing mock option in the Python and TypeScript SDKs.Ruby SDK: fluent wrapper for shared trace function keys
client.get_function(key) returns a wrapper bound to that trace function key, so you can wrap multiple methods or classes without repeating the key on every call.client.get_function in the Python SDK and client.getFunction in TypeScript.DashboardPlugins
Accurate experiment counts with multi-label traces
Experiment pass/fail counts now correctly deduplicate traces that have labels from multiple sources (human review, approved agent, unapproved agent). Previously, a trace with both a human and an agent label could be double-counted in experiment totals. The viewer now picks the highest-priority label per trace: human labels take precedence over approved agent labels, which take precedence over unapproved ones.Experiments auto-label replayed traces
When you run an experiment through the assistant, replayed traces now receive agent labels automatically. The experiment viewer shows pass/fail results immediately after a replay completes, without requiring a manual labeling step first.Replay with mocks: shared-key spans return the correct output
Fixed an off-by-one in the replay mockTree when the function under test and one of its children share onetraceFunctionKey (the canonical getFunction(key).withSpan(...) pattern). The marked child was returning the root’s historical output instead of its own. The mockTree is now keyed by (traceFunctionKey, spanName, callIndex), which also unblocks recursive same-key replays.Mocked non-async Promise-returning functions stay Promises
If you wrap afunction fetchX() { return fetch(...) } (no async, but returns a Promise) and mock it during replay, the mocked return is now a Promise, not a raw value. Downstream .then(...) callers no longer crash. Detected at wrap time./bitfab:assistant experiments auto-pick parallel or serial
Phase 5 of the assistant skill now checks whether subagent worktrees inherit bypass permissions before forking parallel experiments. If permissions.defaultMode: "bypassPermissions" is set in committed .claude/settings.json or ~/.claude/settings.json, experiments fork to worktree-isolated subagents; otherwise they run serially in the main agent. Cursor and Codex always run serial since they don’t support worktree-isolated subagent calls.Dashboard
Organization switcher fix
Fixed the organization switcher dropdown not appearing in the header. After upgrading to Clerk v7, the switcher silently returned no memberships, making it impossible to switch between teams. The switcher now reliably shows all your organizations, with your personal workspace listed first and the rest sorted alphabetically.Live agent activity in Studio
The Studio home page now shows what your coding agent is doing in real time. While the assistant is working, the agent card highlights green and displays the current tool action (e.g., “Reading traces…”, “Creating grader…”). When the agent finishes or goes idle, the card fades back to its neutral state. Activity persists across page navigation within Studio, so you won’t lose track of the agent’s progress.Studio detects which coding agent opened it
When Studio is launched from Cursor or Codex, the UI now shows that agent’s logo and name instead of defaulting to Claude Code. The welcome page, header, and “Return to” button all reflect the agent that started the session.Replay failure handling in the assistant skill
/bitfab:assistant now separates infrastructure failures (missing DB rows, rejected writes) from real regressions during replay, and keeps unreplayable traces out of the pass-rate. When a child span fails environmentally, it suggests either flipping the span to mockOnReplay or pointing replay at the trace’s source environment.Replay mocks return the correct child span’s output
Fixed ordering bugs inmock: "marked" that caused a marked child span to return a sibling’s historical output instead of its own. Upgrade to TypeScript SDK 0.12.1 if you’re using mock: "marked" on 0.12.0.Mock child spans during replay
When you replay a recorded trace against new code, child spans sometimes fail locally for reasons unrelated to what you’re iterating on, like a paid API key you don’t have set, a flaky external service, or a production database row that isn’t seeded in your local environment. Replay now supports skipping those children and returning their recorded outputs instead, so the root function can still run.Pass amock strategy to replay() to control it. "none" (default) runs every child for real. "all" returns the historical output for every descendant. "marked" only short-circuits descendants you’ve tagged at definition time, leaving everything else to run real, which is the iteration-friendly mode.Tag a span with mockOnReplay: true in TypeScript or mock_on_replay=True in Python:mock: "marked" (TS) or mock="marked" (Python). The flagged child returns its recorded output and downstream spans run real code, so you can iterate on the analysis or formatting steps without standing up the upstream dependency.When the assistant skill is replaying a function and a child span fails environmentally, it’ll now suggest this fix directly. Full docs: TypeScript SDK and Python SDK reference under “Mocking child spans during replay”.Reliable focus restoration for macOS terminals
When clicking “Return to coding agent” in Studio, focus now reliably returns to the correct terminal app. The previous approach could target the wrong window if the terminal’s environment was modified (common inside Claude Code). The plugin now identifies your terminal by walking the process tree to find the parent application. For iTerm2 users with multiple windows, focus targets the exact session pane.Persistent Studio session for the assistant flow
Addstudio to any /bitfab:assistant invocation (e.g., /bitfab:assistant studio) to keep a single Studio window open for the entire flow. Dataset review and experiment results open inside the same window instead of launching separate ones, so you stay in one place while iterating. Without the studio argument, the flow works exactly as before.See which template renders each span at a glance
Iterating on the right template is faster when you can tell which one runs for the span you’re looking at. In the template preview, click or arrow-key through any span in the trace viewer and the matching card in the left rail lights up in that span’s color. That’s the template to edit.Know when your coding agent is mid-edit
Stay out of the agent’s way and watch its work land in context. When your agent saves a template, the studio names who is editing (Claude Code, Cursor, or Codex), pulses the affected card in the rail, and outlines the exact region inside the rendered span, even on instant saves, so you don’t miss it.Chat session capture
Bitfab plugins can now capture your coding-agent chat sessions and send them to the dashboard. Session capture is opt-in: enable it by settingBITFAB_CAPTURE_SESSIONS=true or adding "captureSessions": true to ~/.config/bitfab/config.json. Nothing is captured until you explicitly turn it on.Once enabled, sessions are only recorded after you invoke a Bitfab tool or slash command in the same conversation, so ordinary non-Bitfab conversations are never captured. Works across Claude Code, Cursor, and Codex.Cross-platform focus restoration
When a plugin opens a browser window (OAuth login, Studio preview), focus now returns to your terminal or editor automatically on Linux and Windows. Previously this only worked on macOS. If platform tools aren’t available (e.g., Wayland on Linux), the handoff completes normally without focus restoration.Studio connection errors surface immediately
When your coding agent opens the Studio preview, connection problems (expired API key, network timeout) are now caught before the browser window opens. Previously, errors could surface mid-session after you’d already started editing.Click-to-target template editing
In the template preview studio, you can now click directly on a rendered span to tell your coding agent exactly which region you want changed. No more “make the user message smaller” guesswork: point at the element and describe the change.Live preview auto-refresh
Templates saved in the studio now re-render in the preview automatically. Previously, you had to reload the page to see your changes.Template reference for coding agents
The newget_template_reference MCP tool returns a catalog of every editable region in the standard template, so coding agents can discover what’s available without you having to describe it.Dashboard
Template preview is faster on large functions
The template preview page loads significantly faster for functions with many spans. Pages that previously made dozens of parallel requests now resolve in a single batched call.Dashboard
Template rendering page: template-first layout
The template rendering page now starts from the templates instead of starting from a trace. You pick a template and see exactly which spans it affects.The new three-column layout shows all templates for a function on the left, the current trace in the center (with non-matching spans dimmed), and affected spans across recent traces on the right. If the current trace has no spans for the selected template, the viewer auto-navigates to one that does.API key context
Coding agents can now callget_api_key_context to find out which organization and environment their API key belongs to before sending traces. No more guesswork.API key descriptions
You can now add a description when creating API keys in the dashboard. Descriptions show up in the key list and are returned byget_api_key_context, so your coding agent can tell you which key it’s using without you having to check.