Skip to main content

​
Overview

Bitfab has four primitives. Everything in the product is one of them, or a view over them.

​
How They Fit Together

The primitives chain into one loop:
  1. Instrument your code with the SDK. Every run of the instrumented function produces a trace.
  2. Collect the interesting traces into a dataset, usually the failures you want to fix and the successes you refuse to break.
  3. Define graders for the properties that matter, and attach them to the dataset. The dataset now has a pass rate instead of an opinion.
  4. Run an experiment: replay the dataset against your changed code, and the graders score the replays. Each replayed trace comes back as fixed, regressed, still passing, or still failing.
  5. Repeat, with the dataset growing every time you find a failure mode it did not cover.
Each primitive belongs to one organization and one trace function key, the name you passed to getFunction() or get_function(). A grader can only be attached to a dataset with the same key, and a dataset can only hold traces from that key.

​
Creating Primitives

Datasets, graders, and experiments are created from your coding agent, in plain language, through the Bitfab plugin:
The web portal deliberately has no create forms for these. Its New buttons explain the agent prompt to use instead. The portal is where you read, review, and manage what already exists. Traces are the exception: they are recorded by your instrumented code, so you create them by instrumenting a workflow and running it.

Datasets

Curate the traces you evaluate against

Graders

Score traces pass or fail automatically

Experiments

Replay a dataset against changed code

​
Where to Go Next