Running an A/B Experiment on a Feature Flag

An A/B experiment in rollfuse is not a separate targeting system: it rides on top of an existing feature flag. The flag still controls which variation a subject sees; the experiment adds a hypothesis, a start/stop lifecycle, and the statistics layer that reads exposures and observations to tell you which variation won. This tutorial creates an experiment on a checkout-button-color flag, starts it, evaluates it from sdk-js, and records a purchase as the conversion metric.

Create the experiment on the flag

An experiment is created under the flag it experiments on, with POST /v1/feature-flags/{feature_flag_id}/experiments. You need the flag's ID and a short hypothesis describing what you expect to change.

const flagId = "flag_checkout_button_color";

const res = await fetch(
  `https://api.rollfuse.com/v1/feature-flags/${flagId}/experiments`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      name: "Checkout button color",
      hypothesis: "Green CTA increases conversion",
    }),
  },
);

const experiment = await res.json();
// experiment.id, experiment.status === "draft"

The experiment is created in a draft state. Nothing is assigned to a variation and nothing is measured yet: sdk-js still evaluates the flag using whatever targeting rules it already has, so creating an experiment has zero effect on production traffic until you start it.

Start the experiment

await fetch(
  `https://api.rollfuse.com/v1/experiments/${experiment.id}/start`,
  {
    method: "POST",
    headers: { Authorization: `Bearer ${token}` },
  },
);

Starting the experiment tells the platform to begin bucketing subjects into the flag's variations under the experiment's allocation, and to start accepting exposure and observation events for it. You can confirm it took effect with GET /v1/experiments/{experiment_id} and checking status === "running".

Evaluate the flag from the client and get a variation

Nothing changes in how the SDK is called. client.evaluate() reads the flag's Configuration, which now includes the running experiment's bucketing rules, and returns the variation assigned to that subject.

import { RollfuseClient } from "@rollfuse/sdk-js";

const client = new RollfuseClient({
  baseUrl: "https://api.rollfuse.com",
  credential: process.env.ROLLFUSE_CREDENTIAL!, // set before running
});

await client.start();

const result = client.evaluate(userId, "checkout-button-color", {
  attributes: { plan: "growth" },
});

// result.value is either "control" (default color) or "green" (treatment)
renderCheckoutButton(result.value);

Because bucketing is deterministic on (subjectKey, flagKey), the same user keeps seeing the same variation across sessions for as long as the experiment runs. The SDK reports the exposure automatically the first time a rule-matched evaluation happens for that subject, via the same asynchronous exposure pipeline used for regular flags; you do not need to call the exposure-events endpoint yourself unless you are recording an exposure that happened outside the SDK's evaluation path (for example, a server-rendered page that decided the variation upstream).

// only needed if you evaluated outside the SDK and need to backfill the exposure
await fetch(
  `https://api.rollfuse.com/v1/experiments/${experiment.id}/exposure-events`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${token}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      subject_key: userId,
      variation: result.value,
    }),
  },
);

Record a conversion when the target action happens

The metric an experiment measures is not inferred: you report it explicitly, at the moment the subject completes the action you defined as the goal. For a checkout button color test, that action is finishing a purchase.

async function onPurchaseComplete(userId, orderTotal) {
  await fetch(
    `https://api.rollfuse.com/v1/experiments/${experiment.id}/observations`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        subject_key: userId,
        metric: "purchase_completed",
        value: orderTotal,
      }),
    },
  );
}

This call belongs in the same code path that already confirms the order, not in a background job that might run for a subject who was never exposed to the experiment. An observation is only useful if it can be joined back to an exposure for the same subject_key; a purchase from a user who never saw the flag evaluated will not count toward either variation.

With the experiment running, exposures flowing from sdk-js, and observations landing on every completed purchase, you have everything the platform needs to compute an outcome. The next step, reading sample-ratio-mismatch and outcome-analysis to decide whether to conclude or abort, is covered in Reading Experiment Results and Deciding When to Conclude.