A/B testing

Collections give you the storage half of an A/B test: multiple variants as labels, each version immutable and addressable, and a one-call promotion when you pick a winner. The measurement half stays in your analytics tool. Floggy never decides a winner and never runs the experiment. It stores the variants and the identifiers you join on.

The identifier triple

Every exposure your site records should carry three ids. Together they are the join key between the content in Floggy and the events in any analytics tool:

  • entryId - which entry.
  • version (or versionId) - the exact immutable snapshot that visitor saw.
  • label - which variant channel it came from (variant-a, variant-b, ...).

Because versions are immutable, the triple is a permanent, unambiguous record of what was shown. Your analytics tool groups conversions by label (or version) and you read the result.

The loop

  1. An agent or pipeline writes two or more versions of an entry and points a label at each (variant-a, variant-b). A create can carry its label inline ({ data, label: "variant-a" }, @floggy/cms 0.2.1+), so the first variant needs one call instead of two. Weights go in each label's meta.
  2. Your site picks a variant per visitor, renders it, and fires an exposure event carrying the triple to your analytics tool.
  3. Your analytics tool (not Floggy) accumulates conversions per variant and decides the winner however you like.
  4. You promote the winning version to production with one API call. The label.updated webhook fires; your site revalidates.

Floggy's role is steps 1 and 4: store the variants, move the label. Steps 2 and 3 are yours.

A variant label is not a publish. variant-a and variant-b are read by name (?label=variant-a); default reads still resolve production, and an entry that only has variant labels stays a draft to anything reading the default. That is the point: you serve variants deliberately during the test, then promote one to production when it wins.

Picking a variant

Assignment must be deterministic per visitor so a returning visitor sees the same variant. The SDK ships a helper for weighted, deterministic assignment:

import { pick } from "@floggy/cms";

// labels: the entry's labels with their meta (from labels list / entry read)
// visitorId: a stable per-visitor id (cookie, user id, ...)
const chosen = pick(labels, visitorId);
// chosen -> { label: "variant-b", version: "v_9", meta: { weight: 70 } }

pick hashes visitorId and distributes deterministically across the labels, honouring meta.weight when present (equal split otherwise). Same visitor, same assignment, every time. Roll your own if you prefer; the only rule is determinism.

Firing the exposure event

Once you have the chosen variant, render chosen.version's data and fire one exposure event carrying the triple. The recipes below are examples of one pattern, not integrations. Floggy does not connect to any of these tools; you send the event from your own code.

The payload is always the same idea: an event name plus the triple as properties.

GA4

gtag("event", "experiment_exposure", {
  entry_id: entryId,
  variant: chosen.label,
  version: chosen.version,
});

PostHog

posthog.capture("experiment_exposure", {
  entryId,
  variant: chosen.label,
  version: chosen.version,
});

Plausible

plausible("experiment_exposure", {
  props: { entryId, variant: chosen.label, version: chosen.version },
});

Umami

umami.track("experiment_exposure", {
  entryId,
  variant: chosen.label,
  version: chosen.version,
});

Then track your conversion events as usual and segment them by variant (or version) in your tool.

Promoting the winner

When your tool says variant-b won, point production at that variant's version:

curl -X PUT ".../collections/landing-pages/entries/<id>/labels/production" \
  -H "Authorization: Bearer $FLOGGY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "version": "v_9", "meta": { "wonExperiment": "bf-headline" } }'

The label.updated webhook fires so your site can revalidate. Keep the losing variants as versions and labels, or clean them up, your call.

Next