> ## Documentation Index
> Fetch the complete documentation index at: https://superflag.sh/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments and metrics

> Run immutable, reproducible feature experiments without turning Superflag into general analytics

# Experiments and metrics

Superflag connects a feature decision to an immutable experiment iteration and
versioned metric definition. Assignment uses the same deterministic evaluator
as ordinary flags; there is no second bucketing system.

## Current model

* an experiment references one flag and immutable metric revisions;
* starting creates an immutable iteration snapshot;
* assignment is deterministic for the configured randomization unit;
* safe allocation ramps preserve existing bucket boundaries;
* a new iteration uses a new salt and population;
* lifecycle, ramp, revision, and configuration updates require version
  preconditions, while creation uses uniqueness and idempotency; and
* optional SDK telemetry records exposure and configured outcomes asynchronously.

Lifecycle is explicit:

```text theme={null}
draft -> running -> paused -> running
draft -> archived
running | paused -> completed -> archived
```

Archived is terminal.

## Metric revisions

Metric definitions support `conversion`, `count`, `sum`, `mean`, and derived
`ratio` kinds. Each definition declares an owner, stable event key, visibility,
desired direction, randomization unit, and bounded attribution window. Running
iterations pin revisions instead of following mutable names.

```json theme={null}
{
  "schemaVersion": 1,
  "owner": "growth",
  "eventKey": "checkout-conversion",
  "kind": "conversion",
  "visibility": "client",
  "randomizationUnit": { "kind": "subject" },
  "desiredDirection": "increase",
  "attributionWindowSeconds": 86400,
  "filters": []
}
```

## Manage from the CLI

```bash theme={null}
superflag metrics create \
  --key checkout-conversion \
  --file checkout-conversion.json \
  --app store \
  --env prod \
  --source agent \
  --idempotency-key checkout-conversion-v1 \
  --dry-run \
  --json

superflag experiments create \
  --file checkout-experiment.json \
  --app store \
  --env prod \
  --idempotency-key checkout-experiment-v1 \
  --json
```

Use `metrics validate|plan|apply|recover` and
`experiments validate|plan|apply|recover` for agent-authored or automated
changes. Plans bind immutable target identity, current revisions, the full
resolved definition, and an integrity hash.

## What the result means

Superflag's feature-decision data answers which variation was assigned, whether
it was actually exposed, which configured outcome followed, and whether the
data is complete enough to inspect. It is not a general event, profile, funnel,
session replay, or arbitrary SQL product.

The first inference engine is a fixed-horizon contract for binary conversion and
numeric means. It exposes sufficient statistics, effect intervals, sample size,
analysis window, freshness, readiness, and health reasons including sample-ratio
mismatch, crossover, missing attribution, late data, stale watermarks, and low
power. It uses family-wise multiple-comparison correction and publishes its
engine version with every result.

Winner selection remains deliberately disabled pending formal statistical
review. Invalid or incomplete analysis cannot trigger an action. Inspect the
authoritative result directly:

```bash theme={null}
superflag experiments results checkout-copy --app store --env prod --json
superflag experiments health checkout-copy --app store --env prod --json
```

Feature insights are bounded rather than general analytics: allow-listed
feature dimensions and measures, a 90-day aggregate window, seven-day redacted
event pages, disclosed approximate-unique error, freshness, query cost, and
opaque cursor checkpoints. Cells below 20 approximate subjects are suppressed.
There is no arbitrary SQL, user profile, funnel,
cohort, or raw targeting-context explorer.

<Card title="Feature telemetry" icon="chart-line" href="/docs/experimentation/telemetry">
  Understand decisions, exposures, outcomes, privacy, and delivery behavior.
</Card>