> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hightouch.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Experiments

> Experiments split an audience into randomized groups, so you can measure the lift a campaign actually drives over a holdout or an alternative treatment.

> [!NOTE]
>
> **Experiments** were previously called **Splits**.

## Before you begin

* A saved [audience](https://docs.hightouch.com/customer-studio/usage.md).
* At least one [sync](https://docs.hightouch.com/customer-studio/syncs.md) or campaign ready to launch.

## Overview

An experiment assigns the members of an audience to distinct, randomized groups. You apply a different treatment to each group — a promotional email to one group and nothing to a holdout, or a different creative to each — and then compare outcomes to see what actually moved the metric.

Randomization is what separates a real effect from a coincidence. Without it, you're left guessing:

* **Correlation** — "Purchases went up after the campaign, but was it the campaign?"
* **Attribution** — "This user converted after seeing the ad, but would they have converted anyway?"

Comparing randomized groups isolates the campaign's impact, so you can measure incrementality, test creatives or channels against each other, and confirm whether personalization outperforms a generic experience.

> [!TIP]
>
> To branch members inside a journey instead of an audience, use an **A/B
> split** tile. See [Journey tiles](https://docs.hightouch.com/customer-studio/journeys/tiles.md#ab-split).

An experiment moves through four stages: split a saved audience into randomized groups, apply a treatment to each group through syncs, choose the metric that defines success, and compare the results across groups once the campaign runs.

## Create an experiment

You build an experiment on the audience you want to test.

1. Open a saved audience and go to the **Experiment** tab.
2. Select **Create experiment**.
3. Under **Split groups**, select **Add split group** for each group you want, then set each group's name and its share of the audience. Use two groups for a simple A/B test, or three or more for A/B/n tests.
4. To measure incremental lift, make one group a **Holdout group**. A holdout receives no treatment and acts as the control you compare the others against.
5. Under **Syncs**, assign each non-holdout group the syncs that deliver its treatment. A group synced to multiple destinations sends all of its members to every destination — the group isn't subdivided between them.

![The Experiment tab showing split groups, their assigned syncs, and measurement configuration](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/hightouch-audiences/splits/experiment-split-groups.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=1131191efd39e385a7208ccbf3b7c489)

> [!WARNING]
>
> Each member is assigned to a group once, based on its primary key. Changing
> the group percentages after launch can reassign members and place a member in
> more than one group. To keep group membership fixed, use an [audience
> snapshot](https://docs.hightouch.com/customer-studio/audience-snapshots.md).

## Configure measurement

Under **Measurement configuration**, define how Hightouch scores the experiment. You set this on the same **Experiment** tab, below the groups.

* **Primary metric** — The event and value that define success, such as the average `QUANTITY_PURCHASED` on a purchase event. This is the outcome compared across groups.
* **Secondary metrics** — Optional additional outcomes to track alongside the primary metric.
* **Start date** and **End date** — The window the experiment measures. An experiment completes at 11:59:59pm on the end date. Leave the end date empty to keep measuring until you set one.
* **Measurement window** — How long after a member enters (or exits) the audience their outcomes still count toward the experiment.

![Measurement configuration showing a primary metric, secondary metrics, start and end dates, and the measurement window](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/hightouch-audiences/splits/experiment-measurement-configuration.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=47de9cda6cceaea68280fbdf991df1e1)

## Measure results

Once the audience has at least one sync involving experiment groups, Hightouch generates a matching **Experiment**. Select **View results** on the Experiment tab to open it. You can also reach every experiment under **Intelligence > [Experiments](https://app.hightouch.com/experiments?from=docs)**.

![The Experiments list under Intelligence](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/campaign-intel/experiments/ci-experiments-nav.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=a7feee8986eeb459347074806413007c)

In the Experiment, you can:

* Compare lift across groups.
* Check statistical significance.
* Analyze results by primary or secondary metric.
* Normalize results per member or against a baseline.

For how to read and configure the results view, see [Experiments in Intelligence](https://docs.hightouch.com/campaign-intelligence/experiments.md).

## Balance groups with stratified sampling

Simple randomization can produce unbalanced groups when your audience spans distinct segments — for example, different regions or loyalty tiers. **Stratified sampling** keeps those segments proportionally represented across groups, so a difference in outcomes reflects the treatment rather than an uneven split.

> [!NOTE]
>
> Stratified sampling is available only for one-time syncs.

To enable it:

1. On the audience's **Experiment** tab, expand **Advanced configuration**.
2. Turn on **Stratified sampling**.
3. Under **Stratification variables**, select one or more columns to balance across (for example, `loyalty_tier`).
4. Save your changes.

![Stratified sampling enabled with a stratification variable selected](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/hightouch-audiences/splits/stratified-sampling.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=7d76107f68a3c74f10bf2372a252d795)

## Holdout group logs

> [!NOTE]
>
> Holdout group logs must be enabled by Hightouch. [Contact us](#pylon-chat) to turn it on.

Holdout group logs record which rows were **excluded** from a sync as part of a holdout group, so data teams can run row-level analysis in the warehouse. Marketers should rely on the Experiment results in Customer Studio instead.

Enabling holdout group logs requires:

* The [Lightning sync engine](https://docs.hightouch.com/syncs/lightning-sync-engine.md).
* A destination with **write access** to your warehouse.

To enable them:

1. Go to **Integrations > Sources** and select a source.
2. Open the **Sync logs** tab.
3. On the **Warehouse sync logs** card, turn on **Audience holdout group logs**.

![The Audience holdout group logs toggle on the Warehouse sync logs card](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/hightouch-audiences/splits/cs-splits-holdout-feature.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=8a76658f85be6cb0433b603e6e56a9e1)

Once enabled, Hightouch logs excluded rows to the `hightouch_audit.audience_holdout` table.

| Column | Description |
| - | - |
| `sync_id` | Sync ID |
| `sync_run_id` | Sync run ID |
| `model_id` | Model or audience ID |
| `timestamp` | Timestamp of the sync |
| `row_id` | Primary key value from the model |
| `fields` | JSON snapshot of model data |
| `split_group` | Name of the experiment group |

## Next steps

* [Experiments in Intelligence](https://docs.hightouch.com/campaign-intelligence/experiments.md) to configure and interpret the results view in depth.
* [Sync audiences](https://docs.hightouch.com/customer-studio/syncs.md) to deliver each group's treatment to a destination.
* [Audience snapshots](https://docs.hightouch.com/customer-studio/audience-snapshots.md) to freeze group membership before you launch.
* [Journey tiles](https://docs.hightouch.com/customer-studio/journeys/tiles.md#ab-split) to run an A/B split inside a journey instead of an audience.