Skip to main content
Experiments were previously called Splits.

​
Before you begin

  • A saved audience.
  • At least one sync or campaign ready to launch.

​
Overview

An experiment assigns the members of an audience to distinct, randomized groups. You apply a different treatment to each group — a promotional email to one group and nothing to a holdout, or a different creative to each — and then compare outcomes to see what actually moved the metric. Randomization is what separates a real effect from a coincidence. Without it, you’re left guessing:
  • Correlation — “Purchases went up after the campaign, but was it the campaign?”
  • Attribution — “This user converted after seeing the ad, but would they have converted anyway?”
Comparing randomized groups isolates the campaign’s impact, so you can measure incrementality, test creatives or channels against each other, and confirm whether personalization outperforms a generic experience.
To branch members inside a journey instead of an audience, use an A/B split tile. See Journey tiles.
An experiment moves through four stages: split a saved audience into randomized groups, apply a treatment to each group through syncs, choose the metric that defines success, and compare the results across groups once the campaign runs.

​
Create an experiment

You build an experiment on the audience you want to test.
  1. Open a saved audience and go to the Experiment tab.
  2. Select Create experiment.
  3. Under Split groups, select Add split group for each group you want, then set each group’s name and its share of the audience. Use two groups for a simple A/B test, or three or more for A/B/n tests.
  4. To measure incremental lift, make one group a Holdout group. A holdout receives no treatment and acts as the control you compare the others against.
  5. Under Syncs, assign each non-holdout group the syncs that deliver its treatment. A group synced to multiple destinations sends all of its members to every destination — the group isn’t subdivided between them.
The Experiment tab showing split groups, their assigned syncs, and measurement configuration
Each member is assigned to a group once, based on its primary key. Changing the group percentages after launch can reassign members and place a member in more than one group. To keep group membership fixed, use an audience snapshot.

​
Configure measurement

Under Measurement configuration, define how Hightouch scores the experiment. You set this on the same Experiment tab, below the groups.
  • Primary metric — The event and value that define success, such as the average QUANTITY_PURCHASED on a purchase event. This is the outcome compared across groups.
  • Secondary metrics — Optional additional outcomes to track alongside the primary metric.
  • Start date and End date — The window the experiment measures. An experiment completes at 11:59:59pm on the end date. Leave the end date empty to keep measuring until you set one.
  • Measurement window — How long after a member enters (or exits) the audience their outcomes still count toward the experiment.
Measurement configuration showing a primary metric, secondary metrics, start and end dates, and the measurement window

​
Measure results

Once the audience has at least one sync involving experiment groups, Hightouch generates a matching Experiment. Select View results on the Experiment tab to open it. You can also reach every experiment under Intelligence > Experiments. The Experiments list under Intelligence In the Experiment, you can:
  • Compare lift across groups.
  • Check statistical significance.
  • Analyze results by primary or secondary metric.
  • Normalize results per member or against a baseline.
For how to read and configure the results view, see Experiments in Intelligence.

​
Balance groups with stratified sampling

Simple randomization can produce unbalanced groups when your audience spans distinct segments — for example, different regions or loyalty tiers. Stratified sampling keeps those segments proportionally represented across groups, so a difference in outcomes reflects the treatment rather than an uneven split.
Stratified sampling is available only for one-time syncs.
To enable it:
  1. On the audience’s Experiment tab, expand Advanced configuration.
  2. Turn on Stratified sampling.
  3. Under Stratification variables, select one or more columns to balance across (for example, loyalty_tier).
  4. Save your changes.
Stratified sampling enabled with a stratification variable selected

​
Holdout group logs

Holdout group logs must be enabled by Hightouch. to turn it on.
Holdout group logs record which rows were excluded from a sync as part of a holdout group, so data teams can run row-level analysis in the warehouse. Marketers should rely on the Experiment results in Customer Studio instead. Enabling holdout group logs requires: To enable them:
  1. Go to Integrations > Sources and select a source.
  2. Open the Sync logs tab.
  3. On the Warehouse sync logs card, turn on Audience holdout group logs.
The Audience holdout group logs toggle on the Warehouse sync logs card Once enabled, Hightouch logs excluded rows to the hightouch_audit.audience_holdout table.

​
Next steps

Last modified on September 17, 2026