Storage

​

What data does Hightouch store?

1. Change data capture
2. In-app observability and debugging

• If you’re on the Free, Starter, or Pro tier, Hightouch hosts and manages a secure encrypted bucket on your behalf. For more information, see the managed storage section.
• If you’re on the Business tier, you can self-host storage in your own infrastructure. For more information, see the self-hosted storage section.

​

Change data capture

​

Observability and debugging

​

Managed storage

​

Data retention

​

Self-hosted storage

• Amazon S3
• Google Cloud Storage (GCS)
• Microsoft Azure Blob Storage

​

Recommended architecture and security practices

• Scope IAM permissions to the specific bucket or container Hightouch uses, and avoid granting access to unrelated buckets or storage accounts.
• Use a dedicated bucket or container per environment (for example, separate prod, staging, and dev buckets) to avoid cross-environment access. If you need to isolate individual workspaces, give each its own bucket, since Hightouch can’t restrict a workspace to a path within a shared bucket.
• Co-locate storage with your warehouse and destinations when possible to reduce latency and egress costs.
• Prefer private connectivity over public internet paths when available: AWS PrivateLink, Azure Private Link, or Google Private Service Connect. If using public endpoints, maintain strict IP allowlists.
• Use encryption at rest with your standard key-management policy (for example SSE-KMS, CMEK, or customer-managed keys) and ensure key policies grant your Hightouch principal the required access.
• Apply lifecycle rules that auto-expire objects according to your retention policy, and enable storage access logging when required for audits.
• After initial setup or credential rotation, run a small validation sync and confirm data is accessible in the sync debugger before broad rollout.

​

Changing your storage bucket

• Workspace name(s) and URL(s) affected
• Whether the workspace is production or non-production

• Cloud provider and region (for example, AWS us-east-1)
• Bucket or container name (for example, my-company-hightouch-prod)

• For AWS: the IAM role ARN (and external ID, if required) with the permissions listed above
• For GCS: the service account email and confirmation that it has the required permissions (storage.objects.list, storage.objects.create, storage.objects.get, storage.objects.delete, storage.buckets.get)
• For Azure: the storage principal and scope

• Whether you’re using the Lightning sync engine, AI Decisioning, or other features that store artifacts in the bucket
• Whether you’re using Snowflake-integrated features such as Snowflake Warehouse Export — these rely on Snowflake external volumes and stages that also need to be updated to point at the new bucket
• Any lifecycle policies on the old or new bucket that automatically delete objects — particularly planner-state files, which can trigger unexpected full resyncs if expired

• A preferred time window (ideally during low traffic) when syncs can be paused and the configuration updated

​

What to expect

• Lightning sync engine: CDC state is stored primarily in your data warehouse, not in the bucket. Changing buckets generally does not require re-seeding CDC, but historical debugger logs from the old bucket won’t carry over to the new one.
• Standard sync engine: CDC relies on snapshots and diff files stored in the bucket. After a bucket change, each affected sync will need a one-time full resync or CDC reset to establish a clean baseline in the new bucket. Plan for higher warehouse and API load during these initial runs.
• AI Decisioning: The bucket stores long-lived ML artifacts and interaction logs. Migrating may require copying these artifacts to the new bucket, or AI Decisioning will rebuild state from your warehouse over time. Call this out in your support request so the team can plan accordingly.
• Snowflake-integrated features (Snowflake Warehouse Export and others): If you use features that rely on Snowflake external volumes or stages, the Hightouch team will also update the associated storage integrations to point at the new bucket. Make sure the new bucket’s cloud and region are compatible with your Snowflake account.

​

Amazon S3

​

Create your S3 bucket

• Block all public access to the bucket.
• Enable Amazon S3 key encryption (SSE-S3). If using SSE-KMS for encryption, you may need to update your IAM policies to grant Hightouch access.
• Prefer bucket policies with IAM condition scoping over ACL-based access controls.
• Disable bucket versioning.
• Configure your bucket object lifecycle, to enhance security and cut down on costs.

​

Authenticate Hightouch with AWS

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucket",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": ["arn:aws:s3:::${bucketName}"]
    },
    {
      "Sid": "ObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": ["arn:aws:s3:::${bucketName}/*"]
    }
  ]
}

​

Configure your bucket in Hightouch

1. In Hightouch, on the Storage tab of the Settings page, select Amazon S3 as the Cloud storage provider.
2. Select your AWS region, enter your Bucket name, and select the AWS credentials you previously set up.
3. Click Save.

​

Common errors

​

Google Cloud Storage

​

Create a bucket

• Enable Uniform bucket-level access and manage access through IAM policies instead of object ACLs.
• Use CMEK where required by your compliance controls, and ensure your service account has access to the key.
• If you use VPC Service Controls, include the bucket and service account in the same perimeter to avoid access denials.

​

Authenticate with Google Cloud

​

Configure your bucket in Hightouch

1. In Hightouch, on the Storage tab of the Settings page, select Google Cloud Storage as the Cloud storage provider.
2. Enter the Project ID and Bucket name and select the Google Cloud credentials you previously set up.
3. Click Save.

​

Microsoft Azure Blob Storage

​

Create a container

​

Authenticate with Azure

• "actions"."Microsoft.Storage/storageAccounts/blobServices/containers/read"
• "dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action"
• "dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write"
• "dataActions"."Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete"

• "actions"."Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action"

​

Configure your container in Hightouch

1. In Hightouch, on the Storage tab of the Settings page, select Microsoft Azure as the Cloud storage provider.
2. Enter the Account name and Container name from the Azure portal (see connecting Hightouch with your Azure account), and select the Azure credentials you previously set up.
3. Click Save.

​

Common errors

• invalid_client: Check that you’ve provided your client secret value, not client secret ID. See the configuration docs for more information.

• This request is not authorized to perform this operation: Check that Hightouch has adequate access to your storage account. If you’ve configured your storage account’s Network access to be from selected virtual networks and IP addresses, be sure to add Hightouch’s IP addresses to your network rules.