> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hightouch.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Amazon Kinesis

> Stream events into Hightouch Events from an Amazon Kinesis data stream.

> [!NOTE]
>
> The Amazon Kinesis event source must be enabled by Hightouch. [Contact us](#pylon-chat) to turn it on.

## Overview

Hightouch can read events from an Amazon Kinesis data stream in real time. After you connect a stream, Hightouch polls every open shard and converts each record into a Hightouch [`track` event](https://docs.hightouch.com/events/event-spec#track-events). From there, you can load events into your [warehouse](https://docs.hightouch.com/events/warehouses/warehouse.md) or [stream them](https://docs.hightouch.com/events/event-streaming.md) to real-time destinations.

## Setup

To get started, [**create an event source**](https://app.hightouch.com/events/sources/new?from=docs) and select **Amazon Kinesis** under **Streaming / Message queues**.

### Configure your credentials

Select an existing AWS credential or create a new one. Hightouch supports two credential types:

* **Cross-account role** (recommended): Create an IAM role in your AWS account that trusts Hightouch, using the **Account ID** and **External ID** shown in Hightouch, then paste the role's ARN into Hightouch.
* **Access key**: Provide the **Access Key ID** and **Secret Access Key** of an IAM user.

For step-by-step instructions on creating either credential type, see the [Amazon Kinesis destination setup](https://docs.hightouch.com/destinations/amazon-kinesis.md#connect-to-your-aws-account). The process is the same; only the required permissions differ.

### Grant permissions

Hightouch follows the principle of least privilege. The IAM role or user needs the following IAM actions on each stream:

| Stream | Action | Details |
| :- | :- | :- |
| Source stream | `kinesis:DescribeStreamSummary` | Checks that the stream exists and has a status of `ACTIVE` or `UPDATING` before Hightouch starts reading. |
| Source stream | `kinesis:ListShards` | Finds every open shard in the stream so Hightouch can poll each one, and discovers child shards after a split or merge. |
| Source stream | `kinesis:GetShardIterator` | Gets a read position in each shard: the latest position when you first connect the stream, or the last checkpoint when Hightouch resumes after a restart. |
| Source stream | `kinesis:GetRecords` | Reads records from each shard so Hightouch can convert them into events. |
| Dead-letter stream (optional) | `kinesis:PutRecords` | Writes records that Hightouch can't process to the [dead-letter stream](#dead-letter-stream). |

You can use the following JSON sample to create your IAM policy. Leave out the second statement if you don't configure a dead-letter stream.

```json theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kinesis:DescribeStreamSummary",
        "kinesis:ListShards",
        "kinesis:GetShardIterator",
        "kinesis:GetRecords"
      ],
      "Resource": "arn:aws:kinesis:YOUR_REGION:YOUR_ACCOUNT_ID:stream/YOUR_STREAM_NAME"
    },
    {
      "Effect": "Allow",
      "Action": ["kinesis:PutRecords"],
      "Resource": "arn:aws:kinesis:YOUR_REGION:YOUR_ACCOUNT_ID:stream/YOUR_DLQ_STREAM_NAME"
    }
  ]
}
```

### Configure your source

Enter the following fields:

* **Stream name**: The name of the Kinesis data stream to read events from.
* **Region**: The AWS region where the stream lives.
* **DLQ stream name** (optional): A Kinesis stream in the same region where Hightouch writes records it can't process. See [Dead-letter stream](#dead-letter-stream).

The stream must have a status of `ACTIVE` or `UPDATING`. Hightouch doesn't support SSH tunnels for Kinesis sources.

### Message format

Hightouch Events requires each Kinesis record's data to be JSON. Records that aren't valid JSON, or that exceed the 96 KB event size limit, can't be processed.

### Dead-letter stream

If Hightouch encounters a fatal error while processing a record, such as data that isn't valid JSON, it writes the record to the dead-letter stream you configured. If you don't configure a dead-letter stream, Hightouch drops those records.

Each dead-letter record uses the original record's partition key. Its data is a JSON object that contains the original record, the error, and the ID of the event source that wrote it:

```json theme={null}
{
  "message": {
    "partitionKey": "user_123",
    "sequenceNumber": "49590338271490256608559692538361571095921575989136588898",
    "shardId": "shardId-000000000000",
    "streamName": "my-stream",
    "approximateArrivalTimestamp": "2026-09-28T14:52:37.648Z",
    "data": "not valid json",
    "truncated": false
  },
  "error": "Unexpected token o in JSON at position 1",
  "hightouchEventSourceId": "YOUR_EVENT_SOURCE_ID"
}
```

Hightouch truncates the original data to 2,048 characters and sets `truncated` to `true` when it's longer.

### Starting position

When you first connect a stream, Hightouch starts reading from the latest position in each shard. Records written to the stream before the source was created aren't ingested. After that, Hightouch checkpoints its position in each shard after every batch, so it resumes where it left off after a restart.

Hightouch also follows resharding. When a shard closes after a split or merge, Hightouch continues reading from its child shards.

## Event structure

Hightouch ingests every Kinesis record as a `track` event:

* The record's partition key and parsed JSON data go into `properties.partitionKey` and `properties.value`.
* Stream metadata (`streamName`, `shardId`, and `sequenceNumber`) goes into `context.kinesis`.
* The record's approximate arrival time becomes the event `timestamp`.

For example, a record written to the `my-stream` stream with partition key `user_123` and this data:

```json theme={null}
{
  "user_id": "user_123",
  "amount": 123.45
}
```

Is ingested as:

```json theme={null}
{
  "type": "track",
  "event": "My Kinesis Source Event",
  "properties": {
    "partitionKey": "user_123",
    "value": {
      "user_id": "user_123",
      "amount": 123.45
    }
  },
  "context": {
    "kinesis": {
      "streamName": "my-stream",
      "shardId": "shardId-000000000000",
      "sequenceNumber": "49590338271490256608559692538361571095921575989136588898"
    }
  },
  "timestamp": "2026-09-28T14:52:37.648Z"
}
```

By default, the event name is your event source's name followed by `Event`, and the message ID is built from the source ID, shard ID, and sequence number.

### Field mappings

You can configure how Hightouch maps fields in each record to Hightouch [`track` events](https://docs.hightouch.com/events/event-spec#track-events). Mappings can be configured for the following fields:

* **Event Name**: a constant value or a field from the record
* **User ID**
* **Anonymous ID**
* **Message ID**
* **Timestamp**

Field paths are relative to the record's JSON data. For example, to use `user_id` from the record above as the user ID, enter `user_id`, not `properties.value.user_id`. If an event ends up with neither a user ID nor an anonymous ID, Hightouch generates a random anonymous ID for it.

You can also use [Functions](https://docs.hightouch.com/events/functions/overview.md) to standardize schemas or enrich event data.

### Schema enforcement

[Event contracts](https://docs.hightouch.com/events/contracts/management.md) can be attached to Amazon Kinesis sources just like any other event source.

## Tips and troubleshooting

If you run into an issue this page doesn't cover, [contact support](#pylon-chat).