> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hightouch.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Microsoft Fabric

> Microsoft Fabric is a comprehensive analytics and data integration solution that includes cloud-based data warehousing as part of its extensive suite of capabilities.

## Overview

Hightouch lets you pull data stored in Microsoft Fabric and push it to downstream destinations. Connecting Hightouch to Microsoft Fabric requires some setup in both platforms. In Microsoft Fabric, you need to create a service principal and grant it access to your Fabric resource.

## Service principal creation

First, we need to create a service principal. These are the credentials Hightouch will use to access your Microsoft Fabric resources.

1. Sign in to the [Microsoft Entra admin center](https://entra.microsoft.com/).
2. On the sidebar, select **Identity** > **Applications** > **App registrations**.
3. Select **New registration**.
4. Pick a name for your application and select **Accounts in this organizational directory only** as the supported account type.
5. Click **Register**.
6. Note your newly created **Application (client) ID** and **Directory (tenant) ID**.
7. Click **Add a certificate or secret**.

From here, you can opt to use either a **Client secret** or a **Certificate**.

### Client secret

1. Select the **Client secrets** tab.
2. Click **New client secret**.
3. Name your client secret and select its expiration date. To maintain access to Fabric, make sure to refresh your client secret prior to its expiration.
4. Note your client secret **Value** now as it cannot be retrieved later. It will be used to connect Hightouch to Microsoft Fabric.

### Certificate

1. Select the **Certificates** tab.
2. Click **Upload certificate** and upload a valid signed certificate.
3. Note your certificate's **Thumbprint**.

For more detailed instructions on service principal creation, you can check out [Microsoft's documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal).

## Fabric configuration

By default, Microsoft Fabric may not be accessible via service principal. To connect, you'll need to take the following steps.

1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Click **Synapse Data Engineering**.
3. In the top right of the window, select **Settings** > **Governance and insights** > **Admin portal**.
4. On the sidebar, select **Tenant settings**.
5. Scroll down to **Developer settings** and select **Service principals can use Fabric APIs**. Make sure the toggle is set to **Enabled**.
6. Next go to **OneLake settings** and select **Users can access data stored in OneLake with apps external to Fabric**. This toggle should be **Enabled** as well.

## Attaching your service principal to your Fabric warehouse

1. Log in to [Microsoft Fabric](https://app.fabric.microsoft.com/).
2. Select the workspace that contains your warehouse.
3. Hover over your warehouse and select the three dots menu.
4. Choose **Manage permissions** from the menu.
5. On the next page, select **Add user**.
6. Type in the name of your service principal and check the box for **Read all data using SQL (ReadData)**.
7. Select **Grant**.

## Connection configuration

In Hightouch, go to the [**Sources** overview page](https://app.hightouch.com/sources?from=docs) and click the **Add source** button. Select **Microsoft Fabric** and follow the steps below.

### Configure your source

Enter the following required fields into Hightouch:

* **Host**: The hostname or IP address of your Microsoft Fabric instance. If using the hostname, make sure to only insert the hostname and not the full URL (remove the leading `http://` and the final `/`).
* **Port**: The port number of your Microsoft Fabric instance. The default is 1433, but yours may be different.
* **Database**: The name of the database in Microsoft Fabric.

Optionally you can enter a **Request timeout duration** and whether to **Trust server certification**.

### Choose your sync engine

For optimal performance, Hightouch [tracks incremental changes](https://docs.hightouch.com/getting-started/concepts.md#change-data-capture) in your data model—such as added, changed, or removed rows—and only syncs those records. You can choose between two different sync engines for this work.

The **Basic engine** requires read-only access to Fabric. Hightouch executes a query in your database, reads all query results, and then determines incremental changes using Hightouch's infrastructure. This engine is easier to set up since it requires read—not write—access to Fabric.

The **Lightning engine** requires read and write access to Fabric. The engine stores previously synced data in a separate schema in Fabric managed by Hightouch. In other words, the engine uses Fabric to track incremental changes to your data rather than performing these calculations in Hightouch. Therefore, these computations are completed more quickly.

![Standard vs Lightning engine comparison](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/get-started/sync-engine-comparison.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=1202b36c58e475d85344db87dde2c825)

> [!TIP]
>
> Hightouch recommends using the Lightning sync engine when syncing more than 100,000 rows of data.

If you select the Basic engine, you can switch to the Lightning engine later. Once you've configured the Lightning engine, you can't move back to the Basic engine without recreating Fabric as a source.

To learn more, including migration steps and tips, check out the [Lightning sync engine docs](https://docs.hightouch.com/syncs/lightning-sync-engine.md).

#### Basic versus Lightning engine comparison

The **Lightning sync engine** requires granting write access to your data warehouse, which makes its setup more involved than the Basic sync engine. However, it is more performant and reliable than the Basic engine. This makes it the ideal choice to guarantee faster syncs, especially with large data models. It also supports more features, such as [Warehouse Sync Logs](https://docs.hightouch.com/syncs/warehouse-sync-logs.md), [Match Booster](https://docs.hightouch.com/match-booster/overview.md), and [Identity Resolution](https://hightouch.com/platform/identity-resolution).

| Criteria | Basic sync engine | Lightning sync engine |
| - | - | - |
| Performance | Slower | Quicker |
| Ideal for large data models (over 100 thousand rows) | No | Yes |
| Reliability | Normal | High |
| Resilience to sync interruptions | Normal | High |
| Extra features | None | Warehouse Sync Logs, Match Booster, Identity Resolution |
| Ease of setup | Simpler | More involved |
| Location of change data capture | Hightouch infrastructure | Fabric schemas managed by Hightouch |
| Required permissions in Fabric | Read-only | Read and write |
| Ability to switch | You can move to the Lightning engine at any time | You can't move to the Basic engine once Lightning is configured |

#### Lightning engine setup

To set up the Lightning engine, run the following commands. Make sure to replace **your\_service\_principal\_or\_user** with the name of the service principal created earlier.

```sql theme={null}
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_audit')
BEGIN
    EXEC('CREATE SCHEMA [hightouch_audit]');
END;
IF NOT EXISTS (SELECT 1 FROM sys.schemas WHERE name = 'hightouch_planner')
BEGIN
    EXEC('CREATE SCHEMA [hightouch_planner]');
END;
GRANT CONTROL ON SCHEMA::hightouch_audit TO [your_service_principal_or_user];
GRANT CONTROL ON SCHEMA::hightouch_planner TO [your_service_principal_or_user];
```

### Provide credentials

Then you'll enter the credentials from the previous service principal creation step

* **Tenant ID** (or Directory ID)
* **Client ID** (or Application ID)

If you chose to create a **Client Secret** for your service principal, you'll enter it here.

If you instead chose to create a **Certificate**, enter in your certificate's **Thumbprint** and the associated **private key**. Make sure that the private key is in PEM format and includes the header and footer.

### Test your connection

When setting up Microsoft Fabric as a source for the first time, Hightouch validates your credentials and access to your database. Once the test passes, click **Continue** to finish setup.

## Create models

Once your source configuration has passed the necessary validation, you've completed setting up Microsoft Fabric as a source. Next, you can set up [models](https://docs.hightouch.com/getting-started/concepts.md#models) to define which data you want to pull from Microsoft Fabric.

The Microsoft Fabric source supports these modeling methods:

* writing a query in [the SQL editor](https://docs.hightouch.com/models/sql-editor.md)
* using the visual [table selector](https://docs.hightouch.com/models/table-selector.md)

A Microsoft Fabric source can also read other lakehouses and warehouses in the same workspace as the warehouse it connects to. This includes data from other workspaces that you've added to a lakehouse as [OneLake shortcuts](https://learn.microsoft.com/en-us/fabric/onelake/onelake-shortcuts). These tables appear in the table selector. In SQL, reference them with three-part names (`database.schema.table`), as described in Microsoft's docs on [cross-database queries](https://learn.microsoft.com/en-us/fabric/data-warehouse/query-warehouse#write-a-cross-database-query). Your service principal needs read access to each lakehouse or warehouse you query.

You can also use Microsoft Fabric as an event storage destination for Hightouch Events. That connection needs different grants. See [Microsoft Fabric prerequisites](https://docs.hightouch.com/events/warehouses/warehouse.md#microsoft-fabric).

## Tips and troubleshooting

Most issues with the Microsoft Fabric source can be resolved by verifying that your Microsoft Fabric configuration is correct. It must allow external connections via service principal.