> ## Documentation Index > Fetch the complete documentation index at: https://docs.hightouch.com/llms.txt > Use this file to discover all available pages before exploring further. # Field mapping > Map each column in your model to the destination field or property it should populate. *Field mapping tells Hightouch which destination field each model column should populate.* You configure it for every sync, after you decide how records are matched. ## Before you begin * A sync with a model and a destination selected. Configure [record matching](https://docs.hightouch.com/syncs/record-matching.md) first if the sync updates existing records. ## How field mapping works A mapping relates one column in your model to one field or property in your destination. When the sync runs, Hightouch reads each mapped column and writes its value to the field you paired it with. > [!NOTE] > > **Columns, fields, and properties:** Hightouch generally uses "column" for an > attribute in your source or model. Destinations may call the same attribute a > "field" or a "property." Every destination supports mapping a column to a field. Many also support an [advanced mapper](#advanced-mapper) that sets a field from a constant value, sync metadata, a [Liquid](https://shopify.dev/api/liquid) template, or a nested object or array. Use the advanced mapper when a destination field needs a value your model doesn't already contain as a column. ## Map columns to fields The mapping section asks which model columns to sync to which destination fields. You can start from either side — whichever matches how you think about the data. ![Mapping model columns to Braze fields](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/field-mapping-braze.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=0308d8ae1898a813c4c2236d705ed6a9) To add a mapping: 1. Click **Add mapping**. 2. Choose a model column and the destination field it should populate, in either order. 3. Repeat for each field you want to populate. Destinations mark required fields so you can confirm you've mapped them before saving. ![Required destination fields marked with an asterisk](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/required-fields.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=2aa9e9f1db27cece1f345c1999fd8dfe) To map many columns at once, click **Suggest mappings**. Hightouch proposes a destination field for each model column by matching similar names — for example, it pairs an `email_address` column with an **Email** field. Review each suggestion before you save, since name matching can pair the wrong fields. ### Create a destination field Some destinations let you populate a field that doesn't exist yet. Where this is supported, enter the new field's name in the destination input and select the option to create it. ![Creating a custom Braze attribute from the destination field input](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/create-custom-field.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=3dc35989cfa84ef6a8d2be92ad8aea04) ## Advanced mapper The advanced mapper sets a destination field from something other than a single column. Open it by selecting a field's value input, then choose how to supply the value. ![The value type selector in the advanced mapper](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/value-type-selector.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=9de496f73633e07c06b0d5e2aeb3ae23) | Value type | Sets the field from | | - | - | | **Column** | A model column, the same as basic mapping. | | **Static value** | A constant you enter, such as a region code. | | **Variable value** | Sync metadata, such as the sync run's timestamp. | | **Template** | A [Liquid](https://shopify.dev/api/liquid) template that transforms one or more columns. | | **Create an object** | A nested object built from columns. See [inline mapping](#inline-mapping). | | **Create an array** | An array of objects built from a column. See [inline mapping](#inline-mapping). | The value types a destination offers depend on what its API accepts. If selecting a field's value input doesn't open these options, the destination doesn't support the advanced mapper. [get in touch](#pylon-chat) if you need it for a destination that doesn't have it. ### Column values Mapping a column value is the same as basic mapping: search for the model column you want and select it for the destination field. ![Column mapping in the advanced mapper](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/column-mapping.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=45d3a530417604e4c1f8ce984be05e3b) #### Don't sync null values By default, Hightouch syncs `null` values from your model to the destination, which can overwrite an existing destination value with an empty one. To change this, select the settings icon on a column mapping and choose how Hightouch handles nulls: * **Always sync** — Send `null` values to the destination. This is the default and can erase existing data. * **Don't sync null values** — Skip `null` values so the sync never writes an empty value to the field. * **Only write to empty fields** — Write the value only when the destination field is currently empty, so the sync never overwrites an existing value. ![The null value handling options for a column mapping](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/null-value-behavior.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=112556baa4586b597afeafd2d273ba49) This setting applies to a single mapping and doesn't affect nested object or array properties. ### Static values A static value sets a field to the same constant for every record. For example, you might set every record's `region` field to `EMEA`. ![Static value mapping in the advanced mapper](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/static-value-updated.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=ab49d9b57a79fe8fb26d2858a1f399c5) 1. In the advanced mapper, select **Static value** and the value's data type: **String**, **Number**, **Boolean**, or **Null**. 2. Enter the value. 3. Click **Apply**. 4. Select the destination field to map the static value to. ### Variable values A variable value sets a field from sync metadata rather than from your data. For example, you might send the sync run's timestamp to an `updated_at` field. The available variables are: * `model-id` (string) * `model-name` (string) * `current-timestamp` (date/time) * `sync-id` (string) * `sync-run-id` (string) * `operation` (string) ![Variable value mapping in the advanced mapper](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/variable-value-updated.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=eeb819c9d150fdd92de3b2f019d8c62c) 1. In the advanced mapper, select **Variable value** and the variable you want, such as **current-timestamp**. 2. Click **Apply**. 3. Select the destination field to map the variable to. Realtime and journey-triggered syncs expose additional event-context variables in the same list. ### Template mapping The template mapper is convenient when your destination requires data in a specific format, and you want to avoid baking this specificity into your model. For example: * [Webflow](https://docs.hightouch.com/destinations/webflow.md) requires slugs not to have any special characters. You can use the template mapper's `regex_replace` function to remove them. * You want to validate email addresses before sending them to Mailchimp or another email service provider. * You want to enable lead routing in your destination using Liquid conditionals without touching your central dbt model. > [!TIP] > > To create nested objects or arrays, you need to use the [inline > mapper](#inline-mapping). The template mapper uses the [Liquid templating language](https://shopify.dev/api/liquid) to transform column values before they reach the destination. To create a template mapping: 1. In the advanced mapper, select **Template**. 2. Build your expression in the editor. Insert model columns from the **Columns** tab, sync metadata from the **Metadata** tab, and functions from the **Filters** tab. 3. Click **Apply**. 4. Select the destination field to map the template to. ![The Template tab of the advanced mapper, showing the Columns tab of the editor](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/template-columns.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=bb5e5d85f1d904c8e6bd6c431ce10522) For example, the expression `{{ row['last_name'] | includes: "Stark" }}` checks whether a row's `last_name` column contains `"Stark"`. If it does, the row is processed; if it doesn't, the row is rejected. #### Reference columns and metadata Reference a model column with `{{ row['column_name'] }}`. To reference [sync metadata](#variable-values) instead, open the **Metadata** tab to see the available context variables, such as Model ID, Sync ID, and Operation. ![The Metadata tab of the template editor](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/template-metadata-updated.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=a573d97cec9d450b8b58d234fc7476cf) These variables are available on the `context` object. For example, `{{ row['id'] | concat: context['sync_run_id'] }}` concatenates a row's ID with the current sync run's ID. The **Filters** tab lists every available function, including built-in Liquid filters and [Hightouch-specific functions](#liquid-inspired-functions-in-the-advanced-mapper). Select a filter to insert it into your template. ![The Filters tab of the template editor](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/template-filters-updated.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=2f9e541d86e0880fdaeb6e52a1923416) #### Test Liquid templates Use the **Liquid template tester** to preview how your expression evaluates against real rows from your model before you save the mapping. The tester runs your template against up to five sample rows and shows the rendered output for each, so you can catch formatting issues, missing values, or logic errors without triggering a sync. To open the tester, click the **Test template** icon in the top-right corner of the template editor. The tester displays the output for each sample row and indicates whether the template parsed successfully. You can edit the template in the tester and apply it when you're satisfied. ![The Liquid template tester showing rendered output for sample rows](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/liquid-template-tester.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=834a237a9d6473160a1698bbdc7cbb84) The template mapper supports all native Liquid features and adds [Liquid-inspired](#liquid-inspired-functions-in-the-advanced-mapper) functions. For the full list, see the [Liquid reference](#liquid-reference) below. ### Inline mapping > [!NOTE] > > The inline mapper is available for certain destinations and must be enabled by > Hightouch. > > [Contact us](#pylon-chat) > > > > to find out whether your destination supports it and to turn it on. When syncing data to a marketing automation or enterprise resource planning (ERP) tool, you may need to nest related entities. For example, you may have nested items in an invoice or nested entities for a user: associated devices, shipping address, pets, etc. The inline mapper lets you create these nested objects and arrays from your model query results *without altering your model definition*. Each destination has its own formatting expectations for these nested entities. For example, in Stripe, an `invoice` object supports an `items` field that expects an [array of items](https://docs.hightouch.com/destinations/netsuite-rest.md#arrays). The inline mapper lets you model your data *once* and format it during sync configuration, rather than creating a separate model for each destination that requires nested objects or arrays. #### Create an object The object inline mapper builds an object from your model's columns and maps it to a destination field. This section walks through constructing an object for the [shipping](https://stripe.com/docs/api/customers/create#create_customer-shipping-address) field in Stripe's customer object. 1. In the destination field input, select the field you want to map an object to, for example, **Shipping Address**. 2. The model column input autopopulates to **Object** mode if the selected field is of type `object`. If the field's type isn't provided, open the advanced mapper and select **Create an object**. ![The Create an object tab of the advanced mapper](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/create-an-object.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=c917ebc0f025dcd5678d1e0874ca1ad3) 3. Map each nested destination field to a model column. ![An object mapping expanded to show its nested field mappings](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/object-mapping.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=f86dce6e21a4df15d1803b35babae5b7) With nested mappings like these, Hightouch syncs a payload such as: ```json theme={null} { "email": "doe@test.com", "shipping": { "address": { "city": "San Francisco", "state": "California", "postal_code": "94105" }, "name": "John Doe", "phone": "123-456-7890" } } ``` #### Create an array The array inline mapper allows you to construct an array of objects and map them to destination fields using a model column that contains an array of objects. This section walks through constructing a [line items](https://stripe.com/docs/api/quotes/object#quote_object-line_items) array for Stripe's quote object. You would use the same steps for any other object arrays you want to create. ![Array inline mapper example](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-inlinemapper-example-2-1.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=6601643f3f5160325d8053253017139b) #### Required data format To use the inline mapper to create an array of objects, the query results from your [model](https://docs.hightouch.com/models/creating-models.md) need to have a certain format. Specifically, they should appear as an array of objects, like the example **items** field in the entity relationship diagram (ERD) above. ```json theme={null} [ { "item_id": "542", "quantity": 50 }, { "item_id": "631", "quantity": 20 } ] ``` If you don't already have a data field formatted like this, you can define your model using a SQL query that constructs an array of objects from other fields. For example, imagine you have a `line_items` data table where each row represents an invoice item. The columns are: * `invoice_id` for the ID of the invoice the item belongs to * `customer_id` for the ID of the customer who purchased the item * `amount` for the cost of item * `description` for a short description of the item For example: | invoice\_id | customer\_id | amount | description | | - | - | - | - | | in\_1 | cust\_abcd | 55 | Waffle Maker | | in\_1 | cust\_abcd | 7 | Dog Chew Toy | | in\_2 | cust\_efgh | 12 | HDMI Cable | | in\_3 | cust\_abcd | 45 | Humidifier Filters | | in\_3 | cust\_abcd | 109 | Electric Kettle | | in\_3 | cust\_abcd | 15 | Wall Calendar | | in\_4 | cust\_ijkl | 25 | Mediterranean Cookbook | | in\_4 | cust\_ijkl | 10 | AAA Batteries | You want to sync an array of items for each invoice for each customer. For example, for `customer_abcd` from the preceding table, they would have two separate invoice objects, each with their own `items` array, one for the invoice with ID `in_1` and one for the `in_3` invoice: ```json theme={null} [ { "invoice_id": "in_1", "customer_id": "cust_abcd", "items": [ { "description": "Waffle Maker", "amount": 55 }, { "description": "Dog Chew Toy", "amount": 7 } ] }, { "invoice_id": "in_3", "customer_id": "cust_abcd", "items": [ { "description": "Humidifier Filters", "amount": 45 }, { "description": "Electric Kettle", "amount": 109 }, { "description": "Wall Calendar", "amount": 15 } ] } ] ``` To make these fields available in the inline mapper to construct an `items` array, you would need to include SQL like this in your model definition: ```SQL theme={null} select inv.customer_id, inv.invoice_id, array_agg(object_construct(it.*)) as items from line_items inv join items it on it.invoice_id = inv.invoice_id group by 1, 2 ``` This query selects the `customer_id` and `invoice_id` as well as all properties—in this case just description and amount—on each item and then aggregates them into an array of objects. The `group by` statement groups them based on the `customer_id` and `invoice_id` so that each aggregated array is for one invoice for one customer. Without the inline mapper, you would need to write more complex SQL manually constructing each item attribute as part of the item object. #### Map fields Once your model query results have an array of objects to pull data from, you can use the inline mapper to configure as many syncs as you need to sync these items. 1. In the sync's mapping section, select the destination field you want to map an array to, for example, **Line items**. 2. The model column input autopopulates to **Array** mode if the selected field is of type `array`. If the field's type isn't provided, open the advanced mapper and select **Create an array**. 3. You will be prompted to pick a column that contains an array of objects. Select your desired column from the list of compatible columns. If no columns appear, ensure your model query results conform to the [required data format](#required-data-format). ![The Create an array tab of the advanced mapper, showing the array column picker](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/syncs/mapping-data/create-an-array.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=e00d7f18da0eb8a11d2b96dc47ecbf10) 4. Select the relevant destination fields you want to map values to and map the nested fields as needed. ![Mapping our example properties to the quotes object in Stripe.](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-inlinemapper-example-2-3.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=1d93562b257e86c2b3321b62252f16ea) The columns available under the array inline mapper are properties of the previously selected column. Hightouch uses the first 100 rows from your query to extract the available properties at each index of the array. To add additional properties that may be missing, type into the input field and select the custom option or press **Enter**. ![Mapping our example properties to the quotes object in Stripe](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-inlinemapper-example-2-4.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=6da0bfce3d1477a563bf5d8c084970d7) > [!NOTE] > > Hightouch only processes properties if the mapped object property at the index > of the array exists. For example, if `your_column[0].price` exists and > `your_column[1].price` doesn't exist, `items[0].price` will be mapped and > `items[1].price` won't be mapped. Hightouch syncs `null` values if you [don't > choose to ignore them](#dont-sync-null-values). For each row in your query, the array created using the inline mapper matches the length of the array in your column. Given the previous example [row](#create-an-array) and the mappings shown in the screenshot above, Hightouch would sync the following payload: ```json theme={null} { "line_items": [ { "price_data": { "currency": "USD", "product": "542" }, "quantity": 50 }, { "price_data": { "currency": "USD", "product": "631" }, "quantity": 20 } ], "customer": 231, "description": "Two items" } ``` #### Nesting depth You may need to sync arrays or objects with more deeply nested objects. For example, you may need to sync a `Shipping Address` object with this format: ```json theme={null} { "name": "Jane Doe", "address": { "address_line_1": "123 High Street", "address_line_2": "", "city": "San Francisco", "state": "California", "postal_code": "94105", "country": "USA" }, "phone_number": "4151234567" } ``` The "address" key expects a further nested object. Hightouch supports nesting up to two levels deep. You can do this by selecting **Create an object** or **Create an array** while already using the inline mapper. ![Mapping an array in the Hightouch UI](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/nested-mapped-fields.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=1c8f6b5cf91792fe4301a58ddb2c25ca) ### Liquid reference This reference goes over the following: * Basic [liquid formatting](#liquid-template-format) rules in the advanced mapper * Available [liquid variables](#liquid-variables-in-the-advanced-mapper) and [functions](#liquid-inspired-functions-in-the-advanced-mapper) * [Extended use cases](#extended-liquid-use-cases) For more detailed information, refer to the [Liquid reference documentation](https://shopify.dev/api/liquid). #### Liquid template format Variables follow this format: ``` {{ row['company'] }} ``` The double handlebars enclose a Liquid snippet that generates an output. The column name, `company` in the preceding example, should be in single quotes inside the square brackets. Hightouch also provides these alternative syntaxes: ``` {{ row.company }} ``` ``` {{ company }} ``` These shorter alternatives can be useful when templating messages, such as when syncing to [Slack](https://docs.hightouch.com/destinations/slack.md#templating-messages), [Mattermost](https://docs.hightouch.com/destinations/mattermost.md#templating-messages), [Microsoft Teams](https://docs.hightouch.com/destinations/microsoft-teams.md#templating-messages), or to the [SMTP Email](https://docs.hightouch.com/destinations/email.md#templating-emails) destination. > [!WARNING] > > If a model column name contains spaces, reference it with the > bracket-and-quote syntax `row['first name']` inside the double handlebars. Functions follow this format: ``` {{ row['last_name'] | includes : 'Stark' }} ``` A single pipe (`|`) comes after the variable and before the name of the function and any necessary parameters. See the [Liquid Reference](https://shopify.dev/api/liquid) or [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet?shpxid=7bd156ec-6E04-4D53-42BC-9C614D0E5E0E) for a complete list of available functions. #### Liquid variables in the advanced mapper You can use metadata variables in your liquid templates, such as a model's name, all model column names, and the timestamp of a sync's last run. ``` $ $ Model Name $ Timestamp ``` You can browse the available columns and metadata variables in the **Columns** and **Metadata** tabs of the template editor. ![Template Mapping](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-liquid-variables.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=c54a5a630541a6dd4891f1f951d425f2) #### Liquid-inspired functions in the advanced mapper Hightouch provides these non-native functions for you to use out of the box: | Function | Syntax | Description | | - | - | - | | `base64_decode` | `row['email']\|b64_decode` | Base64 decode input | | `base64_encode` | `row['email']\|b64_encode` | Base64 encode input | | `cast` | `row['phone']\|cast:'string'` | Cast input to specified type. Allowable types are 'string', 'number', 'boolean' | | `includes` | `row['last_name']\|includes:'Stark'` | Check input for a substring. Row will be rejected if substring isn't found | | `json_construct` | `\| json_construct: 'city', row['city'], 'state', row['state_province']` | Construct JSON object from key/value arguments | | `MD5` | `row['email']\| md5` | Hash input using MD5 algorithm | | `null_if_empty` | `row['email']\| null_if_empty` | Replace empty strings with `null` | | `parse` | `row['first_name']\|parse` | Parse a JSON-formatted string to a JSON object | | `push` | `row['first_name']\|push : ''` | Append an item to an array | | `regex_replace` | `row['last_name']\|regex_replace: '<[a-zA-Z]>', ''` | Search and replace substrings of input using [RegEx](https://regexr.com/) | | `regex_test` | `row['last_name']\|regex_test: '<[a-zA-Z]>'` | Check input for a [RegEx](https://regexr.com/) match. Row will be rejected if no match is found using [RegEx](https://regexr.com/) | | `sha256` | `row['email']\|sha256` | Hash input using SHA-256 | | `sha512` | `row['email']\|sha512` | Hash input using SHA-512 | | `to_date` | `row['dob']\|to_date` | Parse input and return a [Date](https://shopify.dev/api/liquid/objects/date) object ([ISO 8601](https://www.iso.org/iso-8601-date-and-time-format.html) format) | | `to_unix` | `row['date_sent']\|to_unix` | Convert input to [Unix](https://www.unixtimestamp.com/index.php) time | | `validate_email` | `row['email']\|validate_email` | Reject emails that aren't [RFC 2822](http://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/) compliant email addresses | #### Liquid function chaining You can use more than one Liquid function in a template by [chaining](https://en.wikipedia.org/wiki/Method_chaining) them. For example, you may want to put a string variable into camel case before hashing it. To chain functions, separate them with a single pipe (`|`): Examples: ``` {{ row['full_name'] | camelcase | sha256 }} ``` ``` {{ row['email'] | downcase | validate_email | sha256 }} ``` ### Extended Liquid use cases See Shopify's [Liquid docs](https://shopify.github.io/liquid/basics/introduction/), [Liquid Cheatsheet](https://www.shopify.com/partners/shopify-cheat-sheet), or [Liquid Reference](https://shopify.dev/api/liquid) for a complete list of available functions. In this section, you can learn how to use Liquid to: * set up [conditional outputs](#conditional-output) * output a `null` value for [empty values](#conditional-output-for-empty-values) * set a [default value](#default-value) * [escape special characters](#json-function) in JSON mappings You can also use Liquid to work with different [data types](https://docs.hightouch.com/models/data-types-casting.md#data-types): * **Numbers**: use [math functions](#math-functions) (such as `abs` and `round`) or format [currency amounts](#currency-amounts) * **Booleans**: output a [conditional boolean value](#conditional-boolean-output) * **Strings**: use [string functions](#string-functions) (such as `capitalize`, `strip`, and `append`) or [isolate parts of a string](#isolate-parts-of-a-string) * **Dates**: use the [date function](#date-function) * **Arrays**: [construct a static array](#construct-a-static-array), [sync an empty array](#sync-an-empty-array), or [convert an array to a string](#convert-an-array-to-a-string) * **Objects**: [convert an object to a string](#json-function) or [isolate object properties](#isolate-object-properties) #### Conditional output You can use Liquid conditionals to specify output based on input. For example, this Liquid snippet: ``` {%- if row['discount_percent'] >= 50 -%} Over half off! {%- elsif row['discount_percent'] == 50 -%} 50% off! {%- elsif row['discount_percent'] > 0 -%} Discounted! {%- else -%} null {%- endif -%} ``` Outputs the following result in a destination column: ![Liquid Conditional Output](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-liquid-conditional-result.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=ed0efc814b5647a53624b1a6fe8b73f6) #### Conditional output for empty values Conditionals are also helpful to guard against missing values. For example, you could use the following Liquid snippet to send a `null` value for empty values. This is helpful if you need to map a column that exists for some rows but not others: ``` {%- if row['column_name'] !=nil -%} {{ row['column_name'] }} {%- else -%} {{ null }} {%- endif -%} ``` #### Conditional boolean output You could also use the [`assign`](https://shopify.dev/docs/api/liquid/tags#assign) function to output a conditional boolean value: ``` {%- if row['column_name'] == "value" -%} {%- assign variable = true -%} {%- else -%} {%- assign variable = false -%} {%- endif -%} {{ variable }} ``` #### Default value You can also use Liquid to set a [default](https://shopify.github.io/liquid/filters/default/) value, in case an input value is `null`, `false`, or empty. For example, this Liquid snippet sets the `revenue` field as `0` if the value is missing or `null` in a model's row: ``` {{ row['revenue'] | default: 0 }} ``` Be sure to change `revenue` to the column name you want to set default values for. #### Math functions See [Liquid Math Functions](https://shopify.dev/api/liquid/filters/math-filters) for a complete list of math functions. In Hightouch, the absolute value and rounding functions are some of the most commonly used: * Use the `abs` function to send an absolute value: ``` {{ row['value'] | abs }} ``` * Use the `round` function to round a value to the nearest integer or a specified number of decimals: ``` {{ row['value'] | round: 2 }} ``` #### Currency amounts Hightouch doesn't support [`money`](https://shopify.dev/docs/api/liquid/filters/money-filters) functions. If you would like to format currency amounts, such as prices, you can set this up manually by using the following template: ``` {%- assign decimals = row['price'] | split: "" | reverse | slice: 0,2 | reverse | join: "" -%} {%- assign units = row['price'] | split: "" | reverse | slice: 3,3 | reverse | join: "" -%} {%- assign thousands = row['price'] | split: "" | reverse | slice: 6,3 | reverse | join: "" -%} {%- assign millions = row['price'] | split: "" | reverse | slice: 9,3 | reverse | join: "" -%} {%- if millions != "" -%} ${{millions}},{{thousands}},{{units}}.{{decimals}} {%- elsif thousands != "" -%} ${{thousands}},{{units}}.{{decimals}} {%- else -%} ${{units}}.{{decimals}} {%- endif -%} ``` To explain this expression: * [`split`](https://shopify.github.io/liquid/filters/split/): converts the value in the `row['price']` model column (either `number` or `string`) to an array * [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the array * [`slice`](https://shopify.github.io/liquid/filters/slice/): isolates the items in groups of threes * [`reverse`](https://shopify.github.io/liquid/filters/reverse/): reverses the order of items in the groups * [`join`](https://shopify.github.io/liquid/filters/join/): transforms the single groups of items back into strings The second part is a [conditional](#conditional-output) that displays the currency amount in the correct format. > [!NOTE] > > If your model column contains currency amounts without decimal places, make > sure to remove the `decimals` variable from the Liquid expression and update > the `slice` functions accordingly. #### String functions See [Liquid String Functions](https://shopify.dev/api/liquid/filters/string-filters) for a complete list of string functions. In Hightouch, the `capitalize`, `strip`, and `append` functions are some of the most commonly used: * Use the `capitalize` function to capitalize the first letter of a string: ``` {{ row['title'] | capitalize }} ``` * Use the `strip` function to remove all whitespace from the left and right sides of a string: ``` {{ row['title'] | strip }} ``` * Use the `append` function to add a file extension to a string: ``` {{ row['file_name'] | append: '.png' }} ``` #### Isolate parts of a string You can use the Liquid [`split` function](https://shopify.github.io/liquid/filters/split/) to isolate a certain part of a string. For example, suppose you have a model column that returns URLs, including query parameters. An example value may look like this: `www.example.com/?gclid-123`. You may want to send the base URL (`www.example.com/`) as one field and one of the query parameters as another field. Query parameters often include click IDs, such as the `gclid` or `fbclid` that you should include on events you send to [Google Ads](https://docs.hightouch.com/destinations/google.md#conversion-events) or [Meta Conversions](https://docs.hightouch.com/destinations/meta-conversions.md#syncing-events) (or [Meta Offline Conversions](https://docs.hightouch.com/destinations/meta-offline#syncing-events)) respectively. For the example value `www.example.com/?gclid-123`, this Liquid snippet would return the base URL, `www.example.com/`: ``` {{ row['column_name'] | split: "?gclid-" | first }} ``` And this Liquid snippet would return the `gclid` value, `123`: ``` {{ row['column_name'] | split: "?gclid-" | last }} ``` #### Date function If your destination expects dates in a different format than your model returns them, you can use the Liquid [`date` function](https://shopify.github.io/liquid/filters/date/) to reformat them. For example, this Liquid snippet ensures that dates from the column are always in the `DD/MM/YYYY` format: ``` {{ row['column_name'] | date: '%d/%m/%Y' }} ``` And this snippet always returns dates in the `MM/DD/YYYY` format: ``` {{ row['column_name'] | date: '%m/%d/%Y' }} ``` You can also include other parameters to include time and timezone data, by adding `%H:%M:%S` and `%z`: ``` {{ row['column_name'] | date: '%s' | date: '%Y-%m-%dT%H:%M:%S%z' }} ``` You can use Liquid to change a timestamp's time and date values. For example, you may want to send a date one day earlier than the original. This snippet subtracts 24 hours (represented as 86,400 seconds) from the original date in the column: ``` {{ row['column_name'] | date: '%s' | minus: 86400 | date: '%Y-%m-%dT%H:%M:%S' }} ``` To add time to a date, replace `minus` with `plus`. For more examples of how to transform dates and times, refer to this [documentation](https://support.sweethawk.com/hc/en-us/articles/4412143470221-Liquid-markup-for-date-and-time#index_3). #### Construct a static array You can sync a static array value to your destination by using this Liquid snippet: ``` {%- assign value = '[' | append: '"value1","value2"' | append: ']' -%} {{ value | parse }} ``` You can replace `'"value1","value2"'` with the values you wish to insert in your static array. #### Sync an empty array You can sync an empty array to your destination by using this Liquid snippet: ``` {{ '[]' | parse }} ``` If you decide to use this with [Liquid conditionals](#conditional-output) or other complex syntax, make sure to [use a hyphen](https://shopify.github.io/liquid/basics/whitespace/) in your tag syntax to strip whitespace. #### Convert an array to a string You can convert an array to a string by using the [`join`](https://shopify.github.io/liquid/filters/join/) function: ``` {{ row['items'] | join: "" }} ``` #### JSON function You can convert an object to a JSON-formatted string by using this Liquid snippet: ``` {{ row['invoice'] | json }} ``` You can also use this function to escape special characters when creating a JSON payload. For example, if you attempt to map this value, `This is the "value" in the column` in the `special_characters` column, it will fail: ``` "column_mapping" : "{{ row.special_characters }}" ``` To successfully sync this value, select the `json` function and use the example format below: ``` "column_mapping" : {{ row.special_characters | json }} ``` #### Isolate object properties If your model columns contain `object` type values, you may want to sync only one or some of the object's properties to your destination. You can do this in the template mapper using the [dot notation](https://en.wikipedia.org/wiki/Property_\(programming\)#Dot_notation). Specifically, you can write a Liquid expression containing the model column's name and the property's name: `{{ row['model_column'].property }}` > [!WARNING] > > You can use dot notation for [field mapping](#map-columns-to-fields) and > [record matching](https://docs.hightouch.com/syncs/record-matching.md), but only for destinations that > support Liquid templating. > > Furthermore, dot notation doesn't work when [testing a row](https://docs.hightouch.com/syncs/overview.md#test-a-row). > To test your sync, trigger a sync run and take a look at the [live debugger](https://docs.hightouch.com/syncs/debugger.md#use-the-live-debugger). For example, suppose you had a model column named `invoice` that contains objects with this structure: ```json theme={null} { "invoice_number": "INV-001", "issue_date": "2023-06-08", "due_date": "2023-06-30", "customer": { "name": "John Doe", "email": "johndoe@example.com", "address": "123 Main St, City, State, ZIP" }, "items": [ { "description": "Product A", "quantity": 2, "unit_price": 10.99, "total_price": 21.98 }, { "description": "Product B", "quantity": 1, "unit_price": 15.99, "total_price": 15.99 } ], "subtotal": 37.97, "tax": 3.6, "total": 41.57, "currency": "GBP" } ``` You may only want to map the `total` column to a `price` field in your destination. You could do this by including the following Liquid in the template mapper: ``` {{ row['invoice'].total }} ``` You can also use this to map nested properties, for example, `customer.name`: ``` {{ row['invoice'].customer.name }} ``` If you want to sync multiple object properties individually, you can use these types of expressions for multiple mappings in your sync configuration. ![Template Mapping](https://mintcdn.com/hightouch-3c037897/L_vQfA0JQEpeGUUo/images/mapping-data/mapping-data-dot-notation.png?fit=max\&auto=format\&n=L_vQfA0JQEpeGUUo\&q=85\&s=4922354f760ae1b9ed21fe39d1b80167) Make sure you've set the model column's type as `Object / Array` in the [model's **Columns** tab](https://docs.hightouch.com/models/data-types-casting.md#casting-from-your-model-configuration). Otherwise, Hightouch syncs an `undefined` value to your destination. > [!WARNING] > > Dot notation only works with model columns that contain single objects. For > arrays of objects, you can use the [inline > mapper](https://docs.hightouch.com/syncs/mapping-data.md#inline-mapping). ## Spread JSON columns into fields If you manage many destination fields in warehouse SQL, you can sync a JSON (object) column and have Hightouch send each of its keys as a separate destination field. This saves you from mapping hundreds of fields by hand. The keys and values come directly from the column. For example, a `lead_json` column containing this object: ```json theme={null} { "first_name": "Mohan", "tier": "gold", "region": "EMEA" } ``` syncs three fields (`first_name`, `tier`, and `region`) to the destination, alongside any fields you map normally. To spread a column: 1. In the sync's mapping section, find **Spread JSON columns into fields**. This option appears on destinations that support it. 2. Select **Add JSON column** and choose the source column that holds the JSON object. 3. Save your configuration. Regular field mappings always take precedence. If you map a field explicitly and the same field also appears in a spread column, the explicit mapping wins. > [!NOTE] > > Hightouch sends each key exactly as it appears in the JSON object, so every > key must match a field name the destination accepts (after any [key or value > transforms](#transform-spread-keys-and-values)). For consistent results, make > sure every row's JSON object uses the same set of keys. ### Set precedence for multiple columns You can spread more than one JSON column into the same destination. Hightouch applies them from top to bottom. If two columns contain the same key, the lower column wins. Drag the handle to the left of a column to change its order. ### Transform spread keys and values Because Hightouch sends keys and values verbatim, each key must match the destination's field name and each value must be in the shape the destination expects. When they don't match, apply a transform to every key or every value in the column instead of reshaping data in your model. For example, your warehouse might use `snake_case` keys while the destination's API field names have no underscores, or it might wrap each value in an object like `{ "dtype": "Integer", "value": 202624 }` when the destination wants the scalar `202624`. To apply a transform: 1. Select the settings icon to the right of the JSON column. 2. Under **Key transform** or **Value transform**, choose a transform: * **Remove special characters** — removes everything except letters and numbers. For example, `SF_ME_PRC_BASE_PRICE` becomes `SFMEPRCBASEPRICE`. * **Custom Liquid template** — rewrites the key or value with a [Liquid](https://docs.hightouch.com/syncs/mapping-data.md#liquid-reference) template. Reference both sides as `{{ key }}` and `{{ value }}`. For example, `{{ key | remove: "_" | downcase }}` turns `SF_ME_PRC_BASE_PRICE` into `sfmeprcbaseprice`. Liquid output is always a string. * **Extract from object** — pulls a field out of an object-shaped value and preserves its type. Enter a dot-separated path. For example, the path `value` extracts the number `202624` from `{ "dtype": "Integer", "value": 202624 }`. If a row's value isn't an object or is missing the path, Hightouch rejects that row. This transform is available on the value side only. 3. Save your configuration. Each column applies at most one key transform and one value transform. **Remove special characters** and **Custom Liquid template** work on both sides; **Extract from object** applies to values only. > [!TIP] > > Marketo matches REST API field names without regard to case but not > punctuation. **Remove special characters** lets a `snake_case` warehouse > column line up with Marketo's field names without renaming columns in your > model. > [!WARNING] > > A transform runs for every key or value in the column on every row, so a > complex Custom Liquid template over columns with many keys can slow down > syncs. ## Verify the mapping Before you rely on the sync, confirm that each field lands with the value and shape you expect. 1. From the sync configuration page, click **Test**. 2. Choose a representative row, including one that exercises any template or inline mapping. 3. Run the test and review the request Hightouch would send. 4. Confirm each destination field holds the expected value, and that objects and arrays match the structure the destination requires. A sync test doesn't evaluate every mapping — dot notation, for example, resolves only on a real run. When a test can't confirm a value, trigger a sync and inspect the payload in the [live debugger](https://docs.hightouch.com/syncs/debugger.md), then check the destination record to confirm the values arrived. ## Field mapping updates If you edit the field mappings in your sync configuration, the sync engine automatically reprocesses the entire [model query result set](https://docs.hightouch.com/models/creating-models.md#preview-model-results) during the next sync run. Depending on the selected [sync mode](https://docs.hightouch.com/syncs/types-and-modes.md#sync-modes), Hightouch syncs the model rows detected as **added** or as **changed** by [change data capture](https://docs.hightouch.com/syncs/cdc.md) to your destination. If the sync uses either **update** or **upsert** mode, Hightouch resyncs previously synced rows (with the updated mappings) as "changed" rows. If the sync uses **insert** mode, Hightouch doesn't reprocess previously synced rows. Instead, it only syncs model rows that are detected as **added** after the field mapping change was saved in your sync configuration. ### Model column changes Most changes to your [model columns](https://docs.hightouch.com/models/creating-models.md) aren't automatically detected by sync configurations. Hightouch handles the changes differently depending on whether columns are added, renamed, or removed from your model definition, or if you change a column's data type. If you add a new column to your model, Hightouch doesn't automatically add it as a field mapping to your sync configurations. This is because Hightouch doesn't know which columns you would like to sync to the destination, and which destination fields they should be mapped to. Follow the [field mapping](#map-columns-to-fields) instructions to include new columns in your sync configuration. > [!NOTE] > > You need to [preview and save](https://docs.hightouch.com/models/creating-models.md#preview-model-results) > your model to have new columns appear in the Hightouch UI. If you rename a column in your model, all syncs that had that column mapped in their configuration fail with an `Unknown field name` error. The error message also contains the original name of the renamed column. To resolve this, remove or update the relevant field mappings in your sync configuration. > [!NOTE] > > Changes to model column names aren't detected for [**All** / > **Mirror**](https://docs.hightouch.com/syncs/types-and-modes.md#sync-modes) type syncs. If you remove a column from your model, Hightouch doesn't automatically remove it from the field mappings in the sync configuration. Make sure to remove the mapping manually. If you change a model column's [data type](https://docs.hightouch.com/models/data-types-casting.md), the sync engine considers this update as a [field mapping update](#field-mapping-updates) and reprocesses all model rows during the next sync run. > [!WARNING] > > As explained in the [primary key > updates](https://docs.hightouch.com/models/creating-models.md#updating-a-primary-key) section, if you > update a model's primary key by selecting a different column, you need to > trigger a [full resync](https://docs.hightouch.com/syncs/resync-reset-clear.md#full-resync) or reset change > data capture (CDC) for all syncs that use that model. Otherwise, [change data > capture](https://docs.hightouch.com/syncs/cdc.md) can't process your model data correctly, which can make > your syncs fail. ### Destination field changes If you rename a field in your destination, you need to update the mapping to this field in your sync configuration. Select the new field name from the dropdown menu and save your changes. ## Next steps * Set when the sync runs in [schedule a sync](https://docs.hightouch.com/syncs/schedule-sync-ui.md). * Resolve a type mismatch between a column and a field with [data types and casting](https://docs.hightouch.com/models/data-types-casting.md). * Check a run and troubleshoot rejected rows in [debug a sync run](https://docs.hightouch.com/syncs/debugger.md).