Documentation

‍

Note: This article covers AWS S3 Data source setup process. You can learn how to load data to AWS S3 Destination here.

How to connect Azure SQL

1. On the Data sources page, click on the Azure SQL tile.
   You can find it using the search field or just open the SQL category.

2. On the Azure SQL authorization page, fill in the following fields:
3. ~Enter the Host.
4. ~Enter the Port.
5. ~Enter the UID.
6. ~Enter the Password.
7. ~Enter the Database Name.
8. ~Enter the Schema.
9. Make sure you have the following permissions:

4. Whitelist the following IP addresses for your cluster:

• ~Main Cluster: {% dropdown-button name="main" %}

{% dropdown-body name="main" %}

• 34.226.37.150
• 18.213.72.135
• 54.146.15.122
• 3.86.170.178
• 23.21.191.65

{% dropdown-end %}

• ~US Cluster: {% dropdown-button name="us" %}

{% dropdown-body name="us" %}

• 34.226.37.150
• 23.21.24.78
• 3.228.239.24
• 52.44.89.42
• 52.45.121.236

{% dropdown-end %}

• ~EU Cluster: {% dropdown-button name="eu" %}

{% dropdown-body name="eu" %}

• 34.226.37.150
• 3.64.235.5
• 3.68.248.254
• 3.68.63.131

{% dropdown-end %}

‍

5. FIll in the form and click the Authorize button.

{%docs-informer info title="Important!"%}

Due to the limitations of the ```ORDER BY``` clause in Azure SQL, you must be careful with the choice of breakdowns such as ```text``` and ```ntext```, as it will become more expensive due to the ```CAST``` operation (e.g, for ```text``` type, we’ll cast it to ```text -> varchar(max)```). Please carefully consider the performance implications and potential alternatives before including these breakdowns in your extraction.

{%docs-informer-end%}

How to extract data from Azure SQL

Step 1 - Select table

1. Select a table from the schema you specified during authorization.

2. Click Continue to proceed to the next step.

Step 2 - Match

1. Enter your Data Table name. Remember, you can't change this later.
2. Select Sync historical data period. This setting allows you to extract historical data from the selected date range during the initial data extraction.

3. Configure your Extraction scheduling. Allows you to set how often your data is extracted to Improvado storage.

4. Select the fields you need to import. By default, all fields are selected.

5. Check the field type. Each field in a data table can be assigned a String, Number, or Date type.
6. ~The date field is a unique date-key in the database and should be singular. You have the flexibility to choose from a wide range of date formats for this field.
7. ~If you have field with numeric values that you don't want to include in your calculations, change the type from Number to Text.

{%docs-informer info%}

You can learn more about how to work with dates in Flat Data here.

{%docs-informer-end%}

6. The Skip rule allows you to exclude empty rows and rows with specific values in a selected field:
7. ~Default to 'Do Not Skip': Ensures all data is initially included.
8. ~Multi-Select Options: Combine 'Skip: empty' and 'Skip: value' for targeted exclusions. Selecting 'Do Not Skip' disables other skip options.
9. ~Custom Rule addition: Add one custom rule; the form is disabled afterward to maintain focus
10. ~Custom Rule deletion:  Hover over and click on the red-highlighted basket icon next to the rule to exclude it.‍

‍Usage Note: the component is designed for straightforward value-based selections and does not support complex expressions like '<10'.

7. Click on the Import N fields button to continue.

Step 3 - Preview

1. Check the structure of data that will be extracted.

2. Once you've reviewed the fields, click the Continue button to confirm settings and proceed with data extraction.

Connection Details page

Overview tab

You can find your Azure SQL connection on the Connections page.

Settings tab

Here you can change Extraction scheduling and Table meta for your data extraction.

‍