Documentation

Note: Azure Blob Storage is currently supported only as a Destination. This guide doesn’t cover the DataPrep setup for Azure Blob Storage.

Improvado no longer supports Azure Blob on static IPs.

Description

‍Azure Blob Storage is a scalable cloud storage service by Microsoft Azure, designed to store large amounts of unstructured data like text or binary data.

Follow our setup guide to connect Azure Blob Storage to Improvado.

Complete configuration

On the Azure Blob Storage connection page, fill in the following fields:

Step 1. Enter a name for your Destination connection in the Title.

Step 2. Choose authorization option:

{%docs-accordion title="SAS Token"%}

• Enter the Account URL. {%dropdown-button name="account-url-sas"%}

{%dropdown-body name="account-url-sas"%}

Account URL must satisfy the following regular expression: ```https://[a-z0-9]*.blob.core.windows.net$```.

The ```[a-z0-9]``` part of Account URL must be between 3 and 24 characters in length.

{%dropdown-end%}

• Enter the SAS Token. {%dropdown-button name="sas-token"%}

{%dropdown-body name="sas-token"%}

Learn how to create a SAS Token with this guide.

{%dropdown-end%}

{%docs-accordion-end%}

{%docs-accordion title="Service Principal"%}

{%docs-informer info title="Note"%}

Make sure your account has the Storage Blob Data Contributor role.

Learn how to create a Service Principal account in the official guide.

{%docs-informer-end%}

• Enter the Account URL. {%dropdown-button name="account-url-principal"%}

{%dropdown-body name="account-url-principal"%}

Account URL must satisfy the following regular expression: ```https://[a-z0-9]*.blob.core.windows.net$```.

The ```[a-z0-9]``` part of Account URL must be between 3 and 24 characters in length.

{%dropdown-end%}

• Enter the Tenant ID. Learn how to find your Tenant ID here.
• Enter the Client ID.
• Enter the Client Secret.

{%docs-accordion-end%}

Step 3. Enter the Container Name. {%dropdown-button name="container-name"%}

{%dropdown-body name="container-name"%}

Container Name length must be between 3 and 63 characters and must satisfy the following regular expression: ```r'^(?!.*--.*)[a-z0-9][a-z0-9]*[a-z0-9]$’```

{%dropdown-end%}

Step 4. Select the necessary Encryption type option from the dropdown. {%dropdown-button name="encryption-type"%}

{%dropdown-body name="encryption-type"%}

Possible options:

• No encryption (default cloud storage encryption is still enabled)
• Customer-provided keys

{%dropdown-end%}

Step 5. (Customer-provided keys only) Enter the Encryption key. {%dropdown-button name="encryption-key"%}

{%dropdown-body name="encryption-key"%}

If you have selected the Default Cloud Storage encryption type, you will not be able to edit this field.

Otherwise, you should enter your AES-256 key, encoded in standard Base64 or resource name of Cloud KMS key used to encrypt the blob’s contents. For more info, see Azure Blob Storage encryption docs.

{%dropdown-end%}

Step 6. Enter the Folder. {%dropdown-button name="folder"%}

{%dropdown-body name="folder"%}

Possible parameters:

```/data_source/data_table_title/report_type/YYYY/MM/DD/timestamp```

• ```data_source``` is a data provider, integration, connector
• ```data_table_title``` is an object that contains all extraction orders with the same granularity (dimensional schema)
• ```report_type``` is a set of such fields as metrics, properties, dimensions, etc.
• ```timestamp``` is the date and time when data load started

If you use ```/YYYY/MM/DD``` settings, the data will be added to folders daily. Each new record will not delete the previous one, even for data that contains no date.

The maximum length is 254 characters.

{%dropdown-end%}

Step 7. Select the necessary File format option from the dropdown. {%dropdown-button name="file-format"%}

{%dropdown-body name="file-format"%}

Possible formats:

• csv
• csv+gzip
• json
• json+gzip
• parquet
• avro

{%dropdown-end%}

Step 8. Enter the Filename. {%dropdown-button name="filename"%}

{%dropdown-body name="filename"%}

Possible parameters:

```filename-YYYY-MM-DD```

• ```filename``` is the same as destination table name

Note: you cannot use ```DD``` for partition by month.

• ```filename-YYYY-MM-DD``` – for partition by day
• ```filename-YYYY-MM``` – for partition by month

Also, you can use “_” instead of “-” or do not use any symbols at all, for example:

• ```filenameYYYY-MM-DD```
• ```filenameYYYYMMDD```

{%dropdown-end%}

Step 9. Select the necessary Separator option from the dropdown. {%dropdown-button name="separator"%}

{%dropdown-body name="separator"%}

Possible delimiters that can separate data in your file:

• comma
• semicolon
• tab

{%dropdown-end%}

Step 10. Select the necessary Partition by option from the dropdown. {%dropdown-button name="partition-by"%}

{%dropdown-body name="partition-by"%}

Possible ways of splitting data:

• Day
• Month

{%dropdown-end%}

Step 11. Select the necessary Use binary strings option from the dropdown. {%dropdown-button name="use-binary-strings"%}

{%dropdown-body name="use-binary-strings"%}

The "Use binary strings" determines whether string fields are encoded in binary format when loading data.

Recommended setting: "No" (default).

{%dropdown-end%}

‍