Documentation

IMPORTANT: This article covers setup of warehouse for load data from Improvado, not customer data warehouse from which data is being extracted. This article doesn't cover setup of customer data warehouse for Data Prep as well.

Required information:

• Title
• Authentication type
• Host
• Port
• User Name
• Password or Private key file
• Folder
• ~a forward slash ( / ) means that you need to enter the root path‍
• File format
• File name
• Separator
• Partition by (the way of splitting data for uploading to the file)
• Use static IP

Authentication type

Possible parameters:

• SFTP auth by login/password
• SFTP auth by Improvado public key
• SFTP auth by private key

See the credentials options below.

Private key

Supported key format: OpenSSH. If your private key is a .ppk file, please convert it to the OpenSSH key format.

Possible SSH key algorithms:

• RSA
• Ed25519
• DSS
• ECDSA

Folder

Possible parameters:

```/{{data_source}}/{{data_table_title}}/{{report_type}}/{{filename}}/{{YYYY}}/{{MM}}/{{DD}}/{{timestamp}}```

• ```{{data_source}}``` is a data provider, integration, connector.
• ```{{data_table_title}}``` is an object that contains all extraction orders with the same granularity (dimensional schema).
• ```{{filename}}``` is the same as destination table name.
• ```{{report_type}}``` is a set of such fields as metrics, properties, dimensions, etc.
• ```{{timestamp}}``` is the date and time when data load started

If you use ```/{{YYYY}}/{{MM}}/{{DD}}``` settings, the data will be added to folders daily. Each new record will not delete the previous one, even for data that contains no date.

File format

Possible formats:

• csv
• csv+gzip
• json
• json+gzip
• parquet

File name

Possible parameters:

```{{filename}}-{{YYYY}}-{{MM}}-{{DD}}```

• ```{{filename}}``` is the same as destination table name

IMPORTANT: you cannot use {{DD}} for partition by month

• ```{{filename}}-{{YYYY}}-{{MM}}-{{DD}}``` – for partition by day
• ```{{filename}}-{{YYYY}}-{{MM}}``` – for partition by month

Also, you can use “_” instead of “-” or do not use any symbols at all, for example:

• ```{{filename}}_{{YYYY}}-{{MM}}-{{DD}}```
• ```{{filename}}{{YYYY}}{{MM}}{{DD}}```

Separator

Possible delimiters that can separate data in your file:

• comma
• semicolon
• tab

Partition by

Possible ways of splitting data:

• Day
• Month

Use static IP

Select Yes for Use static IP option if you allow Improvado to connect your database by the static IPs mentioned on the Destination connection page.

Select No if you have permitted access to your database from any IP. In this case, Improvado will connect your database using dynamic IPs not listed on the Destination connection page.

How to provide credentials (3 options)

Option 1

If you choose “SFTP auth by login/password” auth type, you must provide the following information:

• Host
• Port
• User Name
• Password

Option 2

If you choose “SFTP auth by Improvado public key” auth type, you must provide the following information:

• Host
• Port
• User Name

Add Improvado SSH public key to your SFTP server (in ```~/.ssh/authorized_keys``` file).

You can copy Improvado public SSH key on the right-hand side of the SFTP connection page.

Option 3

If you choose “SFTP auth by private key” auth type, you must provide the following information:

• Host
• Port
• User Name
• Private key
• Password (if your private key has a passphrase)

Upload your SSH private key generated by following command, e.g.:

ssh-keygen -t rsa -b 4096