Documentation

Note: This article covers AWS S3 Data source setup process. You can learn how to load data to AWS S3 Destination here.

How to set up AWS S3 integration

1. On the Data sources page, click on the AWS S3 tile. You can find it using the search field or just open the Flat Data category.

2. On the AWS S3 authorization page enter the Connection name. It’s necessary for differentiating between different AWS S3 connections within your workspace.
3. Fill in the form and click the Create button.

Step 1. Import page

1. Click to select a file from your computer, or drag and drop your file to this area

{%docs-informer info title="Note"%}

• Supported file formats: CSV, TSV, XLS, XLSX
• Maximum size: 10 GB
• A file must begin with column headers starting in column A.

{%docs-informer-end%}

2. After picking the file a processing bar will show the upload progress.
3. ~You can click on the “trash bin” icon to cancel the uploading process of a file
4. After your file has been uploaded and checked for security, let’s move to the settings.
5. Click on the Continue button.

Step 2. Match page

1. Select the fields you need to import. By default, all fields are selected.
2. Check the type. Each field in a data table can be assigned a text, number, or date type.
3. ~The date field is a unique date-key in the database and should be singular. Also, you have the flexibility to choose from a wide range of date formats for this field.
4. ~If you have field with numeric values that you don't want to include in your calculations, change the type from Number to Text.

{%docs-informer info title="Note"%} You can learn more about how to work with dates in Flat Data here. {%docs-informer-end%}

4. The skip rule allows you to exclude empty rows and rows with specific values in a selected field
5. ~Default to 'Do Not Skip': Ensures all data is initially included.
6. ~Multi-Select Options: Combine 'Skip: empty' and 'Skip: value' for targeted exclusions. Selecting 'Do Not Skip' disables other skip options.
7. ~Custom Rule addition: Add one custom rule; the form is disabled afterward to maintain focus
8. ~Custom Rule deletion:  Hover over and click on the red-highlighted basket icon next to the rule to exclude it.‍

‍Usage Note: the component is designed for straightforward value-based selections and does not support complex expressions like '<10'.

6. Click on the Import N fields button to continue.

Step 3. Preview page

1. Check the Data Structure Preview.
2. ~In this step, you see the first row of data that will be extracted after you click the Continue button.

2. Click the Continue button to confirm settings and proceed with data extraction.
3. You'll see your imported file on the Connections page.

Connection Details page

You’ll be redirected to the Settings tab of the Connection. Here, you can see your AWS S3 instance credentials with an option to regenerate your AWS S3 password.

‍Note: You might see loading in the Connection settings section, that means that your AWS S3 instance is still being created. It might take some time.

Order Details page

Uploaded files

Here you can see the list of files uploaded to the AWS S3 Folder path corresponding to the Data table name.

‍Note: File retention for AWS S3 integration is two weeks. Files uploaded more than two weeks ago are deleted.

Settings tab

Here you can copy your Folder path and Table meta for your data extraction.