Documentation

How to connect Amazon Athena

1. On the Data sources page, click on the Amazon Athena tile.
   You can find it using the search field or by just opening the SQL category.

2. On the Amazon Athena authorization page, fill in the following fields:
3. ~Enter the Amazon Account ID.
4. ~Enter the Access Key ID.
5. ~Enter the Access Secret Key.
6. ~Enter the Region.
7. ~Enter the Catalog Name.
8. ~Enter the Database name.‍‍
9. ~Enter the Workgroup name (Optional).
10. There is a sample of AWS permissions that should be granted to credentials to have successful extraction:

{
 "Version": "2012-10-17",
 "Statement": [
  {
   "Action": [
    "athena:StartQueryExecution",
    "athena:GetQueryExecution",
    "athena:GetQueryResults",
    "athena:ListTableMetadata",
    "athena:GetTableMetadata"
   ],
   "Effect": "Allow",
   "Resource": "arn:aws:athena:us-east-1::workgroup/"  # or * for all or default workgroup
  },
  {
   "Action": [
    "glue:GetTable*",
    "glue:GetDatabases",
    "glue:GetDatabase"
   ],
   "Effect": "Allow",
   "Resource": [
    "arn:aws:glue:us-east-1::catalog",
    "arn:aws:glue:us-east-1::database/",
    "arn:aws:glue:us-east-1::table//",  # if you want to share exact tables
    "arn:aws:glue:us-east-1::table//*"  # to share permissions to see all tables
   ]
  },
  {
   "Action": [
    "s3:GetBucketLocation",
    "s3:ListBucket",
    "s3:GetObject*"
   ],
   "Effect": "Allow",
   "Resource": [
    "arn:aws:s3:::",
    "arn:aws:s3:::/*"
   ]
  },
  {
   "Action": [
    "s3:GetBucketLocation",
    "s3:ListBucket",
    "s3:ListBucketMultipartUploads",
    "s3:GetObject",
    "s3:PutObject",
    "s3:AbortMultipartUpload",
    "s3:ListMultipartUploadParts",
    "s3:DeleteObject"
   ],
   "Effect": "Allow",
   "Resource": [
    "arn:aws:s3:::",
    "arn:aws:s3:::/*"
   ]
  }
 ]
}

4. Whitelist the following IP addresses for your cluster:

• ~Main Cluster: {% dropdown-button name="main" %}

{% dropdown-body name="main" %}

• 34.226.37.150
• 18.213.72.135
• 54.146.15.122
• 3.86.170.178
• 23.21.191.65

{% dropdown-end %}

• ~US Cluster: {% dropdown-button name="us" %}

{% dropdown-body name="us" %}

• 34.226.37.150
• 23.21.24.78
• 3.228.239.24
• 52.44.89.42
• 52.45.121.236

{% dropdown-end %}

• ~EU Cluster: {% dropdown-button name="eu" %}

{% dropdown-body name="eu" %}

• 34.226.37.150
• 3.64.235.5
• 3.68.248.254
• 3.68.63.131

{% dropdown-end %}

5. Fill in the form and click the Authorize button.

How to extract data from Amazon Athena

Step 1 - Select table

1. Select a table from the schema you specified during authorization.

2. Click Continue to proceed to the next step.

Step 2 - Match

1. Enter your Data Table name. Remember, you can't change this later.
2. Select Sync historical data period. This setting allows you to extract historical data from the selected date range during the initial data extraction.

3. Configure your Extraction scheduling. Allows you to set how often your data is extracted to Improvado storage.

4. Select the fields you need to import. By default, all fields are selected.

5. Check the field type. Each field in a data table can be assigned a String, Number, or Date type.
6. ~The date field is a unique date-key in the database and should be singular. You have the flexibility to choose from a wide range of date formats for this field.
7. ~If you have field with numeric values that you don't want to include in your calculations, change the type from Number to Text.

Note: You can learn more about how to work with dates in Flat Data here.

6. The Skip rule allows you to exclude empty rows and rows with specific values in a selected field:
7. ~Default to 'Do Not Skip': Ensures all data is initially included.
8. ~Multi-Select Options: Combine 'Skip: empty' and 'Skip: value' for targeted exclusions. Selecting 'Do Not Skip' disables other skip options.
9. ~Custom Rule addition: Add one custom rule; the form is disabled afterward to maintain focus
10. ~Custom Rule deletion: Hover over and click on the red-highlighted basket icon next to the rule to exclude it.‍

‍Usage Note: the component is designed for straightforward value-based selections and does not support complex expressions like '<10'.

7. Click on the Import N fields button to continue.

Step 3 - Preview

1. Check the structure of data that will be extracted.

2. Once you've reviewed the fields, click the Continue button to confirm settings and proceed with data extraction.

Connection Details page

Overview tab

You can find your Amazon Athena connection on the Connections page.

Settings tab

Here you can change Extraction scheduling and Table meta for your data extraction.

‍