Learn how to use AWS IAM roles to allow Firebolt to access your data lake in Amazon S3.
<bucket>
and <prefix>
with the actual bucket name and path prefix.
_firebolt-s3-access_
.
CREDENTIALS
clause to enable secure data ingestion.
arn:aws:iam::123456789012:role/your_role_name
. Use the ARN value in the Firebolt CREDENTIALS
clause as the AWS_ROLE_ARN
, as shown in the following sections.COPY FROM
statement, the Firebolt Load Data wizard, or an external table definition. The following sections explain how to configure IAM roles for each method.
COPY FROM
COPY FROM
statement. If you specified an external ID, make sure to specify it in addition to the role ARN. When you use the COPY FROM
statement to load data from your source, Firebolt assumes the IAM role to obtain permissions to read from the location specified in the COPY FROM
statement.
For a step-by-step guide, see The simplest COPY FROM workflow.
Example
The following code example loads data from a CSV file in an Amazon S3 bucket into the tutorial
table in Firebolt, using an AWS IAM role for authentication, treating the first row as a header, and automatically creating the table if it does not exist:
external_id
in the CREDENTIALS
of the CREATE EXTERNAL TABLE
statement. Firebolt assumes this IAM role when using an INSERT INTO
statement to load data into a fact or dimension table.
Example
The following code example creates an external table which maps to Parquet files stored in an Amazon S3 bucket, using an AWS IAM role for access, and extracts partition values for c_type
from the file path based on a specified regex pattern:
COPY FROM
, external tables, read TVFs) include both URL
and PATTERN
components. COPY FROM
and external tables manage them through separate URL
and PATTERN
parameters. Read TVFs use URL
parameters with optional glob patterns. For example, if passing url = 's3://foo/bar/*parquet'
, then foo/bar/
becomes the URL and *parquet
becomes the pattern.
When listing files from S3, only the URL
portion is sent in the AWS request. The pattern is later applied by Firebolt to filter the results.
So, if your IAM policy restricts access using a s3:prefix
condition, ensure that your IAM policy’s prefix condition matches the URL
parameter of your COPY FROM
query, external table definition, or read TVF. If you require broader access permissions, you may need to remove the condition entirely.