Skip to main content

Connecting S3

This assumes you have already created an S3 bucket and have a pipeline setup to move Parquet, JSON, or CSV files into it.

You will need to provide both an AWS Access Key ID and AWS Secret Access Key for authentication. The IAM user associated with the credentials must be granted read and list permissions for the bucket and its objects.

Create a Policy​

  1. Log in to your Amazon AWS account and open the IAM console.
  2. In the IAM dashboard, select Policies, then click Create Policy.
  3. Select the JSON tab, then paste the following JSON into the Policy editor (be sure to substitute in your bucket name):
{
    "Version": "2012-10-17",
    "Statement": [
        {
        "Effect": "Allow",
        "Action": [
                "s3:GetObject",
                "s3:ListBucket"
        ],
        "Resource": [
                "arn:aws:s3:::{your-bucket-name}/*",
                "arn:aws:s3:::{your-bucket-name}"
        ]
        }
    ]
}
  1. Give your policy a descriptive name, then click Create policy.
  2. In the IAM dashboard, click Users. Select an existing IAM user or create a new one by clicking Add users.
  3. If you are using an existing IAM user, click the Add permissions dropdown menu and select Add permissions. If you are creating a new user, you will be taken to the Permissions screen after selecting a name.
  4. Select Attach policies directly, then find and check the box for your new policy. Click Next, then Add permissions.
  5. After successfully creating your user, select the Security credentials tab and click Create access key. You will be prompted to select a use case and add optional tags to your access key. Click Create access key to generate the keys.

Make note of the following information:

  1. Bucket name
  2. Access Key ID
  3. AWS Secret Access Key
  4. File extension

Follow instructions to share your data source configuration with Motif.