Connecting to AWS S3 Storage

Amazon S3 is AWS’s object storage solution for the cloud. Object storage is optimized for storing massive amounts of unstructured data.

AWS S3 Permissions

Connect Publisher to AWS S3 storage to access and share your data. The configuration process involves setting up connection details and choosing data update preferences.

Prerequisites

Before starting, gather these required details:

• S3 bucket name

• Bucket region

• AWS Access Key ID

• AWS Secret Access Key

Configuration Process

Initial Setup

From the Publisher interface, select "Data Sources" in the navigation bar and click the "Connect Data Source" button. Choose "AWS S3 Object Store" from the available connectors.

Connection Details

Enter your AWS S3 configuration:

• Storage Bucket name

• Access Key ID (or "Anonymous" for public access)

• Secret Access Key (or "Anonymous" for public access)

• Bucket Region (e.g., us-east-1)

Note: For anonymous access to public buckets, use "Anonymous" for both the Access Key ID and Secret Access Key.

For more information about creating AWS access keys, consult AWS documentation.

Data Update Configuration

Configure how Publisher monitors your S3 bucket for changes:

• S3 Event Notifications with SQS:

  • Create a dedicated SQS queue for S3 bucket notifications
  • If not already done, grant Amazon S3 permissions to publish messages to SQS.
  • Provide the SQS queue name

• Scheduled Updates:

  • Choose an update interval (30 seconds to daily)

  • Updates can be configured later if needed

Finalizing Setup

Provide a descriptive name for your data source. After connection, you have the option to create and share a collection from this data source.