> For the complete documentation index, see [llms.txt](https://docs.omics.ai/products/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.omics.ai/products/workbench/instruments/connecting-a-storage-account.md).

# Connecting a Storage Account

When a storage account is connected, Workbench indexes the directories and identifies the samples and files. The following sections detail how to set up storage accounts for different providers.

1. [AWS](#aws-s3)
2. [Azure Storage Account](#azure-storage-account)
3. [Google Cloud Storage](#google-cloud-storage)

## General Setup

1. Log in to your [Workbench account](https://workbench.omics.ai)
2. Click on **Data** from the left navigation panel
3. Click the **Connect Data** button
4. Continue onto the section that corresponds to the cloud you are configuring

Each provider form ends with an optional **Automatic Reindexing** section. See [Keeping samples up to date](#keeping-samples-up-to-date) below before deciding whether to fill it in.

***

## AWS S3

The following guide outlines the key concepts and minimal configuration needed to establish a secure connection between Instruments and your AWS S3 resources.

### Connect AWS S3

1. Select **Amazon AWS** as the provider.
2. Enter your **Access Key ID** and **Secret Access Key**.
3. Specify the **Bucket** (for example `s3://storage-bucket`) and **Region**.
4. Optionally, enter an **Event Queue URL** under **Automatic Reindexing**. See [Automatic reindexing on AWS](#automatic-reindexing-on-aws).
5. Click **Save** to connect the storage account.

### Required Permissions

To use AWS S3 with Workbench, the following are required:

* `s3:ListBucket`: To list objects within the bucket.
* `s3:GetObject`: To read object contents.

#### Example IAM Policy

Below is an IAM policy that grants the minimum required permissions:

```json
{
  "Version": "2012-10-17",
   "Statement": [
        {
            "Sid": "WorkbenchInstrumentsStorageConnector",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKET_NAME",
                "arn:aws:s3:::BUCKET_NAME/*"
            ]
        }
    ]
}
```

Replace **BUCKET\_NAME** with your actual S3 bucket name.

If you need to restrict access to a specific folder or prefix within your bucket, you can modify the resource path:

```json
"Resource": [
  "arn:aws:s3:::BUCKET_NAME/prefix/*"
]
```

### Generating an Access Key

1. Log in to the [AWS Management Console](https://aws.amazon.com/console/).
2. Navigate to **IAM** (Identity and Access Management).
3. Select **Users** and choose the user for whom you want to create an access key.
4. Go to the **Security Credentials** tab and click **Create Access Key**.
5. Copy the **Access Key ID** and **Secret Access Key** to use in Workbench.

{% hint style="info" %}
For more details, refer to the [AWS Documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html).
{% endhint %}

### Automatic reindexing on AWS

The **Event Queue URL** field enables near-real-time reindexing through S3 Event Notifications. To set it up:

1. Create an SQS queue in the **same region** as the bucket.
2. On the bucket, add an **Event Notification** filtered to `s3:ObjectCreated:*` and `s3:ObjectRemoved:*` events, with the SQS queue as its destination.
3. Grant the IAM user whose access key you entered above permission to **receive and delete messages** on the queue (`sqs:ReceiveMessage` and `sqs:DeleteMessage`).
4. Paste the queue's URL into **Event Queue URL**.

Workbench reads the queue with the same **Access Key ID** and **Secret Access Key** as the bucket.

***

## Azure Storage Account

The following guide outlines the key concepts and minimal configuration needed to establish a secure connection between Instruments and your Azure resources.

### Using OAuth2

1. Select **Microsoft Azure** as the provider.
2. Under **Method**, choose **OAuth 2.0**.
3. Enter your **Tenant ID**, **Client ID**, and **Client Secret**.
4. Provide the **Storage Account Name** and **Storage Container Name**.
5. Optionally, fill in **Automatic Reindexing**. See [Automatic reindexing on Azure](#automatic-reindexing-on-azure).
6. Click **Save**.

The **Method** dropdown offers three authentication methods: **OAuth 2.0**, **Shared Access Signature**, and **Access Key**.

### Required Permissions

To use Azure Storage with Workbench, the following permissions are required:

* `Storage Blob Data Reader`: To list and read blob data within the storage account.

### Creating a Service Principal

1. Log in to the [Azure Portal](https://portal.azure.com/).
2. Navigate to **Azure Active Directory > App Registrations > New Registration**.
3. Provide a name for the application and click **Register**.
4. Go to the **Manage** > **Certificates & Secrets** tab and create a **Client Secret**.
5. Navigate to your Storage Account and open the **Access Control (IAM)** tab.
6. Click **Add Role Assignment**, select **Storage Blob Data Reader**, and assign it to the newly created service principal.

{% hint style="info" %}
For detailed steps, refer to the [Azure Documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/authorize-access-azure-active-directory).
{% endhint %}

### Using SAS Token

1. Select **Microsoft Azure** as the provider.
2. Under **Method**, choose **Shared Access Signature**.
3. Enter the **SAS URL**.
4. Specify the **Storage Account Name** and **Storage Container Name**.
5. Click **Save**.

### Generating a SAS Token for a Container

1. Log in to the [Azure Portal](https://portal.azure.com/).
2. Navigate to your **Storage Account** > **Storage Browser** and go to **Blob Containers** section.
3. Locate the desired blob container in the table.
4. In the last column of the table, click on the ellipsis "`...`" to open the context menu.
5. Click on the **Shared Access Signature** option.
6. Specify the permissions required (e.g., **Read** and **List**).
7. Set the **Start** and **Expiry** time for the token.
8. Click **Generate SAS Token and URL**.
9. Copy the generated **SAS Token URL**.

{% hint style="info" %}
Note: SAS tokens can expire. Ensure the token's validity matches the intended usage period
{% endhint %}

### Using Access Key

1. Select **Microsoft Azure** as the provider.
2. Under **Method**, choose **Access Key**.
3. Enter the **Access Key**.
4. Specify the **Storage Account Name** and **Storage Container Name**.
5. Click **Save**.

### Retrieving the Storage Account Access Key

Follow these steps to retrieve the access key for your Azure Storage Account:

1. Log in to the [Azure Portal](https://portal.azure.com/).
2. Navigate to your **Storage Account**.
3. Under **Security + Networking**, select **Access keys**.
4. Copy the **Key1** or **Key2** under the **Key** section.
5. Use the copied key as the **Access Key** in Workbench.

{% hint style="info" %}
For additional details, visit the [Azure Storage Account Key Documentation](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage).
{% endhint %}

The service account is now ready to list and read objects from the specified bucket.

### Automatic reindexing on Azure

The **Automatic Reindexing** section of the Azure form is available for all three authentication methods. It enables near-real-time reindexing through Azure Event Grid. To set it up:

1. Create an **Azure Storage Queue**.
2. On the storage account, create an **Event Grid event subscription** filtered to `Microsoft.Storage.BlobCreated` and `Microsoft.Storage.BlobDeleted` events for this container, with the Storage Queue as its destination.
3. In Workbench, enter the queue's name in **Event Queue Name** and its **Connection String**.

***

## Google Cloud Storage

### Steps to Connect GCS

1. Select **Google Cloud** as the provider.
2. Upload your **Service Account JSON**.
3. Enter the **Project ID** and **Bucket** (for example `gs://storage-bucket`).
4. Optionally, enter a **Pub/Sub Subscription Name** under **Automatic Reindexing**. See [Automatic reindexing on GCP](#automatic-reindexing-on-gcp).
5. Click **Save** to connect the storage account.

The Service Account JSON file can be at most 1 MB.

### Required Permissions

To properly connect Google Cloud Storage with Workbench, the service account needs the following permissions:

* `storage.buckets.list` - To list buckets
* `storage.objects.get` - To read object contents
* `storage.objects.list` - To list objects within buckets
* `serviceusage.services.use` - To use GCP services

### Creating a Service Account with Required Permissions

#### Option 1: Using Built-in Roles

The easiest approach is to assign two built-in roles to your service account:

* **Storage Admin** (`roles/storage.admin`) - Provides full control of storage resources
* **Service Usage Consumer** (`roles/serviceusage.serviceUsageConsumer`) - Allows service usage

1. Log in to the [Google Cloud Console](https://console.cloud.google.com/).
2. Navigate to **IAM & Admin** > **Service Accounts**.
3. Click **Create Service Account**.
4. Enter a name and description for the service account, then click **Create and Continue**.
5. Add the **Storage Admin** role to grant storage permissions.
6. Add the **Service Usage Consumer** role.
7. Click **Done** to finish creating the service account.
8. Select the newly created service account from the list and navigate to the **Keys** tab.
9. Click **Add Key** > **Create New Key**.
10. Choose the **JSON** format and click **Create**.
11. Save the downloaded JSON key securely for use in Workbench.

#### Option 2: Using a Custom Role

If you prefer more granular control, you can create a custom role with only the required permissions:

1. Navigate to **IAM & Admin** > **Roles**.
2. Click **Create Role**.
3. Enter a name and description for the role.
4. Click **Add Permissions** and add these exact permissions:
   * `storage.buckets.list`
   * `storage.objects.get`
   * `storage.objects.list`
   * `serviceusage.services.use`
5. Click **Create** to create the custom role.
6. Go back to **IAM & Admin** > **Service Accounts**.
7. Create a new service account or select an existing one.
8. Assign your newly created custom role to this service account.
9. Generate and download a JSON key as described above.

### Granting Bucket-Specific Access

If you want to limit access to a specific bucket only:

1. Navigate to the **Cloud Storage** section in the Google Cloud Console.
2. Select the bucket you want the service account to access.
3. Go to the **Permissions** tab and click **Grant Access**.
4. Enter the service account email in the **New Principals** field.
5. Assign only the **Storage Admin** role (or your custom role) to the service account at the bucket level.
   * Note: You do not need to add the Service Usage Consumer role at the bucket level.
6. Click **Save** to apply the permissions.

The service account is now ready to list and read objects from the specified bucket.

### Automatic reindexing on GCP

The **Pub/Sub Subscription Name** field enables near-real-time reindexing through GCS bucket notifications. To set it up:

1. Create a Pub/Sub topic.
2. Add a notification config on the bucket that publishes `OBJECT_FINALIZE` and `OBJECT_DELETE` events to the topic, for example:

   ```bash
   gcloud storage buckets notifications create gs://BUCKET_NAME \
     --topic=TOPIC_NAME \
     --event-types=OBJECT_FINALIZE,OBJECT_DELETE
   ```
3. Create a **pull** subscription on that topic.
4. Grant the service account whose JSON key you uploaded the `roles/pubsub.subscriber` role on the subscription.
5. Enter the subscription's name in **Pub/Sub Subscription Name**.

***

## Keeping samples up to date

Workbench discovers new samples and files by **indexing** the storage account. Indexing happens in three ways:

* **Hourly**: every storage account is reindexed automatically about once an hour. This needs no setup.
* **Automatic reindexing**: if you configured the optional **Automatic Reindexing** section for your provider, changes to the bucket or container trigger a reindex in near real time.
* **Manual**: you can start a reindex at any time from the storage account's samples page.

{% hint style="warning" %}
Without automatic reindexing, files uploaded in the last hour may not be indexed yet. A workflow launched before they're indexed won't see them. Reindex manually before launching if you've just uploaded data.
{% endhint %}

### Checking index freshness and reindexing manually

1. Click **Data** in the left navigation and select the storage account.
2. The samples page header shows the indexing status:
   * **Last indexed** followed by how long ago, for example "Last indexed 5 minutes ago".
   * **Not yet indexed** if the account has never been indexed.
   * **Indexing…** with a spinner while a reindex is running.
3. Click **Reindex** to start a reindex. A "Reindex started" message confirms it.

The page refreshes on its own every few seconds. New samples appear and the spinner clears without reloading the page.

{% hint style="info" %}
The **Reindex** button, and the icons to edit or delete a storage account, need the **Manage Storage** capability. The indexing status is visible to everyone in the workspace.
{% endhint %}

## Editing a storage account

1. Click **Data** in the left navigation and select the storage account.
2. Click the **edit** (pencil) icon in the samples page header.
3. Update the fields you need and click **Save**.

When editing:

* The location fields can't be changed: **Bucket** and **Region** (AWS), **Project ID** and **Bucket** (GCP), and **Storage Account Name** and **Storage Container Name** (Azure). To point at a different location, connect a new storage account.
* Secret fields (such as **Secret Access Key**, **Service Account JSON**, **SAS URL**, **Access Key**, **Client Secret** and **Connection String**) show a masked placeholder. Leave one blank to keep its current value.
* You can add or change the **Automatic Reindexing** settings on an existing account.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.omics.ai/products/workbench/instruments/connecting-a-storage-account.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.