BigQuery Permissions

Configuring BigQuery permissions correctly enables Publisher to securely access and query your Google Cloud data. These permissions establish the service account credentials, access roles, and project configurations needed for proper data interaction.

This guide covers both specific dataset access and project-wide configurations, explaining the core concepts and minimum permissions required for each scenario.

Concepts

• Service Account (SA): This is an identity generated in Google Cloud that can be used to interact with GCP services. Each SA has a unique email (identity) and one or more JSON keys.

• Data Project: A project that houses the target dataset.

• Quota project: A project that you would like to bill BigQuery to and consume quota resources.

• Permission: Granular actions that a user can perform on a given resource.

• Role: A collection of permissions typically needed for specific interactions against one or more resources: “Data Viewer,” “Storage Reader,” etc.

Minimal Configuration

This setup represents the minimal permissions needed to interact with a SPECIFIC BigQuery Data Source without creating custom roles. It conceptually segregates the Quota Project and the Data Project, but they can be the same project.

Creating Credentials

Create a new Service Account (SA) in any project.

For more information on creating a new service account within GCP, please refer to Google's IAM Guide.

gcloud iam service-accounts create publisher-connectors \ 
    --display-name="Publisher Data Connections"

Generate a new JSON key and download it.

gcloud iam service-accounts keys create \
  publisher-connectors-key.json \
--iam-account=publisher-connectors@${PROJECT}.iam.gserviceaccount.com

Adding Permissions to Quota Project

1. Navigate to the quota project's IAM and Admin Page.

2. Grant the following Roles to the SA you created:

   1. Service Usage Consumer — This allows the SA to consume resources.

   2. BigQuery Read Session User — This allows the SA to start a BigQuery read session with the storage API. Without this, they can interact with BQ via the normal Query interface but cannot list the table's contents through our DLCON (although they will get the metadata).

   3. BigQuery Job User — This allows the user to run jobs.

Data Project

1. Navigate to the BigQuery console.

2. Open the dataset you want to grant access to.

3. Click on “Sharing." then click on “Permissions.”

4. Add the SA from the "Creating Credentials" step above and grant it the following role.

   1. BigQuery Data Viewer — This grants the SA the ability to read metadata and table data from any table in this dataset.

Generalized Project Configuration

This setup represents the permissions needed to expose all BigQuery DataSets in a project.

Creating Credentials

1. Create a new Service Account (SA) in any project.

2. Generate a new JSON key and download it.

Adding Permissions to Project

1. Navigate to the quota project's IAM and Admin Page.

2. Grant the following Roles to the SA you created:

   1. Service Usage Consumer — This allows the SA to consume resources.

   2. BigQuery Read Session User — This allows the SA to start a BigQuery read session with the storage API. Without this, they can interact with BQ via the normal Query interface but cannot list the table's contents through our DLCON (although they will get the metadata).

   3. BigQuery Job User — This allows the user to run jobs.

   4. BigQuery Data Viewer — This grants the SA the ability to read metadata and table data from any table in this dataset.