Share
Authentication for GCP-hosted Services Setup Guide

Google Cloud Platform (GCP) is a set of computing, networking, storage, big data, machine learning, and management services from Google that provide access to Google Cloud resources such as computers and hard disk drives, and virtual resources such as virtual machines (VMs). Read Set up the Google Cloud Platform if you are new to GCP.

Any Google Cloud resource you use or allocate must belong to a project. A project organizes your resources, manages permissions, enables APIs, and so on. For example, creating a project enables BigQuery APIs, which allow you to create datasets to store data in BigQuery.

Google’s Identity and Access Management (IAM) enables you to provide access to Google Cloud resources, such as Cloud Storage buckets and projects to authorized users (called principals), which may be a user account, a service account (for applications and compute workloads), a Google group, a Google Workspace account, or a Cloud Identity domain. IAM allows you to authorize principals through roles, which are a set of permissions. For example, the Project Creator role contains the resourcemanager.projects.create permission that allows a principal to create a project. The assigned roles can be modified later.

Prerequisites

  • An active Google user account or service account. Refer to the sections, User accounts or Service accounts if you want to add the user account or service account to your project.

  • The user has the iam.serviceAccountAdmin privileges, for creating service accounts. Read Required roles for the roles needed to manage service accounts. Refer to the section, Creating a Google service account if you do not already have a service account.

  • The user has the resourcemanager.projectIamAdmin privileges, for adding and editing roles for any account.

  • The user has an Owner role in the Google Cloud Platform, for creating projects and datasets.

  • The user has the appropriate role for accessing the Google Cloud Storage (GCS) resources for the required GCP project. You can assign the Storage Admin role or create a custom role and assign it.

  • The user has the appropriate role for connecting to the GCP-hosted service.

Authentication Using User Accounts

To connect to GCP-hosted services such as BigQuery using user accounts, you must add the user accounts to the project and assign the appropriate roles to access the GCP-hosted service linked to the project. In order to do this, you must sign in to your Google account in Hevo and authorize Hevo to access your data.

Adding a user account to a project

Perform the following steps to add a Google user account to a project:

  1. Log in to your Google Cloud Platform with an Owner Role.

    Note: Click here and check the role against your name. If you are not an Owner, contact your account Owner to perform the following steps.

  2. In the left navigation pane, click IAM & Admin, and then click IAM.

    Click IAM

  3. Click on the project drop-down.

    Select project

  4. Select the Name of the project in which you want to add the user account.

    Select project name

  5. Click + ADD in the IAM & Admin page.

    Add new user

  6. Add the following details of the new user:

    New user details

    • New principals: The email ID(s) of the user(s).

    • Role: The roles you want to assign to the user(s). For example, you need to assign the Storage Admin and BigQuery Admin roles to a user account, to allow Hevo to load data to your BigQuery Destination.

  7. Click SAVE.

Modifying roles for an existing user account

You can modify the roles for existing user accounts in a project to assign the appropriate roles for accessing the required data.

To do this:

  1. In the IAM & Admin, IAM page:

  2. Click the PERMISSIONS tab.

  3. Click the Edit icon for the user to whom the roles are to be assigned.

    Edit an existing user

  4. In the Role drop-down, select the required role for the account. This could be one of the predefined roles or a custom role. For example, select Cloud Storage in the left pane and then select the Storage Admin role. Hevo requires this role for creating a GCS bucket.

    Select a role

  5. If you want to assign more roles to the user, click + ADD ANOTHER ROLE. For example, for connecting to BigQuery APIs, select BigQuery in the left pane and then, select BigQueryAdmin.

    Add another role

  6. Click SAVE.

Authentication using Google Service Accounts

As a user with the Owner role, you can create service accounts for your team members to access the GCP-hosted services such as BigQuery. Once you have done this, you must assign the appropriate roles to your service account and download the key file in JSON format, which is used to authenticate Hevo while configuring BigQuery as a Destination or a Source.

(Optional) Creating a Google service account

If you have created a service account, you can skip to the next section to assign the appropriate roles for Hevo to access your data.

Perform the following steps to create a Google service account:

  1. Log in to your Google Cloud Platform with an Owner Role.

    Note: Click here and check your assigned role. If you are not an Owner, contact your account owner to perform the following steps.

  2. In the left navigation pane, click IAM & Admin.

  3. Under IAM & Admin, click Service Accounts.

  4. Select the project in the drop-down at the top of the page.

  5. Click + CREATE SERVICE ACCOUNT.

    Create a service account

  6. In the Create service account page, under the Service account details section, enter a Service account name and a description, and click CREATE AND CONTINUE.

    Enter service account details

The Service account ID is automatically created based on the service account name.

Assigning roles to a Google service account

Perform the following steps to assign roles to new and existing service accounts:

  • For an existing service account:

    1. In your GCP console, under IAM & Admin, click IAM.

    2. In the PERMISSIONS tab, check the Role column to see the assigned roles.

      View your assigned roles

    3. If the required roles are not assigned, click the Edit icon for the service account.

    4. In the Edit permissions dialog that opens up, click + ADD ANOTHER ROLE to add a new row.

      New roles for service account

    5. Select the required role from the Role drop-down.

    6. Click SAVE.

  • For a newly created account:

    1. After you have created the service account, continue to the Grant this service account access to project section.

      Grant access to the service account

    2. Select the role for the related GCP-hosted service. For example, to connect to BigQuery, select BigQuery Admin in the Role drop-down.

    3. Click + ADD ANOTHER ROLE to add a row. For example, Storage Admin in the Role drop-down for accessing the GCS resources for the project.

    4. Click CONTINUE.

    5. Click DONE.

Downloading the key file

Service account credentials are usually stored as a key file in JSON format. You need this file to connect your service account to Hevo.

To download the file:

  1. Log in to the Google Cloud Platform as an Owner.

  2. Navigate to the IAM, Service Accounts page.

  3. Click on the newly created service account.

  4. Navigate to the KEYS tab.

  5. Click ADD KEY, and then Create new key.

    Create key file

  6. Select the Key type as JSON, and click CREATE. This downloads the key file.

    Note: Hevo supports only JSON format for the key file.

    Select key file as JSON

You need to use this key file to connect GCP-hosted services to Hevo through the service account.

Last updated on May 30, 2023

Tell us what went wrong

Skip to the section