Configure BigQuery
Overview
Connecting to your Google BigQuery data warehouse requires a few simple steps to get you started.
1. Create a BigQuery Warehouse
2. Configure BigQuery in Toric
3. Create an Automation
1. Create a BigQuery Warehouse
Before you configure Google BigQuery in Toric, you will need to create a BigQuery data warehouse.
1.1 General steps to Create a BigQuery Project
1. Go to the Google Cloud Console
2. Click the project drop-down and select or create the project that you want to use with BigQuery (detailed documentation here).
3. Click the hamburger menu (top left) and select the BigQuery option in the list.
4. Click the Create dataset button (detailed documentation here)
5. In the Create dataset dialog:
5.1 Enter a dataset name.
5.2 Select a data location.
5.3 (Optional) To create a dataset with a specific expiration time, select Custom expiration time and enter a number of days or a date.
5.4 Click Create dataset.
6. (Optional) In the dataset's details page, you can do the following:
6.1 View and modify the dataset's access controls.
6.2 View and modify the dataset's expiration time.
6.3 View and modify the dataset's labels.
7. Open the kebab (three dots) menu on the dataset you created, and click Create a table (detailed documentation here). Use the Schema editor to add fields and select their data type.
That's it! You have successfully set up warehouse(table) in Google BigQuery. For a more detailed set of steps please see the Google support guide here.
1.2 Create a BigQuery Service Account
First you will need to enable the BigQuery Data Transfer API.
1. Open the hamburger menu and select API & Services.
2. Click the ENABLE APIS AND SERVICES, then click Enable.
Once a warehouse is created, and the API services are enabled, you need to generate a private key service account. Please follow the steps to create a Service Account and generate a Private Key.
1. In your BigQuery project, click the hamburger menu and select IAM & Admin.
2. Click the Service accounts tab (detailed documentation here).
3. Click the Create service account button.
4. In the Create service account dialog:
4.1 Enter a name for the service account.
4.2 Select a role for the service account.
4.3 (Optional) Add any additional roles or permissions. The most important in the following list is BigQuery Admin.
4.4 (Optional) Grant users access to this service account.
4.5 Click Create.
5. In the Service account permissions dialog: click Continue.
6. In the Grant users access to this service account dialog:
6.1 (Optional) Select the users or groups that you want to grant access to the service account.
6.2 Click Done.
7. In the Service account details page:
7.1 Click the Create key button.
7.2 In the Create key dialog, select the key type and click Create.
7.3 The private key file will be downloaded to your computer. Save it in a secure location, you will need to copy the Private key later in step 2.
Note: We recommend using a password vault like 1Password to store all your credentials.
2. Configure Big Query in Toric
Login to Toric and follow these steps to setup a BigQuery Connector:
2.1 Enter a Configuration Name
Type a unique name for your BigQuery connector.
2.2 Copy your BigQuery Private Key
Your private key should be an entire JSON object, copy the entire contents in the Private Key Content box.
Note: Make there you do not copy extra spaces, this may lead to an invalid configuration.
2.3 Validate your Configuration
Use the Validate Configuration button to make sure all fields are completed correctly. Invalid fields will be highlighted in red, and a successful validation will show in green.
Click on Add Configuration to finalize the creation of your BigQuery Connector.
3. Connect to an External Warehouse
Navigate to the Data Warehouses section to create a New Connected Warehouse.
In the pop-up, enter a Warehouse Name.
Note: We recommend using a unique name as you can create multiple warehouse connections.
Use the Application dropdown to select a data warehouse provider, in this case BigQuery. Then use the Configuration dropdown to select one of the options from step 2.1.
Finally, use the Table dropdown to select where to write your data.
Related articles
Next steps
Other Topics
