GCP Pub/Sub Connector

Overview

Google Cloud Platform (GCP) is the basis for creating, enabling, and using all Google Cloud services including managing APIs, enabling billing, adding and removing collaborators, and managing permissions for Google Cloud resources.

Prerequisites

• A Google account.
• A Cloud Console project.

Step 1 — Create a Project

1. Go to the Manage resources page in the Cloud Console.

2. On the Select organization drop-down list at the top of the page, select the organization in which you want to create a project. If you are a free trial user, skip this step, as this list does not appear.

3. Click Create Project.

4. In the New Project window that appears, enter a project name and select a billing account as applicable. A project name can contain only letters, numbers, single quotes, hyphens, spaces, or exclamation points, and must be between 4 and 30 characters.

5. Enter the parent organization or folder in the Location box. That resource will be the hierarchical parent of the new project.

6. When you're finished entering new project details, click Create.

Step 2 — Create a Topic

1. Go to the Pub/Sub topics page in the Cloud Console.

2. Click Create topic. In the Topic ID field, provide a unique topic name, for example, MyTopic.

3. Click Save.

Step 3 — Add a Subscription

1. Display the menu for the topic you just created and click Create subscription.

1. Type a name for the subscription, such as MySub.

2. Leave the delivery type as Pull.

1. Click Create.

Step 4 — Publish a Message to the Topic

1. Go to the Topic details page for the topic you just created.

2. Click the Messages tab near the bottom of the page, then click Publish Message.

3. Enter a message in the Message field.

4. Click Publish.

Step 5 — Create a Service Account Key

1. In the Cloud Console, go to the Service Accounts page.

2. Select a project.

3. Click the email address of the service account that you want to create a key for.

4. Click the Keys tab.

5. Click the Add key drop-down menu, then select Create new key.

6. Select JSON as the Key type and click Create. Clicking Create downloads a service account key file.

⚠ After you download the key file, you cannot download it again.

Step 6 — Pull a Message from the Subscription

1. In the Cloud Console, go to the Pub/Sub subscriptions page.

2. Click the subscription ID.

3. In the Subscription details page, click View messages.

4. Click Pull.

Step 7 — Configure the GCP Pub/Sub Connector in Bloo

All connector configuration is done from the Datasource page in Bloo.

Field	Description
Connector Name	Enter a name for the connector.
Project Id	Enter the project name.
Subscription Id	Enter the Subscription ID of the topic you created.
Credentials File	Enter the details from the credentials.json file, downloaded while creating the project.
Number of Processes	Total number of processes that can be run at a time.

1. Click Save after entering all the required details.

2. Bloo will validate the configuration automatically.

3. Navigate to Collection Status and confirm the connector is listed with a status of Active. This signifies the connector is configured successfully and data is ready to ingest.

✓ When the connector appears in Collection Status with status Active, it is configured successfully and data is ready to ingest.

Related Links

Bloo — Collection Status | Bloo — Connector Validation | GCP Pub/Sub — Quickstart | GCP — Create service account keys