AWS Kinesis Connector

AWS Kinesis integrates with Bloo using the pull method via the Amazon Kinesis Data Streams API. Bloo consumes data records from your Kinesis stream in real time, normalizing and monitoring log data pushed to the stream by your application.

Before You Begin — Prerequisites

You will need the following before configuring the connector in Bloo. Your Kinesis stream must already exist and your application must be pushing data to it.

Requirement	Where to Get It	Notes
AWS Account	Sign in at console.aws.amazon.com	Must have IAM admin access to create users and policies
Access Key ID	Generated from IAM → Users → Security credentials (Step 1 below)	Created for an IAM user with a custom Kinesis consumer policy
Secret Access Key	Generated alongside Access Key ID	Shown only once — copy and store securely
AWS Region	The region where your Kinesis stream is located	Example: us-east-1

Note For reference, see the AWS documentation on Amazon Kinesis Data Streams (https://docs.aws.amazon.com/streams/latest/dev/introduction.html) and Controlling Access to Kinesis Data Streams (https://docs.aws.amazon.com/streams/latest/dev/controlling-access.html).

Step 1 — Create an IAM User with a Custom Kinesis Policy

Bloo reads from your Kinesis stream as a consumer. Follow these steps to create an IAM user with the exact permissions Bloo needs.

1. Sign in to the AWS Management Console at

   https://console.aws.amazon.com

2. In the search bar, type IAM and select the IAM service.

1. In the left sidebar, click Users then click Add users.

1. Enter a username (for example, kinesis-bloo-user). Under Select AWS access type, check the checkbox next to Access key - Programmatic access. Then click Next: Permissions.

1. On the Set permissions page, select Attach policies directly, then click Create policy. This opens the policy editor in a new tab.

1. In the policy editor, switch to the JSON tab and paste the following policy.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:CreateStream",
                "kinesis:DeleteStream",
                "kinesis:Put*",
                "kinesis:Get*",
                "kinesis:DescribeStreamSummary",
                "kinesis:DescribeStream"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}Copy

Note The permissions above are the minimum required for Bloo to read from your Kinesis stream. CreateStream, DeleteStream, and Put* permissions are not needed by the connector.

1. Enter a name for the policy (for example, BlooKinesisReadPolicy) and click Create policy. Return to the user creation tab.

2. Back on the Set permissions page, refresh the policy list and search for the policy you just created (BlooKinesisReadPolicy). Check the box next to it to select it. Click Next: Tags — add tags if needed or skip this section. Click Next: Review to review the user details, then click Create user.

1. Copy the Access Key ID and Secret Access Key shown on the confirmation screen. Click Download .csv file to save them securely. These are the credentials you will enter into Bloo.

Warning The Secret Access Key is shown only once. Download the .csv file now or copy it immediately — you cannot retrieve it again after leaving this screen.

Step 2 — Configure the Connector in Bloo

With the credentials from Step 1 and your stream details, set up the connector on the Datasource page in Bloo.

1. In Bloo, navigate to the Datasource page and click Add Datasource.

2. Search for and select AWS Kinesis.

3. Fill in the configuration fields as described below.

Field	Description	Example
Connector Name	A unique display name for this connector instance in Bloo.	Kinesis-Prod
Stream Name	Enter the exact name of your Kinesis data stream.	Test stream
Access Key ID	Enter the Access Key ID generated in Step 1.	Paste from Step 1
Secret Access Key	Enter the Secret Access Key generated in Step 1.	Paste from Step 1
Region	Enter the AWS region where your Kinesis stream is located.	us-east-1

1. Click Save. The connector proceeds directly to validation.

Confirmed Go to Collection Status (https://dnif.it/kb/operations/collection-status/) in Bloo and verify the AWS Kinesis connector shows status Active.

Related: How to Configure Connectors Collection Status