Set up Kafka on Instaclustr Cloud
This guide goes through the steps to create a Kafka cluster in Instaclustr and to connect it to RisingWave for data ingestion. For more information regarding the data ingestion from Instaclustr Cloud, please refer to Instaclustr Documentation.Sign up for an Upstash Cloud account
Start by signing up for a free Instaclustr account, which will grant you access to Kafka services. To create your account, visit Instaclustr Cloud.
Create a Kafka cluster
Once you are logged in, follow the instructions in Getting Started with Apache Kafka to create a Kafka cluster in the Instaclustr Cloud.

Connect and interact with your Kafka cluster
You can connect with the Kafka cluster using various clients and languages such as Java, C#, Python, and Ruby to produce and consume Kafka messages using the Connection Info page that contains a list of your node addresses, authentication credentials to connect to your cluster, and a few connection examples for popular clients Kafka supports. Therefore, you can extract real-time flight data from the AviationStck API and feed it into a Kafka topic in the Kafka cluster deployed in the Instaclustr Cloud.
Ingest data from Instaclustr Kafka
Create a RisingWave project
You can create a RisingWave project and connect to it by following the steps in the Quick Start in the RisingWave documentation.Create a source
Once you have successfully deployed the RisingWave project and connected to it, proceed to create a source in RisingWave to ingest data from Instaclustr Kafka.Create a materialized view
Let’s create a materialized view namedaviation_mv
based on the source aviation_source
, to transform and cast certain columns from the source data, effectively modifying their data types.