@font-face {
    font-family: 'Monument Grotesk';
    src: url('https://fonts.gstatic.com/s/inter/v18/UcCo3FwrK3iLTcviYwYZ8UA3.woff2');
    font-weight: normal;
    font-style: normal;
    font-display: swap;
  }
  
  h1,
  h2,
  h3 {
    font-family: 'monument grotesk';
    letter-spacing: normal !important;
  }

Describes how Change Data Capture (CDC) is handled in RisingWave

Overview

Change data capture with RisingWave

RisingWave

Join Slack community

GitHub

Guides

Apache Iceberg™

SQL/UDFs

Python SDK

Client Libraries

RisingWave Console

Changelog

Sign up free

Blog

Archived

RisingWave is an event stream processing and management platform. It offers an unified experience for real-time data ingestion, stream processing, data persistence, and low-latency serving.

What is RisingWave?

This guide aims to provide a quick and easy way to get started with RisingWave.

Quick start

RisingWave excels in a variety of real-time data processing scenarios, making it an ideal choice for several categories of use cases, including:

RisingWave use cases

This topic introduces the RisingWave Premium Edition and offers a complete list of all premium features.

Premium edition

RisingWave Premium Edition

This topic describes the elastic disk cache feature of RisingWave.

Elastic disk cache

Discover the wide range of real-world use cases where RisingWave delivers concrete value with the list of runnable demos.

Example repository

This topic provides an overview of RisingWave's stream processing performance characteristics based on rigorous testing using the industry-standard Nexmark benchmark and an extended set of real-world SQL queries.

Stream processing benchmarks

Stream processing benchmarks: Nexmark results

This topic provides an overview of RisingWave's ad-hoc query serving performance characteristics (latency and throughput) based on the industry-standard sysbench benchmark.

Query serving benchmarks

Query serving benchmarks: sysbench results

Describes how to use the purpose-built Python SDK by RisingWave to build event-driven applications.

You can use third-party PostgreSQL drivers to seamlessly interact with RisingWave from your applications.

Use RisingWave in your Node.js application

Node.js

Describes how to use third-party Python drivers to interact with RisingWave.

Python

Ruby

Java

Learn about RisingWave Console, an on-premise UI for RisingWave observability and management. Discover its key features, benefits, available editions, and how to get help or contribute.

Introduction

Understand the essential requirements for installing and running RisingWave Console to connect to your RisingWave clusters.

Prerequisites for RisingWave Console

Step-by-step guide to install and set up RisingWave Console using Docker, binary, or Docker Compose. Learn about key configuration variables and initial login.

Installation and setup

Learn how to connect your existing RisingWave instances to RisingWave Console by adding cluster details, testing connections, and managing these configurations.

Connect to your RisingWave clusters

Learn how to view your connected RisingWave clusters in RisingWave Console, access detailed information, and use its operational tools.

Manage and monitor clusters in RisingWave Console

Discover how to use RisingWave Console's SQL Console to query your RisingWave data, explore schemas, view query history, and visualize streaming data flows.

Use the SQL console

Learn about different ways to use Iceberg with RisingWave

Interact with Iceberg

Read from Iceberg

Read data from Iceberg tables

Write to Iceberg

Write data to Iceberg tables

Iceberg table engine

Data processing is a crucial step in converting raw data into valuable insights.

Overview of data processing

Similar to traditional databases, RisingWave supports data updates and deletions within tables. Additionally, when data in a table is modified or deleted, any materialized views built on that table will be updated accordingly.

Deletes and updates

Transactions

This guide introduces how to maintain a wide table whose columns come from different sources. Traditional data warehouses or ETL use a join query for this purpose. However, streaming join brings issues such as low efficiency and high memory consumption.

Maintain wide table with table sinks

In stream processing, watermarks are integral when using event time processing logic with event time based operations. Watermarks are like markers or signals that track the progress of event time, allowing you to process events within their corresponding time windows. A watermark is an estimate of the maximum event time observed so far, or a threshold indicating that events received so far have a timestamp later than or equal to the current watermark. Events that arrive with a timestamp earlier than the current watermark are considered late and are not processed within its time window.

Watermarks

Emit on window close

Indexes in a database are designed to increase the speed at which the database management system (DBMS) can locate and retrieve the desired data from a table or a materialized view.

Indexes

This guide describes how to leverage the time travel feature. This feature helps access historical data at a specific time.

Time travel queries

Learn how to store data persistently inside RisingWave.

Learn how to use the Iceberg table engine in RisingWave to store data natively in the Iceberg format.

RisingWave allows you to access and use insights from your streaming data immediately. It also functions like any other database, allowing you to query batch or raw data that you've inserted.

Access and use your processed data

Query with SELECT statements

A foreign data wrapper in PostgreSQL allows you to directly virtualize data stored in an external database as a local external table, also known as a foreign table. This tutorial will demonstrate how to interact between PostgreSQL and RisingWave. In this example, RisingWave will use CDC (Change Data Capture) to extract data from PostgreSQL, and analyze it using a materialized view. Then PostgreSQL will directly retrieve the computation results stored in RisingWave.

RisingWave as Postgres FDW

RisingWave as a PostgreSQL foreign data wrapper

Subscription is used to pull data change records for a specific table or materialized view (MV).

Subscription

RisingWave supports delivering data to downstream systems via its sink connectors.

Overview of data delivery

This guide describes how to sink data from RisingWave to Apache Iceberg using the Iceberg sink connector in RisingWave.

Apache Iceberg

Sink data from RisingWave to Apache Iceberg

This topic describes how to sink data from RisingWave to a Kafka broker and how to specify security (encryption and authentication) settings.

Apache Kafka

Sink to Kafka

 This guide describes how to sink data from RisingWave to ClickHouse using the ClickHouse sink connector in RisingWave.

ClickHouse

Sink data from RisingWave to ClickHouse

This guide describes how to sink data from RisingWave to Elasticsearch using the Elasticsearch sink connector in RisingWave.

Elasticsearch

Sink data from RisingWave to Elasticsearch

PostgreSQL

Sink data from RisingWave to PostgreSQL

This guide will introduce how to sink data from RisingWave to JDBC-available databases using the JDBC sink connector.

MySQL

Sink data from RisingWave to MySQL with the JDBC connector

SQL references

Overview of the SQL commands supported by RisingWave.

SQL commands

The `ALTER CONNECTION` command modifies the definition of a connection. To use this command, you must own the connection.

ALTER CONNECTION

The `ALTER DATABASE` command modifies the definition of a database.

ALTER DATABASE

The `ALTER FUNCTION` command modifies the definition of a function. To use this command, you must own the function.

ALTER FUNCTION

The `ALTER INDEX` command modifies the definition of an index.

ALTER INDEX

The `ALTER MATERIALIZED VIEW` command modifies the metadata of a materialized view. To use this command, you must own the materialized view.

ALTER MATERIALIZED VIEW

The `ALTER SCHEMA` command modifies the definition of a schema. To use this command, you must own the schema.

ALTER SCHEMA

Use the `ALTER SECRET` command to alter the secrets that store sensitive credentials.

ALTER SECRET

The `ALTER SINK` command modifies the metadata of a sink.

ALTER SINK

The `ALTER SOURCE` command modifies the definition of a source.

ALTER SOURCE

The `ALTER SUBSCRIPTION` command modifies the definition of a subscription.

ALTER SUBSCRIPTION

ALTER SWAP

The `ALTER SYSTEM` command modifies the value of a server configuration parameter.

ALTER SYSTEM

The `ALTER TABLE` command modifies the definition of a table.

ALTER TABLE

Use the `ALTER USER` command to modify the name, password, system permissions, and other properties of an existing user.

ALTER USER

The `ALTER VIEW` command modifies the metadata of a view. To use this command, you must own the view.

ALTER VIEW

Use the `AS CHANGELOG` clause to convert a changelog operation in a stream into a column.

AS CHANGELOG

RisingWave supports read-only transactions. You can use the `BEGIN READ ONLY` command to start a read-only transaction.

BEGIN

Use `CANCEL JOBS/JOB` to cancel specific streaming jobs that are in progress.

CANCEL JOBS

You can use the `COMMENT ON` command to add comments on tables or columns in RisingWave. Comments are stored as metadata of tables or columns.

COMMENT ON

RisingWave supports read-only transactions. You can use the `COMMIT` command to commit the current transaction.

COMMIT

The `CREATE AGGREGATE` command can be used to create [user-defined aggregate functions](/sql/udfs/user-defined-functions) (UDAFs). Currently, UDAFs are only supported in Python and JavaScript as embedded UDFs.

CREATE AGGREGATE

Use the `CREATE CONNECTION` command to create a reusable catalog for connector parameters.

CREATE CONNECTION

Use the `CREATE DATABASE` command to create a new database.

CREATE DATABASE

The `CREATE FUNCTION` command can be used to create [user-defined functions](/sql/udfs/user-defined-functions) (UDFs).

CREATE FUNCTION

Use the `CREATE INDEX` command to construct an [index](/processing/indexes) on a table or a materialized view.

CREATE INDEX

Use the `CREATE MATERIALIZED VIEW` command to create a materialized view. A materialized view can be created based on sources, tables, materialized views, or indexes.

CREATE MATERIALIZED VIEW

Use the `CREATE SCHEMA` command to create a new schema.

CREATE SCHEMA

Use the `CREATE SECRET` command to create secrets to store sensitive credentials securely.

CREATE SECRET

Use the `CREATE SINK INTO` command to create a sink into RisingWave's table.

CREATE SINK INTO

Use the `CREATE SINK` command to create a sink. A sink is an external target where you can send data processed in RisingWave. You can create a sink from a table or a materialized view.

CREATE SINK

A source is a resource that RisingWave can read data from. You can create a source in RisingWave using the `CREATE SOURCE` command.

CREATE SOURCE

Use the `CREATE TABLE` command to create a new table. Tables consist of fixed columns and insertable rows. 

CREATE TABLE

Use the `CREATE USER` command to create a new user account in RisingWave.

CREATE USER

Use the `CREATE VIEW` command to create a non-materialized view, which runs every time the view is referenced in a query. A non-materialized view can be created based on sources, tables, views, or indexes.

CREATE VIEW

Use the `DELETE` command to permanently remove rows from a table.

DELETE

Use the `DESCRIBE` command to check the information or execution fragments of a relation (table, source, sink, view, materialized view).

DESCRIBE

Use the `DISCARD` command to reset the state of current session

DISCARD

Use the `DROP AGGREGATE` command to remove an existing [user-defined aggregate function (UDAF)](/sql/udfs/user-defined-functions). The usage is similar to `DROP FUNCTION`, except that it's for aggregate functions.

DROP AGGREGATE

Use the `DROP CONNECTION` command to remove a connection.

DROP CONNECTION

Use the `DROP DATABASE` command to remove a database from your RisingWave instance.

DROP DATABASE

DROP FUNCTION

Use the `DROP INDEX` command to remove an index from a table or a materialized view.

DROP INDEX

Use the `DROP MATERIALIZED VIEW` command to remove a materialized view from the database.

DROP MATERIALIZED VIEW

Use the `DROP SCHEMA` command to remove a schema from a database.

DROP SCHEMA

Use the `DROP SECRET` command to drop the secrets that store sensitive credentials.

DROP SECRET

Use the `DROP SINK` command to remove a [sink](/sql/commands/sql-create-sink) if you no longer need to deliver data to the sink.

DROP SINK

Use the `DROP SOURCE` command to remove a [source](/sql/commands/sql-create-source) if you no longer need the data inflow from the source.

Get started

Working with data

Deploy & operate

Performance

Troubleshooting

Reference

Cloud

Change data capture with RisingWave