Connect to Apache Iceberg

This guide explains how to connect RisingWave to your Apache Iceberg tables for batch data ingestion. RisingWave supports reading data from Iceberg tables stored on S3-compatible storage (including AWS S3 and MinIO) and Google Cloud Storage (GCS).

​

Prerequisites

• An existing Apache Iceberg table.
• Access credentials for the underlying storage system (e.g., S3 access key and secret key).
• Network connectivity between RisingWave and your storage system.
• Know your Iceberg catalog type.

​

Connect to Iceberg

RisingWave supports connecting to Iceberg using CREATE SOURCE. Currently, RisingWave does not support creating a table backed by an Iceberg table using CREATE TABLE ... WITH (connector=...). If you create a table to store data, and ingest from the Iceberg source, the table is not an Iceberg table.

​

Basic connection example (S3)

The following example shows how to connect to an Iceberg table stored on S3:

CREATE SOURCE iceberg_source
WITH (
    connector = 'iceberg',
    warehouse.path = 's3://your-bucket/path/to/iceberg/warehouse',
    database.name = 'your_iceberg_db',
    table.name = 'your_iceberg_table',
    s3.endpoint = 'http://your-s3-endpoint:port', -- e.g., 'http://minio:9000'
    s3.access.key = 'YOUR_ACCESS_KEY',
    s3.secret.key = 'YOUR_SECRET_KEY',
    s3.region = 'your-s3-region'  -- Optional if endpoint is specified
) ;

Replace the placeholders with your actual values.

​

Query data

You can select from an Iceberg source to query data from the Iceberg table in RisingWave.

SELECT * FROM iceberg_source;

​

Create streaming jobs

You can create a materialized view or a sink from an Iceberg source to incrementally process and transform its data, similar to how you would with other supported sources.

Note that streaming ingestion only works for append-only iceberg tables. Any delete or update operations will be ignored.

​

Data type mapping

RisingWave converts data types from Iceberg to RisingWave according to the following table:

​

Time travel

RisingWave’s Iceberg source supports time travel. Please refer to Apache Iceberg configuration options for more details.

​

System tables

RisingWave offers system tables to query Iceberg metadata. Please refer to Apache Iceberg configuration options for more details.

​

What’s next?

• All configuration options: Apache Iceberg configuration options