Amazon S3 Tables catalog

Amazon S3 Tables is an AWS-native Iceberg catalog service that uses the REST protocol with AWS SigV4 authentication. You can use it with RisingWave when creating an Iceberg SOURCE, SINK, or CONNECTION (for internal Iceberg tables).

New to S3 Tables and need to create a table bucket / namespace / table first? See Create Amazon S3 Tables with AWS CLI.

Required REST parameters for S3 Tables:

Parameter	Description	Value for S3 Tables
`catalog.uri`	The S3 Tables REST endpoint for your region.	`https://s3tables.<your-region>.amazonaws.com/iceberg`
`catalog.rest.signing_region`	The AWS region for signing requests.	e.g., `us-east-1`
`catalog.rest.sigv4_enabled`	Enables SigV4 signing. Must be `true`.	`true`
`catalog.rest.signing_name`	The service name for signing requests.	`s3tables`

For a complete list of all catalog-related parameters, see the main Catalog configuration page.

Internal tables (recommended)
Write (CREATE SINK)
Read (CREATE SOURCE)

Use S3 Tables as the catalog for RisingWave-managed (internal) Iceberg tables by creating an Iceberg CONNECTION, then creating tables with ENGINE = iceberg.

CREATE CONNECTION s3_tables_conn WITH (
    type = 'iceberg',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.region = '<your-region>',
    s3.access.key = '...',
    s3.secret.key = '...',
    enable_config_load = false,

    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables'
);

SET iceberg_engine_connection = 'public.s3_tables_conn';

-- `commit_checkpoint_interval` controls Iceberg commit frequency. Default: about every 60 seconds; set to 1 for faster commits and visibility.
CREATE TABLE my_internal_iceberg_table (
  id INT PRIMARY KEY,
  name VARCHAR,
  value INT
)
WITH (commit_checkpoint_interval = 1)
ENGINE = iceberg;

Deliver data from RisingWave to an S3 Tables-managed Iceberg table using CREATE SINK.

CREATE SINK my_s3_tables_sink FROM my_mv
WITH (
    connector = 'iceberg',
    type = 'upsert',
    primary_key = 'id',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.access.key = '...',
    s3.secret.key = '...',
    s3.region = '<your-region>',
    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables',
    database.name = '<your-database-name>',
    create_table_if_not_exists = 'true',
    table.name = '<your-table-name>'
);

Ingest data from an S3 Tables-managed Iceberg table into RisingWave using CREATE SOURCE.

CREATE SOURCE my_s3_tables_source
WITH (
    connector = 'iceberg',
    warehouse.path = 'arn:aws:s3tables:<your-region>:<account-id>:bucket/<bucket-name>',
    s3.access.key = '...',
    s3.secret.key = '...',
    s3.region = '<your-region>',
    catalog.type = 'rest',
    catalog.uri = 'https://s3tables.<your-region>.amazonaws.com/iceberg',
    catalog.rest.signing_region = '<your-region>',
    catalog.rest.sigv4_enabled = true,
    catalog.rest.signing_name = 's3tables',
    database.name = '<your-database-name>',
    table.name = '<your-table-name>'
);

Interact with Apache Iceberg

Nimtable: Iceberg o11y platform

Amazon S3 Tables catalog