Skip to main content
Amazon S3 is an object storage service that offers industry-leading scalability, data availability, security, and performance.

​
Syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector='s3',
   connector_parameter = 'value', ...
);

​
Parameters

Parameter namesDescription
connectorRequired. Support the S3 connector only.
s3.region_nameRequired. The service region.
s3.bucket_nameRequired. The name of the bucket where the sink data is stored in.
s3.pathRequired. The directory where the sink file is located.
s3.credentials.accessConditional. Your AWS access key ID. Required if enable_config_load is false or not set.
s3.credentials.secretConditional. Your AWS secret access key. Required if enable_config_load is false or not set.
enable_config_loadOptional. Set to true to use the default AWS credential provider chain for authentication. This is useful for environments where credentials are provided through IAM roles or environment variables. If this is set to true, do not provide s3.credentials.access and s3.credentials.secret. You must also set the DISABLE_DEFAULT_CREDENTIAL=false environment variable on the Meta and Streaming Nodes.
s3.endpoint_urlOptional. The host URL for an S3-compatible object storage server. This allows users to use a different server instead of the standard S3 server.
s3.assume_roleOptional. Specifies the ARN of an IAM role to assume when accessing S3. It allows temporary, secure access to S3 resources without sharing long-term credentials.
typeRequired. Defines the type of the sink. The S3 sink only supports append-only
max_row_countOptional. Maximum number of rows per output file. Defaults to 10240. A file is finalized when this threshold or rollover_seconds is reached first.
rollover_secondsOptional. Maximum duration (in seconds) before the current file is finalized and a new one is started. Defaults to 10. A file is finalized when this threshold or max_row_count is reached first.
In RisingWave Cloud, the default AWS credential provider chain is disabled. Provide s3.credentials.access and s3.credentials.secret (or use a supported assume-role setup). These credentials cannot be omitted. The enable_config_load option is supported only in self-hosted deployments.

​
Example

CREATE SINK s3_sink AS SELECT v1
FROM t 
WITH (
    connector='s3',
    s3.path = '<test_path>',
    s3.region_name = '<region_name>',
    s3.bucket_name = '<bucket_name>',
    s3.credentials.access = '<access_key_id>',
    s3.credentials.secret = '<secret_access_key>',
    s3.endpoint_url = '<endpoint_url>',
    type = 'append-only',
)FORMAT PLAIN ENCODE PARQUET(force_append_only=true);

​
Advanced topics

For more information about encode Parquet or JSON, see Sink data in parquet or json format. For more information about batching strategy, see Batching strategy for file sink.

​
Multiple output files during backfill

When a sink is created, RisingWave performs an initial backfill of the upstream table. Because a file is finalized when either rollover_seconds (default: 10) or max_row_count (default: 10240) is reached, a table with more than 10,240 rows will produce multiple output files under the configured s3.path. Seeing roughly 10K rows in a single file does not mean data is missing. To verify that all rows were delivered, list all objects under the sink path and sum their row counts: 
aws s3 ls s3://<bucket>/<path>/ --recursive
Adjust max_row_count and rollover_seconds when creating the sink if you need different file sizes.