Amazon DynamoDB is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. It provides consistent, single-digit millisecond response times and offers built-in security, backup and restore, and in-memory caching.

PREMIUM EDITION FEATURE

This feature is only available in the premium edition of RisingWave. The premium edition offers additional advanced features and capabilities beyond the free and community editions. If you have any questions about upgrading to the premium edition, please contact our sales team at sales@risingwave-labs.com.

PUBLIC PREVIEW

This feature is in the public preview stage, meaning it’s nearing the final product but is not yet fully stable. If you encounter any issues or have feedback, please contact us through our Slack channel. Your input is valuable in helping us improve the feature. For more information, see our Public preview feature list.

Syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector = 'dynamodb',
   connector_parameter = 'value', ...
)
FORMAT data_format ENCODE data_encode [ (
    format_parameter = 'value'
) ];

Parameters

FieldNote
tableRequired. Name of the DynamoDB table where you want to write the data.
primary_keyRequired. A pair of columns representing the partition key and sort key of DynamoDB, e.g., key1,key2, separated by comma.
aws.regionRequired. AWS region where your DynamoDB table is hosted.
aws.endpoint_urlOptional. Custom endpoint URL for connecting to DynamoDB. Useful for testing with DynamoDB Local.
aws.credentials.access_key_idConditional. AWS access key ID for accessing DynamoDB. Must be specified when using access key ID and secret key.
aws.credentials.secret_access_keyConditional. AWS secret access key for accessing DynamoDB. Must be specified when using access key ID and secret key.
aws.credentials.role.arnConditional. ARN of the IAM role to assume for accessing DynamoDB. Must be specified when using AssumeRole.
aws.credentials.role.external_idConditional. External ID for assuming the IAM role specified in aws.credentials.role.arn.
aws.profileOptional. The name of the AWS CLI profile to use for accessing DynamoDB. If specified, it overrides the default profile.
dynamodb.max_batch_item_numsOptional. The maximum number of items in the BatchWriteItem operation. It must be larger than 1, and less than or equal to 25. The default is 25.
dynamodb.max_future_send_numsOptional. The maximum number of concurrent write futures in DynamoDB. It must be less than 360, and the default is 256. This is derived from user-defined max_parallelism_units (40000 by default). If the write throughput of RisingWave exceeds the max_parallelism_units set in DynamoDB, an error would be reported.

Partition key and sort key mapping

For the DynamoDB sink to function correctly, it is required that the source table in RisingWave has a composite primary key comprising two columns, corresponding to the partition key and sort key defined in the target DynamoDB table.

For example, if you are sinking data into a DynamoDB table named movies that is configured with a partition key title and a sort key year, you must ensure that the RisingWave table schema also defines a composite primary key with the same columns. Below is an example schema definition for the RisingWave table:

CREATE TABLE IF NOT EXISTS movies (
    year integer,
    title varchar,
    description varchar,
    primary key (year, title)
);

This makes sure that the data structure in RisingWave aligns with the key definition in DynamoDB, thus preventing any further potential errors.

Data type mapping

RisingWave Data TypeDynamoDB Data Type
booleanBool
smallintnumber (N)
integernumber (N)
bigintnumber (N)
numericnumber (N)
realnumber (N)
double precisionnumber (N)
serialnumber (N)
character varying (varchar)string (S)
byteabinary
datestring (S)
time without time zonestring (S)
timestamp without time zonestring (S)
timestamp with time zonestring (S)
intervalstring (S)
structmap (M)
arraylist (L)
JSONBstring (S)

The struct datatype in RisingWave will map to map (M) in DynamoDB in a recursive way. Refer to source code for details.