Support for certain earlier version will end following the release of v2.6, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our version support policy.
SQL features
- SQL commands:
- Technical preview: Supports using
CREATE INDEXto create vector indexes, including flat and HNSW methods, and allows creating vector indexes on function expressions. #22432, #22022, #22808 CREATE SINKsupportsauto.schema.changeoption for Elasticsearch, Redshift, and Snowflake. #22611ALTER SOURCEandALTER TABLEsupportCONNECTOR WITHclause. #20780DROP CONNECTIONsupportsCASCADEoption to automatically drop dependent objects, except for Iceberg connections. #22557DROP FUNCTIONsupportsCASCADEoption to automatically remove dependent objects. #22991- Supports using multiple version columns to resolve upsert conflicts. #22931
- Technical preview: Supports using
- SQL functions & operators:
- Supports new data type
vector(n). #22019 - Supports
jsonb_to_arrayfunction to convert a JSONB array into a SQL array of JSONB elements. #22777 openai_embeddingaccepts a single constant JSONBconfigparameter instead of separateapi_keyandmodelarguments. #22960- Supports
arg_minandarg_maxaggregate functions. #22895
- Supports new data type
- System catalog:
- Adds system function
has_database_privilege. #22576
- Adds system function
Connectors
- Public preview: Iceberg table engine supports
copy-on-writewrite mode for enhanced compaction. #22713 - Supports Redshift and Snowflake v2 sink connectors. You can use intermediate table and leverage S3 to improve sink performance, and auto schema change is supported to automatically add new columns. #22984
- Iceberg table engine and Iceberg sink support
VACUUM [FULL] schema_name.object_namesyntax. #22942 - NATS connector no longer creates stream automatically unless new option
allow_create_streamis explicitly set totrue. #22315 - Kafka source supports altering
group.id.prefixparameter at runtime. #22492 - For Kafka source, when querying with a timestamp beyond the latest message, it starts from the latest offset instead of resetting to the earliest. #22413
- MQTT source supports multiple comma-separated topics. #22074
- Postgres CDC supports parallelized CDC backfill. #22536
- For PostgreSQL CDC, if you’ve already created a publication, fewer permissions are required. #22807
- Supports TOASTed data from PostgreSQL when using the CDC connector. #22852
Installation and deployment
- Supports deploying lakekeeper as an Iceberg REST catalog via Docker compose or Kubernetes. #23045
- Supports using
hashicorp_vaultas secret backend with Token or AppRole authentication methods. #22627
Cluster configuration changes
- Supports Memory-Only Mode for faster, low-latency queries by fully loading operator states into memory. You can configure it at the operator level to balance performance and memory usage. #22857
- Adds session variable
streaming_separate_consecutive_jointo separate consecutive stream joins to improve performance. #22259 - Adds session variable
streaming_join_encodingto switch join encoding type. #22416 - Adds session variable
streaming_separate_sinkto separate sink. #22827 - Adds
protect_drop_table_with_incoming_sinkto block table drops if incoming sinks exist. #22347 - Adds
streaming_now_progress_ratioto decouplenow_timestampfromepoch_timestamp. #22487
Access control list (ACL)
- Adds a new superuser
rwadminfor the Cloud control plane. #22327
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.6.0 standalone - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
Support for certain earlier version will end following the release of v2.5, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our version support policy.
SQL features
-
SQL commands:
- Technical preview: Adds option
backfill_orderto define the ingestion order of streams during backfill in case of join amplification. #20967 - Supports
ALTER DEFAULT PRIVILEGEScommand to set privileges for objects created in the future. #22191 - The
publicschema is now managed byGRANT/REVOKEcommand. New non-super users should first be granted privileges before creating objects. #20201 - Supports
ALTER COLUMN TYPEsyntax inALTER TABLEto change the data type of struct columns. Changing between primitive types (e.g.,int→bigint) is not supported. #20940 SHOW PROCESSLISTnow displays queries from all serving nodes, making it easier to troubleshoot distributed queries. #21952- Supports
CONNECTOR WITHclause forALTER SINKcommand. #20691 - Supports per-database configuration for
barrier_interval_msandcheckpoint_frequency. #22178
- Technical preview: Adds option
-
SQL functions & operators:
- Technical preview: Supports
approx_count_distinctto estimate the number of distinct values. #21516, #21545 - Technical preview: Supports
map_filterfunction to filter map data. #22077 - Supports
array_flattento flatten an array of arrays. #21640 - Supports
openai_embeddingfunction to generate embeddings. #22242 - Supports
WINDOWclause and window function over named window. #22024 - Supports the concatenation operator (
||), position, and overlay functions forbyteadata type. #22076
- Technical preview: Supports
-
System catalog:
- Adds system catalog rw_fragment_backfill_progress to track the progress of each backfill partition. #22124
Connectors
- Technical preview: Webhook source supports receiving batches of updates at once. #22075
- Technical preview: Supports
sink_decouplefor creating sink into table. #21784 - Breaking change: Adds escape syntax and usage instructions for the TEMPLATE encoding format. #21431
- Supports Iceberg compaction for Iceberg table engine. #22043
- Supports
hosted_catalog=trueto enable RisingWave-managed SQL catalog in Iceberg table engine. #21351 - Supports
enable_config_loadfor Iceberg sink, source, connection, and Iceberg table engine. #21938 - Supports
partition_byfor Iceberg table engine as the Iceberg sink. #21594 - Supports using
ALTER SINK ... CONNECTOR WITHto update connector-specific properties without recreating the sink. #20691
Cluster configuration changes
- Technical preview: Adds session variable
streaming_enable_unaligned_jointo control whether the streaming joins should be unaligned. #20386
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.5.0 standalone - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
Support for certain earlier version will end following the release of v2.4, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our version support policy.
SQL features
- SQL commands:
- Supports
EXPLAIN ANALYZEcommand to analyze the actual runtime performance of a streaming job. #20715, #20966 - Supports configurable
streaming_parallelismsettings for tables, indexes, materialized views, sinks, and sources. #21366 - Supports
DESCRIBE FRAGMENTScommand to check the execution fragments of an existing job. #21354 - Supports NOT NULL constraints in table schemas for both batch and streaming operations. #20822
- Supports
GRANTandREVOKEprivileges on connection, function and secret. #20755 ALTER SYSTEMnow applies session variable changes immediately to the current session. #21294
- Supports
- System catalog:
Connectors
- Deprecation notice: Deprecates the Pulsar Iceberg reader and removed the
icelakedependency. #20844 - Deprecation notice: The legacy S3 source is now fully deprecated in v2.4.0. #20658
- Supports creating an append-only table with the Iceberg engine. #21811
- Adds
commit_retry_numoption to configure the number of commit retries on Iceberg failures. #20433 - Supports Redis Pub/Sub messaging in sink connectors. #20991
- Supports Iceberg sink and Iceberg source on Azure Blob. #21468
- Supports exactly once consistency semantics for Iceberg sink. #19771
Cluster configuration changes
- Adds system parameter
per_database_isolationto enable per-database failure isolation. #20872 - Adds session variable
streaming_enable_materialized_expressionsto enable materialized expressions. #21552 - Adds session variable
streaming_force_filter_inside_jointo force filter to be pushed down into inner join. #21289
Fixes
- Security for logical views has been improved to correctly prevent unauthorized access. #20941
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.4.0 standalone - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
Support for certain earlier version will end following the release of v2.3, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our version support policy.
SQL features
- Query syntax:
-
SQL commands:
- Public preview: Supports specifying
resource_groupforCREATE DATABASEcommand. #20751 - Allows enabling or disabling the creation of embedded Python, JavaScript, and WebAssembly UDFs. #20893
- Supports async and batched functions for embedded JavaScript UDFs. #20403
- Rejects schema-qualified CURSOR names in DECLARE statements to align with Postgres specifications. #20221
CREATE TABLEcommand supports partially specifying the schema instead of using*to ingest all columns. #20203DROP SCHEMAcommand supports theCASCADEoption to drop the specified schema and all its dependencies. #19702- Supports switching databases during an active session using either
SET DATABASE TOorUSE. #19786 - Supports
SET PARALLELISMclause forALTER SOURCEcommand. #20606 - Supports
IF NOT EXISTSoption forCREATE FUNCTION/CREATE AGGREGATE. #20079 - Supports
GRANTorREVOKEprivileges on view. #20670
- Public preview: Supports specifying
-
SQL functions & operators:
- Supports IGNORE NULLS for
first_valueandlast_valuewindow functions. #19874
- Supports IGNORE NULLS for
- System catalog:
Connectors
- Deprecation notice: The legacy S3 source will be fully deprecated in v2.4.0. While the properties and enumerator will be retained to ensure that the metadata can start normally, it may disrupt other streaming jobs that still rely on the legacy source. #20658
- Public preview: Support ingesting data from PostgreSQL 17 using CDC. #20668
- Supports creating connections for Iceberg. #20189
- Supports AWS S3 Tables integration for the Iceberg source, sink, and table engine. #21018
- Supports creating Iceberg sources and sinks with GCS. #19999
- Supports generated columns for tables on CDC sources. #20380
- Supports reading Parquet files from GCS and Azure blob. #19974, #20046
- Removes schema from DML statements for JDBC sinks if not specified one. #20632
- Adds
partition_byto enable partitioning using column names or transformations for Iceberg sinks. #20495 - Adds
jdbc.auto.commitfor JDBC sinks to automatically commit transactions. #20597 - Adds
snowflakecatalog for Iceberg source. #20421 - Adds
aws.msk.signer_timeout_secto set timeout limit for loading AWS credentials of AWS MSK. #20610 jdbc.query.timeoutfor JDBC sinks is reduced from 600s to 60s. #20641
Installation and deployment
- For the production cluster, a license key with
Paidtier and CPU limit of 4 is set by default to allow the free trial of paid features on a small scale. #20296 - Adds a function
rw_license()to retrieve the license information. #20629
Cluster configuration changes
- Adds runtime parameter
sink_rate_limit. #19660
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.3.0 standalone - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
-
SQL commands:
- Public preview: Supports
EXPLAIN FORMAT JSON. #19041 - Public preview: Supports
EXPLAIN FORMAT (XML | YAML). #19400 - Supports
ALTER...SWAP WITH...syntax for tables, materialized views, views, sources, sinks, and subscriptions. #19172 - Introduces a new connection that allows for reusing connector parameters, replacing the deprecated AWS PrivateLink connection. #19270
- Supports creating connections for schema registries. #19685
- Ensures results returned by cursors are sorted by the primary key. #18801
- Supports
ALTER SECRET. #19495
- Public preview: Supports
- System catalog:
Connectors
- Public preview: Adds a native Postgres sink connector. #19328
- Public preview: Supports
mysql_querytable-valued function. #19071 - Supports using the streaming developer configuration
stream_switch_jdbc_pg_to_nativeto switch fromconnector='jdbc'toconnector='postgres'for JDBC Postgres sinks. #19703 - Supports nested
Reffor Avro encoding, except for circular dependency. #19601, #19701, #19746 - Adds parameter
messages_as_jsonbfor source’s Protobuf encoding when creating a source or table. #19935 - Supports
_rw_timestampsystem column for tables. #19232 - Allows Iceberg REST catalog to be specified as
catalog.credential,catalog.scope,catalog.oauth2_server_uri, orcatalog.token. #19406 - Supports configuring parameter
commit_checkpoint_intervalwhen creating Iceberg engine tables. The default value is60. #19738 - Supports
INCLUDE partitionwhen creating MQTT sources to include which topic each message is from. #19017 - Supports
INCLUDE subjectwhen creating NATS sources. #19708 - Supports webhook sources. #19272
- Adds Google BigQuery sink parameter
bigquery.credentials. #19798
Cluster configuration changes
- Adds session variable
batch_expr_strict_modeto control if the query fails or fills the table withNULLvalues during expression evaluation failures. #19562 - Adds runtime parameter
backfill_rate_limitto configure source backfill. #19445 - Adds runtime parameter
dml_rate_limit. #19679
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.2.0-standalone single_node - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
- Query syntax:
- Public preview: Supports
ASOF JOINto join and find the closest matching record by the event time or another ordered property. #18683 - Supports
AGGREGATE:prefixed scalar function in streaming aggregation. #18205 - Supports using user-defined aggregate functions as window function. #18181
- Supports blocking subscription cursors and configuring cursor timeouts. #18675
- Public preview: Supports
- SQL commands:
- Enhances observability of cursors and subscription cursors by improving output results of
SHOW SUBSCRIPTION CURSORSandSHOW CURSORScommands. #18896
- Enhances observability of cursors and subscription cursors by improving output results of
- SQL functions & operators:
- Public preview: Supports the TVF
postgres_query. #18811
- Public preview: Supports the TVF
- System catalog:
Connectors
- Breaking change: Changes
scan.startup.mode=latestfor NATS source connector to start consuming from next available message instead of last one. #18733 - Public preview: Supports shared Kafka sources, which can be disabled by session variable
streaming_use_shared_source. #18749 - Supports recursively scanning file sources. #18324
- Supports schemaless ingestion for data in JSON format from Kafka sources by using
INCLUDE payloadclause. #18437 - Adds a set of options for NATS source connector based on the async_nats crate. #17615
- Adds a required option,
consumer.durable_name, for NATS source connector. #18873 - Supports option
max_packet_sizefor MQTT sources. #18520 - Supports option
database.encryptfor SQL Server CDC sources. #18912 - Supports ingesting data from a partitioned table for PostgreSQL CDC sources. #18456
- Supports option
auto.schema.changefor PostgreSQL CDC sources to enable replicating Postgres table schema change. #18760 - Requires upstream table name to also be prefixed with database name when creating a SQL Server CDC table. #18868
- Adds
JSONencode for file sinks, allowing users to sink JSON files into object storage. #18744 - Supports
create_table_if_not_existsoption for Iceberg sink connector. #18362 - Supports WebHDFS sinks. #18293
- Removes option
bulk_write_max_entriesfor MongoDB sink and optiondefault_max_batch_rowsfor DynamoDB sink. Adds optionsmax_batch_item_numsandmax_future_send_numsfor DynamoDB sink. #17645 - Sets sink decoupling as the default policy for MongoDB, DynamoDB, and Redis sink connectors. #17645
- Supports option
routing_columnfor ElasticSearch sinks, allowing a column to be set as a routing key. #18698 - Supports specifying batching strategy when sinking data in Parquet format. #18472
Installation and deployment
- Adds a CLI argument of
--license-key-pathfor the Meta Node, enabling a background task to watch and reload license key from the specified file. #18768
Cluster configuration changes
- When
visibility_modeis set toall, the latest uncommitted data will be queried, but consistency is no longer guaranteed between the tables. #18230 - Supports
SET TIME ZONE INTERVAL '+00:00' HOUR TO MINUTEas equivalent toSET TIME ZONE UTC. #18705 - The etcd metastore is fully deprecated and unsupported. Users previously utilizing etcd metastore must manually migrate to a SQL backend (PostgreSQL, MySQL, or SQLite) to upgrade to v2.1.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.1.0-standalone single_node - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
- Query syntax:
- SQL commands:
- Breaking change:
DECLARE cursor_name SUBSCRIPTION CURSORis the same asDECLARE cursor_name SUBSCRIPTION CURSOR since now(), which will be consumed from the current time.DECLARE cursor_name SUBSCRIPTION CURSOR FULLwill start consuming data from stock. The type of operation has changed tovarchar. It is one ofInsert,Delete,UpdateInset, orUpdateDelete. #18217 - Public preview: Supports managing database credentials securely. #17456
SHOW CURSORSwill return all query cursors in the current session.SHOW SUBSCRIPTION CURSORSwill return all subscription cursors and subscriptions in the current session. #18217- Allows
ALTER TABLEon tables with generated columns. #17652 - Allows dropping generated columns from tables created with a schema registry. #17689
- Supports using scalar functions with list inputs as aggregate functions. #17622
- Supports altering the backfill rate limit for materialized views. #17911
- Breaking change:
- SQL functions & operators:
- Public preview: Supports
approx_percentile(). #17814, #17873. - Public preview: Supports native
maptype and related functions formap, and ingestingAVRO MAPtype into RisingWavemaptype. #17986 - Public preview: Supports scanning a directory of parquet files. #17811
- Supports
pg_index_column_has_property()to query index column properties. #17275 - Supports continuous timestamp generation in streaming mode. #17371
- Supports
acosd(). #9876 - Supports function
rw_recovery_status()andpg_is_in_recovery()to retrieve the Meta Node status. #17641
- Public preview: Supports
- System catalog:
Connectors
- Public preview: Supports ingesting Avro map type for source connectors. #17980
- Public preview: Supports encoding
parquetfor file source. #17201 - Public preview: Supports batch reading S3 Parquet files. #17625, #17673.
- Supports AWS Glue schema registry with
aws.glue.schema_arnparameter. #17605 - Supports creating tables and sources with
format upsert encode protobuf. #17624 - Supports ingesting Avro Union type for source connectors. #17485
- Supports reading files compressed in gzip format. #16538
- Adds the option to use a semicolon as the delimiter for CSV encode. #17356
- Uses OpenDAL to connect to S3 object store state backend. #18011
- Public preview: Supports replicating DDL for MySQL CDC source. #17876
- Supports parameter
refresh.interval.secoption for S3, GCS, and POSIX sources. #18184 - Supports parameter
group.id.prefixfor Kafka sources. #18115 - Validates slot name of PostgreSQL CDC sources. #17949
- Supports altering
backfill_rate_limitof CDC tables. #17989 - Public preview: Supports sinking data to file systems in parquet format. #17311
- Supports upsert Protobuf type sinks, which requires
KEY ENCODE TEXT. #18024 - Adds option
jsonb.handling.modeunderWITHoptions for sinks in JSON format. #17693 - Public preview: Supports Azure Blob sinks. #18244
- Public preview: Supports MongoDB sinks. #17102
- Supports Azure Blob file sources. #18295
- Supports glue catalog for iceberg sink and source. #17477
- Adds
jdbc.query.timeoutfor JDBC sinks to set the timeout for queries. #18430 - Changes default Kafka sink message timeout from five seconds to five minutes. #18304
- Adds new parameters
retry_on_conflict,batch_size_kb,batch_num_messages, andconcurrent_requestsfor ElasticSearch sink. #17867 - Supports parameter
bigquery.retry_timesfor BigQuery sink. #17237 - Supports parameter
bigquery.auto_create_tablefor BigQuery sink. #17393 - Supports parameter
doris.partial_columnsfor Doris sink. #16821 - Supports ClickHouse sink checkpoint decouple. #17491
- Sets sink decouple as default for all sinks. #18182
- Uses S3’s SQS notification to complete the import of data from Snowflake instead of the Snowflake HTTP client. #17627
- Ensures at-least-once delivery semantic and eventual consistency for Kinesis sink. #17983
- Supports backfilling by consuming a fixed snapshot of upstream table and then the upstream data epoch by epoch. #17735
Installation and deployment
- Supports configuring the SQL metastore using username, password, and database separately. #17530
- Supports more seamless scaling-in in Kubernetes deployments. #17802
Cluster configuration changes
- Breaking change: Refactors
streaming_rate_limitintosource_rate_limitandbackfill_rate_limit. #17796 - Breaking change: Adds a default soft and hard limit on actor count per worker parallelism. When the hard limit is reached, streaming workloads will fail. #18383
- Introduces
batch.developer.exchange_connection_pool_sizeandstreaming.developer.exchange_connection_pool_sizeto configure streaming and batch remote exchange between two nodes. #17768 - Introduces system parameter
license_keyused to enable enterprise features. #17396
Fixes
- Deletes related cursors when deleting a subscription. #17232
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.0.0-standalone single_node - Prebuilt all-in-one library for Linux.
- Source code (zip).
- Source code (tar.gz).
- risectl - a CLI tool for managing and accessing RisingWave clusters.
SQL features
- Query syntax:
- SQL commands:
- Supports specifying the authorization and omitting the schema name when creating a schema. #16806.
- Supports session window in batch and emit-on-window-close mode. #17098.
- Supports fetching multiple rows from a subscription cursor. #16764.
- Supports user-defined aggregate functions for embedded Python and JavaScript UDFs. #16874.
- SQL functions & operators:
- System catalog:
- Supports
rw_catalog.actor_id_to_ddlandrw_catalog.fragment_id_to_ddl. #17229.
- Supports
Connectors
- Avro schemas with
"default": "NaN"and positive and negative infinities, are supported asfloatanddoubletypes. #17309. - Supports ingesting simple
AVRO MAPtypes asJSONB. #16948. - Supports ingesting
avro uuidtypes asvarchar. #17069. - Supports ingesting
avrowith internalReftypes. #17052. - Adds
awsprefix to AWS related parameters when creating a source or sink. #16671. - Supports using AWS IAM to connect to Amazon MSK. #16625.
- Adds
pubsub.parallelismunder theWITHoption for Google PubSub source. #16733. - Supports
INCLUDE TIMESTAMP [AS]clause for MySQL, PostgreSQL, and MongoDB CDC tables.#16833. - Supports additional metadata columns for CDC tables. #17051.
- Automatically maps upstream table schema when creating MySQL and PostgreSQL tables. #16986.
- Sets a network timeout for JDBC sink connections. #17244.
- Enables sink decouple by default for Kafka, Kinesis, Pulsar, Google Pub/Sub, NATS, MQTT, ClickHouse sinks. #17221.
- Supports the
KEY ENCODEclause when creating a sink. #16377. - Supports
FORMAT PLAIN ENCODE AVROfor Kafka sinks. #17216. - Supports DynamoDB sink. #16670.
- Supports Microsoft SQL Server sinks for self-hosted SQL Server and Azure SQL. #17154.
- Supports OpenSearch sink. #16330.
- Supports checkpoint decouple for StarRocks sinks. #16816.
- Supports checkpoint decouple for Delta Lake sinks. #16777.
- Supports sinking serial types. #16969.
Cluster configuration changes
- Sets arrangement backfill as the default. #14846.
- Supports spill hash join to avoid OOM issues. #17122.
- Supports spill hash aggregation for batch queries. #16771.
- Changes the algorithm that calculates the reserve memory size. #16992.
Bug fixes
- Improves error message and location of the cursor. #16959.
- Improves error message when trying to create a CDC source with columns. #16636.
- Allows
GRANTandREVOKEprivileges on views. #16699.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.10.0 single_node - Prebuilt all-in-one library for Linux.
- Source code (zip).
- Source code (tar.gz).
- risectl - a CLI tool for managing and accessing RisingWave clusters.
v1.9.0 was skipped due to some critical bugs.
SQL features
- Query syntax:
- Supports non-append-only temporal joins, where the outer side is not required to be append-only. #16286.
- SQL commands:
- Supports
DISCARD ALLcommand for Npgsql compatibility. #16432. - Supports creating, dropping, altering, and showing subscriptions. #14831.
- Supports cursors for subscription queues. #15180.
- Supports altering stream rate limit for sources and tables with a source. #16399.
- Supports
RECOVERcommand to trigger an ad-hoc recovery. #16259.
- Supports
- SQL functions & operators:
- Supports
jsonb_populate_record()andjsonb_populate_recordset(). #13421.
- Supports
- System catalog:
Connectors
- Provides stable support for SQLAlchemy 2.0. #29.
- Deprecates
s3connector. #16337. - Supports generated columns for non-shared CDC tables. #16522.
- Supports time travel for Iceberg sources. #15866.
- Blocks sink creation until backfill is completed by default. #16249.
- Supports Kafka connector parameter
properties.request.required.acks. #16482. - Adds connector parameter
ssl.modefor PostgreSQL and Neon source connector. #15690. - Adds connector parameter
ssl.modefor MySQL source connector. #16579. - Supports parameters
snapshot.intervalandsnapshot.batch_sizeunderWITHoptions when creating a table from a CDC source. #16426. - Supports implicitly converting
numerictypes from PostgreSQL sources intorw_int256orvarchar. #16346. - Supports configuring the timeout of CDC sources. #16598.
- Supports
timestamptz.handling.modeformatting option when creating a source withPLAIN,UPSERT, orDEBEZIUM JSONformats. #16265. - Only uses fragment ID as group ID for Kafka sources. #16111.
- Supports cluster URLs for Redis sink connector. #16034.
- Supports creating Delta sinks with GCS. #16182.
- Supports Snowflake sink connector. #15429.
- Supports creating
upserttype BigQuery sinks. #15780.
Installation and deployment
- Sets PostgreSQL as the default meta store when deploying with Docker Compose. #16724.
Cluster configuration changes
- Supports using
ALTER SYSTEMto set a system-wide default value for a session parameter. #16062. - Modifies the meaning of
streaming_rate_limit=0, which now means pausing the snapshot read stream for backfill, and pausing source read for sources. This statement previously disabled the rate limit within the session. #16333. - Supports configuring the reserved memory bytes of the Compute Node by using
RW_RESERVED_MEMORY_BYTESruntime parameter andreserved-memory-bytesstartup option. #16433. - Introduce new timeout and retry configurations for ObjectStore and deprecate ambiguous timeout configurations. #16231.
Fixes
- Properly convert
-inf,+inf, andnantypes tonullfor JDBC sinks. #16230. - Handles sinking
-inf,+inf, andnantypes for ClickHouse, Doris, and StarRocks sink connectors. #15664. - Fixes an issue where
DELETEevents could not be sinked if the primary key isuuidtype for JDBC sinks.#16447. - Fixes an issue where
enumtypes from PostgreSQL could not be ingested asvarchartypes. #16423. - Fixes sources with
encode avroon decimal ingesting. #16202. - Fixes sources with
encode avroon bytes/fixed/decimal default value. #16414.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.9.1-standalone single_node - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
- Query syntax:
- Supports
RANGEframes in window function calls. #14416.
- Supports
- SQL commands:
- Adds support for embedded Rust UDFs. #14903.
- Adds support for embedded Python UDFs. #15168.
- Supports refreshing the schema of a table created using an external connection to get the latest schema. #15025.
- Supports refreshing the schema of a source to get the latest schema. #15541.
- Adds a description column to the system parameters table. #15113.
- Supports authenticating with OAuth token acquired from the Cloud when creating a user. #13151.
- SQL functions & operators:
- System catalog:
- Adds missing columns for
pg_catalog.pg_index,rw_catalog.rw_columnsandinformation_schema.columns, and system viewpg_catalog.pg_partitioned_table. #15151. - Supports
pg_catalog.pg_constraintfor DBeaver compatibility. #15227. - Supports
pg_catalog.pg_stat_get_numscansfor DBeaver. #15642. - Supports system table
rw_depend. #15385. - Supports
pg_settingscatalog. #15108.
- Adds missing columns for
Connectors
- Breaking change: Sinks created from v1.6 and earlier that have
decoupleenabled may cause compatibility issues. Check if you have any sinks with this configuration by using the internal tablerw_sink_decouplebefore upgrading to v1.8. #15613. - Avro tables and sources now require a schema registry during creation. #15256.
- Supports using Karapace when specifying a schema registry when creating a Kafka source. #15486.
- Supports Protobuf data format for NATS JetStream source. #15378.
- Supports Confluent schema registry for Kafka sinks when using
FORMAT PLAIN ENCODE PROTOBUF. #15546. - Adds Kafka sink and source parameter
enable.ssl.certificate.verification. #15073. - Supports
max_batch_rowsandrequest_timeoutparameters for Cassandra and ScyllaDB sources. #15516. - Adds built-in MongoDB CDC source connector. #14966.
- Adds
ignore_optionparameter for sources created using Debezium format. #15304. - Supports batch read from Iceberg source. #15214.
- Supports automatically deriving columns from Iceberg source. #15415.
- Supports JDBC catalog for Iceberg sources. #15551.
- Adds JDBC and Hive catalogs for Iceberg sink. #14885.
Installation and deployment
- Supports tab-completion for
SETandALTER SYSTEM SETcommands inpsqlclient. #15123. - Supports SQL meta store. #16019.
Bug fixes
- Fixes an issue where built-in CDC connectors do not accept empty passwords. #15411.
- Fixes an issue where materialized views created on a shared CDC source were allowed. #15635.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.8.0-standalone single_node - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
- Query syntax:
- SQL commands:
- Supports changing the schema registry by redefining the
format_encode_options. #14057.
- Supports changing the schema registry by redefining the
- SQL functions & operators:
- System catalog:
- Change
rw_streaming_parallelismto allow queries on streaming job parallelism with job name and type. Adds system viewrw_fragment_parallelismto allow for queries on parallelism information at fragment level #14789, #14261. - Adds
relpersistenceinpg_classcatalog. #14400. - Supports
pg_get_viewdef(). #14336.
- Change
Connectors
- Cassandra and ScyllaDB sinks no longer support
timestamptype. #14413. - Updates StarRocks sink connector parameters. #14823.
- Introduces
snapshotoption to allow users to disable CDC backfill and to only consume from the latest changelog. #14718. - Sets the default value of
transactionalparameter totruefor MySQL and Postgres CDC shared sources. #14899.
Installation and deployment
- [Pre-Release] Supports the standalone mode to run RisingWave in a single process. #14951.
- Supports Alibaba Cloud OSS as the storage backend.
Cluster configuration changes
- Introduce a session variable
batch_enable_distributed_dmlto enable batch ingesting. #14630. - Changes wording from
AUTOtoADAPTIVEparallelism. #14414. - Supports adaptive scaling for streaming jobs by default. #14873.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.7.0-standalone single_node - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl - a CLI tool for managing and accessing RisingWave clusters
SQL features
- Query syntax:
- Allows
NOWin upper bound condition for temporal filters. #13985. - Supports temporal filters with multiple
ORexpressions. #14382. - Supports
<expr> [ NOT ] SIMILAR TO <pat> [ ESCAPE <esc_text> ]clause. #14000. - Breaking change: Fixes the correctness of
SOME,ALL, andANYexpressions. Drop and recreate any materialized views that use these expressions. #14221. - Supports array subquery and \du command. #14044.
- Supports
SET PARALLELISMclause forALTERcommands. #14240.
- Allows
- SQL commands:
- SQL functions & operators:
- Breaking change:
0b10is now interpreted as binary10instead of0 as b10. Integer literals can be given in hex0x, oct0o, and bin0b. #14262. - Supports interval type as input for
to_char(). #14071. - Supports
NULLand fraction expression as direct arguments of ordered-set aggregate functions. #14080.
- Breaking change:
- System catalog:
- Add system view
rw_streaming_parallelism. #14261.
- Add system view
Connectors
- Adds CDC backfill support for Postgres so users can ingest multiple PostgreSQL tables with a single replication slot. #13958.
- Support multi-table transactions from upstream MySQL & Postgres CDC. Specify
transactional = truein theWITHoptions to enable it. #14375. - Renames
scan.startup.timestamp_millistoscan.startup.timestamp.millisfor Kafka, Pulsar, and NATS source. #13656. - Adds
properties.ssl.endpoint.identification.algorithmparameter for Kafka source and sink.#13990. - Supports
FORMAT PLAIN ENCODE PROTOBUFsyntax for Kafka sink. #12858. - Supports GCS file source. #13414.
- Breaking change: For ClickHouse sinks,
timestamptzcan be sinked toDateTime64.timestampcannot be sinked and has to be converted totimestamptzfirst before being sinked. #13672. - For Elasticsearch sinks, the default es.type is set as
_docfor Elasticsearch 6.x and 7.x, and is removed in Elasticsearch 8.x. RisingWave’s Elasticsearch sink will now send JSONB as a JSON string, and Elasticsearch will convert it into an object. #14273. connector = 'iceberg_java'is deprecated, and users can only Iceberg sinks with the Rust version of Iceberg. Similarly, the DeltaLake sink will also use the Rust version implementation. #14277.- Supports StarRocks sink. #12681.
Installation and deployment
- Allows for
storage.prefetch_buffer_capacity_mbto be configured in the TOML file to prevent out-of-memory issues. #13558. - Supports Huawei Cloud OBS as the storage backend. #13844.
Cluster configuration changes
- Supports setting
statement_timeoutvalue for queries. #13933. - Exposes SSL functionality through
RW_SSL_CERTandRW_SSL_KEYenvironment variables to configure SSL certificates and key file location. #14062.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.6.0 playground - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl-v1.6.0-x86_64-unknown-linux.tar.gz
SQL features
- SQL Commands:
- Supports
SET SCHEMAsyntax forALTER {TABLE t | [MATERIALIZED] VIEW (m)v | SOURCE src | SINK sink | CONNECTION c | FUNCTION f( argument_type [, ...] )}. #13341. - Supports
OWNER TOsyntax for ALTER DATABASE | SCHEMA | TABLE | [MATERIALIZED] VIEW | SOURCE | SINK. #13216. - Supports
RENAME TOsyntax forALTER { DATABASE db | SCHEMA s}. #13713. - Supports
KILLcommand. #13434 - Supports
SHOW PROCESSLISTcommand. #13287. - Supports
SET TO DEFAULTcommand. #13693. - Supports
SHOW COLUMNSandDESCRIBEfrom sinks and views. #13626.
- Supports
- SQL functions & operators
- Supports list and struct types for
jsonb_aggandjsonb_object_agg. #13299. - Supports
jsonb_build_arrayandjsonb_build_object. #13198. - Supports
to_jsonb. #13161. - Supports JSON path operators and functions. #13568.
- Supports array operators
@>and<@. #13253. - Fixes the correctness of case expressions. Previously if there were multiple matching values, the last one would match. #13890. The fix introduces a breaking change. It is recommended to drop and recreate any materialized views that contain
CASEexpressions. If your instance enters a crash-loop, we suggest upgrading to v1.5.2, and dropping the corresponding materialized view that containsCASEexpressions.
- Supports list and struct types for
- System catalog
Sources & sink
- The load generator can generate
timestamptzcolumns. #13451. - Adds option
[properties.fetch.queue.backoff.ms](http://properties.fetch.queue.backoff.ms)for Kafka source. #13321. - Supports creating multiple CDC tables that share the same source, which allows for incremental and lock-free snapshot loading. #12535.
CREATE SINKstatements no longer need to wait for backfill to complete. #13665
Deployment
- Adds a docker-compose file for standalone mode. #13233.
Cluster configuration changes
- Adds support for system parameter
pause_on_next_bootstrap. #11936
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.5.0 playground - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl-v1.5.0-x86_64-unknown-linux.tar.gz
SQL features
- Query syntax:
- Supports using subqueries in
UPDATEandDELETEstatements. #12995
- Supports using subqueries in
- SQL commands
- SQL functions & operators:
- Supports
substringandsubstrfunctions forbyteadata type. #13088. - Supports functions
jsonb_pretty,jsonb_object,jsonb_strip_nulls, andjsonb_extract_path. #13050, #13036, #13169, #13143. - Supports jsonb
@>,<@,?,?|, ?&,#>,#>>,-and#-operators. #13056, #13110, #13118. - Supports
greatestandleastfunctions. #12838. - Supports
regexp_split_to_arrayfunction. #12844. - Supports
bit_andandbit_oraggregate functions in materialized views. #12758. - Supports
jsonb_aggandjsonb_object_aggin streaming mode. #12836. - Supports general
rankanddense_rankwindow functions. #13183.
- Supports
- System catalog:
Sources & sinks
- Adds
google.protobuf.Anysupport for Protobuf sources. #12291. - Adds
schemas.enablesupport for Kafka sinks with upsert JSON. #12113. - Adds support for Kafka sinks with upsert Avro using schema registry. #13007.
server.idoption is now optional for MySQL CDC source. #13031- Enables
timestamptz.handling.modeoption to control the timestamptz output format for certain sinks. #13109. - Adds the
streamfield and support for multiple inputs for thesubjectfield for NATS source connector. #12799. - Adds new option
properties.allow.auto.create.topicsfor Kafka sink. #12766. - Adds support for
s3_v2source connector, a more efficient version of the S3 source. #12595. - Adds support for Google BigQuery sink.#12873.
- Adds support for Redis sink. #11999,#13003.
Deployment
- Release RisingWave all-in-one binary with connector libraries. #13133
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.4.0 playground - Prebuilt all-in-one library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl-v1.4.0-x86_64-unknown-linux.tar.gz
SQL features
- SQL commands
- SQL functions & operators
- Supports
array_min. #12071 - Supports
array_max. #12100 - Supports
array_sort. #12189 - Supports
array_sum. #12162 formatfunction supports variable inputs. #12178- Regular expression functions support back reference, positive, negative lookahead, and positive, negative lookbehind. #12329
- Supports
||operator for concatenating JSONB data. #12502 - Supports
bool_andandbool_orin materialized views. #11956
- Supports
- Query syntax:
- Supports
WITH ORDINALITYclause. #12273
- Supports
- System catalog
Sources & sinks
- Generated columns defined with non-deterministic functions cannot be part of the primary key. #12181
- Adds new
properties.enable.auto.commitparameter for the Kafka consumer, which sets theenable.auto.commitparameter for the Kafka client. #12223 - Adds
privatelink.endpointparameter to the WITH clause, to support private link for Kafka connector on GCP and AWS. #12266 - Adds parameters
message.timeout.msandmax.in.flight.requests.per.connectionfor Kafka sources. #12574 - Allows Kinesis source to start ingesting data from a specific timestamp.
sequence_numberis no longer supported as a startup mode option. #12241 - Allow optional
FORMAT DEBEZIUM ENCODE JSONafter the connector definition of CDC tables. Allow optionalFORMAT NATIVE ENCODE NATIVEafter the connector definition of Nexmark sources or tables. #12306 - Allows multiple URLs when defining schema registries. #11982
- Adds support for sinking data to versions 7 and 8 of Elasticsearch. #10357, #10415, #1303
- Adds support for sinking append-only data to the NATS messaging system. #11924
- Adds support for sinking data to Doris. #12336
- Adds support for sinking data to Apache Pulsar. #12286
- Adds support for sinking data to Cassandra and ScyllaDB. #11878
- Adds support for creating upsert Iceberg sinks. #12576
- Supports specifying the
sink_decouplesession variable asdefault,trueandenable, orfalseanddisable. #12544 - A
varcharcolumn in RisingWave can sink into auuidcolumn in Postgres. #12704 - New syntaxes for specifying data format and data encoding when creating a Kafka, Kinesis, and Pulsar sink. #12556
Administration & observability
- Supports querying from
information_schema.views, which contains formations about views defined in the database. #12045
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.3.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl-v1.3.0-x86_64-unknown-linux.tar.gz
- risingwave-connector-v1.3.0.tar.gz
SQL features
-
SQL commands:
-
Breaking change: Syntax of emit-on-window-close has changed. If your application contains integration code, please update your code accordingly. #11363
In v1.1:
In v1.2 and onwards:
- Privileges for tables can now be granted or revoked. #11725
-
The default
DISTRIBUTED BYcolumns have been changed from the whole index columns into the first index column. #11865 -
Supports
ALTER SOURCE ADD COLUMN. #11350 -
Supports
SHOW JOBSandCANCEL JOBS, with which you can show the in-progress streaming jobs and cancel jobs by their IDs. #11854 -
Supports
[I]LIKEinSHOWcommands. #11791
-
Breaking change: Syntax of emit-on-window-close has changed. If your application contains integration code, please update your code accordingly. #11363
In v1.1:
-
SQL functions & operators
- Supports lambda functions via
array_transform. #11888, #11937 - Supports
to_date(). #11241 - The
to_char()function now supportstimestamptzinput. #11778 - Supports
scale,min_scale, andtrim_scale. #11663 - Supports
regexp_replace. #11819 - Supports
regexp_count. #11975 - Supports
[NOT] ILIKEexpressions. #11743 - Supports
[!]~~[*]operators, which are equivalent to[NOT] [I]LIKE. #11748 - Supports
IS JSONpredicate. #11831
- Supports lambda functions via
- Query syntax:
- System catalog
- Adds support for transactions for single-table CDC data. #11453
Sources & sinks
- Adds a new parameter
schema.registry.name.strategyto the Kafka connector, with which you can specify naming strategies for schema registries. #11384 - Breaking Change: Implements a Rust-native Iceberg sink connector to improve stability and performance. The connector introduces new parameters. Applications that rely on the previous version of the feature (specifically, the version included in RisingWave v1.0 and v1.1) may no longer function correctly. To restore functionality to your applications, please carefully review the syntax and parameters outlined on this page and make any necessary revisions to your code. #11326
- Adds support for sinking data to ClickHouse. #11240
- Beta: An enhancement has been made to the mysql-cdc connector to improve data ingestion performance. It achieves so by optimizing the data backfilling logic for CDC tables. This feature is not enabled by default. To enable it, run this command:
SET cdc_backfill="true";#11707 - Adds a parameter
client.idfor Kafka sources. #11911
Deployment
- Supports HDFS as the storage backend for deployments via Docker Compose. #11632
Administration & observability
- Adds a new system parameter
max_concurrent_creating_streaming_jobs, with which users can specify the maximum number of streaming jobs that can be created concurrently. #11601 - Improves the calculation logic of the Mem Table Size (Max) metric in the RisingWave Dashboard. #11442
- Adds new metrics to RisingWave Dashboard:
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.2.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
- risectl-v1.2.0-x86_64-unknown-linux.tar.gz
- risingwave-connector-v1.2.0.tar.gz
SQL features
-
SQL commands:
-
DROPcommands now support theCASCADEoption, which drops the specified item and all its dependencies. #11250 -
CREATE TABLEnow supports theAPPEND ONLYclause, allowing the definition of watermark columns on the table. #11233 -
Supports new commands
START TRANSACTION,BEGIN, andCOMMITfor read-only transactions. #10735 -
Supports
SHOW CLUSTERto show the details of your RisingWave cluster, including the address of the cluster, its state, the parallel units it is using, and whether it’s streaming data, serving data, or unschedulable. #10656, #10932
-
-
SQL functions:
-
Supports new window functions:
lead()andlag(). #10915 -
Supports new aggregate functions:
first_value()andlast_value(), which retrieve the first and last values within a specific ordering from a set of rows. #10740 -
Supports the
grouping()function to determine if a column or expression in theGROUP BYclause is part of the current grouping set or not. #11006 -
Supports the
set_config()system administration function. #11147 -
Supports the
sign()mathematical function. #10819 -
Supports
string_agg()withDISTINCTandORDER BY, enabling advanced string concatenation with distinct values and custom sorting. #10864 -
Supports the co-existence of
string_agg()and other aggregations withDISTINCT. #10864 -
Supports the
zone_stringparameter in thedate_trunc(),extract(), anddate_part()functions, ensuring compatibility with PostgreSQL. #10480-
Breaking change: Previously, when the input for
date_truncwas actually a date, the function would cast it to a timestamp and record the choice in the query plan. However, after this release, new query plans will cast the input totimestamptzinstead. As a result, some old SQL queries, especially those saved as views, may fail to bind correctly and require type adjustments. It’s important to note that old query plans will continue working because the casting choice is recorded with a cast to timestamp. Before this release:After this release:Now, the result ofdate_truncincludes the timezone offset (+00:00) in the output, making it consistent with the behavior in PostgreSQL.
-
Breaking change: Previously, when the input for
-
round()now accepts a negative value and rounds it to the left of the decimal point. #10961 -
to_timestamp()now returnstimestamptz. #11018
-
Supports new window functions:
-
Query clauses
-
SELECTnow supports theEXCEPTclause which excludes specific columns from the result set. #10438, #10723 -
SELECTnow supports theGROUPING SETSclause which allows users to perform aggregations on multiple levels of grouping within a single query. #10807 - Supports index selection for temporal joins. #11019
-
Supports
CUBEin group-by clauses to generate multiple grouping sets. #11262
-
-
Patterns
- Supports multiple rank function calls in TopN by group. #11149
-
System catalog
- Supports querying
created_atandinitialized_atfrom RisingWave relations such as sources, sinks, and tables in RisingWave catalogs. #11199
- Supports querying
Connectors
- Supports specifying more Kafka parameters when creating a source or sink. #11203
-
JDBC sinks used for upserts must specify the downstream primary key via the
primary_keyoption. #11042 -
access_keyand its correspondingsecret_keyare now mandatory for all AWS authentication components. #11120
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.1.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
SQL features
-
SQL command:
- Supports the
GROUPING SETSclause. #10807
- Supports the
-
SQL function:
-
Adds the
current_setting()function to get the current value of a configuration parameter. #10051 -
Adds new array functions:
array_position(),array_replace(),array_ndims(),array_lower(),array_upper(),array_length(), andarray_dims(). #10166, #10197 -
Adds new aggregate functions:
percentile_cont(),percentile_disc(), andmode(). #10252 -
Adds new system functions:
user(),current_user(), andcurrent_role(). #10366 -
Adds new string functions:
left()andright(). #10765 -
Adds new bytea functions:
octet_length()andbit_length(). #10462 -
array_length()andcardinality()return integer instead of bigint. #10267 -
Supports the
row_numberwindow function that doesn’t match the TopN pattern. #10869
-
Adds the
- User-defined function:
- System catalog:
-
Supports
GROUP BYoutput alias or index. #10305 -
Supports using scalar functions in the
FROMclause. #10317 - Supports tagging the created VPC endpoints when creating a PrivateLink connection. #10582
Connectors
-
Breaking change: When creating a source or table with a connector whose schema is auto-resolved from an external format file, the syntax for defining primary keys within column definitions is replaced with the table constraint syntax. #10195
OldNew
-
Breaking change: Modifies the syntax for specifying data and encoding formats for a source in
CREATE SOURCEandCREATE TABLEcommands. For v1.0.0, the old syntax is still accepted but will be deprecated in the next release. #10768 Old syntax - part 1:New syntax - part 1:Old syntax - part 2:New syntax - part 2: - Supports sinking data to Delta Lake. #10374, #10580
- Supports sinking data to AWS Kinesis. #10437
-
Supports
BYTESas a row format. #10592 - Supports specifying schema for the PostgreSQL sink. #10576
- Supports using the user-provided publication to create a PostgreSQL CDC table. #10804
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.0.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
Installation
- Now, you can easily install RisingWave on your local machine with Homebrew by running
brew install risingwave. See Run RisingWave.
Administration
- Adds the
pg_indexesanddattablespacesystem catalogs. #9844, #9822 - Now, the
SHOW PARAMETERScommand will display the mutability of each system parameter. #9526
SQL features
- Experimental features: Adds support for 256-bit integers. #9146, #9184, #9186, #9191, #9217
- Indexes can be created on expressions. #9142
- Adds support for expressions in aggregate function arguments. #9955
- Adds support for
VALUESclause. #8751 - Adds support for generated columns, which are special columns computed from other columns in a table or source. #8700, #9580
- Adds support for using expressions in the
ORDER BYandPARTITION BYclauses. #9827 - New SQL commands
CREATE CONNECTIONandSHOW CONNECTIONS: Creates an AWS PrivateLink connection and show all available connections. #8907DROP CONNECTION: Drops a connection. #9128SHOW FUNCTIONS: Shows existing user-defined functions. #9398DROP FUNCTIONS: Drops a user-defined function. #9561SHOW CREATE SOURCEandSHOW CREATE SINK: Shows the SQL statement used to create a source or sink. #9083SHOW INDEXES: Shows all indexes on a particular table. #9835
- SQL functions
- Adds support for trigonometric functions. #8838, #8918, #9064, #9203, #9259
- Adds support for degrees and radians functions. #9108
- Adds support for the
lag()andlead()window functions and theOVER()andEMIT ON WINDOW CLOSEclause. #9597, #9622, #9701 - Adds support for new aggregate functions, including
bool_and,bool_or,jsonb_agg, andjsonb_object_agg. #9452 - Adds support for
max(),min(), andcount()for timestamptz data. #9165 - Adds support for microseconds and milliseconds for
to_char()andto_timestamp(). #9257 - Adds support for multibyte Unicode in
overlay()andascii()functions. #9321 - Adds support for the
string_to_array()function. #9289 - Adds support for
array_positions(). #9152 - Adds support for
cardinality(). #8867 - Adds support for
array_remove(). #9116 - Adds support for
trim_array(). #9265 - Adds support for array range access. #9362
- Adds support for JSONB in UDF. #9103
- Adds support for
btrim()and updatestrim()to PostgreSQL standard syntax. #8985 - Adds support for
date_part(). #8830 - Expands
extract()with more fields. #8830 - Adds support for
proctime(), which returns the system time with time zone when a record is processed. #9088 - Adds support for
translate(),@(), andceiling(). #8998 - Adds support for
encode()anddecode(). #9351 - Adds support for the
intersectoperator. #9573 - Adds support for the default escape
\in alikeexpression. #9624 - Adds support for the
IS [NOT] UNKNOWNcomparison predicate. #9965 - Adds support for the
starts_with()string function and^@. #9967 - Adds support for unary
trunc,ln,log10(log),exp,cbrt(||/) mathematical functions. #9991
Connectors
- Adds support for ingesting CDC data from TiDB and sinking data to TiDB with the JDBC connector. #8708
- Adds support for ingesting CDC data from Citus. #8988
- Adds support for loading Pulsar secret key file from AWS S3. #8428, #8222
- Adds support for using an established AWS PrivateLink connection in a
CREATE SOURCE,CREATE TABLE, orCREATE SINKstatement for a Kafka source/sink. #9119, #9128, #9728, #9263 - Deprecates the
use_transactionfield in the Kafka sink connector. #9207 - Adds support for zstd compression type for Kafka connector. #9297
- Deprecates the
upsertproperty in the Kafka connector as it can be inferred from the row format. #9457 - Adds a new field
properties.sync.call.timeoutin the WITH clause of the Kafka source connector to control the timeout. #9005 - Adds support for a new row format
DEBEZIUM_MONGO_JSONin the Kafka source connector. #9250 - Adds CSV format support for the Kafka source connector. #9875
Cluster configuration changes
--data_directoryand--state_storemust be specified on CLI of the Meta Node, or the cluster will fail to start. #9170- Clusters will refuse to start if the specified object store URL identified by
state_storeanddata_directoryis occupied by another instance. Do not share the object store URL between multiple clusters. #9642
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.19.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
Starting from this version, we’ll respect semantic versioning conventions by using the middle number (
y , instead of z, in x.y.z) to indicate minor versions. That is why this is v0.18.0, not v0.1.18.Administration and troubleshooting
- Improves error messages by including the location of the statement in question. #8646
- Initial values of immutable system parameters can be specified via the meta-node command line. The initial values provided in the configuration file will be ignored. #8366
SQL features
- Adds initial support for user-defined functions. #8597 #8644 #8255 #7943
- Adds support for JSONB data type. #8256 #8181
- Adds support for
NULLS { FIRST | LAST }inORDER BYclauses. #8485 - New commands:
- New functions:
array_length: Returns the length of an array. #8636- String functions implemented with the help of ChatGPT. #8767 #8839
chr(integer)->varcharstarts_with(varchar, varchar)->booleaninitcap(varchar)->varcharlpad(varchar, integer)->varcharlpad(varchar, integer, varchar)->varcharrpad(varchar, integer)-> varcharrpad(varchar, integer, varchar)->varcharreverse(varchar)->varcharstrpos(varchar, varchar)->integerto_ascii(varchar)->varcharto_hex(integer)->varcharto_hex(bigint)->varchar)
- Improves the data type values of columns returned by
DESCRIBE. #8819 UPDATEcommands cannot update primary key columns. #8569- Adds support for microsecond precision for intervals. #8501
- Adds an optional parameter
offsettotumble()andhop()functions. #8490 - Data records that has null time values will be ignored by time window functions. #8146
- Improves the behaviors of the
expoperator when the operand is too large or small. #8309 - Supports process time temporal join, which enables the joining of an append-only stream (such as Kafka) with a temporal table (e.g. a materialized view backed by MySQL CDC). This feature ensures that any updates made to the temporal table will not affect previous results obtained from the temporal join. Supports
FOR SYSTEM_TIME AS OF NOW()syntax to express process time temporal join. #8480
Connectors
- Adds a new field
basetimeto the load generator connector for generating timestamp data. The load generator will take this field asnowand generates data accordingly. #8619 - Empty cells in CSV are now parsed as null. #8709
- Adds the Iceberg connector. #8508
- Adds support for the upsert type to the Kafka sink connector. #8168
- Removes the message name parameter for Avro data. #8124
- Adds support for AWS PrivateLink for Kafka source connector. #8247
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.18.0 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
Administration
- Adds a system catalog view
rw_catalog.rw_ddl_progress, with which users can view the progress of aCREATE INDEX,CREATE SINK, orCREATE MATERIALIZED VIEWstatement. #7914 - Adds the
pg_conversionandpg_enumsystem catalogs. #7964, #7706
SQL features
- Adds the
exp()function. #7971 - Adds the
pow()function. #7789 - Adds support for displaying primary keys in
EXPLAINstatements. #7590 - Adds support for descending order in
CREATE INDEXstatements. #7822 - Adds
SHOW PARAMETERSandALTER SYSTEMcommands to display and update system parameters. #7882, #7913
Connectors
- Adds a new parameter
match_patternto the S3 connector. With the new parameter, users can specify the pattern to filter files that they want to ingest from S3 buckets. For documentation updates, see Ingest data from S3 buckets. #7565 - Adds the PostgreSQL CDC connector. Users can use this connector to ingest data and CDC events from PostgreSQL directly. For documentation updates, see Ingest data from PostgreSQL CDC. #6869, #7133
- Adds the MySQL CDC connector. Users can use this connector to ingest data and CDC events from MySQL directly. For documentation updates, see Ingest data from MySQL CDC. #6689, #6345, #6481, #7133
- Adds the JDBC sink connector, with which users can sink data to MySQL, PostgreSQL, or other databases that are compliant with JDBC. #6493
- Add new parameters to the Kafka sink connector.
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.17 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
Administration
- Adds support for aborting a query in local mode with
Ctrl + C. #7444
SQL features
- Adds support for the
to_timestampfunction. #7060 - Adds support for the
RETURNINGclause in DML statements. #7094 - Breaking change: Deprecates
CREATE MATERIALIZED SOURCE. To create a materialized source, create a table and include the newly added connector settings. #7281, #7110 - Adds support for the
candiflags inregex_match()andregex_matches()functions. #7135 - Adds support for
SHOW CREATE TABLE. You can use this statement to show the definition of a table. #7152 - Adds support for the
pg_stat_activitysystem catalog and several system functions. #7274 - Adds the
_rw_kafka_timestampparameter to show the timestamps of Kafka messages. Users can now specify the scope of Kafka messages by timestamps. #7275, #7150 - Adds support for displaying PostgreSQL and RisingWave versions in
version(). #7314 - Adds support for displaying internal tables using the
SHOW INTERNAL TABLESstatement. #7348 - Adds support for
SET VISIBILITY_MODEYou can use this session variable to configure whether only checkpoint data is readable for batch query. #5850 - Adds support for
SET STREAMING_PARALLELISM. You can use this session variable to configure parallelism for streaming queries. #7370
Connectors
- Adds support for generating array and struct data using the datagen connector. #7099
- Adds the S3 source connector, with which users can ingest data in CSV format from S3 locations. For data ingestion from files, CSV is the only supported format and the files must be placed on S3. #6846
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.16 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
Installation and deployment
- Parallelism and available memory of Compute Nodes are now command-line arguments and removed from the configuration file. #6767
- The default barrier interval is set to 1 second. #6553
- Adds support for meta store backup and recovery. #6737
SQL features
- Adds support for
SHOW CREATE MATERIALIZED VIEWandSHOW CREATE VIEWto show how materialized and non-materialized views are defined. #6921 - Adds support for
CREATE TABLE IF NOT EXISTS. #6643 - A sink can be created from a SELECT query. #6648
- Adds support for struct casting and comparison. #6552
- Adds pg_catalog views and system functions. #6982
- Adds support for
CREATE TABLE AS. #6798 - Ads the initial support for batch query on Kafka source. #6474
- Adds support for
SET QUERY_EPOCHto query historical data based on meta backup. #6840
Connectors
- Improves the handling of schema errors for Avro and Protobuf data. #6821
- Adds two options to the datagen connector to make it possible to generate increasing timestamp values. #6591
Observability
- Adds metrics for the backup manager in Grafana. #6898
- RisingWave Dashboard can now fetch data from Prometheus and visualize it in charts. #6602
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.15 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
SQL features
PRIMARY KEYconstraint checks can be performed on materialized sources and tables but not on non-materialized sources. For tables or materialized sources that enabledPRIMARY KEYconstraints, if you insert data to an existing key, the new data will overwrite the old data. #6320 #6435- Adds support for timestamp with time zone data type. You can use this data type in time window functions, and convert between it and timestamp (without time zone). #5855 #5910 #5968
- Adds support for
UNIONandUNION ALLoperators. #6363 #6397 - Implements the
rank()function to support a different mode of Top-N queries. #6383 - Adds support for logical views (
CREATE VIEW). #6023 - Adds the
data_trunc()function. #6365 - Adds the system catalog schema. #6227
- Displays error messages when users enter conflicting or redundant command options. #5933
Connectors
- Adds support for the Maxwell Change Data Capture (CDC) format. #6057
- Protobuf schema files can be loaded from Web locations in
s3://,http://, orhttps://formats. #6114 #5964 - Adds support for Confluent Schema Registry for Kafka data in Avro and Protobuf formats. #6289
- Adds two options to the Kinesis connector. Users can specify the startup mode and optionally the sequence number to start with. #6317
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.14 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
SQL features
-
SQL commands:
- Improves the formatting of response messages of
EXPLAINstatements. #5541
- Improves the formatting of response messages of
-
SQL functions:
-
to_char()now supports specifying output format in lowercase. #5032to_char(timestamp '2006-01-02 15:04:05', 'yyyy-mm-dd hh24:mi:ss')→2006-01-02 15:04:05 -
generate_seriesnow supports negative steps. #5231 - Adds support for sum/min/max functions over interval-type data. #5105, #5549
- Adds support for array concatenation. #5060, #5345
- Adds support for specifying empty arrays. #5402
-
Casting from array to varchar is now supported. #5081
array[1,2]::varchar→{1,2} -
Casting from varchar to integer allows leading and trailing spaces. #5452
' 1 '::int→1
-
-
Adds new system catalog and psql meta-commands. #5127, #5742
\d: Lists all relations in the current database. (Materialized) sources are not supported yet.\dt: Lists all tables in the current database.\dm: Lists all materialized views in the current database.\di: Lists all indexes in the current database.pg_catalog.pg_index: Contains information about indexes.
Connectors
- Nested columns are now supported for the datagen connector. #5550
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.13 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
SQL features
- SQL commands:
EXPLAINnow supports specifying options. Supported options:trace,verbose, andtype. Unlike PostgreSQL, each option should be separated by a comma and wrapped by parentheses as a whole. #4730- Adds support for
ALTER USER. #4261 CREATE/ALTER USERnow has new optionsCREATEUSERandNOCREATEUSER, which specify whether or not the user has the privilege to create, alter, or drop other users. #4447- Adds support for EXPLAIN CREATE SINK. #4430
- SQL functions:
- Adds support for new system information functions:
current_schema,current_schema(), andsession_user. #4358
- Adds support for new system information functions:
- The
pg_namespacecatalog now has a new namespace columnnspaclfor storing access privileges. #4326
Connectors
-
Some connector parameters were renamed. The old parameter names are still functional but may be deprecated in the future. #4503
-
Kafka & Redpanda
kafka.brokers->properties.bootstrap.serverkafka.topic->topickafka.scan.startup.mode->scan.startup.modekafka.time.offset->scan.startup.timestamp_milliskafka.consumer.group->consumer.group.id
-
Kinesis
kinesis.stream.name->streamkinesis.stream.region->aws.regionkinesis.endpoint->endpointkinesis.credentials.access->aws.credentials.access_key_idkinesis.credentials.secret->aws.credentials.secret_access_keykinesis.credentials.session_token->aws.credentials.session_tokenkinesis.assumerole.arn->aws.credentials.role.arnkinesis.assumerole.external_id->aws.credentials.role.external_id
-
Pulsar
pulsar.topic->topicpulsar.admin.url->admin.urlpulsar.service.url->service.urlpulsar.scan.startup.mode->scan.startup.modepulsar.time.offset->scan.startup.timestamp_millis
-
Kafka & Redpanda
-
The row format name,
debezium json, for CDC stream sources, has been renamed todebezium_json. #4494
Configuration changes
- The default batch query execution mode was changed from distributed to local. #4789
Assets
- Run this version from Docker:
docker run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/risingwavelabs/risingwave:v0.1.12 playground - Prebuilt library for Linux is not available in this release.
- Source code (zip)
- Source code (tar.gz)
SQL features
- New SQL functions:
overlay(): Replaces a substring. #3671generate_series(): Generates a series of values from the starting point to the ending point. #4030regexp_match(),regexp_matches(): Compares a string against a regular expression pattern and returns matched substrings. #3702 #4062string_agg (): Concatenates the values into a string. #3952 #4183current_database(): Returns the current database. #3650
- New SQL commands:
- Support for lookup joins. Currently, lookup joins can only be performed in local query mode. To use lookup joins, users need to set the configuration parameter
rw_batch_enable_lookup_jointo true. #3859 #3763 - An
ORDER BYclause in theCREATE MATERIALIZED VIEWstatement is allowed but not considered as part of the definition of the materialized view. It is only used in the initial creation of the materialized view. It is not used during refreshes. This is a behavior change due to the introduction of parallel table scans. #3670 - Support for filter clauses on aggregate functions. #4114
Connectors
- RisingWave can now sink data to Kafka topics in append-only mode and Debezium mode. #3923 #3682 #3674
- Syntax change for
CREATE SOURCE: A parameter name is no longer wrapped by single quotation marks. #3997. See the example:- Old:
CREATE SOURCE s1 WITH ( 'connector' = 'kafka', 'kafka.topic' = 'kafka_1_partition_topic', 'kafka.brokers' = '127.0.0.1:29092' ) ROW FORMAT json; - New:
CREATE SOURCE s WITH ( connector = 'kafka', kafka.topic = 'kafka_1_partition_topic', kafka.brokers = '127.0.0.1:29092' ) ROW FORMAT json;
- Old:
Assets
- Run this version from Docker:
run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/risingwavelabs/risingwave:v0.1.11 playground - Prebuilt library for Linux
- Source code (zip)
- Source code (tar.gz)
SQL features
SQL operators and functions
- Support string concatenation operator
||. #3147 - Support interval comparison. #3222
- Support dividing intervals by integers, floats, or decimals. #3441
- Intervals multiplying intervals by floats. #3641
- Support more temporal operations. #3472
- Support Common Table Expressions (CTEs) as input of time window functions. #3188
- Add these new functions:
concat()for concatenating strings #3091repeat()for repeating string #3148octet_length()and bit_length() for getting string length #3526Row()for constructing rows #2914 #3152pg_typeof()for getting data types of values #3494current_database()for getting the name of the current database in the session #3650approx_count_distinct()for distinct counting #3121unnest()for expanding nested tables to rows #3017
- Support
count(),min(), andmax()functions on these data types: interval, timestamp, varchar, and date. #3069
SQL commands
- Support
EXPLAIN CREATE INDEX. #3229 - Add cascade and restrict options in
REVOKEcommands. #3363 - Expand the
CREATE TABLEsyntax to support creating append-only tables. #3058 - Support the
CREATE USERcommand and user authentication. #3074
Data types
- Support implicit casts from single-quoted literals. #3487
- Add string as an alias for data type varchar. #3094
- Support string intervals. #3037
Database management
- Add the default super user “postgres”. #3127
- The default schema name is changed to “public” from “dev”. #3166
Connectors
- Add random seed for the Datagen Source Connector. #3124