> ## Documentation Index > Fetch the complete documentation index at: https://docs.risingwave.com/llms.txt > Use this file to discover all available pages before exploring further. # Release notes > This page summarizes changes in each version of RisingWave, including new features and important bug fixes. Support for certain earlier versions will end following the release of v3.0. Please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). ## SQL features * SQL commands: * Supports `SHOW CURSORS` and subscription cursors across all frontend nodes. [#24107](https://github.com/risingwavelabs/risingwave/pull/24107) * Supports `DROP SECRET CASCADE` to automatically drop dependent objects when dropping a secret. [#24581](https://github.com/risingwavelabs/risingwave/pull/24581) * Supports altering the retention period of a subscription with `ALTER SUBSCRIPTION ... SET (retention = ...)`. [#24842](https://github.com/risingwavelabs/risingwave/pull/24842) * Supports altering the backfill parallelism for streaming jobs. [#24323](https://github.com/risingwavelabs/risingwave/pull/24323) * Supports `BACKUP` and meta snapshot management commands. [#25042](https://github.com/risingwavelabs/risingwave/pull/25042) * Supports `ALTER WATERMARK` for live tables without recreating them. [#25651](https://github.com/risingwavelabs/risingwave/pull/25651) * Supports `ALTER SOURCE CONNECTOR` to update CDC source connection passwords. [#25435](https://github.com/risingwavelabs/risingwave/pull/25435) * Supports TLS for webhook source endpoints and multi-column webhook source tables. [#25443](https://github.com/risingwavelabs/risingwave/pull/25443) * Supports atomically adding columns to Iceberg tables. [#24756](https://github.com/risingwavelabs/risingwave/pull/24756) * Supports using secrets as arguments in UDF call expressions. [#25212](https://github.com/risingwavelabs/risingwave/pull/25212) * Adds the sink's target table to the system catalog and `DESCRIBE` statement output. [#24893](https://github.com/risingwavelabs/risingwave/pull/24893) * Schema owners now automatically have the `USAGE` privilege on their own schema. [#24692](https://github.com/risingwavelabs/risingwave/pull/24692) * `statement_timeout` values without a unit are now interpreted as milliseconds, matching PostgreSQL behavior. [#25557](https://github.com/risingwavelabs/risingwave/pull/25557) * Duplicate CTE names within the same `WITH` clause are now rejected at parse time. [#24676](https://github.com/risingwavelabs/risingwave/pull/24676) * SQL functions & operators: * Adds `crc32()` and `crc32c()` functions for computing CRC checksums on `bytea` values. [#24964](https://github.com/risingwavelabs/risingwave/pull/24964) * `jsonb_agg` now supports a wildcard (`*`) parameter. [#25282](https://github.com/risingwavelabs/risingwave/pull/25282) * Adds array overlap predicates for checking element membership across arrays. [#25597](https://github.com/risingwavelabs/risingwave/pull/25597) * System catalog: * Adds column `schema_name` to `rw_catalog.rw_materialized_views` and `rw_catalog.rw_subscriptions`, enabling schema-based filtering without extra joins. [#25248](https://github.com/risingwavelabs/risingwave/pull/25248) * `rw_fragment_backfill_progress` now includes locality backfill progress. [#24955](https://github.com/risingwavelabs/risingwave/pull/24955) * Adds system catalog `rw_iceberg_compaction_schedules` to view scheduled Iceberg compaction jobs. [#25013](https://github.com/risingwavelabs/risingwave/pull/25013) ## DataFusion engine * DataFusion is now enabled by default for batch queries. [#25198](https://github.com/risingwavelabs/risingwave/pull/25198) * Adds support for table functions in DataFusion. [#24715](https://github.com/risingwavelabs/risingwave/pull/24715) * Adds support for `LogicalExpand` operator in DataFusion. [#24767](https://github.com/risingwavelabs/risingwave/pull/24767) * Maps additional array functions to DataFusion equivalents. [#24859](https://github.com/risingwavelabs/risingwave/pull/24859) * Improves OOM handling and adds memory reserve headroom for DataFusion operators. [#24736](https://github.com/risingwavelabs/risingwave/pull/24736), [#24883](https://github.com/risingwavelabs/risingwave/pull/24883) ## Optimizer * Supports materialized view selection for batch queries: when an equivalent pre-computed MV exists, batch queries automatically use it for improved performance. [#24611](https://github.com/risingwavelabs/risingwave/pull/24611), [#24751](https://github.com/risingwavelabs/risingwave/pull/24751), [#24782](https://github.com/risingwavelabs/risingwave/pull/24782), [#25046](https://github.com/risingwavelabs/risingwave/pull/25046) ## Connectors * **PostgreSQL CDC**: * Supports PostgreSQL 18. [#24765](https://github.com/risingwavelabs/risingwave/pull/24765) * Monitors upstream max LSN for more accurate replication lag tracking. [#24919](https://github.com/risingwavelabs/risingwave/pull/24919) * Maps upstream PostgreSQL composite types to `text`. [#25153](https://github.com/risingwavelabs/risingwave/pull/25153) * Ingests upstream PostgreSQL `pgvector` type. [#25184](https://github.com/risingwavelabs/risingwave/pull/25184) * Supports TOAST unchanged-value placeholders for vector columns. [#25863](https://github.com/risingwavelabs/risingwave/pull/25863) * **MySQL CDC**: Adds `RW_JAVA_LOG` environment variable to control Java connector logging granularity. [#24404](https://github.com/risingwavelabs/risingwave/pull/24404) * **SQL Server CDC**: * Adds consume-offset and replication lag monitors. [#25124](https://github.com/risingwavelabs/risingwave/pull/25124) * Supports tables with multiple CDC capture instances. [#25317](https://github.com/risingwavelabs/risingwave/pull/25317) * **Kafka**: * Adds Kafka consumer lag catalog for monitoring consumer group lag. [#24708](https://github.com/risingwavelabs/risingwave/pull/24708) * Supports altering Kafka SSL configurations at runtime. [#25717](https://github.com/risingwavelabs/risingwave/pull/25717) * **Iceberg**: * Supports configurable write-Parquet properties for Iceberg sink and compaction. [#24612](https://github.com/risingwavelabs/risingwave/pull/24612) * Supports sort order keys for Iceberg sink. [#25202](https://github.com/risingwavelabs/risingwave/pull/25202) * Supports Iceberg V3 table format with primary-key index. [#25346](https://github.com/risingwavelabs/risingwave/pull/25346) * Adds Iceberg source ingestion metrics. [#25122](https://github.com/risingwavelabs/risingwave/pull/25122) * **JDBC sink**: Supports altering the JDBC sink URL, username, and password at runtime. [#24846](https://github.com/risingwavelabs/risingwave/pull/24846) * **Delta Lake sink**: Supports exactly-once delivery. [#25159](https://github.com/risingwavelabs/risingwave/pull/25159) * **HTTP sink** (new): Supports writing data to HTTP endpoints, including dynamic URL targets. [#25396](https://github.com/risingwavelabs/risingwave/pull/25396), [#25896](https://github.com/risingwavelabs/risingwave/pull/25896) * **WebSocket source** (new): Supports ingesting data from WebSocket endpoints with async acknowledgments. [#25444](https://github.com/risingwavelabs/risingwave/pull/25444) * **PubSub source**: Supports adaptive splits and streaming pull for improved throughput. [#25074](https://github.com/risingwavelabs/risingwave/pull/25074) * **Pulsar source**: Exposes configurable operation retry controls. [#25467](https://github.com/risingwavelabs/risingwave/pull/25467) * **MQTT source**: Supports Protobuf encoding. [#25403](https://github.com/risingwavelabs/risingwave/pull/25403) * **File source**: Supports case-insensitive column name matching for Parquet files. [#24734](https://github.com/risingwavelabs/risingwave/pull/24734) * **Deprecated**: Maxwell, Canal, and Citus CDC sources are disabled. [#25534](https://github.com/risingwavelabs/risingwave/pull/25534) ## Cluster configuration changes * Adds `join_hash_map_evict_interval_rows` to the `[streaming.developer]` configuration section. This controls how often the join hash map cache is evicted in hash join and as-of join executors. Eviction runs after every `join_hash_map_evict_interval_rows` rows processed. The default is `16`, matching the previous hard-coded value. Values smaller than `1` are clamped to `1` by the executor. [#25210](https://github.com/risingwavelabs/risingwave/pull/25210) * Supports setting session parameters via the PostgreSQL `StartupMessage` `options` field. [#24754](https://github.com/risingwavelabs/risingwave/pull/24754) * Adds `RW_TRACING_EXTRA_ATTRIBUTES` environment variable for injecting custom attributes into distributed traces. [#24946](https://github.com/risingwavelabs/risingwave/pull/24946) * Default adaptive parallelism is now capped at 64 workers to prevent over-allocation on large clusters. [#24678](https://github.com/risingwavelabs/risingwave/pull/24678) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v3.0.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v3.0.0/risingwave-v3.0.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v3.0.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v3.0.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v3.0.0/risectl-v3.0.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v2.8.0...v3.0.0). Support for certain earlier versions will end following the release of v2.8. Please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). Versions `v2.8.0-rc.1`, `v2.8.0`, and `v2.8.1` are deprecated. We recommend upgrading to `v2.8.2` or later. ## SQL features * SQL commands: * Supports watermark derivation for AsOf joins, enabling time-based windowed asynchronous joins to propagate watermarks from both input streams through equality and inequality join conditions. [#23803](https://github.com/risingwavelabs/risingwave/pull/23803) ## Connectors * MySQL CDC source now monitors the progress of binlog offsets in both upstream and downstream processes. [#23692](https://github.com/risingwavelabs/risingwave/pull/23692) * Supports `geometry` type for PostgreSQL CDC, mapping upstream PostGIS geometry columns to RisingWave `bytea` (stored as raw EWKB bytes) for both snapshot and incremental paths. [#24491](https://github.com/risingwavelabs/risingwave/pull/24491) * Supports TCP keepalive configuration for PostgreSQL Rust sink and JDBC sink connections to prevent silent connection drops. [#24388](https://github.com/risingwavelabs/risingwave/pull/24388) * Iceberg: Enforces merge-on-read for append-only tables to ensure data consistency. [#24487](https://github.com/risingwavelabs/risingwave/pull/24487) * Redshift and Snowflake v2 sinks: Adds new parameters `writer_target_interval_seconds` (alias: `schedule_seconds`) to configure the interval for writing data to the target, and `write_intermediate_interval_seconds` to configure the interval for writing data to intermediate storage. [#23086](https://github.com/risingwavelabs/risingwave/pull/23086) * Adds a new optional `schema.registry.ca_pem_path` parameter for Confluent Schema Registry connections. Set it to a file path to load a custom root CA certificate, or to `'ignore'` to skip TLS certificate verification (not recommended for production). Supports secret management via `SECRET ... AS FILE`. [#17826](https://github.com/risingwavelabs/risingwave/pull/17826) ## Cluster configuration changes * Adds session variable `enable_serverless_backfill` to enable serverless backfill. Defaults to `false`. [#24686](https://github.com/risingwavelabs/risingwave/pull/24686) * Adds configuration option `time_travel_vacuum_max_version_count` (default: `10000`) to limit the number of metadata versions retained during time travel vacuum operations. [#24626](https://github.com/risingwavelabs/risingwave/pull/24626) * Improves locality backfill derivation for aggregation nodes, extending index-based optimizations to the aggregation backfill pipeline. [#24710](https://github.com/risingwavelabs/risingwave/pull/24710) * Single-node deployments now correctly enforce the `rwu_limit` specified in the license key. [#24796](https://github.com/risingwavelabs/risingwave/pull/24796) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.8.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.8.0/risingwave-v2.8.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.8.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.8.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.8.0/risectl-v2.8.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.7...release-2.8). Support for certain earlier versions will end following the release of v2.7. Please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). Versions `v2.7.0-rc.1`, `v2.7.0`, `v2.7.1`, `v2.7.2`, and `v2.7.3` are deprecated. We recommend upgrading to `v2.7.4` or later. ## Upgrade-time behavior changes In v2.7.0, following the refactoring of `sink-into-table`, the system will automatically migrate fragments of all legacy tables to a unified pattern. * **Migration process**: Upon upgrading and starting the cluster, a background task will automatically initiate the migration for all tables. You can see meta logs like: * `"migrated table fragments table_id={table_id}"` indicate a successful migration. * `"failed to migrate legacy table fragments"` indicate an error on migration. * **Impact**: Migration errors triggering these logs will not affect existing jobs or system availability. The system will continuously retry the migration. * If the migration fails but does not impact your usage, please report the issue via Slack or GitHub issue and wait for bug fix in the next version. * To disable automatic migration, set `enable_legacy_table_migration = false` in the `[meta]` section of your `toml` file (requires meta node restart). * **Limitations & Workaround**: You cannot create a new `sink-into-table` or drop an existing one on a table that has not completed migration. If the background migration is stuck or taking too long and you urgently need to perform these DDL operations, you can force a migration by running: ``` ALTER TABLE {table_name} ADD COLUMN {col_name} int; ALTER TABLE {table_name} DROP COLUMN {col_name}; ``` After this, you can use it as a normal table. ## SQL features * SQL commands: * Supports vector index Top-N lookups in streaming queries, enabling MVs and sinks to enrich each row with nearest-neighbor results. [#23288](https://github.com/risingwavelabs/risingwave/pull/23288) * Supports `ALTER FRAGMENT ... SET PARALLELISM` command to alter the parallelism of streaming fragments. [#23523](https://github.com/risingwavelabs/risingwave/pull/23523) * Reduces reserved keywords in `EXPLAIN` options (e.g., `BACKFILL`, `TRACE`) by parsing them as identifiers. [#23208](https://github.com/risingwavelabs/risingwave/pull/23208) * Adds schema name to `name` column in the output of `SHOW [TABLE|VIEW|SOURCE|SINK|...]` commands. [#23119](https://github.com/risingwavelabs/risingwave/pull/23119) * When `DROP`/`CANCEL` takes longer than `slow_ddl_notification_secs` (30s), you can run `RECOVER` for immediate effect. [#23613](https://github.com/risingwavelabs/risingwave/pull/23613) * Supports `ALTER TABLE ... ADD COLUMN` command on tables with incoming sinks. [#23766](https://github.com/risingwavelabs/risingwave/pull/23766) * Supports querying vector index in append-only streaming. [#23288](https://github.com/risingwavelabs/risingwave/pull/23288) * SQL functions & operators: * Supports parsing `STRUCT< >` as empty struct type. [#23099](https://github.com/risingwavelabs/risingwave/pull/23099) * Supports `rw_cluster_id()` function to return the cluster's unique identifier as a `VARCHAR`. [#23260](https://github.com/risingwavelabs/risingwave/pull/23260) * System catalog: * Adds system catalog `rw_recovery_info` to display the database-level recovery state. [#23728](https://github.com/risingwavelabs/risingwave/pull/23728) * Adds column `parallelism_policy` in system catalog `rw_fragments`. [#23524](https://github.com/risingwavelabs/risingwave/pull/23524) ## Connectors * **Breaking change:** When decoding Protobuf messages, RisingWave returns `NULL` for missing `optional` fields and default values for all other fields. [#23120](https://github.com/risingwavelabs/risingwave/pull/23120) * Supports using `CREATE SOURCE` directly for UPSERT-format sources requiring a primary key, without creating a table first. [#22856](https://github.com/risingwavelabs/risingwave/pull/22856) * File sink now uses UUIDs instead of `executor_id` in output file names. [#23981](https://github.com/risingwavelabs/risingwave/pull/23981) * Supports on-demand manual refresh for Iceberg batch tables with position deletes and equity deletes. [#23527](https://github.com/risingwavelabs/risingwave/pull/23527) * **Technical preview**: Supports periodic automatic refresh for batch tables with persistent state tracking and improved observability. [#23737](https://github.com/risingwavelabs/risingwave/pull/23737) * **Technical preview**: Supports files-with-delete Iceberg compaction and introduces additional compaction parameters. [#23660](https://github.com/risingwavelabs/risingwave/pull/23660), [#23809](https://github.com/risingwavelabs/risingwave/pull/23809) * Supports Azure Data Lake Storage Gen2 for Iceberg sink when using REST catalog. [#23350](https://github.com/risingwavelabs/risingwave/pull/23350) * Supports configuring S3 and Glue with independent credentials for Iceberg connector when using Glue catalog. [#23775](https://github.com/risingwavelabs/risingwave/pull/23775) * Support assuming IAM roles for S3 and Glue when `enable_config_load` is enabled for Iceberg connector when using Glue catalog. [#23788](https://github.com/risingwavelabs/risingwave/pull/23788) * Supports key-pair authentication using RSA private keys for Snowflake sink. [#23525](https://github.com/risingwavelabs/risingwave/pull/23525) * Supports JDBC sink connector for SQL Server. [#23416](https://github.com/risingwavelabs/risingwave/pull/23416) * Changes the default value of `commit_checkpoint_interval` from 10 to 60 for Iceberg sink. [#23552](https://github.com/risingwavelabs/risingwave/pull/23552) * Supports `is_exactly_once` for Iceberg sink by default. [#23553](https://github.com/risingwavelabs/risingwave/pull/23553) * Supports vended credentials for Iceberg connector when using REST catalog. [#23567](https://github.com/risingwavelabs/risingwave/pull/23567) * Supports configuring `postgres.is.aws.rds` to specify if the upstream PostgreSQL is hosted on AWS RDS. [#23420](https://github.com/risingwavelabs/risingwave/pull/23420) * Supports using Base64-encoded JSON strings for BigQuery credentials. [#23100](https://github.com/risingwavelabs/risingwave/pull/23100) * Supports TLS connections to Redis sink using `rediss://` protocol. [#23223](https://github.com/risingwavelabs/risingwave/pull/23223) * Supports Redis Streams for Redis sink. [#23412](https://github.com/risingwavelabs/risingwave/pull/23412) * Supports `COPY () TO STDOUT` syntax to load data from PostgreSQL tables. [#23198](https://github.com/risingwavelabs/risingwave/pull/23198) * Supports ack for Pulsar source, enabling control over broker resending of unacknowledged messages. [#22158](https://github.com/risingwavelabs/risingwave/pull/22158) * Supports `force_compaction` on sinks to enable buffering and compaction of updates when the stream key is the same as the primary key. [#23681](https://github.com/risingwavelabs/risingwave/pull/23681) ## Installation and deployment * Counts the CPU cores of all nodes (instead of only compute nodes) in the cluster against the `cpu_core_limit` specified in the license key. [#23390](https://github.com/risingwavelabs/risingwave/pull/23390) * Changes `cpu_core_limit` in the license key to `rwu_limit` to limit both CPU cores and total cluster memory. [#23403](https://github.com/risingwavelabs/risingwave/pull/23403) * **Breaking change**: Requires the correct `aud` claim for authentication. [#23155](https://github.com/risingwavelabs/risingwave/pull/23155) * Supports LDAP authentication to verify users against an external LDAP directory server. [#23150](https://github.com/risingwavelabs/risingwave/pull/23150) ## Cluster configuration changes * Adds session variable `enable_index_selection` to control whether to enable index selection for both streaming and batch queries. [#23092](https://github.com/risingwavelabs/risingwave/pull/23092) * **Technical preview**: Adds session variable `enable_locality_backfill` to control locality backfilling, which extends index-based optimizations to the entire backfill pipeline. [#23275](https://github.com/risingwavelabs/risingwave/pull/23275) * Adds configuration option `RW_SSL_ENFORCE` to control SSL enforcement for security. [#23306](https://github.com/risingwavelabs/risingwave/pull/23306) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.7.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.7.0/risingwave-v2.7.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.7.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.7.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.7.0/risectl-v2.7.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.6...release-2.7). Support for certain earlier version will end following the release of v2.6, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). ## SQL features * SQL commands: * **Technical preview**: Supports using `CREATE INDEX` to create vector indexes, including flat and HNSW methods, and allows creating vector indexes on function expressions. [#22432](https://github.com/risingwavelabs/risingwave/pull/22432), [#22022](https://github.com/risingwavelabs/risingwave/pull/22022), [#22808](https://github.com/risingwavelabs/risingwave/pull/22808) * `CREATE SINK` supports `auto.schema.change` option for Elasticsearch, Redshift, and Snowflake. [#22611](https://github.com/risingwavelabs/risingwave/pull/22611) * `ALTER SOURCE` and `ALTER TABLE` support `CONNECTOR WITH` clause. [#20780](https://github.com/risingwavelabs/risingwave/pull/20780) * `DROP CONNECTION` supports `CASCADE` option to automatically drop dependent objects, except for Iceberg connections. [#22557](https://github.com/risingwavelabs/risingwave/pull/22557) * `DROP FUNCTION` supports `CASCADE` option to automatically remove dependent objects. [#22991](https://github.com/risingwavelabs/risingwave/pull/22991) * Supports using multiple version columns to resolve upsert conflicts. [#22931](https://github.com/risingwavelabs/risingwave/pull/22931) * SQL functions & operators: * Supports new data type `vector(n)`. [#22019](https://github.com/risingwavelabs/risingwave/pull/22019) * Supports `jsonb_to_array` function to convert a JSONB array into a SQL array of JSONB elements. [#22777](https://github.com/risingwavelabs/risingwave/pull/22777) * `openai_embedding` accepts a single constant JSONB `config` parameter instead of separate `api_key` and `model` arguments. [#22960](https://github.com/risingwavelabs/risingwave/pull/22960) * Supports `arg_min` and `arg_max` aggregate functions. [#22895](https://github.com/risingwavelabs/risingwave/pull/22895) * System catalog: * Adds system function `has_database_privilege`. [#22576](https://github.com/risingwavelabs/risingwave/pull/22576) * Adds system function `rw_test_feature` and system catalog `rw_features`. [#22526](https://github.com/risingwavelabs/risingwave/pull/22526) ## Connectors * **Public preview:** Iceberg table engine supports `copy-on-write` write mode for enhanced compaction. [#22713](https://github.com/risingwavelabs/risingwave/pull/22713) * Supports Redshift and Snowflake v2 sink connectors. You can use intermediate table and leverage S3 to improve sink performance, and auto schema change is supported to automatically add new columns. [#22984](https://github.com/risingwavelabs/risingwave/pull/22984) * Iceberg table engine and Iceberg sink support `VACUUM [FULL] schema_name.object_name` syntax. [#22942](https://github.com/risingwavelabs/risingwave/pull/22942) * NATS connector no longer creates stream automatically unless new option `allow_create_stream` is explicitly set to `true`. [#22315](https://github.com/risingwavelabs/risingwave/pull/22315) * Kafka source supports altering `group.id.prefix` parameter at runtime. [#22492](https://github.com/risingwavelabs/risingwave/pull/22492) * For Kafka source, when querying with a timestamp beyond the latest message, it starts from the latest offset instead of resetting to the earliest. [#22413](https://github.com/risingwavelabs/risingwave/pull/22413) * MQTT source supports multiple comma-separated topics. [#22074](https://github.com/risingwavelabs/risingwave/pull/22074) * Postgres CDC supports parallelized CDC backfill. [#22536](https://github.com/risingwavelabs/risingwave/pull/22536) * For PostgreSQL CDC, if you’ve already created a publication, fewer permissions are required. [#22807](https://github.com/risingwavelabs/risingwave/pull/22807) * Supports TOASTed data from PostgreSQL when using the CDC connector. [#22852](https://github.com/risingwavelabs/risingwave/pull/22852) ## Installation and deployment * Supports deploying lakekeeper as an Iceberg REST catalog via Docker compose or Kubernetes. [#23045](https://github.com/risingwavelabs/risingwave/pull/23045) * Supports using `hashicorp_vault` as secret backend with Token or AppRole authentication methods. [#22627](https://github.com/risingwavelabs/risingwave/pull/22627) ## Cluster configuration changes * Supports Memory-Only Mode for faster, low-latency queries by fully loading operator states into memory. You can configure it at the operator level to balance performance and memory usage. [#22857](https://github.com/risingwavelabs/risingwave/pull/22857) * Adds session variable `streaming_separate_consecutive_join` to separate consecutive stream joins to improve performance. [#22259](https://github.com/risingwavelabs/risingwave/pull/22259) * Adds session variable `streaming_join_encoding` to switch join encoding type. [#22416](https://github.com/risingwavelabs/risingwave/pull/22416) * Adds session variable `streaming_separate_sink` to separate sink. [#22827](https://github.com/risingwavelabs/risingwave/pull/22827) * Adds `protect_drop_table_with_incoming_sink` to block table drops if incoming sinks exist. [#22347](https://github.com/risingwavelabs/risingwave/pull/22347) * Adds `streaming_now_progress_ratio` to decouple `now_timestamp` from `epoch_timestamp`. [#22487](https://github.com/risingwavelabs/risingwave/pull/22487) ## Access control list (ACL) * Adds a new superuser `rwadmin` for the Cloud control plane. [#22327](https://github.com/risingwavelabs/risingwave/pull/22327) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.6.0 standalone` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.6.0/risingwave-v2.6.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.6.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.6.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.6.0/risectl-v2.6.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.5...release-2.6). Support for certain earlier version will end following the release of v2.5, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). ## SQL features * SQL commands: * **Technical preview**: Adds option `backfill_order` to define the ingestion order of streams during backfill in case of join amplification. [#20967](https://github.com/risingwavelabs/risingwave/pull/20967) * Supports `ALTER DEFAULT PRIVILEGES` command to set privileges for objects created in the future. [#22191](https://github.com/risingwavelabs/risingwave/pull/22191) * The `public` schema is now managed by `GRANT` / `REVOKE` command. New non-super users should first be granted privileges before creating objects. [#20201](https://github.com/risingwavelabs/risingwave/pull/20201) * Supports `ALTER COLUMN TYPE` syntax in `ALTER TABLE` to change the data type of struct columns. Changing between primitive types (e.g., `int` → `bigint`) is not supported. [#20940](https://github.com/risingwavelabs/risingwave/pull/20940) * `SHOW PROCESSLIST` now displays queries from all serving nodes, making it easier to troubleshoot distributed queries. [#21952](https://github.com/risingwavelabs/risingwave/pull/21952) * Supports `CONNECTOR WITH` clause for `ALTER SINK` command. [#20691](https://github.com/risingwavelabs/risingwave/pull/20691) * Supports per-database configuration for `barrier_interval_ms` and `checkpoint_frequency`. [#22178](https://github.com/risingwavelabs/risingwave/pull/22178) * SQL functions & operators: * **Technical preview**: Supports `approx_count_distinct` to estimate the number of distinct values. [#21516](https://github.com/risingwavelabs/risingwave/pull/21516), [#21545](https://github.com/risingwavelabs/risingwave/pull/21545) * **Technical preview**: Supports `map_filter` function to filter map data. [#22077](https://github.com/risingwavelabs/risingwave/pull/22077) * Supports `array_flatten` to flatten an array of arrays. [#21640](https://github.com/risingwavelabs/risingwave/pull/21640) * Supports `openai_embedding` function to generate embeddings. [#22242](https://github.com/risingwavelabs/risingwave/pull/22242) * Supports `WINDOW` clause and window function over named window. [#22024](https://github.com/risingwavelabs/risingwave/pull/22024) * Supports the concatenation operator (`||`), position, and overlay functions for `bytea` data type. [#22076](https://github.com/risingwavelabs/risingwave/pull/22076) * System catalog: * Adds system catalog rw\_fragment\_backfill\_progress to track the progress of each backfill partition. [#22124](https://github.com/risingwavelabs/risingwave/pull/22124) ## Connectors * **Technical preview**: Webhook source supports receiving batches of updates at once. [#22075](https://github.com/risingwavelabs/risingwave/pull/22075) * **Technical preview**: Supports `sink_decouple` for creating sink into table. [#21784](https://github.com/risingwavelabs/risingwave/pull/21784) * **Breaking change**: Adds escape syntax and usage instructions for the TEMPLATE encoding format. [#21431](https://github.com/risingwavelabs/risingwave/pull/21431) * Supports Iceberg compaction for Iceberg table engine. [#22043](https://github.com/risingwavelabs/risingwave/pull/22043) * Supports `hosted_catalog=true` to enable RisingWave-managed SQL catalog in Iceberg table engine. [#21351](https://github.com/risingwavelabs/risingwave/pull/21351) * Supports `enable_config_load` for Iceberg sink, source, connection, and Iceberg table engine. [#21938](https://github.com/risingwavelabs/risingwave/pull/21938) * Supports `partition_by` for Iceberg table engine as the Iceberg sink. [#21594](https://github.com/risingwavelabs/risingwave/pull/21594) * Supports using `ALTER SINK ... CONNECTOR WITH` to update connector-specific properties without recreating the sink. [#20691](https://github.com/risingwavelabs/risingwave/pull/20691) ## Cluster configuration changes * **Technical preview**: Adds session variable `streaming_enable_unaligned_join` to control whether the streaming joins should be unaligned. [#20386](https://github.com/risingwavelabs/risingwave/pull/20386) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.5.0 standalone` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.5.0/risingwave-v2.5.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.5.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.5.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.5.0/risectl-v2.5.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.4...release-2.5). Support for certain earlier version will end following the release of v2.4, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). ## SQL features * SQL commands: * Supports `EXPLAIN ANALYZE` command to analyze the actual runtime performance of a streaming job. [#20715](https://github.com/risingwavelabs/risingwave/pull/20715), [#20966](https://github.com/risingwavelabs/risingwave/pull/20966) * Supports configurable `streaming_parallelism` settings for tables, indexes, materialized views, sinks, and sources. [#21366](https://github.com/risingwavelabs/risingwave/pull/21366) * Supports `DESCRIBE FRAGMENTS` command to check the execution fragments of an existing job. [#21354](https://github.com/risingwavelabs/risingwave/pull/21354) * Supports **NOT NULL** constraints in table schemas for both batch and streaming operations. [#20822](https://github.com/risingwavelabs/risingwave/pull/20822) * Supports `GRANT` and `REVOKE` privileges on connection, function and secret. [#20755](https://github.com/risingwavelabs/risingwave/pull/20755) * `ALTER SYSTEM` now applies session variable changes immediately to the current session. [#21294](https://github.com/risingwavelabs/risingwave/pull/21294) * System catalog: * Adds system catalogs `iceberg_tables` and `iceberg_namespace_properties`. [#21400](https://github.com/risingwavelabs/risingwave/pull/21400) * Adds system function `has_function_privilege`. [#20755](https://github.com/risingwavelabs/risingwave/pull/20755) ## Connectors * **Deprecation notice:** Deprecates the Pulsar Iceberg reader and removed the `icelake` dependency. [#20844](https://github.com/risingwavelabs/risingwave/pull/20844) * **Deprecation notice:** The legacy S3 source is now fully deprecated in v2.4.0. [#20658](https://github.com/risingwavelabs/risingwave/pull/20658) * Supports creating an append-only table with the Iceberg engine. [#21811](https://github.com/risingwavelabs/risingwave/pull/21811) * Adds `commit_retry_num` option to configure the number of commit retries on Iceberg failures. [#20433](https://github.com/risingwavelabs/risingwave/pull/20433) * Supports Redis Pub/Sub messaging in sink connectors. [#20991](https://github.com/risingwavelabs/risingwave/pull/20991) * Supports Iceberg sink and Iceberg source on Azure Blob. [#21468](https://github.com/risingwavelabs/risingwave/pull/21468) * Supports exactly once consistency semantics for Iceberg sink. [#19771](https://github.com/risingwavelabs/risingwave/pull/19771) ## Cluster configuration changes * Adds system parameter `per_database_isolation` to enable per-database failure isolation. [#20872](https://github.com/risingwavelabs/risingwave/pull/20872) * Adds session variable `streaming_enable_materialized_expressions` to enable materialized expressions. [#21552](https://github.com/risingwavelabs/risingwave/pull/21552) * Adds session variable `streaming_force_filter_inside_join` to force filter to be pushed down into inner join. [#21289](https://github.com/risingwavelabs/risingwave/pull/21289) ## Fixes * Security for logical views has been improved to correctly prevent unauthorized access. [#20941](https://github.com/risingwavelabs/risingwave/pull/20941) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.4.0 standalone` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.4.0/risingwave-v2.4.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.4.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.4.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.4.0/risectl-v2.4.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.3...release-2.4). Support for certain earlier version will end following the release of v2.3, please plan your upgrade to a newer version soon to ensure continued support and updates. For more details, see our [version support policy](/changelog/release-support-policy). ## SQL features * Query syntax: * Supports cross-database queries in both streaming and ad-hoc modes. [#20261](https://github.com/risingwavelabs/risingwave/pull/20261) * Supports grouping by `window_start` and `window_end` at the same time in emit-on-window-close queries. [#19998](https://github.com/risingwavelabs/risingwave/pull/19998) * Supports `ASOF JOIN` for batch queries to match the nearest record with possible variations in case of ties. [#19790](https://github.com/risingwavelabs/risingwave/pull/19790) * SQL commands: * **Public preview:** Supports specifying `resource_group` for `CREATE DATABASE` command. [#20751](https://github.com/risingwavelabs/risingwave/pull/20751) * Allows enabling or disabling the creation of embedded Python, JavaScript, and WebAssembly UDFs. [#20893](https://github.com/risingwavelabs/risingwave/pull/20893) * Supports async and batched functions for embedded JavaScript UDFs. [#20403](https://github.com/risingwavelabs/risingwave/pull/20403) * Rejects schema-qualified CURSOR names in DECLARE statements to align with Postgres specifications. [#20221](https://github.com/risingwavelabs/risingwave/pull/20221) * `CREATE TABLE` command supports partially specifying the schema instead of using `*` to ingest all columns. [#20203](https://github.com/risingwavelabs/risingwave/pull/20203) * `DROP SCHEMA` command supports the `CASCADE` option to drop the specified schema and all its dependencies. [#19702](https://github.com/risingwavelabs/risingwave/pull/19702) * Supports switching databases during an active session using either `SET DATABASE TO` or `USE`. [#19786](https://github.com/risingwavelabs/risingwave/pull/19786) * Supports `SET PARALLELISM` clause for `ALTER SOURCE` command. [#20606](https://github.com/risingwavelabs/risingwave/pull/20606) * Supports `IF NOT EXISTS` option for `CREATE FUNCTION`/`CREATE AGGREGATE`. [#20079](https://github.com/risingwavelabs/risingwave/pull/20079) * Supports `GRANT` or `REVOKE` privileges on view. [#20670](https://github.com/risingwavelabs/risingwave/pull/20670) * SQL functions & operators: * Supports IGNORE NULLS for `first_value` and `last_value` window functions. [#19874](https://github.com/risingwavelabs/risingwave/pull/19874) * System catalog: * **Public preview:** Adds system catalog `rw_resource_groups`. [#20920](https://github.com/risingwavelabs/risingwave/pull/20920) * **Public preview:** Adds system catalog `rw_streaming_jobs`. [#20207](https://github.com/risingwavelabs/risingwave/pull/20207) ### Connectors * **Deprecation notice:** The legacy S3 source will be fully deprecated in v2.4.0. While the properties and enumerator will be retained to ensure that the metadata can start normally, it may disrupt other streaming jobs that still rely on the legacy source. [#20658](https://github.com/risingwavelabs/risingwave/pull/20658) * **Public preview:** Support ingesting data from **PostgreSQL 17** using CDC. [#20668](https://github.com/risingwavelabs/risingwave/pull/20668) * Supports creating connections for Iceberg. [#20189](https://github.com/risingwavelabs/risingwave/pull/20189) * Supports AWS S3 Tables integration for the Iceberg source, sink, and table engine. [#21018](https://github.com/risingwavelabs/risingwave/pull/21018) * Supports creating Iceberg sources and sinks with GCS. [#19999](https://github.com/risingwavelabs/risingwave/pull/19999) * Supports generated columns for tables on CDC sources. [#20380](https://github.com/risingwavelabs/risingwave/pull/20380) * Supports reading Parquet files from GCS and Azure blob. [#19974](https://github.com/risingwavelabs/risingwave/pull/19974), [#20046](https://github.com/risingwavelabs/risingwave/pull/20046) * Removes schema from DML statements for JDBC sinks if not specified one. [#20632](https://github.com/risingwavelabs/risingwave/pull/20632) * Adds `partition_by` to enable partitioning using column names or transformations for Iceberg sinks. [#20495](https://github.com/risingwavelabs/risingwave/pull/20495) * Adds `jdbc.auto.commit` for JDBC sinks to automatically commit transactions. [#20597](https://github.com/risingwavelabs/risingwave/pull/20597) * Adds `snowflake` catalog for Iceberg source. [#20421](https://github.com/risingwavelabs/risingwave/pull/20421) * Adds `aws.msk.signer_timeout_sec` to set timeout limit for loading AWS credentials of AWS MSK. [#20610](https://github.com/risingwavelabs/risingwave/pull/20610) * `jdbc.query.timeout` for JDBC sinks is reduced from 600s to 60s. [#20641](https://github.com/risingwavelabs/risingwave/pull/20641) ### Installation and deployment * For the production cluster, a license key with `Paid` tier and CPU limit of 4 is set by default to allow the free trial of paid features on a small scale. [#20296](https://github.com/risingwavelabs/risingwave/pull/20296) * Adds a function `rw_license()` to retrieve the license information. [#20629](https://github.com/risingwavelabs/risingwave/pull/20629) ### Cluster configuration changes * Adds runtime parameter `sink_rate_limit`. [#19660](https://github.com/risingwavelabs/risingwave/pull/19660) * Adds system parameter `adaptive_parallelism_strategy`. [#20589](https://github.com/risingwavelabs/risingwave/pull/20589) ### Assets * Run this version from Docker:
`docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.3.0 standalone` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.3.0/risingwave-v2.3.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.3.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.3.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.3.0/risectl-v2.3.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.2...release-2.3). ## SQL features * SQL commands: * **Public preview:** Supports `EXPLAIN FORMAT JSON`. [#19041](https://github.com/risingwavelabs/risingwave/pull/19041) * **Public preview:** Supports `EXPLAIN FORMAT (XML | YAML)`. [#19400](https://github.com/risingwavelabs/risingwave/pull/19400) * Supports `ALTER...SWAP WITH...` syntax for tables, materialized views, views, sources, sinks, and subscriptions. [#19172](https://github.com/risingwavelabs/risingwave/pull/19172) * Introduces a new connection that allows for reusing connector parameters, replacing the deprecated AWS PrivateLink connection. [#19270](https://github.com/risingwavelabs/risingwave/pull/19270) * Supports creating connections for schema registries. [#19685](https://github.com/risingwavelabs/risingwave/pull/19685) * Ensures results returned by cursors are sorted by the primary key. [#18801](https://github.com/risingwavelabs/risingwave/pull/18801) * Supports `ALTER SECRET`. [#19495](https://github.com/risingwavelabs/risingwave/pull/19495) * System catalog: * Adds system catalog `rw_internal_table_info`. [#19642](https://github.com/risingwavelabs/risingwave/pull/19642) * Adds system catalog `rw_rate_limit`. [#19466](https://github.com/risingwavelabs/risingwave/pull/19466) ## Connectors * **Public preview:** Adds a native Postgres sink connector. [#19328](https://github.com/risingwavelabs/risingwave/pull/19328) * **Public preview:** Supports `mysql_query` table-valued function. [#19071](https://github.com/risingwavelabs/risingwave/pull/19071) * Supports using the streaming developer configuration `stream_switch_jdbc_pg_to_native` to switch from `connector='jdbc'` to `connector='postgres'` for JDBC Postgres sinks. [#19703](https://github.com/risingwavelabs/risingwave/pull/19703) * Supports nested `Ref` for Avro encoding, except for circular dependency. [#19601](https://github.com/risingwavelabs/risingwave/pull/19601), [#19701](https://github.com/risingwavelabs/risingwave/pull/19701), [#19746](https://github.com/risingwavelabs/risingwave/pull/19746) * Adds parameter `messages_as_jsonb` for source’s Protobuf encoding when creating a source or table. [#19935](https://github.com/risingwavelabs/risingwave/pull/19935) * Supports `_rw_timestamp` system column for tables. [#19232](https://github.com/risingwavelabs/risingwave/pull/19232) * Allows Iceberg REST catalog to be specified as  `catalog.credential`, `catalog.scope`, `catalog.oauth2_server_uri`, or `catalog.token`. [#19406](https://github.com/risingwavelabs/risingwave/pull/19406) * Supports configuring parameter `commit_checkpoint_interval` when creating Iceberg engine tables. The default value is `60`. [#19738](https://github.com/risingwavelabs/risingwave/pull/19738) * Supports `INCLUDE partition` when creating MQTT sources to include which topic each message is from. [#19017](https://github.com/risingwavelabs/risingwave/pull/19017) * Supports `INCLUDE subject` when creating NATS sources. [#19708](https://github.com/risingwavelabs/risingwave/pull/19708) * Supports **webhook sources**. [#19272](https://github.com/risingwavelabs/risingwave/pull/19272) * Adds Google BigQuery sink parameter `bigquery.credentials`. [#19798](https://github.com/risingwavelabs/risingwave/pull/19798) ## Cluster configuration changes * Adds session variable `batch_expr_strict_mode` to control if the query fails or fills the table with `NULL` values during expression evaluation failures. [#19562](https://github.com/risingwavelabs/risingwave/pull/19562) * Adds runtime parameter `backfill_rate_limit` to configure source backfill. [#19445](https://github.com/risingwavelabs/risingwave/pull/19445) * Adds runtime parameter `dml_rate_limit`. [#19679](https://github.com/risingwavelabs/risingwave/pull/19679) ## Assets * Run this version from Docker:
`docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.2.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.2.0/risingwave-v2.2.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.2.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.2.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.2.0/risectl-v2.2.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.1...release-2.2). ## SQL features * Query syntax: * **Public preview:** Supports `ASOF JOIN` to join and find the closest matching record by the event time or another ordered property. [#18683](https://github.com/risingwavelabs/risingwave/pull/18683) * Supports `AGGREGATE:` prefixed scalar function in streaming aggregation. [#18205](https://github.com/risingwavelabs/risingwave/pull/18205) * Supports using user-defined aggregate functions as window function. [#18181](https://github.com/risingwavelabs/risingwave/pull/18181) * Supports blocking subscription cursors and configuring cursor timeouts. [#18675](https://github.com/risingwavelabs/risingwave/pull/18675) * SQL commands: * Enhances observability of cursors and subscription cursors by improving output results of `SHOW SUBSCRIPTION CURSORS` and `SHOW CURSORS` commands. [#18896](https://github.com/risingwavelabs/risingwave/pull/18896) * SQL functions & operators: * **Public preview:** Supports the TVF `postgres_query`. [#18811](https://github.com/risingwavelabs/risingwave/pull/18811) * System catalog: * Supports `pg_catalog.pg_sequence`. [#18714](https://github.com/risingwavelabs/risingwave/pull/18714) * Supports `information_schema.table_constraints`. [#18716](https://github.com/risingwavelabs/risingwave/pull/18716) * Supports `information_schema.schemata`. [#18709](https://github.com/risingwavelabs/risingwave/pull/18709) ## Connectors * **Breaking change:** Changes `scan.startup.mode=latest` for NATS source connector to start consuming from next available message instead of last one. [#18733](https://github.com/risingwavelabs/risingwave/pull/18733) * **Public preview:** Supports shared Kafka sources, which can be disabled by session variable `streaming_use_shared_source`. [#18749](https://github.com/risingwavelabs/risingwave/pull/18749) * Supports recursively scanning file sources. [#18324](https://github.com/risingwavelabs/risingwave/pull/18324) * Supports schemaless ingestion for data in JSON format from Kafka sources by using `INCLUDE payload` clause. [#18437](https://github.com/risingwavelabs/risingwave/pull/18437) * Adds a set of options for NATS source connector based on the async\_nats crate. [#17615](https://github.com/risingwavelabs/risingwave/pull/17615) * Adds a required option, `consumer.durable_name`, for NATS source connector. [#18873](https://github.com/risingwavelabs/risingwave/pull/18873) * Supports option `max_packet_size` for MQTT sources. [#18520](https://github.com/risingwavelabs/risingwave/pull/18520) * Supports option `database.encrypt` for SQL Server CDC sources. [#18912](https://github.com/risingwavelabs/risingwave/pull/18912) * Supports ingesting data from a partitioned table for PostgreSQL CDC sources. [#18456](https://github.com/risingwavelabs/risingwave/pull/18456) * Supports option `auto.schema.change` for PostgreSQL CDC sources to enable replicating Postgres table schema change. [#18760](https://github.com/risingwavelabs/risingwave/pull/18760) * Requires upstream table name to also be prefixed with database name when creating a SQL Server CDC table. [#18868](https://github.com/risingwavelabs/risingwave/pull/18868) * Adds `JSON` encode for file sinks, allowing users to sink JSON files into object storage. [#18744](https://github.com/risingwavelabs/risingwave/pull/18744) * Supports `create_table_if_not_exists` option for Iceberg sink connector. [#18362](https://github.com/risingwavelabs/risingwave/pull/18362) * Supports WebHDFS sinks. [#18293](https://github.com/risingwavelabs/risingwave/pull/18293) * Removes option `bulk_write_max_entries` for MongoDB sink and option `default_max_batch_rows` for DynamoDB sink. Adds options `max_batch_item_nums` and `max_future_send_nums` for DynamoDB sink. [#17645](https://github.com/risingwavelabs/risingwave/pull/17645) * Sets sink decoupling as the default policy for MongoDB, DynamoDB, and Redis sink connectors. [#17645](https://github.com/risingwavelabs/risingwave/pull/17645) * Supports option `routing_column` for ElasticSearch sinks, allowing a column to be set as a routing key. [#18698](https://github.com/risingwavelabs/risingwave/pull/18698) * Supports specifying batching strategy when sinking data in Parquet format. [#18472](https://github.com/risingwavelabs/risingwave/pull/18472) ## Installation and deployment * Adds a CLI argument of `--license-key-path` for the Meta Node, enabling a background task to watch and reload license key from the specified file. [#18768](https://github.com/risingwavelabs/risingwave/pull/18768) ## Cluster configuration changes * When `visibility_mode` is set to `all`, the latest uncommitted data will be queried, but consistency is no longer guaranteed between the tables. [#18230](https://github.com/risingwavelabs/risingwave/pull/18230) * Supports `SET TIME ZONE INTERVAL '+00:00' HOUR TO MINUTE` as equivalent to `SET TIME ZONE UTC`. [#18705](https://github.com/risingwavelabs/risingwave/pull/18705) * The etcd metastore is fully deprecated and unsupported. Users previously utilizing etcd metastore must manually migrate to a SQL backend (PostgreSQL, MySQL, or SQLite) to upgrade to v2.1. ## Assets * Run this version from Docker:
`docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.1.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.1.0/risingwave-v2.1.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.1.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.1.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.1.0/risectl-v2.1.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-2.0...release-2.1). ## SQL features * Query syntax: * **Public preview:** Supports `AS CHANGELOG` to convert any stream into an append-only changelog. [#17132](https://github.com/risingwavelabs/risingwave/pull/17132) * Supports time travel query to access historical data at a specific point in time. [#17665](https://github.com/risingwavelabs/risingwave/pull/17665), [#17621](https://github.com/risingwavelabs/risingwave/pull/17621). * Supports `CORRESPONDING` specification in set operations. [#17891](https://github.com/risingwavelabs/risingwave/pull/17891) * SQL commands: * **Breaking change:** `DECLARE cursor_name SUBSCRIPTION CURSOR` is the same as `DECLARE cursor_name SUBSCRIPTION CURSOR since now()`, which will be consumed from the current time. `DECLARE cursor_name SUBSCRIPTION CURSOR FULL` will start consuming data from stock. The type of operation has changed to `varchar`. It is one of `Insert`, `Delete`, `UpdateInset`, or `UpdateDelete`. [#18217](https://github.com/risingwavelabs/risingwave/pull/18217) * **Public preview:** Supports managing database credentials securely. [#17456](https://github.com/risingwavelabs/risingwave/pull/17456) * `SHOW CURSORS` will return all query cursors in the current session. `SHOW SUBSCRIPTION CURSORS` will return all subscription cursors and subscriptions in the current session. [#18217](https://github.com/risingwavelabs/risingwave/pull/18217) * Allows `ALTER TABLE` on tables with generated columns. [#17652](https://github.com/risingwavelabs/risingwave/pull/17652) * Allows dropping generated columns from tables created with a schema registry. [#17689](https://github.com/risingwavelabs/risingwave/pull/17689) * Supports using scalar functions with list inputs as aggregate functions. [#17622](https://github.com/risingwavelabs/risingwave/pull/17622) * Supports altering the backfill rate limit for materialized views. [#17911](https://github.com/risingwavelabs/risingwave/pull/17911) * SQL functions & operators: * **Public preview:** Supports `approx_percentile()`. [#17814](https://github.com/risingwavelabs/risingwave/pull/17814), [#17873](https://github.com/risingwavelabs/risingwave/pull/17873). * **Public preview:** Supports native `map` type and related functions for `map`, and ingesting `AVRO MAP` type into RisingWave `map` type. [#17986](https://github.com/risingwavelabs/risingwave/pull/17986) * **Public preview:** Supports scanning a directory of parquet files. [#17811](https://github.com/risingwavelabs/risingwave/pull/17811) * Supports `pg_index_column_has_property()` to query index column properties. [#17275](https://github.com/risingwavelabs/risingwave/pull/17275) * Supports continuous timestamp generation in streaming mode. [#17371](https://github.com/risingwavelabs/risingwave/pull/17371) * Supports `acosd()`. [#9876](https://github.com/risingwavelabs/risingwave/pull/9876) * Supports function `rw_recovery_status()` and `pg_is_in_recovery()` to retrieve the Meta Node status. [#17641](https://github.com/risingwavelabs/risingwave/pull/17641) * System catalog: * Adds `append_only` column in RisingWave catalogs `rw_tables` and `rw_materialized_views`. [#17598](https://github.com/risingwavelabs/risingwave/pull/17598) * Adds RisingWave catalog `rw_catalog.rw_secrets`. [#17726](https://github.com/risingwavelabs/risingwave/pull/17726) ## Connectors * **Public preview:** Supports ingesting Avro map type for source connectors. [#17980](https://github.com/risingwavelabs/risingwave/pull/17980) * **Public preview:** Supports encoding `parquet` for file source. [#17201](https://github.com/risingwavelabs/risingwave/pull/17201) * **Public preview:** Supports batch reading S3 Parquet files. [#17625](https://github.com/risingwavelabs/risingwave/pull/17625), [#17673](https://github.com/risingwavelabs/risingwave/pull/17673). * Supports AWS Glue schema registry with `aws.glue.schema_arn` parameter. [#17605](https://github.com/risingwavelabs/risingwave/pull/17605) * Supports creating tables and sources with `format upsert encode protobuf`. [#17624](https://github.com/risingwavelabs/risingwave/pull/17624) * Supports ingesting Avro Union type for source connectors. [#17485](https://github.com/risingwavelabs/risingwave/pull/17485) * Supports reading files compressed in gzip format. [#16538](https://github.com/risingwavelabs/risingwave/pull/16538) * Adds the option to use a semicolon as the delimiter for CSV encode. [#17356](https://github.com/risingwavelabs/risingwave/pull/17356) * Uses OpenDAL to connect to S3 object store state backend. [#18011](https://github.com/risingwavelabs/risingwave/pull/18011) * **Public preview:** Supports replicating DDL for MySQL CDC source. [#17876](https://github.com/risingwavelabs/risingwave/pull/17876) * Supports parameter `refresh.interval.sec` option for S3, GCS, and POSIX sources. [#18184](https://github.com/risingwavelabs/risingwave/pull/18184) * Supports parameter `group.id.prefix` for Kafka sources. [#18115](https://github.com/risingwavelabs/risingwave/pull/18115) * Validates slot name of PostgreSQL CDC sources. [#17949](https://github.com/risingwavelabs/risingwave/pull/17949) * Supports altering `backfill_rate_limit` of CDC tables. [#17989](https://github.com/risingwavelabs/risingwave/pull/17989) * **Public preview:** Supports sinking data to file systems in parquet format. [#17311](https://github.com/risingwavelabs/risingwave/pull/17311) * Supports upsert Protobuf type sinks, which requires `KEY ENCODE TEXT`. [#18024](https://github.com/risingwavelabs/risingwave/pull/18024) * Adds option `jsonb.handling.mode` under `WITH` options for sinks in JSON format. [#17693](https://github.com/risingwavelabs/risingwave/pull/17693) * **Public preview:** Supports Azure Blob sinks. [#18244](https://github.com/risingwavelabs/risingwave/pull/18244) * **Public preview:** Supports MongoDB sinks. [#17102](https://github.com/risingwavelabs/risingwave/pull/17102) * Supports Azure Blob file sources. [#18295](https://github.com/risingwavelabs/risingwave/pull/18295) * Supports glue catalog for iceberg sink and source. [#17477](https://github.com/risingwavelabs/risingwave/pull/17477) * Adds `jdbc.query.timeout` for JDBC sinks to set the timeout for queries. [#18430](https://github.com/risingwavelabs/risingwave/pull/18430) * Changes default Kafka sink message timeout from five seconds to five minutes. [#18304](https://github.com/risingwavelabs/risingwave/pull/18304) * Adds new parameters `retry_on_conflict`, `batch_size_kb`, `batch_num_messages`, and `concurrent_requests` for ElasticSearch sink. [#17867](https://github.com/risingwavelabs/risingwave/pull/17867) * Supports parameter `bigquery.retry_times` for BigQuery sink. [#17237](https://github.com/risingwavelabs/risingwave/pull/17237) * Supports parameter `bigquery.auto_create_table` for BigQuery sink. [#17393](https://github.com/risingwavelabs/risingwave/pull/17393) * Supports parameter `doris.partial_columns` for Doris sink. [#16821](https://github.com/risingwavelabs/risingwave/pull/16821) * Supports ClickHouse sink checkpoint decouple. [#17491](https://github.com/risingwavelabs/risingwave/pull/17491) * Sets sink decouple as default for all sinks. [#18182](https://github.com/risingwavelabs/risingwave/pull/18182) * Uses S3's SQS notification to complete the import of data from Snowflake instead of the Snowflake HTTP client. [#17627](https://github.com/risingwavelabs/risingwave/pull/17627) * Ensures at-least-once delivery semantic and eventual consistency for Kinesis sink. [#17983](https://github.com/risingwavelabs/risingwave/pull/17983) * Supports backfilling by consuming a fixed snapshot of upstream table and then the upstream data epoch by epoch. [#17735](https://github.com/risingwavelabs/risingwave/pull/17735) ## Installation and deployment * Supports configuring the SQL metastore using username, password, and database separately. [#17530](https://github.com/risingwavelabs/risingwave/pull/17530) * Supports more seamless scaling-in in Kubernetes deployments. [#17802](https://github.com/risingwavelabs/risingwave/pull/17802) ## Cluster configuration changes * **Breaking change:** Refactors `streaming_rate_limit` into `source_rate_limit` and `backfill_rate_limit`. [#17796](https://github.com/risingwavelabs/risingwave/pull/17796) * **Breaking change:** Adds a default soft and hard limit on actor count per worker parallelism. When the hard limit is reached, streaming workloads will fail. [#18383](https://github.com/risingwavelabs/risingwave/pull/18383) * Introduces `batch.developer.exchange_connection_pool_size` and `streaming.developer.exchange_connection_pool_size` to configure streaming and batch remote exchange between two nodes. [#17768](https://github.com/risingwavelabs/risingwave/pull/17768) * Introduces system parameter `license_key` used to enable enterprise features. [#17396](https://github.com/risingwavelabs/risingwave/pull/17396) ## Fixes * Deletes related cursors when deleting a subscription. [#17232](https://github.com/risingwavelabs/risingwave/pull/17232) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v2.0.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v2.0.0/risingwave-v2.0.0-x86_64-unknown-linux-all-in-one.tar.gz). * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.0.0.zip). * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v2.0.0.tar.gz). * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v2.0.0/risectl-v2.0.0-x86_64-unknown-linux.tar.gz). See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.10...release-2.0). ## SQL features * Query syntax: * SQL commands: * Supports specifying the authorization and omitting the schema name when creating a schema. [#16806](https://github.com/risingwavelabs/risingwave/pull/16806). * Supports session window in batch and emit-on-window-close mode. [#17098](https://github.com/risingwavelabs/risingwave/pull/17098). * Supports fetching multiple rows from a subscription cursor. [#16764](https://github.com/risingwavelabs/risingwave/pull/16764). * Supports user-defined aggregate functions for embedded Python and JavaScript UDFs. [#16874](https://github.com/risingwavelabs/risingwave/pull/16874). * SQL functions & operators: * Supports `has_table_privilege()`, `has_schema_privilege()`, and `has_any_column_privilege()`. [#16674](https://github.com/risingwavelabs/risingwave/pull/16674). * Supports `quote_literal()` and `quote_nullable()`. [#16807](https://github.com/risingwavelabs/risingwave/pull/16807). * Supports `pg_get_keywords()`. [#17033](https://github.com/risingwavelabs/risingwave/pull/17033). * Supports `jsonb_set()`. [#17124](https://github.com/risingwavelabs/risingwave/pull/17124). * Allows the specified time zone of `AT TIME ZONE` to be non-literal. [#17395](https://github.com/risingwavelabs/risingwave/pull/17395) * System catalog: * Supports `rw_catalog.actor_id_to_ddl` and `rw_catalog.fragment_id_to_ddl`. [#17229](https://github.com/risingwavelabs/risingwave/pull/17229). ## Connectors * Avro schemas with `"default": "NaN"` and positive and negative infinities, are supported as `float` and `double` types. [#17309](https://github.com/risingwavelabs/risingwave/pull/17309). * Supports ingesting simple `AVRO MAP` types as `JSONB`. [#16948](https://github.com/risingwavelabs/risingwave/pull/16948). * Supports ingesting `avro uuid` types as `varchar`. [#17069](https://github.com/risingwavelabs/risingwave/pull/17069). * Supports ingesting `avro` with internal `Ref` types. [#17052](https://github.com/risingwavelabs/risingwave/pull/17052). * Adds `aws` prefix to AWS related parameters when creating a source or sink. [#16671](https://github.com/risingwavelabs/risingwave/pull/16671). * Supports using AWS IAM to connect to Amazon MSK. [#16625](https://github.com/risingwavelabs/risingwave/pull/16625). * Adds `pubsub.parallelism` under the `WITH` option for Google PubSub source. [#16733](https://github.com/risingwavelabs/risingwave/pull/16733). * Supports `INCLUDE TIMESTAMP [AS]` clause for MySQL, PostgreSQL, and MongoDB CDC tables.[#16833](https://github.com/risingwavelabs/risingwave/pull/16833). * Supports additional metadata columns for CDC tables. [#17051](https://github.com/risingwavelabs/risingwave/pull/17051). * Automatically maps upstream table schema when creating MySQL and PostgreSQL tables. [#16986](https://github.com/risingwavelabs/risingwave/pull/16986). * Sets a network timeout for JDBC sink connections. [#17244](https://github.com/risingwavelabs/risingwave/pull/17244). * Enables sink decouple by default for Kafka, Kinesis, Pulsar, Google Pub/Sub, NATS, MQTT, ClickHouse sinks. [#17221](https://github.com/risingwavelabs/risingwave/pull/17221). * Supports the `KEY ENCODE` clause when creating a sink. [#16377](https://github.com/risingwavelabs/risingwave/pull/16377). * Supports `FORMAT PLAIN ENCODE AVRO` for Kafka sinks. [#17216](https://github.com/risingwavelabs/risingwave/pull/17216). * Supports DynamoDB sink. [#16670](https://github.com/risingwavelabs/risingwave/pull/16670). * Supports Microsoft SQL Server sinks for self-hosted SQL Server and Azure SQL. [#17154](https://github.com/risingwavelabs/risingwave/pull/17154). * Supports OpenSearch sink. [#16330](https://github.com/risingwavelabs/risingwave/pull/16330). * Supports checkpoint decouple for StarRocks sinks. [#16816](https://github.com/risingwavelabs/risingwave/pull/16816). * Supports checkpoint decouple for Delta Lake sinks. [#16777](https://github.com/risingwavelabs/risingwave/pull/16777). * Supports sinking serial types. [#16969](https://github.com/risingwavelabs/risingwave/pull/16969). ## Cluster configuration changes * Sets arrangement backfill as the default. [#14846](https://github.com/risingwavelabs/risingwave/pull/14846). * Supports spill hash join to avoid OOM issues. [#17122](https://github.com/risingwavelabs/risingwave/pull/17122). * Supports spill hash aggregation for batch queries. [#16771](https://github.com/risingwavelabs/risingwave/pull/16771). * Changes the algorithm that calculates the reserve memory size. [#16992](https://github.com/risingwavelabs/risingwave/pull/16992). ## Bug fixes * Improves error message and location of the cursor. [#16959](https://github.com/risingwavelabs/risingwave/pull/16959). * Improves error message when trying to create a CDC source with columns. [#16636](https://github.com/risingwavelabs/risingwave/pull/16636). * Allows `GRANT` and `REVOKE` privileges on views. [#16699](https://github.com/risingwavelabs/risingwave/pull/16699). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.10.0 single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.9.1/risingwave-v1.9.1-x86_64-unknown-linux-all-in-one.tar.gz). * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.10.0.zip). * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.10.0.tar.gz). * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v1.10.0/risectl-v1.10.0-x86_64-unknown-linux.tar.gz). See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.9...release-1.10). v1.9.0 was skipped due to some critical bugs. ## SQL features * Query syntax: * Supports non-append-only temporal joins, where the outer side is not required to be append-only. [#16286](https://github.com/risingwavelabs/risingwave/pull/16286). * SQL commands: * Supports `DISCARD ALL` command for Npgsql compatibility. [#16432](https://github.com/risingwavelabs/risingwave/pull/16432). * Supports creating, dropping, altering, and showing subscriptions. [#14831](https://github.com/risingwavelabs/risingwave/pull/14831). * Supports cursors for subscription queues. [#15180](https://github.com/risingwavelabs/risingwave/pull/15180). * Supports altering stream rate limit for sources and tables with a source. [#16399](https://github.com/risingwavelabs/risingwave/pull/16399). * Supports `RECOVER` command to trigger an ad-hoc recovery. [#16259](https://github.com/risingwavelabs/risingwave/pull/16259). * SQL functions & operators: * Supports `jsonb_populate_record()` and `jsonb_populate_recordset()`. [#13421](https://github.com/risingwavelabs/risingwave/pull/13421). * System catalog: * Supports system table `rw_iceberg_files` for displaying the files of an Iceberg source or table. [#16180](https://github.com/risingwavelabs/risingwave/pull/16180). * Supports system table `rw_iceberg_snapshot` for listing all snapshots. [#16175](https://github.com/risingwavelabs/risingwave/pull/16175). ## Connectors * Provides stable support for SQLAlchemy 2.0. [#29](https://github.com/risingwavelabs/sqlalchemy-risingwave/pull/29). * Deprecates `s3` connector. [#16337](https://github.com/risingwavelabs/risingwave/pull/16337). * Supports generated columns for non-shared CDC tables. [#16522](https://github.com/risingwavelabs/risingwave/pull/16522). * Supports time travel for Iceberg sources. [#15866](https://github.com/risingwavelabs/risingwave/pull/15866). * Blocks sink creation until backfill is completed by default. [#16249](https://github.com/risingwavelabs/risingwave/pull/16249). * Supports Kafka connector parameter `properties.request.required.acks`. [#16482](https://github.com/risingwavelabs/risingwave/pull/16482). * Adds connector parameter `ssl.mode` for PostgreSQL and Neon source connector. [#15690](https://github.com/risingwavelabs/risingwave/pull/15690). * Adds connector parameter `ssl.mode` for MySQL source connector. [#16579](https://github.com/risingwavelabs/risingwave/pull/16579). * Supports parameters `snapshot.interval` and `snapshot.batch_size` under `WITH` options when creating a table from a CDC source. [#16426](https://github.com/risingwavelabs/risingwave/pull/16426). * Supports implicitly converting `numeric` types from PostgreSQL sources into `rw_int256` or `varchar`. [#16346](https://github.com/risingwavelabs/risingwave/pull/16346). * Supports configuring the timeout of CDC sources. [#16598](https://github.com/risingwavelabs/risingwave/pull/16598). * Supports `timestamptz.handling.mode` formatting option when creating a source with `PLAIN`, `UPSERT`, or `DEBEZIUM JSON` formats. [#16265](https://github.com/risingwavelabs/risingwave/pull/16265). * Only uses fragment ID as group ID for Kafka sources. [#16111](https://github.com/risingwavelabs/risingwave/pull/16111). * Supports cluster URLs for Redis sink connector. [#16034](https://github.com/risingwavelabs/risingwave/pull/16034). * Supports creating Delta sinks with GCS. [#16182](https://github.com/risingwavelabs/risingwave/pull/16182). * Supports Snowflake sink connector. [#15429](https://github.com/risingwavelabs/risingwave/pull/15429). * Supports creating `upsert` type BigQuery sinks. [#15780](https://github.com/risingwavelabs/risingwave/pull/15780). ## Installation and deployment * Sets PostgreSQL as the default meta store when deploying with Docker Compose. [#16724](https://github.com/risingwavelabs/risingwave/pull/16724). ## Cluster configuration changes * Supports using `ALTER SYSTEM` to set a system-wide default value for a session parameter. [#16062](https://github.com/risingwavelabs/risingwave/pull/16062). * Modifies the meaning of `streaming_rate_limit=0`, which now means pausing the snapshot read stream for backfill, and pausing source read for sources. This statement previously disabled the rate limit within the session. [#16333](https://github.com/risingwavelabs/risingwave/pull/16333). * Supports configuring the reserved memory bytes of the Compute Node by using `RW_RESERVED_MEMORY_BYTES` runtime parameter and `reserved-memory-bytes` startup option. [#16433](https://github.com/risingwavelabs/risingwave/pull/16433). * Introduce new timeout and retry configurations for ObjectStore and deprecate ambiguous timeout configurations. [#16231](https://github.com/risingwavelabs/risingwave/pull/16231). ## Fixes * Properly convert `-inf`, `+inf`, and `nan` types to `null` for JDBC sinks. [#16230](https://github.com/risingwavelabs/risingwave/pull/16230). * Handles sinking `-inf`, `+inf`, and `nan` types for ClickHouse, Doris, and StarRocks sink connectors. [#15664](https://github.com/risingwavelabs/risingwave/pull/15664). * Fixes an issue where `DELETE` events could not be sinked if the primary key is `uuid` type for JDBC sinks.[#16447](https://github.com/risingwavelabs/risingwave/pull/16447). * Fixes an issue where `enum` types from PostgreSQL could not be ingested as `varchar` types. [#16423](https://github.com/risingwavelabs/risingwave/pull/16423). * Fixes sources with `encode avro` on decimal ingesting. [#16202](https://github.com/risingwavelabs/risingwave/pull/16202). * Fixes sources with `encode avro` on bytes/fixed/decimal default value. [#16414](https://github.com/risingwavelabs/risingwave/pull/16414). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.9.1-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.9.1/risingwave-v1.9.1-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.9.1.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.9.1.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v1.9.1/risectl-v1.9.1-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.8...release-1.9). ## SQL features * Query syntax: * Supports `RANGE` frames in window function calls. [#14416](https://github.com/risingwavelabs/risingwave/pull/14416). * SQL commands: * Adds support for embedded Rust UDFs. [#14903](https://github.com/risingwavelabs/risingwave/pull/14903). * Adds support for embedded Python UDFs. [#15168](https://github.com/risingwavelabs/risingwave/pull/15168). * Supports refreshing the schema of a table created using an external connection to get the latest schema. [#15025](https://github.com/risingwavelabs/risingwave/pull/15025). * Supports refreshing the schema of a source to get the latest schema. [#15541](https://github.com/risingwavelabs/risingwave/pull/15541). * Adds a description column to the system parameters table. [#15113](https://github.com/risingwavelabs/risingwave/pull/15113). * Supports authenticating with OAuth token acquired from the Cloud when creating a user. [#13151](https://github.com/risingwavelabs/risingwave/pull/13151). * SQL functions & operators: * Supports ruby-pg. [#14859](https://github.com/risingwavelabs/risingwave/pull/14859). * Supports `VARIADIC` arguments for the functions `format`, `concat_ws`, `jsonb_build_array`, `jsonb_build_object`, `jsonb_extract_path`, `jsonb_extract_path_text`. [#14753](https://github.com/risingwavelabs/risingwave/pull/14753). * Supports `concat` function. [#14753](https://github.com/risingwavelabs/risingwave/pull/14753). * System catalog: * Adds missing columns for  `pg_catalog.pg_index`, `rw_catalog.rw_columns` and `information_schema.columns`, and system view `pg_catalog.pg_partitioned_table`. [#15151](https://github.com/risingwavelabs/risingwave/pull/15151). * Supports `pg_catalog.pg_constraint` for DBeaver compatibility. [#15227](https://github.com/risingwavelabs/risingwave/pull/15227). * Supports `pg_catalog.pg_stat_get_numscans` for DBeaver. [#15642](https://github.com/risingwavelabs/risingwave/pull/15642). * Supports system table `rw_depend`. [#15385](https://github.com/risingwavelabs/risingwave/pull/15385). * Supports `pg_settings` catalog. [#15108](https://github.com/risingwavelabs/risingwave/pull/15108). ## Connectors * **Breaking change:** Sinks created from v1.6 and earlier that have `decouple` enabled may cause compatibility issues. Check if you have any sinks with this configuration by using the internal table `rw_sink_decouple` before upgrading to v1.8. [#15613](https://github.com/risingwavelabs/risingwave/pull/15613). * Avro tables and sources now require a schema registry during creation. [#15256](https://github.com/risingwavelabs/risingwave/pull/15256). * Supports using Karapace when specifying a schema registry when creating a Kafka source. [#15486](https://github.com/risingwavelabs/risingwave/pull/15486). * Supports Protobuf data format for NATS JetStream source. [#15378](https://github.com/risingwavelabs/risingwave/pull/15378). * Supports Confluent schema registry for Kafka sinks when using `FORMAT PLAIN ENCODE PROTOBUF`. [#15546](https://github.com/risingwavelabs/risingwave/pull/15546). * Adds Kafka sink and source parameter `enable.ssl.certificate.verification`. [#15073](https://github.com/risingwavelabs/risingwave/pull/15073). * Supports `max_batch_rows` and  `request_timeout` parameters for Cassandra and ScyllaDB sources. [#15516](https://github.com/risingwavelabs/risingwave/pull/15516). * Adds built-in MongoDB CDC source connector. [#14966](https://github.com/risingwavelabs/risingwave/pull/14966). * Adds `ignore_option` parameter for sources created using Debezium format. [#15304](https://github.com/risingwavelabs/risingwave/pull/15304). * Supports batch read from Iceberg source. [#15214](https://github.com/risingwavelabs/risingwave/pull/15214). * Supports automatically deriving columns from Iceberg source. [#15415](https://github.com/risingwavelabs/risingwave/pull/15415). * Supports JDBC catalog for Iceberg sources. [#15551](https://github.com/risingwavelabs/risingwave/pull/15551). * Adds JDBC and Hive catalogs for Iceberg sink. [#14885](https://github.com/risingwavelabs/risingwave/pull/14885). ## Installation and deployment * Supports tab-completion for `SET` and `ALTER SYSTEM SET` commands in `psql` client. [#15123](https://github.com/risingwavelabs/risingwave/pull/15123). * Supports SQL meta store. [#16019](https://github.com/risingwavelabs/risingwave/pull/16019). ## Bug fixes * Fixes an issue where built-in CDC connectors do not accept empty passwords. [#15411](https://github.com/risingwavelabs/risingwave/pull/15411). * Fixes an issue where materialized views created on a shared CDC source were allowed. [#15635](https://github.com/risingwavelabs/risingwave/pull/15635). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.8.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.8.0/risingwave-v1.8.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.8.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.8.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v1.8.0/risectl-v1.8.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.7...release-1.8). ## SQL features * Query syntax: * Supports using `*` when creating a table or source with schema from an external connector to read all columns. [#14644](https://github.com/risingwavelabs/risingwave/pull/14644). * Supports `INCLUDE` clause to add additional connector columns. [#14215](https://github.com/risingwavelabs/risingwave/pull/14215). * Supports `INCLUDE HEADER` clause to specify desired keys in header and make it a column. [#14628](https://github.com/risingwavelabs/risingwave/pull/14628). * SQL commands: * Supports changing the schema registry by redefining the `format_encode_options`. [#14057](https://github.com/risingwavelabs/risingwave/pull/14057). * SQL functions & operators: * Adds experimental support for JavaScript UDF. [#14513](https://github.com/risingwavelabs/risingwave/pull/14513). * Adds experimental support for Rust UDF. [#14271](https://github.com/risingwavelabs/risingwave/pull/14271). * Supports implicit type cast for UDF. [#14458](https://github.com/risingwavelabs/risingwave/pull/14458). * Supports named SQL UDF. [#14806](https://github.com/risingwavelabs/risingwave/pull/14806). * Supports `encrypt` and `decrypt` functions. [#14717](https://github.com/risingwavelabs/risingwave/pull/14717). * Supports `make_date()`, `make_time()`, and `make_timestamp()`. [#14827](https://github.com/risingwavelabs/risingwave/pull/14827). * System catalog: * Change `rw_streaming_parallelism` to allow queries on streaming job parallelism with job name and type. Adds system view `rw_fragment_parallelism` to allow for queries on parallelism information at fragment level [#14789](https://github.com/risingwavelabs/risingwave/pull/14789), [#14261](https://github.com/risingwavelabs/risingwave/pull/14261). * Adds `relpersistence` in `pg_class` catalog. [#14400](https://github.com/risingwavelabs/risingwave/pull/14400). * Supports `pg_get_viewdef()`. [#14336](https://github.com/risingwavelabs/risingwave/pull/14336). ## Connectors * Cassandra and ScyllaDB sinks no longer support `timestamp` type. [#14413](https://github.com/risingwavelabs/risingwave/pull/14413). * Updates StarRocks sink connector parameters. [#14823](https://github.com/risingwavelabs/risingwave/pull/14823). * Introduces `snapshot` option to allow users to disable CDC backfill and to only consume from the latest changelog. [#14718](https://github.com/risingwavelabs/risingwave/pull/14718). * Sets the default value of `transactional` parameter to `true` for MySQL and Postgres CDC shared sources. [#14899](https://github.com/risingwavelabs/risingwave/pull/14899). ## Installation and deployment * \[Pre-Release] Supports the standalone mode to run RisingWave in a single process. [#14951](https://github.com/risingwavelabs/risingwave/pull/14951). * Supports Alibaba Cloud OSS as the storage backend. ## Cluster configuration changes * Introduce a session variable `batch_enable_distributed_dml` to enable batch ingesting. [#14630](https://github.com/risingwavelabs/risingwave/pull/14630). * Changes wording from `AUTO` to `ADAPTIVE` parallelism. [#14414](https://github.com/risingwavelabs/risingwave/pull/14414). * Supports adaptive scaling for streaming jobs by default. [#14873](https://github.com/risingwavelabs/risingwave/pull/14873). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.7.0-standalone single_node` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.7.0/risingwave-v1.7.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.7.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.7.0.tar.gz) * [risectl - a CLI tool for managing and accessing RisingWave clusters](https://github.com/risingwavelabs/risingwave/releases/download/v1.7.0/risectl-v1.7.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.6...release-1.7). ## SQL features * Query syntax: * Allows `NOW` in upper bound condition for temporal filters. [#13985](https://github.com/risingwavelabs/risingwave/pull/13985). * Supports temporal filters with multiple `OR` expressions. [#14382](https://github.com/risingwavelabs/risingwave/pull/14382). * Supports ` [ NOT ] SIMILAR TO [ ESCAPE ]` clause. [#14000](https://github.com/risingwavelabs/risingwave/pull/14000). * **Breaking change**: Fixes the correctness of `SOME`, `ALL`, and `ANY` expressions. Drop and recreate any materialized views that use these expressions. [#14221](https://github.com/risingwavelabs/risingwave/pull/14221). * Supports array subquery and \du command. [#14044](https://github.com/risingwavelabs/risingwave/pull/14044). * Supports `SET PARALLELISM` clause for `ALTER` commands. [#14240](https://github.com/risingwavelabs/risingwave/pull/14240). * SQL commands: * Technical preview feature: Supports `CREATE SINK INTO TABLE` Multiple sinks can use the same table as the destination. [#13185](https://github.com/risingwavelabs/risingwave/pull/13185), [#13659](https://github.com/risingwavelabs/risingwave/pull/13659). * SQL functions & operators: * **Breaking change**: `0b10` is now interpreted as binary `10` instead of `0 as b10`. Integer literals can be given in hex `0x`, oct `0o`, and bin `0b`. [#14262](https://github.com/risingwavelabs/risingwave/pull/14262). * Supports interval type as input for `to_char()`. [#14071](https://github.com/risingwavelabs/risingwave/pull/14071). * Supports `NULL` and fraction expression as direct arguments of ordered-set aggregate functions. [#14080](https://github.com/risingwavelabs/risingwave/pull/14080). * System catalog: * Add system view `rw_streaming_parallelism`. [#14261](https://github.com/risingwavelabs/risingwave/pull/14261). ## Connectors * Adds CDC backfill support for Postgres so users can ingest multiple PostgreSQL tables with a single replication slot. [#13958](https://github.com/risingwavelabs/risingwave/pull/13958). * Support multi-table transactions from upstream MySQL & Postgres CDC. Specify `transactional = true` in the `WITH` options to enable it. [#14375](https://github.com/risingwavelabs/risingwave/pull/14375). * Renames `scan.startup.timestamp_millis` to `scan.startup.timestamp.millis` for Kafka, Pulsar, and NATS source. [#13656](https://github.com/risingwavelabs/risingwave/pull/13656). * Adds `properties.ssl.endpoint.identification.algorithm` parameter for Kafka source and sink.[#13990](https://github.com/risingwavelabs/risingwave/pull/13990). * Supports `FORMAT PLAIN ENCODE PROTOBUF` syntax for Kafka sink. [#12858](https://github.com/risingwavelabs/risingwave/pull/12858). * Supports GCS file source. [#13414](https://github.com/risingwavelabs/risingwave/pull/13414). * **Breaking change:** For ClickHouse sinks, `timestamptz` can be sinked to `DateTime64`. `timestamp` cannot be sinked and has to be converted to `timestamptz` first before being sinked. [#13672](https://github.com/risingwavelabs/risingwave/pull/13672). * For Elasticsearch sinks, the default es.type is set as `_doc` for Elasticsearch 6.x and 7.x, and is removed in Elasticsearch 8.x. RisingWave's Elasticsearch sink will now send JSONB as a JSON string, and Elasticsearch will convert it into an object. [#14273](https://github.com/risingwavelabs/risingwave/pull/14273). * `connector = 'iceberg_java'` is deprecated, and users can only Iceberg sinks with the Rust version of Iceberg. Similarly, the DeltaLake sink will also use the Rust version implementation. [#14277](https://github.com/risingwavelabs/risingwave/pull/14277). * Supports StarRocks sink. [#12681](https://github.com/risingwavelabs/risingwave/pull/12681). ## Installation and deployment * Allows for `storage.prefetch_buffer_capacity_mb` to be configured in the TOML file to prevent out-of-memory issues. [#13558](https://github.com/risingwavelabs/risingwave/pull/13558). * Supports Huawei Cloud OBS as the storage backend. [#13844](https://github.com/risingwavelabs/risingwave/pull/13844). ## Cluster configuration changes * Supports setting `statement_timeout` value for queries. [#13933](https://github.com/risingwavelabs/risingwave/pull/13933). * Exposes SSL functionality through `RW_SSL_CERT` and `RW_SSL_KEY` environment variables to configure SSL certificates and key file location. [#14062](https://github.com/risingwavelabs/risingwave/pull/14062). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.6.0 playground` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.6.0/risingwave-v1.6.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.6.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.6.0.tar.gz) * [risectl-v1.6.0-x86\_64-unknown-linux.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.6.0/risectl-v1.6.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/release-1.5...release-1.6). ## SQL features * SQL Commands: * Supports `SET SCHEMA` syntax for `ALTER {TABLE t | [MATERIALIZED] VIEW (m)v | SOURCE src | SINK sink | CONNECTION c | FUNCTION f( argument_type [, ...] )}`. [#13341](https://github.com/risingwavelabs/risingwave/pull/13341). * Supports `OWNER TO` syntax for ALTER DATABASE | SCHEMA | TABLE | \[MATERIALIZED] VIEW | SOURCE | SINK. [#13216](https://github.com/risingwavelabs/risingwave/pull/13216). * Supports `RENAME TO` syntax for `ALTER { DATABASE db | SCHEMA s}`. [#13713](https://github.com/risingwavelabs/risingwave/pull/13713). * Supports `KILL` command. [#13434](https://github.com/risingwavelabs/risingwave/pull/13434) * Supports `SHOW PROCESSLIST` command. [#13287](https://github.com/risingwavelabs/risingwave/pull/13287). * Supports `SET TO DEFAULT` command. [#13693](https://github.com/risingwavelabs/risingwave/pull/13693). * Supports `SHOW COLUMNS` and `DESCRIBE` from sinks and views. [#13626](https://github.com/risingwavelabs/risingwave/pull/13626). * SQL functions & operators * Supports list and struct types for `jsonb_agg` and `jsonb_object_agg`. [#13299](https://github.com/risingwavelabs/risingwave/pull/13299). * Supports `jsonb_build_array` and `jsonb_build_object`. [#13198](https://github.com/risingwavelabs/risingwave/pull/13198). * Supports `to_jsonb`. [#13161](https://github.com/risingwavelabs/risingwave/pull/13161). * Supports JSON path operators and functions. [#13568](https://github.com/risingwavelabs/risingwave/pull/13568). * Supports array operators `@>` and `<@`. [#13253](https://github.com/risingwavelabs/risingwave/pull/13253). * Fixes the correctness of case expressions. Previously if there were multiple matching values, the last one would match. [#13890](https://github.com/risingwavelabs/risingwave/pull/13890). **The fix introduces a breaking change**. It is recommended to drop and recreate any materialized views that contain `CASE` expressions. If your instance enters a crash-loop, we suggest upgrading to v1.5.2, and dropping the corresponding materialized view that contains `CASE` expressions. * System catalog * Adds columns `rw_version`, `total_memory_bytes`, `total_cpu_cores`, and `started_at`, and all nodes in system table `rw_worker_nodes`. [#13487](https://github.com/risingwavelabs/risingwave/pull/13487). * Adds system table `rw_internal_tables`. [#13272](https://github.com/risingwavelabs/risingwave/pull/13272). * Supports sink columns in `rw_columns` and `information_schema.columns`. [#13626](https://github.com/risingwavelabs/risingwave/pull/13626). ## Sources & sink * The load generator can generate `timestamptz` columns. [#13451](https://github.com/risingwavelabs/risingwave/pull/13451). * Adds option `[properties.fetch.queue.backoff.ms](http://properties.fetch.queue.backoff.ms)` for Kafka source. [#13321](https://github.com/risingwavelabs/risingwave/pull/13321). * Supports creating multiple CDC tables that share the same source, which allows for incremental and lock-free snapshot loading. [#12535](https://github.com/risingwavelabs/risingwave/pull/12535). * `CREATE SINK` statements no longer need to wait for backfill to complete. [#13665](https://github.com/risingwavelabs/risingwave/pull/13665) ## Deployment * Adds a docker-compose file for standalone mode. [#13233](https://github.com/risingwavelabs/risingwave/pull/13233). ### Cluster configuration changes * Adds support for system parameter `pause_on_next_bootstrap`. [#11936](https://github.com/risingwavelabs/risingwave/pull/11936) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.5.0 playground` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.5.0/risingwave-v1.5.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.5.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.5.0.tar.gz) * [risectl-v1.5.0-x86\_64-unknown-linux.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.5.0/risectl-v1.5.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v1.4-rc...v1.5.0-rc). ## SQL features * Query syntax: * Supports using subqueries in `UPDATE` and `DELETE` statements. [#12995](https://github.com/risingwavelabs/risingwave/pull/12995) * SQL commands * Supports `COMMENT ON` clause for tables and columns. [#12849](https://github.com/risingwavelabs/risingwave/pull/12849). * Supports persistent background materialized views. [#12167](https://github.com/risingwavelabs/risingwave/pull/12167). * Supports exposing hidden columns and distribution keys when using `SHOW COLUMNS FROM` command. [#12839](https://github.com/risingwavelabs/risingwave/pull/12839) * Supports exposing hidden columns when using `DESCRIBE` command. [#12839](https://github.com/risingwavelabs/risingwave/pull/12839) * SQL functions & operators: * Supports `substring` and `substr` functions for `bytea` data type. [#13088](https://github.com/risingwavelabs/risingwave/pull/13088). * Supports functions `jsonb_pretty`,  `jsonb_object`, `jsonb_strip_nulls`, and `jsonb_extract_path`. [#13050](https://github.com/risingwavelabs/risingwave/pull/13050), [#13036](https://github.com/risingwavelabs/risingwave/pull/13036), [#13169](https://github.com/risingwavelabs/risingwave/pull/13169), [#13143](https://github.com/risingwavelabs/risingwave/pull/13143). * Supports jsonb `@>`, `<@`, `?`, `?|, ?&`, `#>`,  `#>>`, `-` and `#-` operators. [#13056](https://github.com/risingwavelabs/risingwave/pull/13056), [#13110](https://github.com/risingwavelabs/risingwave/pull/13110), [#13118](https://github.com/risingwavelabs/risingwave/pull/13118). * Supports `greatest` and `least` functions. [#12838](https://github.com/risingwavelabs/risingwave/pull/12838). * Supports `regexp_split_to_array` function. [#12844](https://github.com/risingwavelabs/risingwave/pull/12844). * Supports `bit_and` and `bit_or` aggregate functions in materialized views. [#12758](https://github.com/risingwavelabs/risingwave/pull/12758). * Supports `jsonb_agg` and `jsonb_object_agg` in streaming mode. [#12836](https://github.com/risingwavelabs/risingwave/pull/12836). * Supports general `rank` and `dense_rank` window functions. [#13183](https://github.com/risingwavelabs/risingwave/pull/13183). * System catalog: * Adds column `parallelism` in system table `rw_fragments`. [#12901](https://github.com/risingwavelabs/risingwave/pull/12901) * Adds columns `is_hidden`, `is_primary_key`, and `is_distribution_key` in `rw_columns` system table. [#12839](https://github.com/risingwavelabs/risingwave/pull/12839) ## Sources & sinks * Adds `google.protobuf.Any` support for Protobuf sources. [#12291](https://github.com/risingwavelabs/risingwave/pull/12291). * Adds `schemas.enable` support for Kafka sinks with upsert JSON. [#12113](https://github.com/risingwavelabs/risingwave/pull/12113). * Adds support for Kafka sinks with upsert Avro using schema registry. [#13007](https://github.com/risingwavelabs/risingwave/pull/13007). * `server.id` option is now optional for MySQL CDC source. [#13031](https://github.com/risingwavelabs/risingwave/pull/13031) * Enables `timestamptz.handling.mode` option to control the timestamptz output format for certain sinks. [#13109](https://github.com/risingwavelabs/risingwave/pull/13109). * Adds the `stream` field and support for multiple inputs for the `subject` field for NATS source connector. [#12799](https://github.com/risingwavelabs/risingwave/pull/12799). * Adds new option `properties.allow.auto.create.topics` for Kafka sink. [#12766](https://github.com/risingwavelabs/risingwave/pull/12766). * Adds support for `s3_v2` source connector, a more efficient version of the S3 source. [#12595](https://github.com/risingwavelabs/risingwave/pull/12595). * Adds support for Google BigQuery sink.[#12873](https://github.com/risingwavelabs/risingwave/pull/12873). * Adds support for Redis sink. [#11999](https://github.com/risingwavelabs/risingwave/pull/11999),[#13003](https://github.com/risingwavelabs/risingwave/pull/13003). ## Deployment * Release RisingWave all-in-one binary with connector libraries. [#13133](https://github.com/risingwavelabs/risingwave/pull/13133) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.4.0 playground` * [Prebuilt all-in-one library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.4.0/risingwave-v1.4.0-x86_64-unknown-linux-all-in-one.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.4.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.4.0.tar.gz) * [risectl-v1.4.0-x86\_64-unknown-linux.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.4.0/risectl-v1.4.0-x86_64-unknown-linux.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v1.3-rc...v1.4-rc). ## SQL features * SQL commands * Beta: Variable `BACKGROUND_DDL` can be set to `true` when creating a materialized view. [#12355](https://github.com/risingwavelabs/risingwave/pull/12355) * `ALTER COLUMN` command can be used for tables with non-schema-registry source. [#12164](https://github.com/risingwavelabs/risingwave/pull/12164) * SQL functions & operators * Supports `array_min`. [#12071](https://github.com/risingwavelabs/risingwave/pull/12071) * Supports `array_max`. [#12100](https://github.com/risingwavelabs/risingwave/pull/12100) * Supports `array_sort`. [#12189](https://github.com/risingwavelabs/risingwave/pull/12189) * Supports `array_sum`. [#12162](https://github.com/risingwavelabs/risingwave/pull/12162) * `format` function supports variable inputs. [#12178](https://github.com/risingwavelabs/risingwave/pull/12178) * Regular expression functions support back reference, positive, negative lookahead, and positive, negative lookbehind. [#12329](https://github.com/risingwavelabs/risingwave/pull/12329) * Supports `||` operator for concatenating JSONB data. [#12502](https://github.com/risingwavelabs/risingwave/pull/12502) * Supports `bool_and` and `bool_or` in materialized views. [#11956](https://github.com/risingwavelabs/risingwave/pull/11956) * Query syntax: * Supports `WITH ORDINALITY` clause. [#12273](https://github.com/risingwavelabs/risingwave/pull/12273) * System catalog * Adds new system function `pg_sleep`. [#12294](https://github.com/risingwavelabs/risingwave/pull/12294) * Adds new system function `_pg_expandarray`. [#12448](https://github.com/risingwavelabs/risingwave/pull/12448) * Adds new storage related system tables: * `rw_hummock_sstables` [#12532](https://github.com/risingwavelabs/risingwave/pull/12532) * `rw_hommock_pinned_versions`, `rw_hommock_pinned_snapshots` [#12285](https://github.com/risingwavelabs/risingwave/pull/12285) * `rw_hummock_branched_objects` , `rw_hummock_current_version` , `rw_hummock_checkpoint_version` , `rw_hummock_version_deltas` [#12309](https://github.com/risingwavelabs/risingwave/pull/12309) * `rw_hummock_meta_configs`, `rw_hummock_compaction_group_configs` [#12337](https://github.com/risingwavelabs/risingwave/pull/12337) ## Sources & sinks * Generated columns defined with non-deterministic functions cannot be part of the primary key. [#12181](https://github.com/risingwavelabs/risingwave/pull/12181) * Adds new `properties.enable.auto.commit` parameter for the Kafka consumer, which sets the `enable.auto.commit` parameter for the Kafka client. [#12223](https://github.com/risingwavelabs/risingwave/pull/12223) * Adds `privatelink.endpoint` parameter to the WITH clause, to support private link for Kafka connector on GCP and AWS. [#12266](https://github.com/risingwavelabs/risingwave/pull/12266) * Adds parameters `message.timeout.ms` and `max.in.flight.requests.per.connection` for Kafka sources. [#12574](https://github.com/risingwavelabs/risingwave/pull/12574) * Allows Kinesis source to start ingesting data from a specific timestamp. `sequence_number` is no longer supported as a startup mode option. [#12241](https://github.com/risingwavelabs/risingwave/pull/12241) * Allow optional `FORMAT DEBEZIUM ENCODE JSON` after the connector definition of CDC tables. Allow optional `FORMAT NATIVE ENCODE NATIVE` after the connector definition of Nexmark sources or tables. [#12306](https://github.com/risingwavelabs/risingwave/pull/12306) * Allows multiple URLs when defining schema registries. [#11982](https://github.com/risingwavelabs/risingwave/pull/11982) * Adds support for sinking data to versions 7 and 8 of Elasticsearch. [#10357](https://github.com/risingwavelabs/risingwave/pull/10357), [#10415](https://github.com/risingwavelabs/risingwave/pull/10415), [#1303](https://github.com/risingwavelabs/risingwave-docs/issues/1303) * Adds support for sinking append-only data to the NATS messaging system. [#11924](https://github.com/risingwavelabs/risingwave/pull/11924) * Adds support for sinking data to Doris. [#12336](https://github.com/risingwavelabs/risingwave/pull/12336) * Adds support for sinking data to Apache Pulsar. [#12286](https://github.com/risingwavelabs/risingwave/pull/12286) * Adds support for sinking data to Cassandra and ScyllaDB. [#11878](https://github.com/risingwavelabs/risingwave/pull/11878) * Adds support for creating upsert Iceberg sinks. [#12576](https://github.com/risingwavelabs/risingwave/pull/12576) * Supports specifying the `sink_decouple` session variable as `default`, `true` and `enable`, or `false` and `disable`. [#12544](https://github.com/risingwavelabs/risingwave/pull/12544) * A `varchar` column in RisingWave can sink into a `uuid` column in Postgres. [#12704](https://github.com/risingwavelabs/risingwave/pull/12704) * New syntaxes for specifying data format and data encoding when creating a Kafka, Kinesis, and Pulsar sink. [#12556](https://github.com/risingwavelabs/risingwave/pull/12556) #### Administration & observability * Supports querying from `information_schema.views`, which contains formations about views defined in the database. [#12045](https://github.com/risingwavelabs/risingwave/pull/12045) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.3.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.3.0/risingwave-v1.3.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.3.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.3.0.tar.gz) * [risectl-v1.3.0-x86\_64-unknown-linux.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.3.0/risectl-v1.3.0-x86_64-unknown-linux.tar.gz) * [risingwave-connector-v1.3.0.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.3.0/risingwave-connector-v1.3.0.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v1.2-rc...v1.3-rc). ## SQL features * SQL commands: * Breaking change: Syntax of emit-on-window-close has changed. If your application contains integration code, please update your code accordingly. [#11363](https://github.com/risingwavelabs/risingwave/pull/11363) In v1.1: ```sql theme={null} CREATE MATERIALIZED VIEW mv EMIT ON WINDOW CLOSE AS SELECT ...; ``` In v1.2 and onwards: ```sql theme={null} CREATE MATERIALIZED VIEW mv AS SELECT ... EMIT ON WINDOW CLOSE; ``` * Privileges for tables can now be granted or revoked. [#11725](https://github.com/risingwavelabs/risingwave/pull/11725) * The default `DISTRIBUTED BY` columns have been changed from the whole index columns into the first index column. [#11865](https://github.com/risingwavelabs/risingwave/pull/11865) * Supports `ALTER SOURCE ADD COLUMN`. [#11350](https://github.com/risingwavelabs/risingwave/pull/11350) * Supports `SHOW JOBS` and `CANCEL JOBS` , with which you can show the in-progress streaming jobs and cancel jobs by their IDs. [#11854](https://github.com/risingwavelabs/risingwave/pull/11854) * Supports `[I]LIKE` in `SHOW` commands. [#11791](https://github.com/risingwavelabs/risingwave/pull/11791) * SQL functions & operators * Supports lambda functions via `array_transform`. [#11888](https://github.com/risingwavelabs/risingwave/pull/11888), [#11937](https://github.com/risingwavelabs/risingwave/pull/11937) * Supports `to_date()`. [#11241](https://github.com/risingwavelabs/risingwave/pull/11241) * The `to_char()` function now supports `timestamptz` input. [#11778](https://github.com/risingwavelabs/risingwave/pull/11778) * Supports `scale`, `min_scale`, and `trim_scale`. [#11663](https://github.com/risingwavelabs/risingwave/pull/11663) * Supports `regexp_replace`. [#11819](https://github.com/risingwavelabs/risingwave/pull/11819) * Supports `regexp_count`. [#11975](https://github.com/risingwavelabs/risingwave/pull/11975) * Supports `[NOT] ILIKE` expressions. [#11743](https://github.com/risingwavelabs/risingwave/pull/11743) * Supports `[!]~~[*]` operators, which are equivalent to `[NOT] [I]LIKE`. [#11748](https://github.com/risingwavelabs/risingwave/pull/11748) * Supports `IS JSON` predicate. [#11831](https://github.com/risingwavelabs/risingwave/pull/11831) * Query syntax: * Adds support for `LIMIT` clauses in streaming queries. [#11566](https://github.com/risingwavelabs/risingwave/pull/11566) * Supports `LATERAL` subqueries. [#11780](https://github.com/risingwavelabs/risingwave/pull/11780) * System catalog * A new group of system catalogs (`rw_relations`, `rw_system_tables`, `rw_types`, `rw_user_secrets`, and `rw_columns`) are available for you to retrieve system data and metadata. [#11334](https://github.com/risingwavelabs/risingwave/pull/11334) * Adds new system function `pg_relation_size()`. [#11687](https://github.com/risingwavelabs/risingwave/pull/11687) * Adds support for transactions for single-table CDC data. [#11453](https://github.com/risingwavelabs/risingwave/pull/11453) ## Sources & sinks * Adds a new parameter `schema.registry.name.strategy` to the Kafka connector, with which you can specify naming strategies for schema registries. [#11384](https://github.com/risingwavelabs/risingwave/pull/11384) * Breaking Change: Implements a Rust-native Iceberg sink connector to improve stability and performance. The connector introduces new parameters. Applications that rely on the previous version of the feature (specifically, the version included in RisingWave v1.0 and v1.1) may no longer function correctly. To restore functionality to your applications, please carefully review the syntax and parameters outlined on this page and make any necessary revisions to your code. [#11326](https://github.com/risingwavelabs/risingwave/pull/11326) * Adds support for sinking data to ClickHouse. [#11240](https://github.com/risingwavelabs/risingwave/pull/11240) * Beta: An enhancement has been made to the mysql-cdc connector to improve data ingestion performance. It achieves so by optimizing the data backfilling logic for CDC tables. This feature is not enabled by default. To enable it, run this command: `SET cdc_backfill="true";` [#11707](https://github.com/risingwavelabs/risingwave/pull/11707) * Adds a parameter `client.id` for Kafka sources. [#11911](https://github.com/risingwavelabs/risingwave/pull/11911) ## Deployment * Supports HDFS as the storage backend for deployments via Docker Compose. [#11632](https://github.com/risingwavelabs/risingwave/pull/11632) ## Administration & observability * Adds a new system parameter `max_concurrent_creating_streaming_jobs`, with which users can specify the maximum number of streaming jobs that can be created concurrently. [#11601](https://github.com/risingwavelabs/risingwave/pull/11601) * Improves the calculation logic of the *Mem Table Size (Max)* metric in the RisingWave Dashboard. [#11442](https://github.com/risingwavelabs/risingwave/pull/11442) * Adds new metrics to RisingWave Dashboard: * *Materialized View Memory Usage* [#10958](https://github.com/risingwavelabs/risingwave/pull/10958) * *Materialized View Read Size*, *Materialized View Write Size* [#11054](https://github.com/risingwavelabs/risingwave/pull/11054) * *Active Sessions* [#11688](https://github.com/risingwavelabs/risingwave/pull/11688) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.2.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.2.0/risingwave-v1.2.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.2.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.2.0.tar.gz) * [risectl-v1.2.0-x86\_64-unknown-linux.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.2.0/risectl-v1.2.0-x86_64-unknown-linux.tar.gz) * [risingwave-connector-v1.2.0.tar.gz](https://github.com/risingwavelabs/risingwave/releases/download/v1.2.0/risingwave-connector-v1.2.0.tar.gz) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v1.1.0...v1.2.0). ## SQL features * SQL commands: * `DROP` commands now support the `CASCADE` option, which drops the specified item and all its dependencies. [#11250](https://github.com/risingwavelabs/risingwave/pull/11250) * `CREATE TABLE` now supports the `APPEND ONLY` clause, allowing the definition of watermark columns on the table. [#11233](https://github.com/risingwavelabs/risingwave/pull/11233) * Supports new commands `START TRANSACTION`, `BEGIN`, and `COMMIT` for read-only transactions. [#10735](https://github.com/risingwavelabs/risingwave/pull/10735) * Supports `SHOW CLUSTER` to show the details of your RisingWave cluster, including the address of the cluster, its state, the parallel units it is using, and whether it's streaming data, serving data, or unschedulable. [#10656](https://github.com/risingwavelabs/risingwave/pull/10656), [#10932](https://github.com/risingwavelabs/risingwave/pull/10932) * SQL functions: * Supports new window functions: `lead()` and `lag()`. [#10915](https://github.com/risingwavelabs/risingwave/pull/10915) * Supports new aggregate functions: `first_value()` and `last_value()`, which retrieve the first and last values within a specific ordering from a set of rows. [#10740](https://github.com/risingwavelabs/risingwave/pull/10740) * Supports the `grouping()` function to determine if a column or expression in the `GROUP BY` clause is part of the current grouping set or not. [#11006](https://github.com/risingwavelabs/risingwave/pull/11006) * Supports the `set_config()` system administration function. [#11147](https://github.com/risingwavelabs/risingwave/pull/11147) * Supports the `sign()` mathematical function. [#10819](https://github.com/risingwavelabs/risingwave/pull/10819) * Supports `string_agg()` with `DISTINCT` and `ORDER BY`, enabling advanced string concatenation with distinct values and custom sorting. [#10864](https://github.com/risingwavelabs/risingwave/pull/10864) * Supports the co-existence of `string_agg()` and other aggregations with `DISTINCT`. [#10864](https://github.com/risingwavelabs/risingwave/pull/10864) * Supports the `zone_string` parameter in the `date_trunc()`, `extract()`, and `date_part()` functions, ensuring compatibility with PostgreSQL. [#10480](https://github.com/risingwavelabs/risingwave/pull/10480) * **Breaking change**: Previously, when the input for `date_trunc` was actually a date, the function would cast it to a timestamp and record the choice in the query plan. However, after this release, new query plans will cast the input to `timestamptz` instead. As a result, some old SQL queries, especially those saved as views, may fail to bind correctly and require type adjustments. It's important to note that old query plans will continue working because the casting choice is recorded with a cast to timestamp. Before this release: ```sql theme={null} SELECT date_trunc('month', date '2023-03-04'); date_trunc --------------------------- 2023-03-01 00:00:00 (1 row) ``` After this release: ```sql theme={null} SELECT date_trunc('month', date '2023-03-04'); date_trunc --------------------------- 2023-03-01 00:00:00+00:00 (1 row) ``` Now, the result of `date_trunc` includes the timezone offset (`+00:00`) in the output, making it consistent with the behavior in PostgreSQL. * `round()` now accepts a negative value and rounds it to the left of the decimal point. [#10961](https://github.com/risingwavelabs/risingwave/pull/10961) * `to_timestamp()` now returns `timestamptz`. [#11018](https://github.com/risingwavelabs/risingwave/pull/11018) * Query clauses * `SELECT` now supports the `EXCEPT` clause which excludes specific columns from the result set. [#10438](https://github.com/risingwavelabs/risingwave/pull/10438), [#10723](https://github.com/risingwavelabs/risingwave/pull/10723) * `SELECT` now supports the `GROUPING SETS` clause which allows users to perform aggregations on multiple levels of grouping within a single query. [#10807](https://github.com/risingwavelabs/risingwave/pull/10807) * Supports index selection for temporal joins. [#11019](https://github.com/risingwavelabs/risingwave/pull/11019) * Supports `CUBE` in group-by clauses to generate multiple grouping sets. [#11262](https://github.com/risingwavelabs/risingwave/pull/11262) * Patterns * Supports multiple rank function calls in TopN by group. [#11149](https://github.com/risingwavelabs/risingwave/pull/11149) * System catalog * Supports querying `created_at` and `initialized_at` from RisingWave relations such as sources, sinks, and tables in RisingWave catalogs. [#11199](https://github.com/risingwavelabs/risingwave/pull/11199) ## Connectors * Supports specifying more Kafka parameters when creating a source or sink. [#11203](https://github.com/risingwavelabs/risingwave/pull/11203) * JDBC sinks used for upserts must specify the downstream primary key via the `primary_key` option. [#11042](https://github.com/risingwavelabs/risingwave/pull/11042) * `access_key` and its corresponding `secret_key` are now mandatory for all AWS authentication components. [#11120](https://github.com/risingwavelabs/risingwave/pull/11120) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.1.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.1.0/risingwave-v1.1.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.1.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.1.0.tar.gz) ## SQL features * SQL command: * Supports the `GROUPING SETS` clause. [#10807](https://github.com/risingwavelabs/risingwave/pull/10807) * SQL function: * Adds the `current_setting()` function to get the current value of a configuration parameter. [#10051](https://github.com/risingwavelabs/risingwave/issues/10051) * Adds new array functions: `array_position()`, `array_replace()`, `array_ndims()`, `array_lower()`, `array_upper()`, `array_length()`, and `array_dims()`. [#10166](https://github.com/risingwavelabs/risingwave/pull/10166), [#10197](https://github.com/risingwavelabs/risingwave/pull/10197) * Adds new aggregate functions: `percentile_cont()`, `percentile_disc()`, and `mode()`. [#10252](https://github.com/risingwavelabs/risingwave/pull/10252) * Adds new system functions: `user()`, `current_user()`, and `current_role()`. [#10366](https://github.com/risingwavelabs/risingwave/pull/10366) * Adds new string functions: `left()` and `right()`. [#10765](https://github.com/risingwavelabs/risingwave/pull/10765) * Adds new bytea functions: `octet_length()` and `bit_length()`. [#10462](https://github.com/risingwavelabs/risingwave/pull/10462) * `array_length()` and `cardinality()` return integer instead of bigint. [#10267](https://github.com/risingwavelabs/risingwave/pull/10267) * Supports the `row_number` window function that doesn't match the TopN pattern. [#10869](https://github.com/risingwavelabs/risingwave/pull/10869) * User-defined function: * Adds support for defining UDFs in Java. [#10095](https://github.com/risingwavelabs/risingwave/pull/10095) * Adds support for more Java UDF and Python UDF data types. [#10399](https://github.com/risingwavelabs/risingwave/pull/10399) * The language parameter is no longer required in `CREATE FUNCTION`. [#10608](https://github.com/risingwavelabs/risingwave/pull/10608) * System catalog: * Adds more columns to `information_schema.columns`: `column_default`, `character_maximum_length`, and `udt_name`. [#10269](https://github.com/risingwavelabs/risingwave/pull/10269) * Adds a new system catalog `pg_proc`. [#10216](https://github.com/risingwavelabs/risingwave/pull/10216) * Adds new RisingWave catalogs: * `rw_table_fragments`, `rw_fragments`, `rw_actors` [#10712](https://github.com/risingwavelabs/risingwave/pull/10712) * `rw_worker_nodes`, `rw_parallel_units` [#10656](https://github.com/risingwavelabs/risingwave/pull/10656) * `rw_connections`, `rw_databases`, `rw_functions`, `rw_indexes`, `rw_materialized_views`, `rw_schemas`, `rw_sinks`, `rw_sources`, `rw_tables`, `rw_users`, `rw_views` [#10593](https://github.com/risingwavelabs/risingwave/pull/10593) * Supports `GROUP BY` output alias or index. [#10305](https://github.com/risingwavelabs/risingwave/pull/10305) * Supports using scalar functions in the `FROM` clause. [#10317](https://github.com/risingwavelabs/risingwave/pull/10317) * Supports tagging the created VPC endpoints when creating a PrivateLink connection. [#10582](https://github.com/risingwavelabs/risingwave/pull/10582) ## Connectors * ***Breaking change***: When creating a source or table with a connector whose schema is auto-resolved from an external format file, the syntax for defining primary keys within column definitions is replaced with the table constraint syntax. [#10195](https://github.com/risingwavelabs/risingwave/pull/10195) ```sql Old theme={null} CREATE TABLE debezium_non_compact (order_id int PRIMARY KEY) WITH ( connector = 'kafka', kafka.topic = 'debezium_non_compact_avro_json', kafka.brokers = 'message_queue:29092', kafka.scan.startup.mode = 'earliest' ) ROW FORMAT DEBEZIUM_AVRO ROW SCHEMA LOCATION CONFLUENT SCHEMA REGISTRY 'http://message_queue:8081'; ``` ```sql New theme={null} CREATE TABLE debezium_non_compact (PRIMARY KEY(order_id)) WITH ( ... ``` * ***Breaking change***: Modifies the syntax for specifying data and encoding formats for a source in `CREATE SOURCE` and `CREATE TABLE` commands. For v1.0.0, the old syntax is still accepted but will be deprecated in the next release. [#10768](https://github.com/risingwavelabs/risingwave/pull/10768) Old syntax - part 1: ```bash theme={null} ROW FORMAT data_format [ MESSAGE 'message' ] [ ROW SCHEMA LOCATION ['location' | CONFLUENT SCHEMA REGISTRY 'schema_registry_url' ] ]; ``` New syntax - part 1: ```sql theme={null} FORMAT data_format ENCODE data_encode ( message = 'message', schema.location = 'location' | schema.registry = 'schema_registry_url' ); ``` Old syntax - part 2: ```sql theme={null} ROW FORMAT csv WITHOUT HEADER DELIMITED BY ','; ``` New syntax - part 2: ```bash theme={null} FORMAT PLAIN ENCODE CSV ( without_header = 'true', delimiter = ',' ); ``` * Supports sinking data to Delta Lake. [#10374](https://github.com/risingwavelabs/risingwave/pull/10374), [#10580](https://github.com/risingwavelabs/risingwave/pull/10580) * Supports sinking data to AWS Kinesis. [#10437](https://github.com/risingwavelabs/risingwave/pull/10437) * Supports `BYTES` as a row format. [#10592](https://github.com/risingwavelabs/risingwave/pull/10592) * Supports specifying schema for the PostgreSQL sink. [#10576](https://github.com/risingwavelabs/risingwave/pull/10576) * Supports using the user-provided publication to create a PostgreSQL CDC table. [#10804](https://github.com/risingwavelabs/risingwave/pull/10804) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v1.0.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v1.0.0/risingwave-v1.0.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.0.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v1.0.0.tar.gz) ## Installation * Now, you can easily install RisingWave on your local machine with Homebrew by running `brew install risingwave`. See [Run RisingWave](/get-started/quickstart#install-and-start-risingwave). ## Administration * Adds the `pg_indexes` and `dattablespace` system catalogs. [#9844](https://github.com/risingwavelabs/risingwave/pull/9844), [#9822](https://github.com/risingwavelabs/risingwave/pull/9822) * Now, the `SHOW PARAMETERS` command will display the mutability of each system parameter. [#9526](https://github.com/risingwavelabs/risingwave/pull/9526) ## SQL features * Experimental features: Adds support for 256-bit integers. [#9146](https://github.com/risingwavelabs/risingwave/pull/9146), [#9184](https://github.com/risingwavelabs/risingwave/pull/9184), [#9186](https://github.com/risingwavelabs/risingwave/pull/9186), [#9191](https://github.com/risingwavelabs/risingwave/pull/9191), [#9217](https://github.com/risingwavelabs/risingwave/pull/9217) * Indexes can be created on expressions. [#9142](https://github.com/risingwavelabs/risingwave/pull/9142) * Adds support for expressions in aggregate function arguments. [#9955](https://github.com/risingwavelabs/risingwave/pull/9955) * Adds support for `VALUES` clause. [#8751](https://github.com/risingwavelabs/risingwave/pull/8751) * Adds support for generated columns, which are special columns computed from other columns in a table or source. [#8700](https://github.com/risingwavelabs/risingwave/pull/8700), [#9580](https://github.com/risingwavelabs/risingwave/pull/9580) * Adds support for using expressions in the `ORDER BY` and `PARTITION BY` clauses. [#9827](https://github.com/risingwavelabs/risingwave/pull/9827) * New SQL commands * `CREATE CONNECTION` and `SHOW CONNECTIONS`: Creates an AWS PrivateLink connection and show all available connections. [#8907](https://github.com/risingwavelabs/risingwave/pull/8907) * `DROP CONNECTION`: Drops a connection. [#9128](https://github.com/risingwavelabs/risingwave/pull/9128) * `SHOW FUNCTIONS`: Shows existing user-defined functions. [#9398](https://github.com/risingwavelabs/risingwave/pull/9398) * `DROP FUNCTIONS`: Drops a user-defined function. [#9561](https://github.com/risingwavelabs/risingwave/pull/9561) * `SHOW CREATE SOURCE` and `SHOW CREATE SINK`: Shows the SQL statement used to create a source or sink. [#9083](https://github.com/risingwavelabs/risingwave/pull/9083) * `SHOW INDEXES`: Shows all indexes on a particular table. [#9835](https://github.com/risingwavelabs/risingwave/pull/9835) * SQL functions * Adds support for trigonometric functions. [#8838](https://github.com/risingwavelabs/risingwave/pull/8838), [#8918](https://github.com/risingwavelabs/risingwave/pull/8918), [#9064](https://github.com/risingwavelabs/risingwave/pull/9064), [#9203](https://github.com/risingwavelabs/risingwave/pull/9203), [#9259](https://github.com/risingwavelabs/risingwave/pull/9259) * Adds support for degrees and radians functions. [#9108](https://github.com/risingwavelabs/risingwave/pull/9108) * Adds support for the `lag()` and `lead()` window functions and the `OVER()` and  `EMIT ON WINDOW CLOSE` clause. [#9597](https://github.com/risingwavelabs/risingwave/pull/9597), [#9622](https://github.com/risingwavelabs/risingwave/pull/9622), [#9701](https://github.com/risingwavelabs/risingwave/pull/9701) * Adds support for new aggregate functions, including `bool_and`, `bool_or`, `jsonb_agg`, and `jsonb_object_agg`. [#9452](https://github.com/risingwavelabs/risingwave/pull/9452) * Adds support for `max()`, `min()`, and `count()` for timestamptz data. [#9165](https://github.com/risingwavelabs/risingwave/pull/9165) * Adds support for microseconds and milliseconds for `to_char()` and `to_timestamp()`. [#9257](https://github.com/risingwavelabs/risingwave/pull/9257) * Adds support for multibyte Unicode in `overlay()` and `ascii()` functions. [#9321](https://github.com/risingwavelabs/risingwave/pull/9321) * Adds support for the `string_to_array()` function. [#9289](https://github.com/risingwavelabs/risingwave/pull/9289) * Adds support for `array_positions()`. [#9152](https://github.com/risingwavelabs/risingwave/pull/9152) * Adds support for `cardinality()`. [#8867](https://github.com/risingwavelabs/risingwave/pull/8867) * Adds support for `array_remove()`. [#9116](https://github.com/risingwavelabs/risingwave/pull/9116) * Adds support for `trim_array()`. [#9265](https://github.com/risingwavelabs/risingwave/pull/9265) * Adds support for array range access. [#9362](https://github.com/risingwavelabs/risingwave/pull/9362) * Adds support for JSONB in UDF. [#9103](https://github.com/risingwavelabs/risingwave/pull/9103) * Adds support for `btrim()` and updates `trim()` to PostgreSQL standard syntax. [#8985](https://github.com/risingwavelabs/risingwave/pull/8985) * Adds support for `date_part()`. [#8830](https://github.com/risingwavelabs/risingwave/pull/8830) * Expands `extract()` with more fields. [#8830](https://github.com/risingwavelabs/risingwave/pull/8830) * Adds support for `proctime()`, which returns the system time with time zone when a record is processed. [#9088](https://github.com/risingwavelabs/risingwave/pull/9088) * Adds support for `translate()`, `@()`, and `ceiling()`. [#8998](https://github.com/risingwavelabs/risingwave/pull/8998) * Adds support for `encode()` and `decode()`. [#9351](https://github.com/risingwavelabs/risingwave/pull/9351) * Adds support for the `intersect` operator. [#9573](https://github.com/risingwavelabs/risingwave/pull/9573) * Adds support for the default escape `\` in a `like` expression. [#9624](https://github.com/risingwavelabs/risingwave/pull/9624) * Adds support for the `IS [NOT] UNKNOWN` comparison predicate. [#9965](https://github.com/risingwavelabs/risingwave/pull/9965) * Adds support for the `starts_with()` string function and `^@`. [#9967](https://github.com/risingwavelabs/risingwave/pull/9967) * Adds support for unary `trunc`, `ln`, `log10` (`log`), `exp`, `cbrt` (`||/`) mathematical functions. [#9991](https://github.com/risingwavelabs/risingwave/pull/9991) ## Connectors * Adds support for ingesting CDC data from TiDB and sinking data to TiDB with the JDBC connector. [#8708](https://github.com/risingwavelabs/risingwave/pull/8708) * Adds support for ingesting CDC data from Citus. [#8988](https://github.com/risingwavelabs/risingwave/pull/8988) * Adds support for loading Pulsar secret key file from AWS S3. [#8428](https://github.com/risingwavelabs/risingwave/pull/8428), [#8222](https://github.com/risingwavelabs/risingwave/pull/8222) * Adds support for using an established AWS PrivateLink connection in a `CREATE SOURCE`, `CREATE TABLE`, or `CREATE SINK` statement for a Kafka source/sink. [#9119](https://github.com/risingwavelabs/risingwave/pull/9119), [#9128](https://github.com/risingwavelabs/risingwave/pull/9128), [#9728](https://github.com/risingwavelabs/risingwave/pull/9728), [#9263](https://github.com/risingwavelabs/risingwave/pull/9263) * Deprecates the `use_transaction` field in the Kafka sink connector. [#9207](https://github.com/risingwavelabs/risingwave/pull/9207) * Adds support for zstd compression type for Kafka connector. [#9297](https://github.com/risingwavelabs/risingwave/pull/9297) * Deprecates the `upsert` property in the Kafka connector as it can be inferred from the row format. [#9457](https://github.com/risingwavelabs/risingwave/pull/9457) * Adds a new field `properties.sync.call.timeout` in the WITH clause of the Kafka source connector to control the timeout. [#9005](https://github.com/risingwavelabs/risingwave/pull/9005) * Adds support for a new row format `DEBEZIUM_MONGO_JSON` in the Kafka source connector. [#9250](https://github.com/risingwavelabs/risingwave/pull/9250) * Adds CSV format support for the Kafka source connector. [#9875](https://github.com/risingwavelabs/risingwave/pull/9875) ## Cluster configuration changes * `--data_directory`and `--state_store`must be specified on CLI of the Meta Node, or the cluster will fail to start. [#9170](https://github.com/risingwavelabs/risingwave/pull/9170) * Clusters will refuse to start if the specified object store URL identified by `state_store` and `data_directory` is occupied by another instance. Do not share the object store URL between multiple clusters. [#9642](https://github.com/risingwavelabs/risingwave/pull/9642) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.19.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.19.0/risingwave-v0.19.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.19.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.19.0.tar.gz) Starting from this version, we’ll respect semantic versioning conventions by using the middle number (`y` , instead of `z,` in `x.y.z`) to indicate minor versions. That is why this is `v0.18.0`, not `v0.1.18`. ## Administration and troubleshooting * Improves error messages by including the location of the statement in question. [#8646](https://github.com/risingwavelabs/risingwave/pull/8646) * Initial values of immutable system parameters can be specified via the meta-node command line. The initial values provided in the configuration file will be ignored. [#8366](https://github.com/risingwavelabs/risingwave/pull/8366) ## SQL features * Adds initial support for user-defined functions. [#8597](https://github.com/risingwavelabs/risingwave/pull/8597) [#8644](https://github.com/risingwavelabs/risingwave/pull/8644) [#8255](https://github.com/risingwavelabs/risingwave/pull/8255) [#7943](https://github.com/risingwavelabs/risingwave/pull/7943) * Adds support for JSONB data type. [#8256](https://github.com/risingwavelabs/risingwave/pull/8256) [#8181](https://github.com/risingwavelabs/risingwave/pull/8181) * Adds support for `NULLS { FIRST | LAST }` in `ORDER BY` clauses. [#8485](https://github.com/risingwavelabs/risingwave/pull/8485) * New commands: * `ALTER SOURCE RENAME TO` [#8778](https://github.com/risingwavelabs/risingwave/pull/8778) * `SET TIME ZONE` [#8572](https://github.com/risingwavelabs/risingwave/pull/8572) * `ALTER RELATION RENAME` [#7745](https://github.com/risingwavelabs/risingwave/pull/7745) * `ALTER TABLE ADD/DROP COLUMN` for regular tables (without connector settings). [#8394](https://github.com/risingwavelabs/risingwave/pull/8394) * New functions: * `array_length` : Returns the length of an array. [#8636](https://github.com/risingwavelabs/risingwave/pull/8636) * String functions implemented with the help of ChatGPT. [#8767](https://github.com/risingwavelabs/risingwave/pull/8767) [#8839](https://github.com/risingwavelabs/risingwave/pull/8839) * `chr(integer)` -> `varchar` * `starts_with(varchar, varchar)` -> `boolean` * `initcap(varchar)` -> `varchar` * `lpad(varchar, integer)` -> `varchar` * `lpad(varchar, integer, varchar)` -> `varchar` * `rpad(varchar, integer)` -> varchar * `rpad(varchar, integer, varchar)` -> `varchar` * `reverse(varchar)` -> `varchar` * `strpos(varchar, varchar)` -> `integer` * `to_ascii(varchar)` -> `varchar` * `to_hex(integer)` -> `varchar` * `to_hex(bigint)` -> `varchar`) * Improves the data type values of columns returned by `DESCRIBE` . [#8819](https://github.com/risingwavelabs/risingwave/pull/8819) * `UPDATE` commands cannot update primary key columns. [#8569](https://github.com/risingwavelabs/risingwave/pull/8569) * Adds support for microsecond precision for intervals. [#8501](https://github.com/risingwavelabs/risingwave/pull/8501) * Adds an optional parameter `offset` to `tumble()` and `hop()` functions. [#8490](https://github.com/risingwavelabs/risingwave/pull/8490) * Data records that has null time values will be ignored by time window functions. [#8146](https://github.com/risingwavelabs/risingwave/pull/8146) * Improves the behaviors of the `exp` operator when the operand is too large or small. [#8309](https://github.com/risingwavelabs/risingwave/pull/8309) * Supports process time temporal join, which enables the joining of an append-only stream (such as Kafka) with a temporal table (e.g. a materialized view backed by MySQL CDC). This feature ensures that any updates made to the temporal table will not affect previous results obtained from the temporal join. Supports `FOR SYSTEM_TIME AS OF NOW()` syntax to express process time temporal join. [#8480](https://github.com/risingwavelabs/risingwave/pull/8480) ## Connectors * Adds a new field `basetime` to the load generator connector for generating timestamp data. The load generator will take this field as `now` and generates data accordingly. [#8619](https://github.com/risingwavelabs/risingwave/pull/8619) * Empty cells in CSV are now parsed as null. [#8709](https://github.com/risingwavelabs/risingwave/pull/8709) * Adds the Iceberg connector. [#8508](https://github.com/risingwavelabs/risingwave/pull/8508) * Adds support for the upsert type to the Kafka sink connector. [#8168](https://github.com/risingwavelabs/risingwave/pull/8168) * Removes the message name parameter for Avro data. [#8124](https://github.com/risingwavelabs/risingwave/pull/8124) * Adds support for AWS PrivateLink for Kafka source connector. [#8247](https://github.com/risingwavelabs/risingwave/pull/8247) See the **Full Changelog** [here](https://github.com/risingwavelabs/risingwave/compare/v0.1.17...v0.18.0). ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.18.0 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.18.0/risingwave-v0.18.0-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.18.0.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.18.0.tar.gz) ## Administration * Adds a system catalog view `rw_catalog.rw_ddl_progress`, with which users can view the progress of a `CREATE INDEX`, `CREATE SINK`, or `CREATE MATERIALIZED VIEW` statement. [#7914](https://github.com/risingwavelabs/risingwave/pull/7914) * Adds the `pg_conversion` and `pg_enum` system catalogs. [#7964](https://github.com/risingwavelabs/risingwave/pull/7964), [#7706](https://github.com/risingwavelabs/risingwave/pull/7706) ## SQL features * Adds the `exp()` function. [#7971](https://github.com/risingwavelabs/risingwave/pull/7971) * Adds the `pow()` function. [#7789](https://github.com/risingwavelabs/risingwave/pull/7789) * Adds support for displaying primary keys in `EXPLAIN` statements. [#7590](https://github.com/risingwavelabs/risingwave/pull/7590) * Adds support for descending order in `CREATE INDEX` statements. [#7822](https://github.com/risingwavelabs/risingwave/pull/7822) * Adds `SHOW PARAMETERS` and `ALTER SYSTEM` commands to display and update system parameters. [#7882](https://github.com/risingwavelabs/risingwave/pull/7882), [#7913](https://github.com/risingwavelabs/risingwave/pull/7913) ## Connectors * Adds a new parameter `match_pattern` to the S3 connector. With the new parameter, users can specify the pattern to filter files that they want to ingest from S3 buckets. For documentation updates, see [Ingest data from S3 buckets](/integrations/sources/s3). [#7565](https://github.com/risingwavelabs/risingwave/pull/7565) * Adds the PostgreSQL CDC connector. Users can use this connector to ingest data and CDC events from PostgreSQL directly. For documentation updates, see [Ingest data from PostgreSQL CDC](/integrations/sources/postgresql-cdc). [#6869](https://github.com/risingwavelabs/risingwave/pull/6869), [#7133](https://github.com/risingwavelabs/risingwave/pull/7133) * Adds the MySQL CDC connector. Users can use this connector to ingest data and CDC events from MySQL directly. For documentation updates, see [Ingest data from MySQL CDC](/ingestion/sources/mysql/mysql-cdc). [#6689](https://github.com/risingwavelabs/risingwave/pull/6689), [#6345](https://github.com/risingwavelabs/risingwave/pull/6345), [#6481](https://github.com/risingwavelabs/risingwave/pull/6481), [#7133](https://github.com/risingwavelabs/risingwave/pull/7133) * Adds the JDBC sink connector, with which users can sink data to MySQL, PostgreSQL, or other databases that are compliant with JDBC. [#6493](https://github.com/risingwavelabs/risingwave/pull/6493) * Add new parameters to the Kafka sink connector. * `force_append_only` : Specifies whether to force a sink to be append-only. [#7922](https://github.com/risingwavelabs/risingwave/pull/7922) * `use_transaction` : Specifies whether to enable Kafka transactions or not. [#7500](https://github.com/risingwavelabs/risingwave/pull/7500) * SSL/SASL parameters: Specifies SSL encryption and SASL authentication settings. [#7540](https://github.com/risingwavelabs/risingwave/pull/7540) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.17 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.17/risingwave-v0.1.17-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.17.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.17.tar.gz) ## Administration * Adds support for aborting a query in local mode with `Ctrl + C`. [#7444](https://github.com/risingwavelabs/risingwave/pull/7444) ## SQL features * Adds support for the `to_timestamp` function. [#7060](https://github.com/risingwavelabs/risingwave/pull/7060) * Adds support for the `RETURNING` clause in DML statements. [#7094](https://github.com/risingwavelabs/risingwave/pull/7094) * Breaking change: Deprecates `CREATE MATERIALIZED SOURCE` . To create a materialized source, create a table and include the newly added connector settings. [#7281](https://github.com/risingwavelabs/risingwave/pull/7281), [#7110](https://github.com/risingwavelabs/risingwave/pull/7110) * Adds support for the `c` and `i` flags in `regex_match()` and `regex_matches()` functions. [#7135](https://github.com/risingwavelabs/risingwave/pull/7135) * Adds support for `SHOW CREATE TABLE` . You can use this statement to show the definition of a table. [#7152](https://github.com/risingwavelabs/risingwave/pull/7152) * Adds support for the `pg_stat_activity` system catalog and several system functions. [#7274](https://github.com/risingwavelabs/risingwave/pull/7274) * Adds the `_rw_kafka_timestamp` parameter to show the timestamps of Kafka messages. Users can now specify the scope of Kafka messages by timestamps. [#7275](https://github.com/risingwavelabs/risingwave/pull/7275), [#7150](https://github.com/risingwavelabs/risingwave/pull/7150) * Adds support for displaying PostgreSQL and RisingWave versions in `version()`. [#7314](https://github.com/risingwavelabs/risingwave/pull/7314) * Adds support for displaying internal tables using the `SHOW INTERNAL TABLES` statement. [#7348](https://github.com/risingwavelabs/risingwave/pull/7348) * Adds support for `SET VISIBILITY_MODE` You can use this session variable to configure whether only checkpoint data is readable for batch query. [#5850](https://github.com/risingwavelabs/risingwave/pull/5850) * Adds support for `SET STREAMING_PARALLELISM` . You can use this session variable to configure parallelism for streaming queries. [#7370](https://github.com/risingwavelabs/risingwave/pull/7370) ## Connectors * Adds support for generating array and struct data using the datagen connector. [#7099](https://github.com/risingwavelabs/risingwave/pull/7099) * Adds the S3 source connector, with which users can ingest data in CSV format from S3 locations. For data ingestion from files, CSV is the only supported format and the files must be placed on S3. [#6846](https://github.com/risingwavelabs/risingwave/pull/6846) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.16 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.16/risingwave-v0.1.16-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.16.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.16.tar.gz) ## Installation and deployment * Parallelism and available memory of Compute Nodes are now command-line arguments and removed from the configuration file. [#6767](https://github.com/risingwavelabs/risingwave/pull/6767) * The default barrier interval is set to 1 second. [#6553](https://github.com/risingwavelabs/risingwave/pull/6553) * Adds support for meta store backup and recovery. [#6737](https://github.com/risingwavelabs/risingwave/pull/6737) ## SQL features * Adds support for `SHOW CREATE MATERIALIZED VIEW` and `SHOW CREATE VIEW` to show how materialized and non-materialized views are defined. [#6921](https://github.com/risingwavelabs/risingwave/pull/6921) * Adds support for `CREATE TABLE IF NOT EXISTS`. [#6643](https://github.com/risingwavelabs/risingwave/pull/6643) * A sink can be created from a SELECT query. [#6648](https://github.com/risingwavelabs/risingwave/pull/6648) * Adds support for struct casting and comparison. [#6552](https://github.com/risingwavelabs/risingwave/pull/6552) * Adds pg\_catalog views and system functions. [#6982](https://github.com/risingwavelabs/risingwave/pull/6982) * Adds support for `CREATE TABLE AS`. [#6798](https://github.com/risingwavelabs/risingwave/pull/6798) * Ads the initial support for batch query on Kafka source. [#6474](https://github.com/risingwavelabs/risingwave/pull/6474) * Adds support for `SET QUERY_EPOCH` to query historical data based on meta backup. [#6840](https://github.com/risingwavelabs/risingwave/pull/6840) ## Connectors * Improves the handling of schema errors for Avro and Protobuf data. [#6821](https://github.com/risingwavelabs/risingwave/pull/6821) * Adds two options to the datagen connector to make it possible to generate increasing timestamp values. [#6591](https://github.com/risingwavelabs/risingwave/pull/6591) ## Observability * Adds metrics for the backup manager in Grafana. [#6898](https://github.com/risingwavelabs/risingwave/pull/6898) * RisingWave Dashboard can now fetch data from Prometheus and visualize it in charts. [#6602](https://github.com/risingwavelabs/risingwave/pull/6602) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.15 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.15/risingwave-v0.1.15-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.15.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.15.tar.gz) ## SQL features * `PRIMARY KEY` constraint checks can be performed on materialized sources and tables but not on non-materialized sources. For tables or materialized sources that enabled `PRIMARY KEY` constraints, if you insert data to an existing key, the new data will overwrite the old data. [#6320](https://github.com/risingwavelabs/risingwave/pull/6320) [#6435](https://github.com/risingwavelabs/risingwave/pull/6435) * Adds support for timestamp with time zone data type. You can use this data type in time window functions, and convert between it and timestamp (without time zone). [#5855](https://github.com/risingwavelabs/risingwave/pull/5855) [#5910](https://github.com/risingwavelabs/risingwave/pull/5910) [#5968](https://github.com/risingwavelabs/risingwave/pull/5968) * Adds support for `UNION` and `UNION ALL` operators. [#6363](https://github.com/risingwavelabs/risingwave/pull/6363) [#6397](https://github.com/risingwavelabs/risingwave/pull/6397) * Implements the `rank()` function to support a different mode of Top-N queries. [#6383](https://github.com/risingwavelabs/risingwave/pull/6383) * Adds support for logical views (`CREATE VIEW`). [#6023](https://github.com/risingwavelabs/risingwave/pull/6023) * Adds the `data_trunc()` function. [#6365](https://github.com/risingwavelabs/risingwave/pull/6365) * Adds the system catalog schema. [#6227](https://github.com/risingwavelabs/risingwave/pull/6227) * Displays error messages when users enter conflicting or redundant command options. [#5933](https://github.com/risingwavelabs/risingwave/pull/5933/) ## Connectors * Adds support for the Maxwell Change Data Capture (CDC) format. [#6057](https://github.com/risingwavelabs/risingwave/pull/6057) * Protobuf schema files can be loaded from Web locations in `s3://`, `http://`, or `https://` formats. [#6114](https://github.com/risingwavelabs/risingwave/pull/6114) [#5964](https://github.com/risingwavelabs/risingwave/pull/5964) * Adds support for Confluent Schema Registry for Kafka data in Avro and Protobuf formats. [#6289](https://github.com/risingwavelabs/risingwave/pull/6289) * Adds two options to the Kinesis connector. Users can specify the startup mode and optionally the sequence number to start with. [#6317](https://github.com/risingwavelabs/risingwave/pull/6317) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.14 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.14/risingwave-v0.1.14-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.14.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.14.tar.gz) ## SQL features * SQL commands: * Improves the formatting of response messages of `EXPLAIN` statements. [#5541](https://github.com/risingwavelabs/risingwave/pull/5541) * SQL functions: * `to_char()` now supports specifying output format in lowercase. [#5032](https://github.com/risingwavelabs/risingwave/pull/5032) `to_char(timestamp '2006-01-02 15:04:05', 'yyyy-mm-dd hh24:mi:ss')` → `2006-01-02 15:04:05` * `generate_series` now supports negative steps. [#5231](https://github.com/risingwavelabs/risingwave/pull/5231) ```sql theme={null} SELECT * FROM generate_series(5,1,-2); generate_series ----------------- 5 3 1 (3 rows) ``` * Adds support for sum/min/max functions over interval-type data. [#5105](https://github.com/risingwavelabs/risingwave/pull/5105), [#5549](https://github.com/risingwavelabs/risingwave/pull/5549) * Adds support for array concatenation. [#5060](https://github.com/risingwavelabs/risingwave/pull/5060), [#5345](https://github.com/risingwavelabs/risingwave/pull/5345) * Adds support for specifying empty arrays. [#5402](https://github.com/risingwavelabs/risingwave/pull/5402) * Casting from array to varchar is now supported. [#5081](https://github.com/risingwavelabs/risingwave/pull/5081) `array[1,2]::varchar` → `{1,2}` * Casting from varchar to integer allows leading and trailing spaces. [#5452](https://github.com/risingwavelabs/risingwave/pull/5452) `' 1 '::int` → `1` * Adds new system catalog and psql meta-commands. [#5127](https://github.com/risingwavelabs/risingwave/pull/5127), [#5742](https://github.com/risingwavelabs/risingwave/pull/5742) * `\d`: Lists all relations in the current database. (Materialized) sources are not supported yet. * `\dt`: Lists all tables in the current database. * `\dm`: Lists all materialized views in the current database. * `\di`: Lists all indexes in the current database. * `pg_catalog.pg_index`: Contains information about indexes. ## Connectors * Nested columns are now supported for the datagen connector. [#5550](https://github.com/risingwavelabs/risingwave/pull/5550) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 risingwavelabs/risingwave:v0.1.13 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.13/risingwave-v0.1.13-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.13.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.13.tar.gz) ## SQL features * SQL commands: * `EXPLAIN` now supports specifying options. Supported options: `trace`, `verbose`, and `type`. Unlike PostgreSQL, each option should be separated by a comma and wrapped by parentheses as a whole. [#4730](https://github.com/risingwavelabs/risingwave/pull/4730) * Adds support for `ALTER USER`. [#4261](https://github.com/risingwavelabs/risingwave/pull/4261) * `CREATE/ALTER USER` now has new options `CREATEUSER` and `NOCREATEUSER`, which specify whether or not the user has the privilege to create, alter, or drop other users. [#4447](https://github.com/risingwavelabs/risingwave/pull/4447) * Adds support for EXPLAIN CREATE SINK. [#4430](https://github.com/risingwavelabs/risingwave/pull/4430) * SQL functions: * Adds support for new system information functions: `current_schema`, `current_schema()`, and `session_user`. [#4358](https://github.com/risingwavelabs/risingwave/pull/4358) * The `pg_namespace` catalog now has a new namespace column `nspacl` for storing access privileges. [#4326](https://github.com/risingwavelabs/risingwave/pull/4326) ## Connectors * Some connector parameters were renamed. The old parameter names are still functional but may be deprecated in the future. [#4503](https://github.com/risingwavelabs/risingwave/pull/4503) * Kafka & Redpanda * `kafka.brokers` -> `properties.bootstrap.server` * `kafka.topic` -> `topic` * `kafka.scan.startup.mode` -> `scan.startup.mode` * `kafka.time.offset` -> `scan.startup.timestamp_millis` * `kafka.consumer.group` -> `consumer.group.id` * Kinesis * `kinesis.stream.name` -> `stream` * `kinesis.stream.region` -> `aws.region` * `kinesis.endpoint` -> `endpoint` * `kinesis.credentials.access` -> `aws.credentials.access_key_id` * `kinesis.credentials.secret` -> `aws.credentials.secret_access_key` * `kinesis.credentials.session_token` -> `aws.credentials.session_token` * `kinesis.assumerole.arn` -> `aws.credentials.role.arn` * `kinesis.assumerole.external_id` -> `aws.credentials.role.external_id` * Pulsar * `pulsar.topic` -> `topic` * `pulsar.admin.url` -> `admin.url` * `pulsar.service.url` -> `service.url` * `pulsar.scan.startup.mode` -> `scan.startup.mode` * `pulsar.time.offset` -> `scan.startup.timestamp_millis` * The row format name, `debezium json`, for CDC stream sources, has been renamed to `debezium_json`. [#4494](https://github.com/risingwavelabs/risingwave/pull/4494) ## Configuration changes * The default batch query execution mode was changed from distributed to local. [#4789](https://github.com/risingwavelabs/risingwave/pull/4789) ## Assets * Run this version from Docker: `docker run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/risingwavelabs/risingwave:v0.1.12 playground` * Prebuilt library for Linux is not available in this release. * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.12.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.12.tar.gz) ## SQL features * New SQL functions: * `overlay()`: Replaces a substring. [#3671](https://github.com/risingwavelabs/risingwave/pull/3671) * `generate_series()`: Generates a series of values from the starting point to the ending point. [#4030](https://github.com/risingwavelabs/risingwave/pull/4030) * `regexp_match()`, `regexp_matches()`: Compares a string against a regular expression pattern and returns matched substrings. [#3702](https://github.com/risingwavelabs/risingwave/pull/3702) [#4062](https://github.com/risingwavelabs/risingwave/pull/4062) * `string_agg ()`: Concatenates the values into a string. [#3952](https://github.com/risingwavelabs/risingwave/pull/3952) [#4183](https://github.com/risingwavelabs/risingwave/pull/4183) * `current_database()`: Returns the current database. [#3650](https://github.com/risingwavelabs/risingwave/pull/3650) * New SQL commands: * `SHOW ALL`: Lists all configuration parameters. Use `SHOW parameter` to show the value of a configuration parameter. [#3694](https://github.com/risingwavelabs/risingwave/pull/3694) [#3664](https://github.com/risingwavelabs/risingwave/pull/3664) * `CREATE SINK`: Sinks data to Kafka. [#3923](https://github.com/risingwavelabs/risingwave/pull/3923) [#3682](https://github.com/risingwavelabs/risingwave/pull/3682) [#3674](https://github.com/risingwavelabs/risingwave/pull/3674) * `EXPLAIN TRACE`: Traces each optimization stage of the optimizer. [#3945](https://github.com/risingwavelabs/risingwave/pull/3945) * Support for lookup joins. Currently, lookup joins can only be performed in local query mode. To use lookup joins, users need to set the configuration parameter `rw_batch_enable_lookup_join` to true. [#3859](https://github.com/risingwavelabs/risingwave/pull/3859) [#3763](https://github.com/risingwavelabs/risingwave/pull/3763) * An `ORDER BY` clause in the `CREATE MATERIALIZED VIEW` statement is allowed but not considered as part of the definition of the materialized view. It is only used in the initial creation of the materialized view. It is not used during refreshes. This is a behavior change due to the introduction of parallel table scans. [#3670](https://github.com/risingwavelabs/risingwave/pull/3670) * Support for filter clauses on aggregate functions. [#4114](https://github.com/risingwavelabs/risingwave/pull/4114) ## Connectors * RisingWave can now sink data to Kafka topics in append-only mode and Debezium mode. [#3923](https://github.com/risingwavelabs/risingwave/pull/3923) [#3682](https://github.com/risingwavelabs/risingwave/pull/3682) [#3674](https://github.com/risingwavelabs/risingwave/pull/3674) * Syntax change for `CREATE SOURCE`: A parameter name is no longer wrapped by single quotation marks. [#3997](https://github.com/risingwavelabs/risingwave/pull/3997). See the example: * Old: `CREATE SOURCE s1 WITH ( 'connector' = 'kafka', 'kafka.topic' = 'kafka_1_partition_topic', 'kafka.brokers' = '127.0.0.1:29092' ) ROW FORMAT json;` * New: `CREATE SOURCE s WITH ( connector = 'kafka', kafka.topic = 'kafka_1_partition_topic', kafka.brokers = '127.0.0.1:29092' ) ROW FORMAT json;` ## Assets * Run this version from Docker: `run -it --pull=always -p 4566:4566 -p 5691:5691 ghcr.io/risingwavelabs/risingwave:v0.1.11 playground` * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.11/risingwave-v0.1.11-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.11.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.11.tar.gz) ## SQL features ### SQL operators and functions * Support string concatenation operator `||`. [#3147](https://github.com/risingwavelabs/risingwave/pull/3147) * Support interval comparison. [#3222](https://github.com/risingwavelabs/risingwave/pull/3222) * Support dividing intervals by integers, floats, or decimals. [#3441](https://github.com/risingwavelabs/risingwave/pull/3441) * Intervals multiplying intervals by floats. [#3641](https://github.com/risingwavelabs/risingwave/pull/3641) * Support more temporal operations. [#3472](https://github.com/risingwavelabs/risingwave/pull/3472) * Support Common Table Expressions (CTEs) as input of time window functions. [#3188](https://github.com/risingwavelabs/risingwave/pull/3188) * Add these new functions: * `concat()` for concatenating strings [#3091](https://github.com/risingwavelabs/risingwave/pull/3091) * `repeat()` for repeating string [#3148](https://github.com/risingwavelabs/risingwave/pull/3148) * `octet_length()` and bit\_length() for getting string length [#3526](https://github.com/risingwavelabs/risingwave/pull/3526) * `Row()` for constructing rows [#2914](https://github.com/risingwavelabs/risingwave/pull/2914) [#3152](https://github.com/risingwavelabs/risingwave/pull/3152) * `pg_typeof()` for getting data types of values [#3494](https://github.com/risingwavelabs/risingwave/pull/3494) * `current_database()` for getting the name of the current database in the session [#3650](https://github.com/risingwavelabs/risingwave/pull/3650) * `approx_count_distinct()` for distinct counting [#3121](https://github.com/risingwavelabs/risingwave/pull/3121) * `unnest()` for expanding nested tables to rows [#3017](https://github.com/risingwavelabs/risingwave/pull/3017) * Support `count()`, `min()`, and `max()` functions on these data types: *interval*, *timestamp*, *varchar*, and *date*. [#3069](https://github.com/risingwavelabs/risingwave/pull/3069) ### SQL commands * Support `EXPLAIN CREATE INDEX`. [#3229](https://github.com/risingwavelabs/risingwave/pull/3229) * Add cascade and restrict options in `REVOKE` commands. [#3363](https://github.com/risingwavelabs/risingwave/pull/3363) * Expand the `CREATE TABLE` syntax to support creating append-only tables. [#3058](https://github.com/risingwavelabs/risingwave/pull/3058) * Support the `CREATE USER` command and user authentication. [#3074](https://github.com/risingwavelabs/risingwave/pull/3074) ### Data types * Support implicit casts from single-quoted literals. [#3487](https://github.com/risingwavelabs/risingwave/pull/3487) * Add string as an alias for data type varchar. [#3094](https://github.com/risingwavelabs/risingwave/pull/3094) * Support string intervals. [#3037](https://github.com/risingwavelabs/risingwave/pull/3037) ### Database management * Add the default super user “postgres”. [#3127](https://github.com/risingwavelabs/risingwave/pull/3127) * The default schema name is changed to “public” from “dev”. [#3166](https://github.com/risingwavelabs/risingwave/pull/3166) ## Connectors * Add random seed for the Datagen Source Connector. [#3124](https://github.com/risingwavelabs/risingwave/pull/3124) ## Assets * [Prebuilt library for Linux](https://github.com/risingwavelabs/risingwave/releases/download/v0.1.10/risingwave-v0.1.10-x86_64-unknown-linux.tar.gz) * [Source code (zip)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.10.zip) * [Source code (tar.gz)](https://github.com/risingwavelabs/risingwave/archive/refs/tags/v0.1.10.tar.gz)