In RisingWave, certain configurations are node-specific and can vary between different nodes. These configurations are stored in a TOML file, which is read at system startup.
risingwave.toml
configuration file. Here’s the steps on how to set them up:
risingwave.toml
file. This file will contain all your node-specific configurations. If it doesn’t exist, create a new one.
risingwave.toml
file. Open the file in a text editor. Each configuration item should be specified in the format config_name = value
. For example:
risingwave.toml
file.
--config-path
command-line argument when starting the node. For example:
RW_CONFIG_PATH
environment variable to the path of your risingwave.toml
file.
For example, in a Kubernetes environment, you can copy the configuration file into the Docker container, or mount a path containing the configuration file into the pod. Then, specify the path to the configuration file using the RW_CONFIG_PATH
environment variable or the --config-path
command-line argument.
risingwave.toml
will override the default values in the source code. If no configuration file is specified, the default values in /risingwave/src/common/src/config.rs
will be used.
For more details about the parameters in the configuration file, see RisingWave configuration files directory. There you’ll find information like the definitions and default values of these parameters.
unsafe_
. Typically these configurations can cause system or data damage if wrongly configured. You may want to contact our technical support before changing the unsafe_
prefixed configurations.
ALTER SYSTEM SET
command.
Example for the system configuration section:
[streaming]
section in the configuration file. For example:
Configuration | Default | Description |
---|---|---|
unsafe_enable_strict_consistency | true | Control the strictness of stream consistency. When set to false, data inconsistency like double-insertion or double-deletion with the same primary keys will be tolerated. |
[storage]
and [storage.xxx]
sections.
What is the file cache and block cache refilling?
When to use the file cache and the block cache refilling?
[storage.data_file_cache]
[storage.meta_file_cache]
[storage.cache_refill]
Configuration | Default | Description |
---|---|---|
dir | "" | The directory for the file cache. If left empty, the file cache will be disabled. |
capacity_mb | 1024 | The file cache capacity in MB. |
file_capacity_mb | 64 | The capacity for each cache file in MB. |
flushers | 4 | Worker count for concurrently writing cache files. |
reclaimers | 4 | Worker count for concurrently reclaiming cache files. |
recover_concurrency | 8 | Worker count for restoring cache when opening. |
insert_rate_limit_mb | 0 | File cache insertion rate limit in MB/s. This option is important as disk bandwidth is usually lower than memory. |
indexer_shards | 64 | The shard number of the indexer. |
compression | ”none” | Compression algorithm for cached data. Supports none, lz4, and zstd. |
Configuration | Default | Description |
---|---|---|
data_refill_levels | [] | Only blocks in the given levels will be refilled. |
timeout_ms | 6000 | The metadata update will be delayed at most timeout_ms to wait for refilling. |
concurrency | 10 | Block refilling concurrency (by unit level). |
unit | 64 | The length of continuous data blocks that can be batched and refilled in one request. |
threshold | 0.5 | Only units whose recently used block ratio exceeds the threshold will be refilled. |
recent_filter_layers | 6 | Number of layers in the recent filter. |
recent_filter_rotate_interval_ms | 10000 | Time interval for rotating recent filter layers. |
[udf]
section in the configuration file. For example:
enable_embedded_python_udf
to true
in the configuration file.