This topic introduces the concept and the tuning method of reserved memory and cache eviction policy.
Details
--reserved-memory-bytes
or the environment variable RW_RESERVED_MEMORY_BYTES
to override the reserved memory configuration for compute nodes. But note that the memory reserved should be at least 512MB.
For instance, suppose you are deploying a compute node on a machine or pod with 64GB of memory. By default, the reserved memory would be calculated as follows:
RW_RESERVED_MEMORY_BYTES=8589934592
as an environment variable or --reserved-memory-bytes=8589934592
as a command line parameter when starting up the compute node. This will allocate 8GB as the reserved memory instead.
Reserved memory setting before 1.10
memory_controller_eviction_factor_XXX
variables in the configuration toml file.
Variables in the configuration toml file
memory_controller_threshold_stable
, RisingWave begins to evict data from the operator cache with mild intensity.memory_controller_threshold_graceful
triggers a more aggressive eviction strategy.memory_controller_threshold_aggressive
, the eviction process becomes most aggressive to free up memory.toml
configuration file.
For more detailed information on how the policy adjusts the intensity of eviction, you can refer to the source code directly.
memory_controller_threshold_stable
is 0.8. RisingWave will aim to use 56% of the total memory stably during normal operations, calculated as: