Skip to main content

Generate test data

The built-in load generator can generate mock data, which can be used in demos and tests. It provides an easy way to simulate the data stream without connecting to an actual external data source.

Use the SQL statement below to connect RisingWave to the built-in load generator.

Syntax

CREATE TABLE source_name ( column_name data_type, ... )
WITH (
connector = ' datagen ',
fields.column_name.column_parameter = ' value ', ... -- Configure the generator for each column. See detailed information below.
datagen.rows.per.second = ' rows_integer ' -- Specify how many rows of records to generate every second. For example, '20'.
) FORMAT PLAIN ENCODE JSON;

WITH options - column_parameter

The following table shows the data types that can be generated for each load generator type.

Generator \ DataNumberTimestampTimestamptzVarcharStructArray
Sequence
Random

Select the type of data to be generated.

The sequence load generator can generate numbers, incremented by 1, from the starting number to the ending number. For example, 1, 2, 3, ... and 1.56, 2.56, 3.56, ...

Specify the following fields for every column.

column_parameterDescriptionValueRequired?
kindGenerator typeSet to sequence.False
Default: random
startStarting number
Must be smaller than the ending number.
Any number of the column data type
Example: 50
False
Default: 0
endEnding number
Must be larger than the starting number.
Any number of the column data type
Example: 100
False
Default: 32767

Example

Here is an example of connecting RisingWave to the built-in load generator.

The following statement creates a source s1 with five columns:

  • i1 — An array of three integers starting from 1 and incrementing by 1
  • v1 — Structs that contain random integers v2 ranging from -10 to 10 and random floating-point numbers v3 ranging from 15 to 55
  • t1 — Random timestamps from as early as 2 hours as 37 minutes prior to the generator execution time
  • z1 - Random timestamps with timezones from as early as 2 hours as 37 minutes prior to the generator execution time
  • c1 — Random strings with each consists of 16 characters
CREATE TABLE s1 (i1 int [], v1 struct<v2 int, v3 double>, t1 timestamp, z1 timestamptz, c1 varchar)
WITH (
connector = 'datagen',

fields.i1.length = '3',
fields.i1._.kind = 'sequence',
fields.i1._.start = '1',

fields.v1.v2.kind = 'random',
fields.v1.v2.min = '-10',
fields.v1.v2.max = '10',
fields.v1.v2.seed = '1',

fields.v1.v3.kind = 'random',
fields.v1.v3.min = '15',
fields.v1.v3.max = '55',
fields.v1.v3.seed = '1',

fields.t1.kind = 'random',
fields.t1.max_past = '2h 37min',
fields.t1.max_past_mode = 'relative',
fields.t1.seed = '3',

fields.z1.kind = 'random',
fields.z1.max_past = '2h 37min',
fields.z1.max_past_mode = 'relative',
fields.z1.seed = '3',

fields.c1.kind = 'random',
fields.c1.length = '16',
fields.c1.seed = '3',

datagen.rows.per.second = '10'
) FORMAT PLAIN ENCODE JSON;

Let's query s1 after a few seconds.

SELECT * FROM s1 ORDER BY i1 LIMIT 20;
     i1     |            v1            |             t1             |                z1                |        c1
------------+--------------------------+----------------------------+----------------------------------+------------------
{1,2,3} | (7,53.96978949033611) | 2023-11-28 13:35:04.967040 | 2023-11-28 21:35:04.967330+00:00 | pGWJLsbmPJZZWpBe
{4,5,6} | (5,44.24453663454818) | 2023-11-28 14:13:15.264457 | 2023-11-28 22:13:15.264481+00:00 | FT7BRdifYMrRgIyI
{7,8,9} | (3,18.808367835800485) | 2023-11-28 15:12:41.918935 | 2023-11-28 23:12:41.919590+00:00 | 0zsMbNLxQh9yYtHh
{10,11,12} | (-4,26.893033246334525) | 2023-11-28 14:55:43.193883 | 2023-11-28 22:55:43.193917+00:00 | zujxzBql3QHxENyy
{13,14,15} | (-3,28.68505963291612) | 2023-11-28 13:35:05.967253 | 2023-11-28 21:35:05.967520+00:00 | aBJTDJpinRv8mLvQ
{16,17,18} | (4,36.32012760913261) | 2023-11-28 14:13:16.264624 | 2023-11-28 22:13:16.264646+00:00 | HVur4zU3hQFgVh74
{19,20,21} | (-10,16.212694434604053) | 2023-11-28 15:12:42.919339 | 2023-11-28 23:12:42.919465+00:00 | LVLAhd1pQvhXVL8p
{22,23,24} | (-8,28.388082274426225) | 2023-11-28 14:55:44.193726 | 2023-11-28 22:55:44.193787+00:00 | siFqrkdlCnNZqAUT
{25,26,27} | (2,40.86763449564268) | 2023-11-28 15:19:51.600898 | 2023-11-28 23:19:51.600977+00:00 | ORjwy3oMNbl1Yi6X
{28,29,30} | (3,29.179236922708526) | 2023-11-28 15:27:49.755084 | 2023-11-28 23:27:49.755105+00:00 | YIVLnWxHyfsiPHQo
{31,32,33} | (6,26.03842827701958) | 2023-11-28 16:07:02.012019 | 2023-11-29 00:07:02.012133+00:00 | lpzCxwpoJp9njIAa
{34,35,36} | (-2,20.2351457847852) | 2023-11-28 14:23:37.167393 | 2023-11-28 22:23:37.167453+00:00 | oW8xmndvmXMRp1Rc
{37,38,39} | (2,36.51138960926262) | 2023-11-28 15:19:52.600699 | 2023-11-28 23:19:52.600741+00:00 | 0m1Qxn96Xeq42H3Z
{40,41,42} | (0,34.2997487580596) | 2023-11-28 15:27:50.754878 | 2023-11-28 23:27:50.754899+00:00 | 1jT3TnEEj56YNa7w
{43,44,45} | (7,39.13577932700749) | 2023-11-28 16:07:03.011837 | 2023-11-29 00:07:03.011905+00:00 | linRToOjph0WlJrd
{46,47,48} | (7,37.43674199879566) | 2023-11-28 14:23:38.167161 | 2023-11-28 22:23:38.167271+00:00 | beql98l3IIkjomTl
{49,50,51} | (1,41.62099792202798) | 2023-11-28 15:24:46.803776 | 2023-11-28 23:24:46.803844+00:00 | xHbIYlJismRlIKFw
{52,53,54} | (9,49.240259695092895) | 2023-11-28 15:39:22.752067 | 2023-11-28 23:39:22.752115+00:00 | TDYNZsSNYMpOpZgC
{55,56,57} | (6,54.64984398127376) | 2023-11-28 13:32:15.049957 | 2023-11-28 21:32:15.050089+00:00 | jqPQM3oyA2lOXLcn
{58,59,60} | (-4,54.197350082045176) | 2023-11-28 14:07:53.278392 | 2023-11-28 22:07:53.278457+00:00 | 72cIOHPb7DE8FTme
(20 rows)
...

Help us make this doc better!