Extract metadata from sources during ingestion.
INCLUDE
clause to extract fields not included in the main message payload (the data defined by your schema) as separate columns. This is useful for accessing metadata like Kafka message offsets, timestamps, or keys.
{ header | key | offset | partition | timestamp | payload | subject | file | database_name | collection_name }
: The field you want to include. The available fields depend on the connector (see Supported Connectors below).
[AS column_name]
: (Optional) The name you want to give to the new column in RisingWave. If you omit AS column_name, RisingWave uses a default name in the format rw_, where:
{connector}
is the connector name (e.g., kafka, pulsar).{col}
is the type of column (e.g., key, offset, timestamp).List[Struct<Varchar, Bytea>]
.
header_col
field can only be defined when including a header.
In this case, the generated column name will have the format _rw_kafka_header_{header col name}_{col type}
, where col type
is the data type of the header column.