This is necessary in order to be able to consume messages from a Kafka service located in a different VPC from the RisingWave cluster in the cloud.

​
Syntax

CREATE CONNECTION [ IF NOT EXISTS ] connection_name
WITH (
    connection_parameter = 'value'
);

​
Parameters

All WITH options are required unless stated otherwise.

Parameter or clauseDescription
connection_nameThe name of the connection to be created.
typeThe type of connection.
providerThe provider of the connection.
service.nameThe service name of the endpoint service.
tagsOptional. The AWS tags used to check for resource leakage. This parameter should have the format: key1=value1, key2=value2, …

NOTE

You can either tag the VPC endpoints by specifying the tags parameter when using the CREATE CONNECTION command or by specifying the environment variable RW_PRIVATELINK_ENDPOINT_DEFAULT_TAGS. When specifying the tags, follow the format of key1=value1, key2=value2, .... If both are specified, the tags specified in the environment variable will be appended to the ones specified by the tags parameter.

​
Example

The statement below creates an AWS PrivateLink connection.

CREATE CONNECTION connection_name WITH (
    type = 'privatelink',
    provider = 'aws',
    service.name = 'com.amazonaws.xyz.us-east-1.abc-xyz-0000'
);

If you are using a cloud-hosted source or sink, such as AWS MSK, there might be connectivity issues when your service is located in a different VPC from where you have deployed RisingWave. To establish a secure, direct connection between these two different VPCs and allow RisingWave to read consumer messages from the broker or send messages to the broker, use the AWS PrivateLink service.

Follow the steps below to create an AWS PrivateLink connection.

  1. Create a target group for each broker. Set the target type as IP addresses and the protocol as TCP. Ensure that the VPC of the target group is the same as your cloud-hosted source.
  2. Create a Network Load Balancer. Ensure that it is enabled in the same subnets your broker sources are in and the Cross-zone load balancing is also enabled.
  3. Create a TCP listener for each MSK broker that corresponds to the target groups created. Ensure the ports are unique.
  4. Complete the health check for each target group.
  5. Create a VPC endpoint service associated with the Network Load Balancer created. Be sure to add the AWS principal of the account that will access the endpoint service to allow the service consumer to connect. See Manage permissions for more details.
  6. Use the CREATE CONNECTION command in RisingWave to create an AWS PrivateLink connection referencing the endpoint service created. Here is an example of creating an AWS PrivateLink connection.
CREATE CONNECTION connection_name WITH (
    type = 'privatelink',
    provider = 'aws',
    service.name = 'com.amazonaws.xyz.us-east-1.abc-xyz-0000'
);
  1. Create a source or sink with AWS PrivateLink connection.

Was this page helpful?

CREATE AGGREGATECREATE DATABASE