Google Cloud Storage is a RESTful online file storage web service for storing and accessing data on Google Cloud Platform infrastructure.

​
Syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector='gcs',
   connector_parameter = 'value', ...
);

​
Parameters

Parameter namesDescription
connectorRequired. Support the GCS connector only.
gcs.bucket_nameRequired. The name of the bucket where the sink data is stored in.
gcs.credentialRequired. Base64-encoded credential key obtained from the GCS service account key JSON file. To get this JSON file, refer to the guides of GCS documentation.
  • To encode it in base64, run the following command: cat ~/Downloads/rwc-byoc-test-464bdd851bce.json | base64 -b 0 | pbcopy, and then paste the output as the value for this parameter.
  • If this field is not specified, ADC (application default credentials) will be used.
gcs.service_accountOptional. The service account of the GCS sink. If gcs.credential or ADC is not specified, the credentials will be derived from the service account.
gcs.pathRequired. The directory where the sink file is located.
typeRequired. Defines the type of the sink. The GCS sink only supports append-only.

​
Example

CREATE SINK gcs_sink AS SELECT v1
FROM t1
WITH (
    connector='gcs',
    gcs.path = '<test_path>',
    gcs.bucket_name = '<bucket_name>',
    gcs.credential = '<account_name>',
    gcs.service_account = '<service_account>'
    type = 'append-only',
)FORMAT PLAIN ENCODE PARQUET(force_append_only=true);

​
Advanced topics

For more information about encode Parquet or JSON, see Sink data in parquet or json format.

For more information about batching strategy, see Batching strategy for file sink.

Google Cloud Storage is a RESTful online file storage web service for storing and accessing data on Google Cloud Platform infrastructure.

​
Syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector='gcs',
   connector_parameter = 'value', ...
);

​
Parameters

Parameter namesDescription
connectorRequired. Support the GCS connector only.
gcs.bucket_nameRequired. The name of the bucket where the sink data is stored in.
gcs.credentialRequired. Base64-encoded credential key obtained from the GCS service account key JSON file. To get this JSON file, refer to the guides of GCS documentation.
  • To encode it in base64, run the following command: cat ~/Downloads/rwc-byoc-test-464bdd851bce.json | base64 -b 0 | pbcopy, and then paste the output as the value for this parameter.
  • If this field is not specified, ADC (application default credentials) will be used.
gcs.service_accountOptional. The service account of the GCS sink. If gcs.credential or ADC is not specified, the credentials will be derived from the service account.
gcs.pathRequired. The directory where the sink file is located.
typeRequired. Defines the type of the sink. The GCS sink only supports append-only.

​
Example

CREATE SINK gcs_sink AS SELECT v1
FROM t1
WITH (
    connector='gcs',
    gcs.path = '<test_path>',
    gcs.bucket_name = '<bucket_name>',
    gcs.credential = '<account_name>',
    gcs.service_account = '<service_account>'
    type = 'append-only',
)FORMAT PLAIN ENCODE PARQUET(force_append_only=true);

​
Advanced topics

For more information about encode Parquet or JSON, see Sink data in parquet or json format.

For more information about batching strategy, see Batching strategy for file sink.