Skip to main content
Amazon S3 is an object storage service that offers industry-leading scalability, data availability, security, and performance.

​
Syntax

CREATE SINK [ IF NOT EXISTS ] sink_name
[FROM sink_from | AS select_query]
WITH (
   connector='s3',
   connector_parameter = 'value', ...
);

​
Parameters

Parameter namesDescription
connectorRequired. Support the S3 connector only.
s3.region_nameRequired. The service region.
s3.bucket_nameRequired. The name of the bucket where the sink data is stored in.
s3.pathRequired. The directory where the sink file is located.
s3.credentials.accessConditional. Your AWS access key ID. Required if enable_config_load is false or not set.
s3.credentials.secretConditional. Your AWS secret access key. Required if enable_config_load is false or not set.
enable_config_loadOptional. Set to true to use the default AWS credential provider chain for authentication. This is useful for environments where credentials are provided through IAM roles or environment variables. If this is set to true, do not provide s3.credentials.access and s3.credentials.secret. You must also set the DISABLE_DEFAULT_CREDENTIAL=false environment variable on the Meta and Streaming Nodes.
s3.endpoint_urlOptional. The host URL for an S3-compatible object storage server. This allows users to use a different server instead of the standard S3 server.
s3.assume_roleOptional. Specifies the ARN of an IAM role to assume when accessing S3. It allows temporary, secure access to S3 resources without sharing long-term credentials.
typeRequired. Defines the type of the sink. The S3 sink only supports append-only

​
Example

CREATE SINK s3_sink AS SELECT v1
FROM t 
WITH (
    connector='s3',
    s3.path = '<test_path>',
    s3.region_name = '<region_name>',
    s3.bucket_name = '<bucket_name>',
    s3.credentials.access = '<access_key_id>',
    s3.credentials.secret = '<secret_access_key>',
    s3.endpoint_url = '<endpoint_url>',
    type = 'append-only',
)FORMAT PLAIN ENCODE PARQUET(force_append_only=true);

​
Advanced topics

For more information about encode Parquet or JSON, see Sink data in parquet or json format. For more information about batching strategy, see Batching strategy for file sink.
⌘I