ArrowFlight table engine

​

Creating a Table

CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...)
    ENGINE = ArrowFlight('host:port', 'dataset_name' [, 'username', 'password']);

• host:port — Address of the remote Arrow Flight server. If the port is omitted, the default port 8815 is used. String.
• dataset_name — Identifier of the dataset on the Flight server (used as a PATH descriptor or in a SELECT * query depending on the arrow_flight_request_descriptor_type setting). String.
• username — Username for basic HTTP authentication. String.
• password — Password for basic HTTP authentication. String.

​

Named Collections

CREATE TABLE remote_flight_data
    ENGINE = ArrowFlight(named_collection_name);

​

Settings

• arrow_flight_request_descriptor_type — Controls how the dataset name is sent to the Flight server. Possible values: path (default, sends as a PATH descriptor) or command (sends as a CMD descriptor with SELECT * FROM <dataset>). Use command for Flight servers that expect SQL commands (e.g., Dremio).
• arrow_flight_request_timeout_sec — Timeout in seconds for a single Arrow Flight request, default 300. It bounds the whole request: for a SELECT that is the entire result stream, not just the wait for the first record batch. 0 means no timeout, in which case a Flight server that accepts a request and never answers blocks the query until the connection is closed.

​

Usage Example

CREATE TABLE remote_flight_data
(
    id UInt32,
    name String,
    value Float64
) ENGINE = ArrowFlight('127.0.0.1:9005', 'sample_dataset');

SELECT * FROM remote_flight_data ORDER BY id;

┌─id─┬─name────┬─value─┐
│  1 │ foo     │ 42.1  │
│  2 │ bar     │ 13.3  │
│  3 │ baz     │ 77.0  │
└────┴─────────┴───────┘

INSERT INTO remote_flight_data VALUES (4, 'qux', 99.9);

​

Notes

• If columns are specified in the CREATE TABLE statement, they must match the schema returned by the Flight server.
• If columns are omitted, the schema is inferred automatically from the remote server.
• Both reading (SELECT) and writing (INSERT) are supported.
• The arrow_flight_request_descriptor_type setting controls whether the dataset name is sent as a PATH descriptor or as a CMD descriptor wrapping a SELECT * query.

​

See Also

• arrowFlight table function
• Arrow Flight Interface
• Apache Arrow Flight SQL
• Arrow format integration in ClickHouse