HTTP(S) dictionary source

Working with an HTTP(S) server depends on how the dictionary is stored in memory. If the dictionary is stored using cache and complex_key_cache, ClickHouse requests the necessary keys by sending a request via the POST method. Example of settings:

DDL
Configuration file

SOURCE(HTTP(
    url 'http://[::1]/os.tsv'
    format 'TabSeparated'
    credentials(user 'user' password 'password')
    headers(header(name 'API-KEY' value 'key'))
))

<source>
    <http>
        <url>http://[::1]/os.tsv</url>
        <format>TabSeparated</format>
        <credentials>
            <user>user</user>
            <password>password</password>
        </credentials>
        <headers>
            <header>
                <name>API-KEY</name>
                <value>key</value>
            </header>
        </headers>
    </http>
</source>

In order for ClickHouse to access an HTTPS resource, you must configure openSSL in the server configuration. Setting fields:

Setting	Description
`url`	The source URL.
`format`	The file format. All the formats described in Formats are supported.
`credentials`	Basic HTTP authentication. Optional.
`user`	Username required for the authentication.
`password`	Password required for the authentication.
`headers`	All custom HTTP headers entries used for the HTTP request. Optional.
`header`	Single HTTP header entry.
`name`	Identifier name used for the header send on the request.
`value`	Value set for a specific identifier name.

When creating a dictionary using the DDL command (CREATE DICTIONARY ...) remote hosts for HTTP dictionaries are checked against the contents of remote_url_allow_hosts section from config to prevent database users to access arbitrary HTTP server. The headers, including their names, are shown as HEADERS ('[HIDDEN]') in the output of SHOW CREATE DICTIONARY, in system.tables and in the query logs, the same way as the password. As with the password, a query that cannot be parsed is logged as is, with only query_masking_rules applied. To display the headers in SHOW CREATE DICTIONARY and system.tables, enable the server setting display_secrets_in_show_and_select and the format setting format_display_secrets_in_show_and_select; the user also needs the displaySecretsInShowAndSelect privilege. These settings do not affect the query logs.

SQL Reference

Data Types

Engines

Functions

Formats

Settings

System Tables

Data Lakes