These parameter descriptions are auto-generated first drafts and are still under review.
auth, engine, execution, and
storage β group related settings, and each block contains scalar values or further nested blocks.
Every file must declare the schema version it targets:
Value types
Each scalar parameter has one of the following types. The Type column in the reference below uses these names.| Type | Description | Example |
|---|---|---|
string | Free-form text. | my-engine |
integer | A whole number. | 8123 |
float | A decimal number. | 0.9 |
boolean | true or false. | true |
duration | A time span written as a number with a unit suffix (s, m, h, d). | 30s, 1h, 1d |
byte size | A size in bytes, optionally with a unit suffix (KB, MB, GB, GiB, β¦). | 8GiB, 100MB |
ulid | A ULID identifier. | 01KP98J0000000000000000000 |
enum | One of a fixed set of values, listed in the parameterβs description. | s3 |
Parameter kinds
The Type column also tells you an entryβs kind:- A scalar holds a single value of one of the types above (for example
stringorduration). - An object (shown as
object) is a nested block of named parameters. - A list (shown as
object[]) is a sequence of objects; every element repeats the same structure.
Overview
auth
| Path | Type | Default Value | Description |
|---|---|---|---|
auth β | object | {} | Authentication settings for the database. |
auth.instance_id β | string | https://localhost | Identifier for this instance. |
auth.mode β | enum | disabled | Selects how clients authenticate. |
auth.native β | object | null | Settings for native authentication, used when auth.mode is native. |
auth.native.initial_user β | object | null | Bootstrap user created on startup so you can connect to a fresh instance. |
auth.native.initial_user.name β | string | Username for the bootstrap user created at startup in native authentication mode. | |
auth.native.initial_user.password β | string | Password for the bootstrap user created at startup in native authentication mode. | |
auth.native.jwt β | object | {} | JWT settings for native mode. |
auth.native.jwt.clock_skew_tolerance β | duration | 30s | Allowed clock skew when validating time-based JWT claims such as exp, nbf, and iat. |
auth.native.jwt.max_token_age β | duration | 1d | Maximum age of a token, measured from its iat (issued-at) claim. |
auth.native.jwt.token_expiry β | duration | 1h | Lifetime of the access tokens that the engine issues. |
auth.native.signing_algorithm β | enum | RS256 | Algorithm used to sign issued tokens. |
auth.native.signing_keys β | object[] | [] | Keys used to sign issued tokens. |
auth.native.signing_keys[*].id β | string | Identifier for this signing key. | |
auth.native.signing_keys[*].private_key_path β | string | Filesystem path to the PEM-encoded private key used for signing. | |
auth.oidc β | object | null | Settings for OIDC authentication, used when auth.mode is oidc. |
auth.oidc.jwt β | object | {} | JWT validation settings for OIDC mode. |
auth.oidc.jwt.clock_skew_tolerance β | duration | 30s | Allowed clock skew when validating time-based JWT claims such as exp, nbf, and iat. |
auth.oidc.jwt.max_token_age β | duration | 1d | Maximum age of a token, measured from its iat (issued-at) claim. |
auth.oidc.postgres_password_fallback_enabled β | boolean | false | When enabled, connections over the Postgres wire protocol can fall back to password-based authentication instead of OIDC tokens. |
auth.oidc.providers β | object[] | [] | Trusted OIDC identity providers. |
auth.oidc.providers[*].discovery β | object | {} | Controls how the engine refreshes the providerβs discovery document. |
auth.oidc.providers[*].discovery.refresh_interval β | duration | 1d | How often the engine re-fetches the providerβs OpenID configuration (discovery) document. |
auth.oidc.providers[*].discovery_url β | string | URL of the providerβs OpenID Connect discovery document β the .../.well-known/openid-configuration endpoint. | |
auth.oidc.providers[*].jit_provisioning β | object | {} | Just-in-time (JIT) provisioning settings. |
auth.oidc.providers[*].jit_provisioning.default_role β | string | public | Role granted to users created through just-in-time provisioning. |
auth.oidc.providers[*].jit_provisioning.enabled β | boolean | false | Whether to create users automatically on first login through this provider. |
auth.oidc.providers[*].jwks β | object | {} | Controls how the engine caches the providerβs JSON Web Key Set (JWKS), which it uses to verify token signatures. |
auth.oidc.providers[*].jwks.cache_ttl β | duration | 1h | How long the engine caches the providerβs JWKS document before re-fetching it. |
auth.oidc.providers[*].name β | string | Name or alias for this provider. | |
auth.oidc.providers[*].username_mapping β | string | Template that maps OIDC token claims to an engine username. |
data_source_encryption
| Path | Type | Default Value | Description |
|---|---|---|---|
data_source_encryption β | object | {} | Encryption of external data-source (location and table) credentials. |
data_source_encryption.decrypted_cache_timeout_ms β | integer | 600000 | How long, in milliseconds, a decrypted data-source credential is cached in memory. |
data_source_encryption.enabled β | boolean | false | Encrypt external data-source credentials with KMS before storing them in the metadata service. |
data_source_encryption.kms_arn β | string | β | ARN of the KMS key used to encrypt data-source credentials. |
data_source_encryption.local_kms_access_key β | string | β | Access key for the local KMS service. |
data_source_encryption.local_kms_endpoint β | string | β | Endpoint URL of the local KMS service. |
data_source_encryption.local_kms_secret_key β | string | β | Secret key for the local KMS service. |
data_source_encryption.skip_s3_access_check β | boolean | false | Skip validating S3 access when creating an external location or table. |
data_source_encryption.use_local_kms β | boolean | false | Use a local KMS-compatible endpoint (for example, localstack) instead of AWS KMS. |
endpoints
| Path | Type | Default Value | Description |
|---|---|---|---|
endpoints β | object | {} | Network listener configuration that defines how clients connect to the engine over HTTP and the PostgreSQL wire protocol. |
endpoints.http β | object | {} | HTTP listener configuration for the query API. |
endpoints.http.listeners β | object[] | [] | List of HTTP listener bindings. |
endpoints.http.listeners[*].path β | string | null | Filesystem path for a Unix-domain-socket HTTP listener. |
endpoints.http.listeners[*].port β | integer | null | TCP port for an HTTP listener (for example, 8123). |
endpoints.http.listeners[*].type β | enum | Listener transport: tcp (network socket) or unix (Unix-domain socket). | |
endpoints.postgres β | object | {} | PostgreSQL wire-protocol listener configuration. |
endpoints.postgres.listeners β | object[] | [] | List of PostgreSQL listener bindings. |
endpoints.postgres.listeners[*].path β | string | null | Not used for PostgreSQL listeners; Unix-domain sockets arenβt supported for the Postgres protocol. |
endpoints.postgres.listeners[*].port β | integer | null | TCP port for PostgreSQL connections (for example, 5432). |
endpoints.postgres.listeners[*].type β | enum | Listener transport for PostgreSQL. |
engine
| Path | Type | Default Value | Description |
|---|---|---|---|
engine β | object | {} | Configuration for the query execution engine β instance identity, node topology, memory limits, tablet eviction, and multi-cluster broadcasting. |
engine.auto_vacuum β | object | null | Background auto-vacuum tuning. |
engine.auto_vacuum.assessment_frequency β | integer | null | How frequently the engine assesses tablets to decide whether an auto-vacuum job is needed. |
engine.auto_vacuum.debug_sleep_before_commit_ms β | integer | null | Debugging knob: artificial delay, in milliseconds, inserted before an auto-vacuum job commits. |
engine.auto_vacuum.enabled β | boolean | null | Whether background auto-vacuum runs. |
engine.auto_vacuum.max_concurrency β | integer | null | Maximum number of auto-vacuum jobs allowed to run concurrently. |
engine.auto_vacuum.max_tablets_in_job β | integer | null | Maximum number of tablets processed in a single auto-vacuum job. |
engine.auto_vacuum.memory_allowance β | float | null | Fraction of engine memory that auto-vacuum may use while running. |
engine.auto_vacuum.min_bad_tablets_threshold β | integer | null | Minimum number of tablets needing cleanup before an auto-vacuum job is triggered. |
engine.auto_vacuum.run_on_first_dml β | boolean | null | Whether to trigger an auto-vacuum assessment on the first DML statement after startup. |
engine.cluster_id β | string | null | Unique identifier for this engine cluster. |
engine.cluster_ordinal β | integer | null | Zero-based ordinal of this cluster within a multi-cluster deployment. |
engine.eviction β | object | {} | Tablet memory-eviction policy, controlling when tablets are evicted from in-memory caches to disk as memory fills. |
engine.eviction.max_tablets_per_mb_of_total_memory β | float | 1.5 | Upper bound on how many tablets the node keeps resident, expressed as tablets per MB of total memory. |
engine.eviction.prefetch_space_ratio_threshold β | float | 0.13 | Fraction of memory (0.0β1.0) reserved for tablet prefetching before eviction pressure throttles it. |
engine.eviction.tablet_eviction_soft_threshold β | float | 0.4 | Memory-usage fraction (0.0β1.0, default 0.4) that governs soft eviction of least-recently-used tablets. |
engine.eviction.tablet_eviction_threshold β | float | 0.2 | Memory-usage fraction (0.0β1.0, default 0.2) that governs hard eviction of tablets from memory to reclaim space. |
engine.eviction.tablet_min_ttl_before_memory_eviction_seconds β | integer | 1800 | Minimum age in seconds a tablet must reach before it becomes eligible for eviction from memory. |
engine.eviction.tablet_ttl_before_full_eviction_seconds β | integer | 21600 | Age in seconds after which an unused tablet is fully evicted from the node β dropped from the local disk cache to reclaim space. |
engine.id β | string | default-engine-id | Human-readable identifier for this engine, shown in logs, metrics, and system views. |
engine.max_server_memory_usage β | byte size | 0B | Maximum memory the server may use (bytes, or a size such as 8GiB). |
engine.max_server_memory_usage_headroom_bytes β | byte size | 0B | Amount of host memory to keep free (bytes, or a size). |
engine.max_server_memory_usage_to_ram_ratio β | float | 0.9 | Fraction of host RAM (0.0β1.0, default 0.9) the engine may use when max_server_memory_usage isnβt set explicitly. |
engine.metrics_collection_frequency_per_minute β | integer | 12 | How many times per minute the engine collects and emits metrics. |
engine.multi_cluster β | object | null | Multi-cluster broadcast configuration for query execution across engine clusters. |
engine.multi_cluster.broadcast_endpoint β | string | Address (host:port) of the multi-cluster broadcast service. | |
engine.multi_cluster.broadcast_ssl_enabled β | boolean | false | Whether to use TLS when connecting to the broadcast endpoint. |
engine.multi_cluster.broadcast_tuple_limit_per_engine_cluster β | integer | 0 | Soft limit on rows broadcast per execution stage across the cluster. |
engine.nodes β | object[] | null | List of engine nodes in this instance. |
engine.nodes[*].aragog_port β | integer | 5678 | TCP port for this nodeβs Aragog distributed-execution service. |
engine.nodes[*].host β | string | Hostname or IP address of this node, used by other nodes and services to reach it. | |
engine.nodes[*].shufflepuff_port β | integer | 16000 | TCP port for this nodeβs Shufflepuff data-shuffle service. |
engine.nodes[*].storage_agent_port β | integer | 3434 | TCP port for this nodeβs Storage Agent (local tablet I/O). |
engine.nodes[*].storage_manager_port β | integer | 1717 | TCP port for this nodeβs Storage Manager (tablet lifecycle and metadata). |
engine.termination_grace_period β | duration | 1m | How long to wait for in-flight queries to finish during graceful shutdown before forcing termination. |
execution
| Path | Type | Default Value | Description |
|---|---|---|---|
execution β | object | {} | Query execution settings β thread limits, tablet handling, hybrid-header compression, AI mutation mode, and admission control. |
execution.admission_controller β | object | {} | Admission control settings that govern how many queries run concurrently and how memory is shared, to avoid out-of-memory conditions and improve throughput. |
execution.admission_controller.enabled β | boolean | false | Enable admission control. |
execution.admission_controller.max_concurrent_admitted_queries β | integer | 100 | Maximum number of concurrently admitted queries; the per-node limit scales with cluster size. |
execution.admission_controller.max_required_relative_memory_for_retry β | float | 0.75 | Cap on the extra memory an out-of-memory retry may request, as a fraction of available memory. |
execution.admission_controller.max_retries_per_query β | integer | 3 | Maximum number of automatic retries when a query fails with an out-of-memory error. |
execution.admission_controller.reduce_required_memory_after_seconds_at_front_of_queue β | integer | 10 | After a query waits this many seconds at the front of the admission queue, its estimated memory requirement is reduced to improve its chance of admission. |
execution.admission_controller.seconds_between_no_admission_warnings β | integer | 3600 | Minimum interval, in seconds, between warnings logged when no query can be admitted. |
execution.admission_controller.seconds_until_no_admission_warning β | integer | 300 | Log a warning when no query has been admitted for this many seconds. |
execution.admission_controller.total_memory_tracker_hard_limit_ratio β | float | 0.9 | Fraction of the memory trackerβs hard limit that admission control may allocate per node. |
execution.ai_mutations_mode β | enum | reevaluate | Execution mode for AI mutation queries: native_only, reevaluate (default), or hybrid. |
execution.allow_amazon_bedrock β | boolean | true | Allow queries that call Amazon Bedrock AI functions. |
execution.enable_shufflepuff β | boolean | true | Enable the Shufflepuff shuffle subsystem used for distributed (multi-node) query execution. |
execution.hybrid_headers_format_version β | integer | 3 | On-disk format version for Hybrid Headers tablet storage. |
execution.hybrid_headers_primary_index_compression_level β | integer | 2 | Compression level for the Hybrid Headers primary index. |
execution.hybrid_headers_primary_index_compression_method β | enum | BROTLI | Compression algorithm for the Hybrid Headers primary index: one of none, gzip, zlib, xz, zstd, brotli, lz4, or snappy. |
execution.max_threads β | integer | 0 | Maximum number of threads used to execute a single query. |
execution.merge_committed_tablets β | boolean | true | Allow background merging of committed tablets during maintenance. |
execution.min_bytes_for_wide_part β | integer | 104857600 | Minimum uncompressed size, in bytes, for a tablet to use the wide format instead of the compact format. |
execution.regexp_cache_max_keys β | integer | 10000 | Maximum number of compiled regular expressions to cache. |
execution.storage_manager_cache_tablets_on_proxy β | boolean | true | Cache tablet-assignment information on the storage-manager proxy to reduce metadata lookups. |
instance
| Path | Type | Default Value | Description |
|---|---|---|---|
instance β | object | {} | Instance identity and deployment topology β the instance ID and whether this is a single-engine or multi-engine deployment. |
instance.account_name β | string | default-account | Name of the account this instance belongs to. |
instance.id β | ulid | 01KP98J0000000000000000000 | Unique instance identifier in ULID format. |
instance.multi_engine β | object | null | Multi-engine settings. |
instance.multi_engine.metadata_endpoint β | string | Address (host:port) of the external Pensieve metadata service. | |
instance.organization_id β | ulid | 01KP98J0000000000000000001 | ULID of the organization this instance belongs to. |
instance.organization_name β | string | default-org | Name of the organization this instance belongs to. |
instance.type β | enum | single_engine | Deployment topology: single_engine (metadata runs locally) or multi_engine (metadata served by an external Pensieve service). |
logging
| Path | Type | Default Value | Description |
|---|---|---|---|
logging β | object | {} | Logging configuration β the default level, output format, per-component overrides, and output sinks. |
logging.components β | object[] | [] | Per-component log-level overrides. |
logging.components[*].level β | enum | Log level for this component, overriding logging.level. | |
logging.components[*].name β | string | Name of the logger component this override applies to. | |
logging.format β | enum | json | Log output format: text (human-readable) or json (structured). |
logging.level β | enum | info | Default log level for all messages: one of trace, debug, info, warn, error, or fatal. |
logging.sinks β | object[] | null | Log output targets. |
logging.sinks[*].file β | object | null | File-sink settings. |
logging.sinks[*].file.path β | string | Filesystem path the file sink writes to. | |
logging.sinks[*].level β | enum | null | Log level for this sink. |
logging.sinks[*].type β | enum | Sink destination: stderr or file. |
otel
planner
| Path | Type | Default Value | Description |
|---|---|---|---|
planner β | object | {} | Query planner configuration. |
planner.automated_column_statistics β | object | {} | Settings for the automated column-statistics cache used by the optimizer. |
planner.automated_column_statistics.cache_max_size_bytes β | integer | 104857600 | Maximum size, in bytes, of the automated column-statistics cache. |
query_log
schema_version
| Path | Type | Default Value | Description |
|---|---|---|---|
schema_version β | string | Version of the configuration schema. |
storage
| Path | Type | Default Value | Description |
|---|---|---|---|
storage β | object | {} | Managed-table storage settings β provider type, bucket/location, provider credentials, and garbage-collection behavior. |
storage.allow_collect_garbage β | boolean | false | Allow manual garbage collection of orphaned tablets via CALL collect_garbage(). |
storage.api_scheme β | string | null | Storage URI scheme (for example, s3://, gs://, or azure://). |
storage.aws β | object | null | AWS settings for S3-backed managed tables. |
storage.aws.intermediary_access_role β | string | null | AWS IAM role assumed for federated, cross-account or cross-tenant S3 access. |
storage.azure β | object | null | Azure settings for Blob-Storage-backed managed tables. |
storage.azure.intermediary_service_principal_client_id β | string | null | Client ID of a federated Azure service principal for cross-tenant access. |
storage.azure.storage_account_name β | string | null | Azure Blob Storage account name for managed tables, accessed via workload identity. |
storage.bucket_name β | string | null | Bucket used for managed-table objects. |
storage.collect_garbage_limit_per_query β | integer | 0 | Maximum tablets cleaned per collect_garbage() call. |
storage.default_s3_endpoint_override β | string | null | Override the S3-compatible endpoint URL, redirecting S3 API calls to a custom or on-premises endpoint. |
storage.download_chunk_prefetch_max_threads β | integer | 96 | Maximum number of threads used to prefetch download chunks from object storage. |
storage.enable_managed_location β | boolean | false | Allow CREATE TABLE to specify a LOCATION for managed tables. |
storage.gc_ttl_seconds β | integer | 604800 | Grace period, in seconds, before a tablet marked for garbage collection is permanently removed from object storage. |
storage.gcp β | object | null | Google Cloud settings for GCS-backed managed tables. |
storage.gcp.intermediary_service_account_id β | string | null | GCP service account used for federated, cross-project or cross-tenant GCS access. |
storage.minio β | object | null | MinIO settings for local or self-hosted S3-compatible storage. |
storage.minio.endpoint β | string | MinIO server endpoint URL (for example, http://localhost:9000). | |
storage.type β | enum | s3 | Object-storage provider for managed tables: s3, gcs, abs, azurite, or minio. |
storage.upload_max_tries β | integer | null | Maximum number of retries for object-storage uploads. |
Details
auth
auth β object, default {}
Authentication settings for the database. Authentication is disabled by default; set auth.mode to enable native or OIDC authentication, then configure the matching block below.
auth.instance_id β string, default https://localhost
Identifier for this instance. In native mode itβs used as the JWT iss (issuer) claim; in both native and OIDC modes itβs used as the expected aud (audience) claim. Defaults to https://localhost.
auth.mode β enum, default disabled
Selects how clients authenticate. Use disabled for no authentication (the default), native to have the engine issue and validate its own tokens, or oidc to validate tokens issued by an external identity provider. When you choose native or oidc, configure the matching block below.
auth.native β object, default null
Settings for native authentication, used when auth.mode is native. In this mode the engine issues and validates its own JWTs.
auth.native.initial_user β object, default null
Bootstrap user created on startup so you can connect to a fresh instance. Provide a name and a password. This is required when you run a single-engine instance with native authentication.
auth.native.initial_user.name β string
Username for the bootstrap user created at startup in native authentication mode.
auth.native.initial_user.password β string
Password for the bootstrap user created at startup in native authentication mode.
auth.native.jwt β object, default {}
JWT settings for native mode. Because the engine issues tokens itself in this mode, these settings control the lifetime and temporal validation of the tokens it generates.
auth.native.jwt.clock_skew_tolerance β duration, default 30s
Allowed clock skew when validating time-based JWT claims such as exp, nbf, and iat. Tokens within this tolerance of the current time are still accepted. Defaults to 30s.
auth.native.jwt.max_token_age β duration, default 1d
Maximum age of a token, measured from its iat (issued-at) claim. The engine rejects tokens older than this even if they havenβt expired. Defaults to 1d.
auth.native.jwt.token_expiry β duration, default 1h
Lifetime of the access tokens that the engine issues. After this duration a token expires and the client must obtain a new one. Defaults to 1h.
auth.native.signing_algorithm β enum, default RS256
Algorithm used to sign issued tokens. Choose one of the RSA algorithms (RS256, RS384, RS512) or ECDSA algorithms (ES256, ES384, ES512). Defaults to RS256.
auth.native.signing_keys β object[], default []
Keys used to sign issued tokens. Each entry points to a private key on disk. Leave the list empty to run in development mode, where the engine generates an ephemeral signing key on startup.
auth.native.signing_keys[*].id β string
Identifier for this signing key. The engine publishes it as the JWT kid (key ID) header so clients can select the correct key when verifying a token.
auth.native.signing_keys[*].private_key_path β string
Filesystem path to the PEM-encoded private key used for signing.
auth.oidc β object, default null
Settings for OIDC authentication, used when auth.mode is oidc. In this mode the engine validates tokens issued by one or more external identity providers and doesnβt issue tokens itself.
auth.oidc.jwt β object, default {}
JWT validation settings for OIDC mode. Because the upstream identity provider issues the tokens, only validation settings apply here β there are no token-issuance options.
auth.oidc.jwt.clock_skew_tolerance β duration, default 30s
Allowed clock skew when validating time-based JWT claims such as exp, nbf, and iat. Tokens within this tolerance of the current time are still accepted. Defaults to 30s.
auth.oidc.jwt.max_token_age β duration, default 1d
Maximum age of a token, measured from its iat (issued-at) claim. The engine rejects tokens older than this even if they havenβt expired. Defaults to 1d.
auth.oidc.postgres_password_fallback_enabled β boolean, default false
When enabled, connections over the Postgres wire protocol can fall back to password-based authentication instead of OIDC tokens. Disabled by default.
auth.oidc.providers β object[], default []
Trusted OIDC identity providers. A single provider is supported at launch; the list form leaves room for multiple providers in the future.
auth.oidc.providers[*].discovery β object, default {}
Controls how the engine refreshes the providerβs discovery document.
auth.oidc.providers[*].discovery.refresh_interval β duration, default 1d
How often the engine re-fetches the providerβs OpenID configuration (discovery) document. Defaults to 1d.
auth.oidc.providers[*].discovery_url β string
URL of the providerβs OpenID Connect discovery document β the .../.well-known/openid-configuration endpoint. The engine reads the providerβs metadata, including its JWKS URL, from this document.
auth.oidc.providers[*].jit_provisioning β object, default {}
Just-in-time (JIT) provisioning settings. When enabled, the engine creates a user automatically the first time someone authenticates through this provider.
auth.oidc.providers[*].jit_provisioning.default_role β string, default public
Role granted to users created through just-in-time provisioning. Defaults to public.
auth.oidc.providers[*].jit_provisioning.enabled β boolean, default false
Whether to create users automatically on first login through this provider. Disabled by default.
auth.oidc.providers[*].jwks β object, default {}
Controls how the engine caches the providerβs JSON Web Key Set (JWKS), which it uses to verify token signatures.
auth.oidc.providers[*].jwks.cache_ttl β duration, default 1h
How long the engine caches the providerβs JWKS document before re-fetching it. Defaults to 1h.
auth.oidc.providers[*].name β string
Name or alias for this provider. The engine uses it to identify the provider in logs and configuration.
auth.oidc.providers[*].username_mapping β string
Template that maps OIDC token claims to an engine username. Reference claims with {{ claim }} syntax β for example {{ email }}, {{ sub }}, or {{ iss }}|{{ sub }} to namespace usernames by issuer.
data_source_encryption
data_source_encryption β object, default {}
Encryption of external data-source (location and table) credentials. When enabled, the engine encrypts S3 credentials with KMS before storing them in the metadata service.
data_source_encryption.decrypted_cache_timeout_ms β integer, default 600000
How long, in milliseconds, a decrypted data-source credential is cached in memory. Defaults to 600000 (10 minutes).
data_source_encryption.enabled β boolean, default false
Encrypt external data-source credentials with KMS before storing them in the metadata service.
data_source_encryption.kms_arn β string, default β
ARN of the KMS key used to encrypt data-source credentials.
data_source_encryption.local_kms_access_key β string, default β
Access key for the local KMS service. Required when use_local_kms is true.
data_source_encryption.local_kms_endpoint β string, default β
Endpoint URL of the local KMS service. Required when use_local_kms is true.
data_source_encryption.local_kms_secret_key β string, default β
Secret key for the local KMS service. Required when use_local_kms is true.
data_source_encryption.skip_s3_access_check β boolean, default false
Skip validating S3 access when creating an external location or table. Useful when credentials are placeholders.
data_source_encryption.use_local_kms β boolean, default false
Use a local KMS-compatible endpoint (for example, localstack) instead of AWS KMS. Intended for self-deployed and test environments.
endpoints
endpoints β object, default {}
Network listener configuration that defines how clients connect to the engine over HTTP and the PostgreSQL wire protocol.
endpoints.http β object, default {}
HTTP listener configuration for the query API.
endpoints.http.listeners β object[], default []
List of HTTP listener bindings. You can define a TCP listener and a Unix-socket listener, each at most once.
endpoints.http.listeners[*].path β string, default null
Filesystem path for a Unix-domain-socket HTTP listener. Required for unix listeners; omit it for tcp listeners.
endpoints.http.listeners[*].port β integer, default null
TCP port for an HTTP listener (for example, 8123). Required for tcp listeners; omit it for unix listeners.
endpoints.http.listeners[*].type β enum
Listener transport: tcp (network socket) or unix (Unix-domain socket).
endpoints.postgres β object, default {}
PostgreSQL wire-protocol listener configuration. Clients connect using standard Postgres drivers and psql.
endpoints.postgres.listeners β object[], default []
List of PostgreSQL listener bindings. TCP only β Unix sockets arenβt supported for the Postgres protocol.
endpoints.postgres.listeners[*].path β string, default null
Not used for PostgreSQL listeners; Unix-domain sockets arenβt supported for the Postgres protocol.
endpoints.postgres.listeners[*].port β integer, default null
TCP port for PostgreSQL connections (for example, 5432). Required for every Postgres listener.
endpoints.postgres.listeners[*].type β enum
Listener transport for PostgreSQL. Only tcp is supported.
engine
engine β object, default {}
Configuration for the query execution engine β instance identity, node topology, memory limits, tablet eviction, and multi-cluster broadcasting.
engine.auto_vacuum β object, default null
Background auto-vacuum tuning. Auto-vacuum compacts and cleans up tablets in the background. Itβs disabled by default; every field is optional and overrides the built-in default only when you set it.
engine.auto_vacuum.assessment_frequency β integer, default null
How frequently the engine assesses tablets to decide whether an auto-vacuum job is needed.
engine.auto_vacuum.debug_sleep_before_commit_ms β integer, default null
Debugging knob: artificial delay, in milliseconds, inserted before an auto-vacuum job commits. Intended for testing only.
engine.auto_vacuum.enabled β boolean, default null
Whether background auto-vacuum runs. Disabled by default.
engine.auto_vacuum.max_concurrency β integer, default null
Maximum number of auto-vacuum jobs allowed to run concurrently.
engine.auto_vacuum.max_tablets_in_job β integer, default null
Maximum number of tablets processed in a single auto-vacuum job.
engine.auto_vacuum.memory_allowance β float, default null
Fraction of engine memory that auto-vacuum may use while running.
engine.auto_vacuum.min_bad_tablets_threshold β integer, default null
Minimum number of tablets needing cleanup before an auto-vacuum job is triggered.
engine.auto_vacuum.run_on_first_dml β boolean, default null
Whether to trigger an auto-vacuum assessment on the first DML statement after startup.
engine.cluster_id β string, default null
Unique identifier for this engine cluster. Required when multi-cluster broadcasting is enabled, where it tags outbound requests for cross-cluster coordination.
engine.cluster_ordinal β integer, default null
Zero-based ordinal of this cluster within a multi-cluster deployment. Required when multi-cluster broadcasting is enabled, to distinguish cluster instances.
engine.eviction β object, default {}
Tablet memory-eviction policy, controlling when tablets are evicted from in-memory caches to disk as memory fills.
engine.eviction.max_tablets_per_mb_of_total_memory β float, default 1.5
Upper bound on how many tablets the node keeps resident, expressed as tablets per MB of total memory. Defaults to 1.5. Caps tablet residency relative to available memory.
engine.eviction.prefetch_space_ratio_threshold β float, default 0.13
Fraction of memory (0.0β1.0) reserved for tablet prefetching before eviction pressure throttles it. Defaults to 0.13.
engine.eviction.tablet_eviction_soft_threshold β float, default 0.4
Memory-usage fraction (0.0β1.0, default 0.4) that governs soft eviction of least-recently-used tablets. Soft-evicted tablets remain available on disk and are re-cached on access.
engine.eviction.tablet_eviction_threshold β float, default 0.2
Memory-usage fraction (0.0β1.0, default 0.2) that governs hard eviction of tablets from memory to reclaim space.
engine.eviction.tablet_min_ttl_before_memory_eviction_seconds β integer, default 1800
Minimum age in seconds a tablet must reach before it becomes eligible for eviction from memory. Defaults to 1800 (30 minutes).
engine.eviction.tablet_ttl_before_full_eviction_seconds β integer, default 21600
Age in seconds after which an unused tablet is fully evicted from the node β dropped from the local disk cache to reclaim space. Defaults to 21600 (6 hours).
engine.id β string, default default-engine-id
Human-readable identifier for this engine, shown in logs, metrics, and system views. Defaults to default-engine-id.
engine.max_server_memory_usage β byte size, default 0B
Maximum memory the server may use (bytes, or a size such as 8GiB). When 0 (the default), the limit is derived from host RAM using max_server_memory_usage_to_ram_ratio and max_server_memory_usage_headroom_bytes.
engine.max_server_memory_usage_headroom_bytes β byte size, default 0B
Amount of host memory to keep free (bytes, or a size). Used with the ratio to cap server memory when max_server_memory_usage isnβt set explicitly. Default 0.
engine.max_server_memory_usage_to_ram_ratio β float, default 0.9
Fraction of host RAM (0.0β1.0, default 0.9) the engine may use when max_server_memory_usage isnβt set explicitly.
engine.metrics_collection_frequency_per_minute β integer, default 12
How many times per minute the engine collects and emits metrics. Default 12 (every five seconds).
engine.multi_cluster β object, default null
Multi-cluster broadcast configuration for query execution across engine clusters. Omit this block for a standalone or single-cluster engine.
engine.multi_cluster.broadcast_endpoint β string
Address (host:port) of the multi-cluster broadcast service. Required and non-empty when multi-cluster broadcasting is enabled.
engine.multi_cluster.broadcast_ssl_enabled β boolean, default false
Whether to use TLS when connecting to the broadcast endpoint. Default false.
engine.multi_cluster.broadcast_tuple_limit_per_engine_cluster β integer, default 0
Soft limit on rows broadcast per execution stage across the cluster. Default 0 (unlimited); set a positive value to cap intermediate result sizes.
engine.nodes β object[], default null
List of engine nodes in this instance. When omitted, a single node on 127.0.0.1 with default ports is used.
engine.nodes[*].aragog_port β integer, default 5678
TCP port for this nodeβs Aragog distributed-execution service. Default 5678.
engine.nodes[*].host β string
Hostname or IP address of this node, used by other nodes and services to reach it.
engine.nodes[*].shufflepuff_port β integer, default 16000
TCP port for this nodeβs Shufflepuff data-shuffle service. Default 16000.
engine.nodes[*].storage_agent_port β integer, default 3434
TCP port for this nodeβs Storage Agent (local tablet I/O). Default 3434.
engine.nodes[*].storage_manager_port β integer, default 1717
TCP port for this nodeβs Storage Manager (tablet lifecycle and metadata). Default 1717.
engine.termination_grace_period β duration, default 1m
How long to wait for in-flight queries to finish during graceful shutdown before forcing termination. Default 1m.
execution
execution β object, default {}
Query execution settings β thread limits, tablet handling, hybrid-header compression, AI mutation mode, and admission control.
execution.admission_controller β object, default {}
Admission control settings that govern how many queries run concurrently and how memory is shared, to avoid out-of-memory conditions and improve throughput.
execution.admission_controller.enabled β boolean, default false
Enable admission control. When enabled, queries are queued and prioritized based on available memory and concurrency limits. Default false.
execution.admission_controller.max_concurrent_admitted_queries β integer, default 100
Maximum number of concurrently admitted queries; the per-node limit scales with cluster size. Default 100.
execution.admission_controller.max_required_relative_memory_for_retry β float, default 0.75
Cap on the extra memory an out-of-memory retry may request, as a fraction of available memory. Default 0.75.
execution.admission_controller.max_retries_per_query β integer, default 3
Maximum number of automatic retries when a query fails with an out-of-memory error. Default 3.
execution.admission_controller.reduce_required_memory_after_seconds_at_front_of_queue β integer, default 10
After a query waits this many seconds at the front of the admission queue, its estimated memory requirement is reduced to improve its chance of admission. Default 10.
execution.admission_controller.seconds_between_no_admission_warnings β integer, default 3600
Minimum interval, in seconds, between warnings logged when no query can be admitted. Default 3600.
execution.admission_controller.seconds_until_no_admission_warning β integer, default 300
Log a warning when no query has been admitted for this many seconds. Default 300.
execution.admission_controller.total_memory_tracker_hard_limit_ratio β float, default 0.9
Fraction of the memory trackerβs hard limit that admission control may allocate per node. Default 0.9.
execution.ai_mutations_mode β enum, default reevaluate
Execution mode for AI mutation queries: native_only, reevaluate (default), or hybrid.
execution.allow_amazon_bedrock β boolean, default true
Allow queries that call Amazon Bedrock AI functions. Enabled by default.
execution.enable_shufflepuff β boolean, default true
Enable the Shufflepuff shuffle subsystem used for distributed (multi-node) query execution. When enabled, the engine registers io_uring buffers at startup, which requires sufficient locked memory (RLIMIT_MEMLOCK). Default true.
execution.hybrid_headers_format_version β integer, default 3
On-disk format version for Hybrid Headers tablet storage. Default 3: version 1 is the original format, 2 adds primary-index compression, and 3 adds compact/subcompact tablets.
execution.hybrid_headers_primary_index_compression_level β integer, default 2
Compression level for the Hybrid Headers primary index. Default 2; the valid range depends on the chosen method.
execution.hybrid_headers_primary_index_compression_method β enum, default BROTLI
Compression algorithm for the Hybrid Headers primary index: one of none, gzip, zlib, xz, zstd, brotli, lz4, or snappy. Default brotli.
execution.max_threads β integer, default 0
Maximum number of threads used to execute a single query. 0 (default) lets the engine choose automatically.
execution.merge_committed_tablets β boolean, default true
Allow background merging of committed tablets during maintenance. Default true.
execution.min_bytes_for_wide_part β integer, default 104857600
Minimum uncompressed size, in bytes, for a tablet to use the wide format instead of the compact format.
execution.regexp_cache_max_keys β integer, default 10000
Maximum number of compiled regular expressions to cache. Default 10000.
execution.storage_manager_cache_tablets_on_proxy β boolean, default true
Cache tablet-assignment information on the storage-manager proxy to reduce metadata lookups. Default true.
instance
instance β object, default {}
Instance identity and deployment topology β the instance ID and whether this is a single-engine or multi-engine deployment.
instance.account_name β string, default default-account
Name of the account this instance belongs to. Defaults to default-account.
instance.id β ulid, default 01KP98J0000000000000000000
Unique instance identifier in ULID format. Set automatically in cloud-managed deployments; override it for custom Firebolt Core setups.
instance.multi_engine β object, default null
Multi-engine settings. Required when instance.type is multi_engine and ignored for single_engine. Configures the connection to a shared, remote metadata service.
instance.multi_engine.metadata_endpoint β string
Address (host:port) of the external Pensieve metadata service. Required when instance.type is multi_engine.
instance.organization_id β ulid, default 01KP98J0000000000000000001
ULID of the organization this instance belongs to. Set automatically in cloud-managed deployments.
instance.organization_name β string, default default-org
Name of the organization this instance belongs to. Defaults to default-org.
instance.type β enum, default single_engine
Deployment topology: single_engine (metadata runs locally) or multi_engine (metadata served by an external Pensieve service). Default single_engine.
logging
logging β object, default {}
Logging configuration β the default level, output format, per-component overrides, and output sinks.
logging.components β object[], default []
Per-component log-level overrides. Each entry sets a level for one logger component, independent of the global default.
logging.components[*].level β enum
Log level for this component, overriding logging.level. One of trace, debug, info, warn, error, or fatal.
logging.components[*].name β string
Name of the logger component this override applies to.
logging.format β enum, default json
Log output format: text (human-readable) or json (structured). Default json.
logging.level β enum, default info
Default log level for all messages: one of trace, debug, info, warn, error, or fatal. Default info. Components and sinks can override it.
logging.sinks β object[], default null
Log output targets. Each sink writes to stderr or a file. When omitted, a single stderr sink at the global level is installed.
logging.sinks[*].file β object, default null
File-sink settings. Required when the sink type is file; must be absent when the type is stderr.
logging.sinks[*].file.path β string
Filesystem path the file sink writes to. Required when the sink type is file.
logging.sinks[*].level β enum, default null
Log level for this sink. Inherits logging.level when omitted. One of trace, debug, info, warn, error, or fatal.
logging.sinks[*].type β enum
Sink destination: stderr or file. Required for each sink.
otel
otel β object, default {}
OpenTelemetry tracing. When enabled, the engine exports traces over OTLP.
otel.enabled β boolean, default false
Whether OpenTelemetry trace export is enabled. Disabled by default.
otel.endpoint β string, default β
OTLP endpoint URL the engine exports traces to.
otel.protocol β string, default http/protobuf
OTLP export protocol. One of http/protobuf (the default) or http/json.
planner
planner β object, default {}
Query planner configuration.
planner.automated_column_statistics β object, default {}
Settings for the automated column-statistics cache used by the optimizer.
planner.automated_column_statistics.cache_max_size_bytes β integer, default 104857600
Maximum size, in bytes, of the automated column-statistics cache. Default 100 MiB (104857600). Raise it to cache more statistics, lower it to reduce memory use.
query_log
query_log β object, default {}
Query history export. When a Kafka broker list is configured, the engine publishes the query log to Kafka.
query_log.kafka_brokers β string, default β
Comma-separated list of Kafka brokers to publish the query log to. Leave empty to disable query-log export.
schema_version
schema_version β string
Version of the configuration schema. Required at the root and must be "1.0". It lets the configuration format evolve through future migrations.
storage
storage β object, default {}
Managed-table storage settings β provider type, bucket/location, provider credentials, and garbage-collection behavior.
storage.allow_collect_garbage β boolean, default false
Allow manual garbage collection of orphaned tablets via CALL collect_garbage(). Default false.
storage.api_scheme β string, default null
Storage URI scheme (for example, s3://, gs://, or azure://). Defaults to the scheme for the configured storage.type; set it only to override that default.
storage.aws β object, default null
AWS settings for S3-backed managed tables. Set this block only when storage.type is s3.
storage.aws.intermediary_access_role β string, default null
AWS IAM role assumed for federated, cross-account or cross-tenant S3 access. Leave unset to use the engineβs own AWS identity.
storage.azure β object, default null
Azure settings for Blob-Storage-backed managed tables. Set this block only when storage.type is abs or azurite.
storage.azure.intermediary_service_principal_client_id β string, default null
Client ID of a federated Azure service principal for cross-tenant access. Leave unset to use the engineβs own workload identity.
storage.azure.storage_account_name β string, default null
Azure Blob Storage account name for managed tables, accessed via workload identity. Required when storage.type is abs.
storage.bucket_name β string, default null
Bucket used for managed-table objects. When set, it overrides the default bucket β useful for Firebolt Core to point at a custom location.
storage.collect_garbage_limit_per_query β integer, default 0
Maximum tablets cleaned per collect_garbage() call. 0 (default) means no per-query limit; set a positive value to process large cleanups in batches.
storage.default_s3_endpoint_override β string, default null
Override the S3-compatible endpoint URL, redirecting S3 API calls to a custom or on-premises endpoint.
storage.download_chunk_prefetch_max_threads β integer, default 96
Maximum number of threads used to prefetch download chunks from object storage.
storage.enable_managed_location β boolean, default false
Allow CREATE TABLE to specify a LOCATION for managed tables. When false (default), managed tables live only in the system-managed bucket.
storage.gc_ttl_seconds β integer, default 604800
Grace period, in seconds, before a tablet marked for garbage collection is permanently removed from object storage. Default 604800 (7 days).
storage.gcp β object, default null
Google Cloud settings for GCS-backed managed tables. Set this block only when storage.type is gcs.
storage.gcp.intermediary_service_account_id β string, default null
GCP service account used for federated, cross-project or cross-tenant GCS access. Leave unset to use the engineβs own workload identity.
storage.minio β object, default null
MinIO settings for local or self-hosted S3-compatible storage. Set this block only when storage.type is minio.
storage.minio.endpoint β string
MinIO server endpoint URL (for example, http://localhost:9000). Required when storage.type is minio.
storage.type β enum, default s3
Object-storage provider for managed tables: s3, gcs, abs, azurite, or minio. Default s3. Set exactly one matching provider block (aws, gcp, azure, or minio).
storage.upload_max_tries β integer, default null
Maximum number of retries for object-storage uploads. Leave unset to use the cloud SDK default.