Overview
Firebolt allows users to specify a custom compression algorithm at both the table level and the column level when creating a table. Compression helps you reduce storage costs and improve query performance by decreasing disk usage and I/O.Supported compression algorithms
Firebolt supports the following compression algorithms:lz4
– Default option; fast compression and decompression, ideal for general workloads.zstd
– Higher compression ratios at the cost of increased CPU usage; useful for large datasets.
Disabling compression with
NONE
is not supported. Firebolt always compresses data using one of the supported algorithms.Specify compression settings
You can specify compression settings for a table, columns, or a combination of both when creating tables:- Table-level compression sets a default compression method for all columns.
- Column-level compression overrides the table-level default for individual columns.
Table-level compression
You specify table-level compression using theWITH
clause in the CREATE TABLE
statement. This compression applies to all columns unless explicitly overridden at the column level.
Syntax:
Use the following syntax to set table-level and/or column-level compression.Compression level
- The optional
COMPRESSION_LEVEL
parameter lets you fine-tune the balance between compression efficiency and CPU usage. - Higher values provide greater comparison but increase CPU overhead.
- Firebolt uses algorithm-specific default levels if no level is specified.
Default behavior
- If you don’t specify column-level compression. Firebolt applies the table-level setting.
- If no table-level setting is provided, Firebolt defaults to
lz4
.
Example:
The following code creates a table with ZTSD compression:Column-level compression
Column-level compression allows you to override table-level compression settings for specific columns. This approach is useful for columns that have different compression requirements, such as large text fields or numerical columns.Syntax:
Compression level
- The
compression_level
parameter fine-tunes compression efficiency versus CPU usage. - Higher compresson levels reduce storage but consume more CPU resources during compression and decompression.
- If not specified, Firebolt applies a default compression level appropriate for the selected algorithm.
Default behavior
- If you do not specify compression explicitly, Firebolt applies the default
lz4
compression algorithm. - When you define table-level compression, all columns inherit this setting unless overridden with column-level compression.
Step-by-step examples of compression
The following examples show how to use table-level and column-level compression in Firebolt. Step 1: Create and populate an uncompressed table Create a base table without custom compression and populate it with data usingGENERATE_SERIES
origin_no_compression
table:
compressed_byte
value of the tables to evaluate the effectiveness of compression settings:
Example results
These are typical results from a single-node engine, showing how different compression methods and column-level settings affect table sizes:compressed_bytes | table_name |
---|---|
4772 | origin_no_compression |
4396 | table_a_b_column_level_zstd |
4584 | table_a_column_level_zstd |
2558 | table_c_column_level_zstd |
763 | table_c_d_column_level_zstd |