| Function | Support | Notes |
|---|---|---|
ANY | Supported | |
APPROX_COUNT_DISTINCT | Supported | |
ARRAY_AGG | Supported | |
AVG | Supported | |
BIT_AND | Supported | |
BIT_OR | Supported | |
BIT_XOR | Supported | |
BOOL_AND | Supported | |
BOOL_OR | Supported | |
COLLECT_LIST | Supported | |
CORR | Supported | |
COUNT | Supported | |
COUNT_IF | Supported | |
COVAR_POP | Supported | |
COVAR_SAMP | Supported | |
EVERY | Supported | |
MAX | Supported | |
MAX_BY | Supported | |
MEAN | Supported | |
MIN | Supported | |
MIN_BY | Supported | |
REGR_AVGX | Supported | |
REGR_AVGY | Supported | |
REGR_COUNT | Supported | |
REGR_INTERCEPT | Supported | |
REGR_R2 | Supported | |
REGR_SLOPE | Supported | |
REGR_SXX | Supported | |
REGR_SXY | Supported | |
REGR_SYY | Supported | |
SOME | Supported | |
STD | Supported | |
STDDEV | Supported | |
STDDEV_POP | Supported | |
STDDEV_SAMP | Supported | |
VAR_POP | Supported | |
VAR_SAMP | Supported | |
VARIANCE | Supported | |
ANY_VALUE | With limitations | The ignore-nulls argument, any_value(col, true), is not supported. Which value a group returns can differ from Spark. |
COLLECT_SET | With limitations | The order of the elements can differ from Spark. |
FIRST_VALUE | With limitations | Supported as a window function. The aggregate form, without OVER, is not. |
GROUPING | With limitations | Returns INTEGER, where Spark returns TINYINT. |
LISTAGG | With limitations | Without a delimiter, values are joined with ,; Spark joins them with no separator. listagg(DISTINCT col) is not supported. |
MEDIAN | With limitations | Not supported for INTERVAL arguments. |
PERCENTILE | With limitations | An array of percentages is not supported, nor are INTERVAL arguments. |
PERCENTILE_CONT | With limitations | Not supported for INTERVAL arguments. |
PERCENTILE_DISC | With limitations | Returns INTEGER, where Spark returns DOUBLE. |
STRING_AGG | With limitations | Names its result column string_agg, where Spark names it listagg. |
SUM | With limitations | A column holding only untyped NULL literals raises an error, where Spark returns NULL. |
APPROX_PERCENTILE, APPROX_TOP_K, APPROX_TOP_K_ACCUMULATE, APPROX_TOP_K_COMBINE, BITMAP_AND_AGG, BITMAP_CONSTRUCT_AGG, BITMAP_OR_AGG, COUNT_MIN_SKETCH, FIRST, GROUPING_ID, HISTOGRAM_NUMERIC, HLL_SKETCH_AGG, HLL_UNION_AGG, KLL_SKETCH_AGG_BIGINT, KLL_SKETCH_AGG_DOUBLE, KLL_SKETCH_AGG_FLOAT, KURTOSIS, LAST, LAST_VALUE, MODE, PERCENTILE_APPROX, SKEWNESS, THETA_INTERSECTION_AGG, THETA_SKETCH_AGG, THETA_UNION_AGG, TRY_AVG, TRY_SUM | Not supported |
Functions
Aggregate functions
Spark aggregate functions supported in SparkSQL and Databricks compatibility modes.
Preview
Spark aggregate functions in SparkSQL and Databricks compatibility modes. Both modes share these functions. Support is one of: Supported, With limitations, or Not supported. Supported means every example in the Spark reference returns what Spark returns; a note under With limitations quotes a call that differs.