Python Polars group_by Aggregation and Sorting

When you need to summarize data by category and then order the results, Polars provides a concise expression-based API for group_by, aggregation, and sorting. This article walks through the core patterns for python polars group_by aggregation and sorting, including multiple aggregations, column naming, and performance considerations.

Basic group_by and Aggregation

The fundamental operation is straightforward: call group_by on one or more columns, then agg with one or more expressions. For example, given a DataFrame of sales records, you can compute total sales per region:

import polars as pl

df = pl.DataFrame({
    "region": ["North", "South", "North", "East", "South", "East"],
    "sales": [100, 200, 150, 300, 250, 400],
})

result = df.group_by("region").agg(pl.col("sales").sum())
print(result)

The output is a new DataFrame with one row per region and a column named sales containing the sum. Polars automatically names the aggregated column after the original column. If you need a different name, use alias:

result = df.group_by("region").agg(pl.col("sales").sum().alias("total_sales"))

Sorting Aggregated Results

After aggregation, you often want the results ordered by the aggregated value. Because group_by returns a new DataFrame, you can apply sort directly on the result:

result = df.group_by("region").agg(pl.col("sales").sum().alias("total_sales"))
sorted_result = result.sort("total_sales", descending=True)
print(sorted_result)

This sorts the aggregated rows by the total_sales column in descending order. You can also sort by multiple columns, for example by region name and then by total sales:

sorted_result = result.sort(["region", "total_sales"])

Sorting after aggregation is the most common pattern because the aggregated values are not available until the agg step completes.

Multiple Aggregations and Column Naming

Often you need more than one summary statistic per group. Pass a list of expressions to agg:

result = df.group_by("region").agg([
    pl.col("sales").sum().alias("total_sales"),
    pl.col("sales").mean().alias("avg_sales"),
    pl.col("sales").count().alias("num_sales"),
])
print(result)

Without alias, Polars would try to reuse the original column name sales for multiple columns, which is invalid. Aliasing keeps each statistic distinct. You can then sort by any of these columns:

result.sort("avg_sales", descending=True)

For a quick reference, here are common aggregations available in Polars expressions:

Use alias to give descriptive names that make sorting and later operations clearer.

Sorting Within Groups

Sometimes you need to sort values inside each group before aggregating. For example, to get the last sale per region, you can sort within the group and then take the last element:

result = df.group_by("region").agg(
    pl.col("sales").sort().last().alias("last_sale")
)

This sorts the sales values in ascending order within each group and selects the last (largest) value. To get the top N per group, combine sort and head:

result = df.group_by("region").agg(
    pl.col("sales").sort(descending=True).head(2).alias("top_two_sales")
)

The result is a list column containing the two largest sales for each region. Sorting within groups is a distinct operation from sorting the aggregated output; it happens inside the agg expression.

Performance: Lazy Evaluation and Expression API

Polars is designed for speed, and its lazy API can optimize the entire pipeline before execution. Instead of chaining eager calls, build a lazy query and collect at the end:

lazy_result = (
    df.lazy()
    .group_by("region")
    .agg(pl.col("sales").sum().alias("total_sales"))
    .sort("total_sales", descending=True)
    .collect()
)

When you call collect, Polars can reorder operations, push down predicates, and reduce memory usage. For large DataFrames, this often outperforms the eager approach because the engine can avoid materializing intermediate results. Sorting after aggregation benefits from the same optimizations.

Another performance point is to avoid unnecessary group_by calls. If you need multiple statistics, combine them in a single agg rather than doing separate group_by operations and joining the results. This reduces the number of passes over the data.

Handling Nulls and Missing Groups

By default, group_by excludes rows where the grouping column is null. If you need to include null as its own group, use nulls_first or include_nulls? In Polars, the group_by method has a parameter nulls_first? Actually, the behavior is that nulls are treated as a separate group when you use group_by on a column that contains nulls. Let's verify: In Polars, group_by includes nulls as a group by default. For example:

df = pl.DataFrame({"cat": ["A", None, "A"], "val": [1, 2, 3]})
result = df.group_by("cat").agg(pl.col("val").sum())
print(result)

This produces two rows: one for "A" and one for null. If you want to drop null groups, filter them out after aggregation:

result = df.group_by("cat").agg(pl.col("val").sum()).filter(pl.col("cat").is_not_null())

When sorting, nulls appear at the end by default in ascending order. Use nulls_last or nulls_first in sort to control placement:

result.sort("total_sales", nulls_last=True)

This is useful when you want to keep missing groups at the bottom.

Common Mistakes with group_by and sort

A frequent error is trying to sort by an aggregated column before naming it. If you use pl.col("sales").sum() without alias, the resulting column is still named sales, so sort("sales") works, but if you later add another aggregation on the same column without alias, you get a duplicate column error. Always alias when using multiple aggregations.

Another mistake is sorting the original DataFrame before group_by when you actually need to sort the aggregated result. Sorting before aggregation only changes the order of rows within each group; it does not affect the order of groups in the output. To order groups, sort after agg.

Finally, be aware that group_by in Polars does not preserve the original row order. The output rows are sorted by the grouping key by default. If you need a custom order, always apply an explicit sort after aggregation. This is especially important when you combine multiple grouping columns; the default order is lexicographic by the group keys.

For complex pipelines, using the lazy API with sort after group_by keeps the intent clear and lets Polars optimize the execution plan. The expression API is consistent between eager and lazy modes, so you can switch without changing the aggregation logic.