Polars vs Pandas in 2026: Performance Benchmarks, Real Numbers, and When to Switch

TL;DR

Polars is the better engine once data passes about 1 GB. In DuckDB Labs’ independent db-benchmark, Polars runs the group-by suite about 7× faster than pandas at 5 GB and the join suite about 40× faster, and pandas can’t finish the 50 GB join at all. Pandas still wins where it always did: small interactive datasets, anything that has to round-trip through scikit-learn or matplotlib, and string-heavy work. Pandas 3.0 (January 2026) closed part of the gap — PyArrow-backed strings and copy-on-write are on by default, and there’s now a pd.col() expressions API that borrows straight from Polars. The pattern that holds up in practice is to run both: Polars for the bulk transforms, Pandas for the last mile. The “should I rewrite everything?” answer is still no.

Updated September 2026: the benchmark section now uses published, reproducible results (DuckDB Labs’ db-benchmark and Polars’ PDS-H) instead of an unreproducible local run. The pandas figures in db-benchmark are from pandas 2.2; the Pandas 3.0 section explains why 3.0 doesn’t change the large-data gap.

Why this comparison keeps coming up

Polars hit 1.0 on July 1, 2024 (announcement). By early 2026 it’s at 1.x with a stable API, lazy execution that actually works, and a streaming engine that handles tables bigger than RAM. The performance gap was always real, but in 2024 the API kept moving and the surrounding tooling was sparse. Now neither of those is a blocker, which is why so many Python data teams are asking the same question: do we keep paying the Pandas tax?

The numbers below come from two public benchmarks you can rerun yourself rather than a one-off local test: DuckDB Labs’ db-benchmark, an independent suite that times group-bys and joins at 0.5, 5 and 50 GB on identical AWS hardware, and Polars’ own PDS-H, 22 analytical queries derived from TPC-H.

Polars vs Pandas: Key Differences

That table summarizes most of the friction. The performance difference is in the numbers below. For a third-party view, the DuckDB Labs db-benchmark tracks group-by and join performance across Polars, Pandas, DuckDB, and others at 0.5 GB, 5 GB, and 50 GB scales.

Performance Benchmarks

Independent: DuckDB Labs db-benchmark. Same machine for every tool (AWS c6id.metal), each suite timed end to end. The totals below sum the first run of every question in the suite, from the published raw timings. pandas 2.2.3 was last run in February 2025, Polars 1.42.1 in July 2026.

Vendor-run: Polars PDS-H. Polars’ own PDS-H benchmark runs 22 TPC-H-derived queries on a 96-vCPU AWS c7a.24xlarge. At scale factor 10 (June 2025: Polars 1.30, pandas 2.2.3), the full set took 3.89 s on Polars’ streaming engine and 365.71 s on pandas. That’s two orders of magnitude, mostly because pandas runs single-threaded on a 96-core machine. Treat it as the vendor’s best case, not a typical workload.

Three things hold across both. First, lazy and streaming execution is where the gap comes from. Polars pushes predicates into the reader, skips unused columns and runs on every core, and pandas does none of that. Second, joins show a bigger gap than group-bys. Third, the gap widens with size until pandas stops completing at all, which is the real argument for switching once your data passes a few GB.

The db-benchmark pandas rows were last run on pandas 2.2, and pandas has since shipped a consequential 3.0, so the honest question is whether it changes the tables above. For the large-data rows, no — and the next section walks through why.

What Pandas 3.0 changed (and what it doesn’t)

Pandas 3.0 landed on January 21, 2026, and it’s the biggest release since the 1.0 cut. It also drops Python 3.10 and older, so the floor is now 3.11. Three defaults changed that touch this comparison directly.

Strings are a real dtype now. pd.Series(["a", "b"]) used to give you object; in 3.0 it infers a dedicated str dtype, PyArrow-backed if PyArrow is installed. That’s the same backend that closed most of the string gap when you had to opt into it with dtype_backend="pyarrow". Now it’s the out-of-the-box default. .str.contains() and .str.lower() run several times faster than the old object path and text columns use roughly half the memory. The practical effect: string-heavy filtering, the one area where Pandas was already competitive, gets more competitive by default, without anyone touching a flag.

Copy-on-write is the default. The SettingWithCopyWarning is gone, and you no longer have to guess whether a slice is a view or a copy. Any indexing operation behaves as a copy at the API level, pandas keeps views under the hood and only copies when it has to, and the net effect is less defensive copying and a modest speedup on chained transforms. The catch, and it’s a real one for anyone porting code: chained assignment (df[df.a > 0]["b"] = 1) silently stops working instead of warning. If your codebase leaned on that pattern, budget real time for the 3.0 upgrade, because the failure is silent.

pd.col() gives Pandas an expressions API. This is the change that most weakens the case for Polars. One of the cleanest arguments for switching was always that Polars expressions compose in ways Pandas indexers never did. Pandas 3.0 now has a col() builder of its own:

import pandas as pd

df = pd.DataFrame({"a": [1, 1, 2], "b": [4, 5, 6]})

# The old lambda dance
df.assign(c=lambda d: d["a"] + d["b"])

# Pandas 3.0 expressions — reads like Polars
df.assign(c=pd.col("a") + pd.col("b"))
df.assign(name_upper=pd.col("name").str.upper())

It supports the usual operators plus Series methods and namespaces (.str, .dt), and it currently works inside assign(), loc, and getitem/setitem. It stays narrower than Polars expressions: there’s no lazy plan behind it, no query optimizer, and no cross-frame reuse. But if you liked Polars mostly for the API and your data fits in memory, pd.col() removes a chunk of the reason to switch.

The engine is the part 3.0 left untouched. Pandas is still single-threaded by default and still eager-only. The group-by and join gap in the tables above comes from Polars running multi-core with a lazy query optimizer that prunes columns and pushes predicates, and PyArrow strings, copy-on-write, and pd.col() touch none of that. db-benchmark hasn’t published pandas 3.0 timings yet, and they’re unlikely to move the large-data rows, because the axes that moved (string backend, copy semantics, expression sugar) are orthogonal to the axes that produce the large-data gap (parallelism, lazy execution, streaming). The verdict below holds. What shrank is the small-and-medium-data case for switching, which is exactly where the advice was already to stay on Pandas.

What the API actually looks like

This is where most of the “should we switch” debate lives. People look at one cherry-picked snippet and conclude either “trivial” or “rewrite everything.” Neither is right.

Here’s the same operation in both: read a CSV, filter to a date window, group by user and event type, average a value, sort the result.

# Pandas
import pandas as pd

df = pd.read_csv("events.csv", parse_dates=["ts"])
out = (
    df[df["ts"].between("2026-01-01", "2026-03-31")]
    .groupby(["user_id", "event_type"], as_index=False)["value"]
    .mean()
    .sort_values("value", ascending=False)
)

# Polars (lazy)
import polars as pl

out = (
    pl.scan_csv("events.csv", try_parse_dates=True)
    .filter(pl.col("ts").is_between(pl.date(2026, 1, 1), pl.date(2026, 3, 31)))
    .group_by(["user_id", "event_type"])
    .agg(pl.col("value").mean())
    .sort("value", descending=True)
    .collect()
)

The shape is similar. The differences that bite during a port:

• Polars uses expressions (pl.col("value").mean()) where Pandas uses string column names plus a method on a Series. Expressions compose, which means complex transforms become readable; the cost is a learning curve and a lot of pl.col(...) typing.
• No .loc, .iloc, or boolean masking on the frame itself. Filtering is .filter(expr), period. Once you’re used to it, .filter reads more clearly than boolean masks.
• scan_* is lazy, read_* is eager. Use scan_* for anything non-trivial. collect() runs the optimized plan.
• Date handling is its own thing. try_parse_dates=True works most of the time but occasionally needs .str.to_datetime() afterward.

When Pandas is the right answer

Plenty of teams rewrite working pipelines because Polars looks cool. A few cases where the switch doesn’t pay:

Small interactive notebooks. If your DataFrame fits in 1 GB and your team thinks in .loc, the speedup is microseconds you’ll never notice. The real bottleneck is Jupyter cell execution and developer fluency.

ML pipelines that touch scikit-learn, statsmodels, or anything plotly. These libraries return Pandas, expect Pandas, and document Pandas. Yes, Polars has .to_pandas() and zero-copy via Arrow, but you’ll be calling it constantly, and every conversion is a tax on readability. If you’re doing more to_pandas() than aggregation, you’re using the wrong tool.

Heavy string manipulation. Polars has solid string operations, but Pandas closed most of this gap with the PyArrow string backend — and as of 3.0 that backend is the default, no dtype_backend="pyarrow" flag required. For complex regex extraction on text-heavy frames, Pandas with PyArrow strings is competitive, and 3.0 makes that the behavior you get without configuring anything.

Code your team already understands. The API gap is real. A senior who can write any Pandas operation in their sleep needs 2-3 weeks of daily use to get fluent in Polars expressions. If the pipeline runs in 90 seconds and isn’t on the critical path, “fast enough” is fast enough.

When Polars is obviously right

The flip side. These are the cases where a rewrite pays off quickly:

ETL on tables larger than 5 GB. This is where the speedup compounds. At db-benchmark’s 5 GB scale, the group-by suite drops from 152 s to 22 s and the join suite from 131 s to 3 s, which is the difference between “schedule it overnight” and “run on demand.”

Anything memory-constrained. Streaming via LazyFrame.collect(engine="streaming") lets you process tables that don’t fit in RAM. Pandas can’t do this without DuckDB, Dask, or chunking gymnastics — and if you’re weighing that streaming engine against DuckDB’s automatic spill-to-disk, I compare the two directly in DuckDB vs Polars.

Pipelines with many sequential transforms. The lazy query planner reorders, fuses, and prunes. The same code in Pandas materializes intermediate results at every step, which is both slow and a memory liability.

New code with no Pandas debt. If you’re starting a project today and the data is non-trivial, Polars is the default. The API is more consistent, null semantics are sane, and you don’t have to remember which methods mutate.

The “use both” pattern

The pattern that holds up is one most teams recognize once they’ve tried it: Polars for the bulk transform, Pandas for the last mile. Concretely:

import polars as pl
import seaborn as sns

# Heavy lifting in Polars
result = (
    pl.scan_parquet("events/*.parquet")
    .filter(pl.col("ts") > "2026-03-01")
    .group_by(["country", "device"])
    .agg([
        pl.col("revenue").sum().alias("rev"),
        pl.col("user_id").n_unique().alias("users"),
    ])
    .sort("rev", descending=True)
    .collect(engine="streaming")
)

# Hand off to the ML / plotting world
df = result.to_pandas()
sns.barplot(df.head(20), x="rev", y="country", hue="device")

Zero-copy through Arrow means to_pandas() is effectively free for numeric and Arrow-backed string columns. The cost shows up only when you’re converting list or struct columns, which most analytics workloads don’t have.

Migration cost, honestly

The migration is often described as “trivial, APIs are similar.” That’s half-true. Here’s where the time actually goes in a typical ETL codebase:

• Index removal. Pandas indexes don’t translate. Anywhere we used the index for joins or alignment, we had to make it an explicit column. About 200 lines of changes, mostly mechanical.
• NaN vs null. Pandas treats NaN as null in some contexts but not others; Polars has real null. Everywhere we relied on .isna() semantics needed verification.
• apply / transform patterns. These mostly become pl.col(...).map_elements(), or much better, native expressions. About 30% of our .apply calls turned out to be expressible natively, which made them 50-100× faster. The rest still work but lose most of the speedup.
• Test fixtures. All our test data was Pandas. Rewriting fixtures took longer than the actual code port.

Total time: about 9 working days for a senior who already knew Polars. Net result: the daily ETL went from 47 minutes to 4 minutes, and we cut the EC2 instance size in half. Worth it. Would not have been worth it for a smaller pipeline.

Common pitfalls

A short list, in case you’re starting:

• pl.col everywhere is verbose. Use pl.col(["a", "b", "c"]) to act on multiple columns at once, and pl.exclude(...) to act on everything else.
• group_by doesn’t sort by default. Pandas does, Polars doesn’t. Add .sort(...) after if you care about deterministic ordering.
• with_columns returns a new frame. Forgetting to assign the result is the most common bug during a port. Polars frames are immutable.
• Benchmark against Pandas 3.0, not 2.x. If you’re comparing on an older Pandas, you’re measuring the object-dtype string path that 3.0 dropped as the default. Install PyArrow, use Pandas 3.0, and the string gap you “found” mostly evaporates before you write a line of Polars.
• Plot libraries hate Polars frames. Always .to_pandas() before plotting. The conversion is fast.

Decision flow

Here’s a sequence of questions to run through when a new pipeline lands on your desk:

1. Is the data under ~1 GB and mostly interactive? Stay on Pandas 3.0. With PyArrow strings and pd.col() you get most of the ergonomics and speed without a rewrite.
2. Does it have to feed scikit-learn, statsmodels, or a plotting library at every step? Pandas, or Polars with a single to_pandas() at the boundary rather than a port of the whole thing.
3. Is it ETL above ~5 GB, or does it blow past RAM? Polars, lazy, with the streaming engine. The speedup compounds here, turning an overnight job into an on-demand one.
4. Is it brand-new code with no Pandas debt? Default to Polars and skip the migration tax entirely.

That covers about 90% of the calls. The remaining 10% are odd corner cases (heavy regex, geospatial work, libraries that only speak Pandas) and they usually answer themselves once you spike the conversion cost.

Polars increasingly competes with DuckDB too — that matchup gets its own treatment in DuckDB vs Polars.

Sources

• Polars documentation — official Polars user guide and Python API reference
• Polars 1.0 announcement — release notes for the July 2024 1.0 cut
• Polars GitHub — source, issues, and release tags
• Pandas documentation — official Pandas docs
• Pandas 3.0 what’s new — copy-on-write default, str dtype default, and the pd.col() expressions API
• Pandas 2.0 what’s new — the earlier PyArrow dtype_backend introduction
• DuckDB Labs db-benchmark — ongoing dataframe benchmarks (group-by and join, 0.5-50 GB); raw timings in time.csv
• Polars PDS-H benchmark — 22 TPC-H-derived queries at SF-10 and SF-100 (Polars 1.30, pandas 2.2.3, June 2025)

FAQ

Is Polars faster than Pandas?

Yes, on large datasets. In DuckDB Labs’ db-benchmark, Polars runs the group-by suite about 7× faster than pandas at 5 GB and the join suite about 40× faster, and pandas can’t complete the 50 GB join. The gap shrinks on small data (under 1 GB) and on heavy string manipulation, where Pandas with the PyArrow backend has closed most of the difference. But for anything above a few GB, the speedup is consistent and dramatic.

Does Pandas 3.0 make Polars unnecessary?

No, but it narrows the case for switching. Pandas 3.0 (January 2026) makes PyArrow-backed strings and copy-on-write the defaults and adds a pd.col() expressions API that reads a lot like Polars. That erases two of the common reasons people cited for moving — slow object-dtype strings and the clunky indexer API — for small and medium data. What 3.0 does not change is the engine: Pandas is still single-threaded and eager-only, so on multi-gigabyte joins, group-bys, and streaming workloads Polars still wins by roughly an order of magnitude. Use Pandas 3.0 for interactive and ML-adjacent work; reach for Polars when the data is large or the pipeline is latency-sensitive.

Should I switch from Pandas to Polars?

It depends on your dataset size and ML pipeline. Switch if you’re starting a new pipeline, working with tables above 5 GB, or hitting memory limits. Don’t switch a working sub-second Pandas script just because Polars exists. If your pipeline is tightly integrated with scikit-learn, statsmodels, or plotting libraries, the constant to_pandas() conversions may negate the benefit. The migration cost is real — plan for about 2 weeks of senior developer time per 4,000 lines, mainly around index removal and NaN-vs-null semantics.

Can Polars replace Pandas completely?

Not yet, especially for the ML last-mile. scikit-learn, statsmodels, matplotlib, seaborn, and most BI tools still expect Pandas DataFrames. Polars has .to_pandas() and .to_numpy() for the handoff (zero-copy for numeric columns), but if you’re converting at every step, you lose the ergonomic advantage. The practical pattern in 2026 is using both: Polars for the heavy transforms and Pandas for the boundary where ML and plotting libraries live. Full replacement will require the ML libraries themselves to adopt Arrow-native inputs, which is happening slowly but isn’t there yet.

What is the difference between Polars and Pandas?

The core difference is the engine: Polars is built on Rust with Apache Arrow columnar memory, while Pandas is built on NumPy with a Python-centric memory model. This leads to several practical differences. Polars is multi-core by default; Pandas is single-threaded. Polars supports lazy execution with a query optimizer that reorders filters, pushes predicates, and skips unused columns; Pandas only does eager evaluation. Polars uses an expressions API (pl.col("x").mean()); Pandas uses indexers (.loc, .iloc) and method chains. Polars has first-class null handling; Pandas has the NaN-as-null ambiguity. And Polars can stream datasets larger than RAM natively, which Pandas cannot without external tools like Dask or DuckDB.

When should I use Polars instead of Pandas?

Use Polars when data is large (>5 GB), when you need streaming/lazy evaluation, when you care about ETL latency, or when you’re starting fresh. Stick with Pandas for interactive small-data work, ML pipelines that depend on scikit-learn, and code your team already understands.

Does Polars work with scikit-learn?

Not directly. Most scikit-learn estimators expect NumPy arrays or Pandas DataFrames. Polars has .to_pandas() and .to_numpy() for the handoff, both of which are zero-copy for numeric columns thanks to the Arrow backend. The pattern is: Polars for preprocessing, convert at the boundary, fit with scikit-learn.

Is Pandas being replaced by Polars?

Not in 2026, and probably not soon. Pandas has 15 years of library inertia. Every Python data tutorial, ML library, and BI tool speaks it. Polars is winning the new-pipeline battle and the performance battle, but Pandas isn’t going anywhere. The realistic outcome is coexistence: Polars where you need speed, Pandas where the surrounding libraries are.

Bottom line

Polars in 2026 is the right default for new data pipelines and a serious option for any production ETL above a few GB. The speedup is real, reproducible in public benchmarks, and largest on the joins and group-bys that dominate production ETL. The migration cost is also real and shouldn’t be hand-waved.

If you’re sitting on Pandas pipelines that run fine, leave them alone. If you’re writing something new, start in Polars. If you’re somewhere in between (a slow daily job that nobody loves), port it and reclaim the time. Just don’t expect the API to be a drop-in. Treat it like learning a sibling language rather than a renamed library.

If you want to see how the Python tooling story is shifting more broadly, I wrote about uv vs pip vs Poetry for package management and Python 3.14 free-threading benchmarks.