Polars vs Pandas in 2026: Performance Benchmarks, Real Numbers, and When to Switch

TL;DR

Polars is the better engine on every benchmark above ~1 GB. In my 240M-row run, joins and group-bys landed roughly 10× faster and Parquet reads about 5×. Pandas still wins where it always did: small interactive datasets, anything that has to round-trip through scikit-learn or matplotlib, and string-heavy work. Pandas 3.0 (January 2026) closed part of the gap — PyArrow-backed strings and copy-on-write are on by default, and there’s now a pd.col() expressions API that borrows straight from Polars. After porting two production pipelines I run both: Polars for the bulk transforms, Pandas for the last mile. The “should I rewrite everything?” answer is still no.

Last updated July 2026, versions and benchmarks re-verified August 2026. The July refresh added the Pandas 3.0 changes (PyArrow strings and copy-on-write now default); the August pass re-checked the version labels and benchmark figures below, and they still hold. The benchmark run is Polars 1.18 vs Pandas 2.2; the engine-level gap is unchanged on Polars 1.43 / Pandas 3.0, for reasons the Pandas 3.0 section explains.

Why this comparison keeps coming up

Polars hit 1.0 on July 1, 2024 (announcement). By early 2026 it’s at 1.x with a stable API, lazy execution that actually works, and a streaming engine that handles tables bigger than RAM. The performance gap was always real, but in 2024 the API kept moving and the surrounding tooling was sparse. Now neither of those is a blocker, which is why every Python data team I talk to is asking the same question: do we keep paying the Pandas tax?

I spent about two months in early 2026 running a serious comparison on a real workload: about 240 million rows of clickstream data spread across 18 Parquet files, with the kinds of joins, aggregations, and filtering that show up in actual ETL. Below is what I measured and where I landed.

Polars vs Pandas: Key Differences

That table summarizes most of the friction. The performace difference is in the numbers below. For a third-party view, the DuckDB Labs db-benchmark tracks group-by and join performance across Polars, Pandas, DuckDB, and others at 0.5 GB, 5 GB, and 50 GB scales.

Performance Benchmarks

Workload: 240M-row Parquet dataset, 7 numeric columns, 3 string columns, ~14 GB on disk. Hardware: M2 Pro, 16-core, 32 GB RAM. Each operation ran 5 times; I report the median.

A few observations from running this. First, lazy mode is not optional. Eager Polars is already fast, but the query optimizer routinely shaved another 30-60% off by reordering filters, pushing predicates into the Parquet reader, and skipping unused columns. Second, the string operation gap is small. If your pipeline is 80% regex parsing, the speedup story falls apart. Third, the join numbers held even when one side was big enough to make Pandas swap to disk.

I ran those numbers on Pandas 2.2 and Polars 1.18. The version labels have moved since (Polars is on 1.43, and Pandas shipped a consequential 3.0), so the honest question is whether any of it changes the table above. For the large-data rows, no — and the next section walks through why.

What Pandas 3.0 changed (and what it doesn’t)

Pandas 3.0 landed on January 21, 2026, and it’s the biggest release since the 1.0 cut. It also drops Python 3.10 and older, so the floor is now 3.11. Three defaults changed that touch this comparison directly.

Strings are a real dtype now. pd.Series(["a", "b"]) used to give you object; in 3.0 it infers a dedicated str dtype, PyArrow-backed if PyArrow is installed. That’s the same backend that closed most of the string gap when you had to opt into it with dtype_backend="pyarrow". Now it’s the out-of-the-box default. .str.contains() and .str.lower() run several times faster than the old object path and text columns use roughly half the memory. The practical effect: the one benchmark row where Pandas was competitive (string contains + filter, 1.3× in my run) gets more competitive by default, without anyone touching a flag.

Copy-on-write is the default. The SettingWithCopyWarning is gone, and you no longer have to guess whether a slice is a view or a copy. Any indexing operation behaves as a copy at the API level, pandas keeps views under the hood and only copies when it has to, and the net effect is less defensive copying and a modest speedup on chained transforms. The catch, and it’s a real one for anyone porting code: chained assignment (df[df.a > 0]["b"] = 1) silently stops working instead of warning. If your codebase leaned on that pattern, budget real time for the 3.0 upgrade, because the failure is silent.

pd.col() gives Pandas an expressions API. This is the change that most weakens the case for Polars. One of the cleanest arguments for switching was always that Polars expressions compose in ways Pandas indexers never did. Pandas 3.0 now has a col() builder of its own:

import pandas as pd

df = pd.DataFrame({"a": [1, 1, 2], "b": [4, 5, 6]})

# The old lambda dance
df.assign(c=lambda d: d["a"] + d["b"])

# Pandas 3.0 expressions — reads like Polars
df.assign(c=pd.col("a") + pd.col("b"))
df.assign(name_upper=pd.col("name").str.upper())

It supports the usual operators plus Series methods and namespaces (.str, .dt), and it currently works inside assign(), loc, and getitem/setitem. It stays narrower than Polars expressions: there’s no lazy plan behind it, no query optimizer, and no cross-frame reuse. But if you liked Polars mostly for the API and your data fits in memory, pd.col() removes a chunk of the reason to switch.

The engine is the part 3.0 left untouched. Pandas is still single-threaded by default and still eager-only. The 10× on group-bys and joins in the table above comes from Polars running multi-core with a lazy query optimizer that prunes columns and pushes predicates, and PyArrow strings, copy-on-write, and pd.col() touch none of that. I didn’t re-run the full 240M-row suite on 3.0 for exactly that reason: the axes that moved (string backend, copy semantics, expression sugar) are orthogonal to the axes that produce the large-data gap (parallelism, lazy execution, streaming). The verdict below holds. What shrank is the small-and-medium-data case for switching, which is exactly where I was already telling people to stay on Pandas.

What the API actually looks like

This is where most of the “should we switch” debate lives. People look at one cherry-picked snippet and conclude either “trivial” or “rewrite everything.” Neither is right.

Here’s the same operation in both: read a CSV, filter to a date window, group by user and event type, average a value, sort the result.

# Pandas
import pandas as pd

df = pd.read_csv("events.csv", parse_dates=["ts"])
out = (
    df[df["ts"].between("2026-01-01", "2026-03-31")]
    .groupby(["user_id", "event_type"], as_index=False)["value"]
    .mean()
    .sort_values("value", ascending=False)
)

# Polars (lazy)
import polars as pl

out = (
    pl.scan_csv("events.csv", try_parse_dates=True)
    .filter(pl.col("ts").is_between(pl.date(2026, 1, 1), pl.date(2026, 3, 31)))
    .group_by(["user_id", "event_type"])
    .agg(pl.col("value").mean())
    .sort("value", descending=True)
    .collect()
)

The shape is similar. The differences that bite during a port:

• Polars uses expressions (pl.col("value").mean()) where Pandas uses string column names plus a method on a Series. Expressions compose, which means complex transforms become readable; the cost is a learning curve and a lot of pl.col(...) typing.
• No .loc, .iloc, or boolean masking on the frame itself. Filtering is .filter(expr), period. After two months I miss boolean-mask filtering exactly zero times. .filter is clearer.
• scan_* is lazy, read_* is eager. Use scan_* for anything non-trivial. collect() runs the optimized plan.
• Date handling is its own thing. try_parse_dates=True works most of the time but occasionally needs .str.to_datetime() afterward.

When Pandas is the right answer

I’ve watched too many teams rewrite working pipelines because Polars looked cool. A few cases where I push back on the switch:

Small interactive notebooks. If your DataFrame fits in 1 GB and your team thinks in .loc, the speedup is microseconds you’ll never notice. The real bottleneck is Jupyter cell execution and developer fluency.

ML pipelines that touch scikit-learn, statsmodels, or anything plotly. These libraries return Pandas, expect Pandas, and document Pandas. Yes, Polars has .to_pandas() and zero-copy via Arrow, but you’ll be calling it constantly, and every conversion is a tax on readability. If you’re doing more to_pandas() than aggregation, you’re using the wrong tool.

Heavy string manipulation. Polars has solid string operations, but Pandas closed most of this gap with the PyArrow string backend — and as of 3.0 that backend is the default, no dtype_backend="pyarrow" flag required. For complex regex extraction on text-heavy frames, I’ve seen Pandas come out ahead by 10-20%, and 3.0 makes that the behavior you get without configuring anything.

Code your team already understands. The API gap is real. A senior who can write any Pandas operation in their sleep needs 2-3 weeks of daily use to get fluent in Polars expressions. If the pipeline runs in 90 seconds and isn’t on the critical path, “fast enough” is fast enough.

When Polars is obviously right

The flip side. These are the cases where I rewrite without hesitating:

ETL on tables larger than 5 GB. This is where the 10× speedup compounds. A 30-minute Pandas job becomes a 3-minute Polars job and your pipeline goes from “schedule it overnight” to “run on demand.”

Anything memory-constrained. Streaming via LazyFrame.collect(engine="streaming") lets you process tables that don’t fit in RAM. Pandas can’t do this without DuckDB, Dask, or chunking gymnastics — and if you’re weighing that streaming engine against DuckDB’s automatic spill-to-disk, I compare the two directly in DuckDB vs Polars.

Pipelines with many sequential transforms. The lazy query planner reorders, fuses, and prunes. The same code in Pandas materializes intermediate results at every step, which is both slow and a memory liability.

New code with no Pandas debt. If you’re starting a project today and the data is non-trivial, Polars is the default. The API is more consistent, null semantics are sane, and you don’t have to remember which methods mutate.

The “use both” pattern

After two months of porting I landed on a pattern most teams will recognize once they’ve tried it: Polars for the bulk transform, Pandas for the last mile. Concretely:

import polars as pl
import seaborn as sns

# Heavy lifting in Polars
result = (
    pl.scan_parquet("events/*.parquet")
    .filter(pl.col("ts") > "2026-03-01")
    .group_by(["country", "device"])
    .agg([
        pl.col("revenue").sum().alias("rev"),
        pl.col("user_id").n_unique().alias("users"),
    ])
    .sort("rev", descending=True)
    .collect(engine="streaming")
)

# Hand off to the ML / plotting world
df = result.to_pandas()
sns.barplot(df.head(20), x="rev", y="country", hue="device")

Zero-copy through Arrow means to_pandas() is effectively free for numeric and Arrow-backed string columns. The cost shows up only when you’re converting list or struct columns, which most analytics workloads don’t have.

Migration cost, honestly

I’d been told the migration was “trivial, APIs are similar.” That was half-true. Here’s what actually took time on a 4,000-line internal ETL package:

• Index removal. Pandas indexes don’t translate. Anywhere we used the index for joins or alignment, we had to make it an explicit column. About 200 lines of changes, mostly mechanical.
• NaN vs null. Pandas treats NaN as null in some contexts but not others; Polars has real null. Everywhere we relied on .isna() semantics needed verification.
• apply / transform patterns. These mostly become pl.col(...).map_elements(), or much better, native expressions. About 30% of our .apply calls turned out to be expressible natively, which made them 50-100× faster. The rest still work but lose most of the speedup.
• Test fixtures. All our test data was Pandas. Rewriting fixtures took longer than the actual code port.

Total time: about 9 working days for a senior who already knew Polars. Net result: the daily ETL went from 47 minutes to 4 minutes, and we cut the EC2 instance size in half. Worth it. Would not have been worth it for a smaller pipeline.

Common pitfalls I hit

A short list, in case you’re starting:

• pl.col everywhere is verbose. Use pl.col(["a", "b", "c"]) to act on multiple columns at once, and pl.exclude(...) to act on everything else.
• group_by doesn’t sort by default. Pandas does, Polars doesn’t. Add .sort(...) after if you care about deterministic ordering.
• with_columns returns a new frame. Forgetting to assign the result is the most common bug during a port. Polars frames are immutable.
• Benchmark against Pandas 3.0, not 2.x. If you’re comparing on an older Pandas, you’re measuring the object-dtype string path that 3.0 dropped as the default. Install PyArrow, use Pandas 3.0, and the string gap you “found” mostly evaporates before you write a line of Polars.
• Plot libraries hate Polars frames. Always .to_pandas() before plotting. The conversion is fast.

Decision flow

Here’s the actual sequence of questions I run through when a new pipeline lands on my desk:

1. Is the data under ~1 GB and mostly interactive? Stay on Pandas 3.0. With PyArrow strings and pd.col() you get most of the ergonomics and speed without a rewrite.
2. Does it have to feed scikit-learn, statsmodels, or a plotting library at every step? Pandas, or Polars with a single to_pandas() at the boundary rather than a port of the whole thing.
3. Is it ETL above ~5 GB, or does it blow past RAM? Polars, lazy, with the streaming engine. The 10× compounds here, turning an overnight job into an on-demand one.
4. Is it brand-new code with no Pandas debt? Default to Polars and skip the migration tax entirely.

That covers about 90% of the calls. The remaining 10% are odd corner cases (heavy regex, geospatial work, libraries that only speak Pandas) and they usually answer themselves once you spike the conversion cost.

Polars increasingly competes with DuckDB too — that matchup gets its own treatment in DuckDB vs Polars.

Sources

• Polars documentation — official Polars user guide and Python API reference
• Polars 1.0 announcement — release notes for the July 2024 1.0 cut
• Polars GitHub — source, issues, and release tags
• Pandas documentation — official Pandas docs
• Pandas 3.0 what’s new — copy-on-write default, str dtype default, and the pd.col() expressions API
• Pandas 2.0 what’s new — the earlier PyArrow dtype_backend introduction
• DuckDB Labs db-benchmark — ongoing dataframe benchmarks (group-by and join, 0.5-50 GB)

FAQ

Is Polars faster than Pandas?

Yes, 3-11x faster on large datasets. In my 240M-row benchmarks, Polars delivered 10x on group-bys and joins, up to 11x on sorting and filtering, and 4.7x on Parquet reads. The gap shrinks on small data (under 1 GB) and on heavy string manipulation, where Pandas with the PyArrow backend has closed most of the difference. But for anything above a few GB, the speedup is consistent and dramatic.

Does Pandas 3.0 make Polars unnecessary?

No, but it narrows the case for switching. Pandas 3.0 (January 2026) makes PyArrow-backed strings and copy-on-write the defaults and adds a pd.col() expressions API that reads a lot like Polars. That erases two of the common reasons people cited for moving — slow object-dtype strings and the clunky indexer API — for small and medium data. What 3.0 does not change is the engine: Pandas is still single-threaded and eager-only, so on multi-gigabyte joins, group-bys, and streaming workloads Polars still wins by roughly an order of magnitude. Use Pandas 3.0 for interactive and ML-adjacent work; reach for Polars when the data is large or the pipeline is latency-sensitive.

Should I switch from Pandas to Polars?

It depends on your dataset size and ML pipeline. Switch if you’re starting a new pipeline, working with tables above 5 GB, or hitting memory limits. Don’t switch a working sub-second Pandas script just because Polars exists. If your pipeline is tightly integrated with scikit-learn, statsmodels, or plotting libraries, the constant to_pandas() conversions may negate the benefit. The migration cost is real — plan for about 2 weeks of senior developer time per 4,000 lines, mainly around index removal and NaN-vs-null semantics.

Can Polars replace Pandas completely?

Not yet, especially for the ML last-mile. scikit-learn, statsmodels, matplotlib, seaborn, and most BI tools still expect Pandas DataFrames. Polars has .to_pandas() and .to_numpy() for the handoff (zero-copy for numeric columns), but if you’re converting at every step, you lose the ergonomic advantage. The practical pattern in 2026 is using both: Polars for the heavy transforms and Pandas for the boundary where ML and plotting libraries live. Full replacement will require the ML libraries themselves to adopt Arrow-native inputs, which is happening slowly but isn’t there yet.

What is the difference between Polars and Pandas?

The core difference is the engine: Polars is built on Rust with Apache Arrow columnar memory, while Pandas is built on NumPy with a Python-centric memory model. This leads to several practical differences. Polars is multi-core by default; Pandas is single-threaded. Polars supports lazy execution with a query optimizer that reorders filters, pushes predicates, and skips unused columns; Pandas only does eager evaluation. Polars uses an expressions API (pl.col("x").mean()); Pandas uses indexers (.loc, .iloc) and method chains. Polars has first-class null handling; Pandas has the NaN-as-null ambiguity. And Polars can stream datasets larger than RAM natively, which Pandas cannot without external tools like Dask or DuckDB.

When should I use Polars instead of Pandas?

Use Polars when data is large (>5 GB), when you need streaming/lazy evaluation, when you care about ETL latency, or when you’re starting fresh. Stick with Pandas for interactive small-data work, ML pipelines that depend on scikit-learn, and code your team already understands.

Does Polars work with scikit-learn?

Not directly. Most scikit-learn estimators expect NumPy arrays or Pandas DataFrames. Polars has .to_pandas() and .to_numpy() for the handoff, both of which are zero-copy for numeric columns thanks to the Arrow backend. The pattern is: Polars for preprocessing, convert at the boundary, fit with scikit-learn.

Is Pandas being replaced by Polars?

Not in 2026, and probably not soon. Pandas has 15 years of library inertia. Every Python data tutorial, ML library, and BI tool speaks it. Polars is winning the new-pipeline battle and the performance battle, but Pandas isn’t going anywhere. The realistic outcome is coexistence: Polars where you need speed, Pandas where the surrounding libraries are.

Bottom line

Polars in 2026 is the right default for new data pipelines and a serious option for any production ETL above a few GB. The 10× speedup is real and durable across the operations you actually run in production. The migration cost is also real and shouldn’t be hand-waved.

If you’re sitting on Pandas pipelines that run fine, leave them alone. If you’re writing something new, start in Polars. If you’re somewhere in between (a slow daily job that nobody loves), port it and reclaim the time. Just don’t expect the API to be a drop-in. Treat it like learning a sibling language rather than a renamed library.

If you want to see how the Python tooling story is shifting more broadly, I wrote about uv vs pip vs Poetry for package management and Python 3.14 free-threading benchmarks. Polars and free-threading actually compose better than I expected, and that’s worth its own post.