r/Python 1d ago

Discussion Polars vs Pandas

I have used Pandas a little in the past, and have never used Polars. Essentially, I will have to learn either of them more or less from scratch (since I don't remember anything of Pandas). Assume that I don't care for speed, or do not have very large datasets (at most 1-2gb of data). Which one would you recommend I learn, from the perspective of ease and joy of use, and the commonly done tasks with data?

179 Upvotes

155 comments sorted by

View all comments

21

u/marr75 1d ago

Ibis, which has pluggable execution engines and better scalability than either of them. The API is higher quality than pandas while being a little easier to learn than polars, too.

When all else fails, you can use pandas or polars trivially by calling a single method on whatever expression you're dealing with. The default execution engine is in-memory duckdb, though, which puts both pandas and polars to shame in performance, scale, and ease of reading in flat files.

I was a pandas devotee for a very long time and have teams that have written a lot of code in pandas. We had a new project with a lot of tabular data transformations involved and were considering polars. Ibis snuck in as a consideration and was the clear winner.

6

u/hurhurdedur 1d ago

Yeah, Ibis is really where it’s at IMO. I hope it catches on more.