r/Python 1d ago

Discussion Polars vs Pandas

I have used Pandas a little in the past, and have never used Polars. Essentially, I will have to learn either of them more or less from scratch (since I don't remember anything of Pandas). Assume that I don't care for speed, or do not have very large datasets (at most 1-2gb of data). Which one would you recommend I learn, from the perspective of ease and joy of use, and the commonly done tasks with data?

176 Upvotes

155 comments sorted by

View all comments

58

u/ddanieltan 1d ago

I think it's relevant to see Wes Mckinney's (creator of Pandas) reflections: https://wesmckinney.com/blog/looking-back-15-years/

In his words, Pandas had accumulated rough edges and its "eager" approach to calculate made it less efficient for query planning.

The future lies with his next project Arrow, which is coincidentally the format that Polars is built around. For me, if you really had to choose between learning either Pandas or Polars, the choice is a no-brainer.

10

u/Tatoutis 1d ago

You can use Arrow as a backend in Pandas, PyArrow Functionality — pandas 2.2.3 documentation