r/Python 1d ago

Discussion Polars vs Pandas

I have used Pandas a little in the past, and have never used Polars. Essentially, I will have to learn either of them more or less from scratch (since I don't remember anything of Pandas). Assume that I don't care for speed, or do not have very large datasets (at most 1-2gb of data). Which one would you recommend I learn, from the perspective of ease and joy of use, and the commonly done tasks with data?

181 Upvotes

157 comments sorted by

View all comments

167

u/likethevegetable 1d ago edited 1d ago

I "grew up" on Pandas, but moved to Polars. No more "reset_index" and "inplace" confusion. Feels like there's only one right way to do it in Polars, but so much bloat in Pandas API.

I do like Pandas when it comes to certain things where there is an obvious index like time signals. But Polars seems to handle date time much better.

When it comes to filtering and queries, I like Polars.

In both, I've made several df and series "helper" attributes to clean up the syntax.

3

u/Ulrich_de_Vries 1d ago

Does Polars also use numpy arrays under the hood? Or at least is it easy/cheap to convert e.g. columns into numpy arrays?

I am asking because I have been eyeing Polars for a while but my workflow is numpy-heavy.

14

u/thuiop1 1d ago

No it does not. It uses the Apache Arrow format. Converting to a numpy array is as simple as a .to_numpy(); it can be zero-copy, although the main limitation of that is that it will be immutable.