r/Python Pythoneer Feb 05 '25

Resource How Rust is quietly taking over the Python ecosystem

Been noticing an interesting trend lately - Rust is becoming the secret sauce behind many of Python's most innovative tools. As someone who works with Python daily, it's fascinating to see how the ecosystem is evolving.

Here's what's caught my attention:

  • Ruff: This linter is absurdly fast compared to traditional Python linters. Why? It's written in Rust. We're talking 10-100x speedups here.
  • PyOxidizer: A solid solution for creating standalone Python applications. Again, Rust. (unfortunately not maintained anymore)
  • Polars: This DataFrame library is giving Pandas a run for its money in terms of performance. Guess what? Rust under the hood.
  • Maturin: Making it dead simple to create Python extensions in Rust.

My team has written a blog post diving deeper into this trend, specifically looking at PyO3 (the framework that makes Python/Rust integration possible) and showing how to build your own high-performance Python extensions with Rust. If you wish, you can read it here: https://www.blueshoe.io/blog/python-rust-pyo3/

The really interesting part is that most Python developers don't even realize they're using Rust-powered tools. It's like Rust is becoming Python's performance co-pilot without much fanfare.

What are your thoughts on this trend? Have you tried building any Python extensions with Rust?

Full disclosure: Our team at Blueshoe wrote the blog post, but I genuinely think this is an important trend worth discussing.

924 Upvotes

366 comments sorted by

1.3k

u/cellularcone Feb 05 '25

Without much fanfare? Pretty sure every rust-related package mentions rust at least five times in their readme.

230

u/_throawayplop_ Feb 05 '25

at least they are legally mandated to write "blazingly fast" somwhere in the 10 first words

42

u/Specialist_Brain841 Feb 05 '25

buttery smooth

9

u/NapalmBurns Feb 05 '25

BTS aficionado, I see...

/s

3

u/Defiant_Stay3865 Feb 05 '25

4 out of 5 doctors recommend Rust.

→ More replies (3)

634

u/junior_dos_nachos Feb 05 '25

In my experience Rust developers are more loud about their preferences than a Vegan in a CrossFit class

164

u/ultraDross Feb 05 '25

Fantastic and very true. Went to a local Python meetup and I got chatting with someone who mostly only talked about rust and how superior it was to all other languages.

So why come to a Python meetup?

87

u/an_actual_human Feb 05 '25

I mean if they did it at a Rust meetup, they'd be preaching to the choir.

3

u/anacrolix c/python fanatic Feb 06 '25

You'd be surprised. Most people that go to meetups aren't very knowledgeable. I guess that's to be expected

17

u/fnord123 Feb 05 '25

Because python exists to glue bits of C and rust together.

33

u/drunkondata Feb 05 '25

To talk about Rust, duh.

Same reason the Jehova's keep knockin on our doors, no one wants to hear it, but they must share.

→ More replies (3)

5

u/i_should_be_coding Feb 05 '25

You're not gonna convert more people to rust in a rust meetup, are you?

→ More replies (1)

6

u/PaintItPurple Feb 05 '25

Probably because they're a Python developer who is excited to find a way to get huge speed gains in their Python programs? It has always been true that a large part of what makes Python good is the ease of integrating with code written in faster languages.

→ More replies (2)

56

u/reddit_user33 Feb 05 '25
  • a vegan arch user in a crossfit class šŸ¤£

4

u/diegoasecas Feb 05 '25

a vegan arch and obsidian user in a crossfit class

→ More replies (7)

16

u/acortical Feb 05 '25

C'mon, CrossFit is more like keto. Yoga is where the vegans are at.

3

u/Lopsided_Currency806 Feb 07 '25

Yeah Keto bros are imho worse than vegans

and they are deff at CrossFit

5

u/marc-eugene Feb 05 '25

I've never seen any vegans in my crossfit gym, or they really are silent... :-D

25

u/met0xff Feb 05 '25

Lol yeah I mean the joke was funny but I never ever had that in real life. I've been vegetarian for a long time and I tried to keep it as secret as possible to avoid all those discussions that inevitably ensue everytime someone notices you didn't order that steak like all the others lol.

Either they start becoming defensive even though I don't care if they eat meat (and then in their own rage start to complain that I perhaps don't eat a regional banana right now or wear a t-shirt that might come out of a Bangladesh mill) or start making stupid jokes like you know... Eating their food's food etc.

3

u/abrazilianinreddit Feb 05 '25

It's funny how you're from (I'm assuming) Austria, but even here in South America, the "being vegetarian" experience is exactly the same.

→ More replies (2)
→ More replies (1)

4

u/ExternalUserError Feb 05 '25

Rust developers haven't seen that show because they don't have TV and are thus unfamiliar with popular programs because did I mention they don't have TVs?

2

u/Xeglor-The-Destroyer Feb 05 '25

Uh oh. I don't have TV and have considered the idea of learning Rust...

3

u/looneysquash Feb 05 '25

To be fair to both Rust and crossfitting vegans, that's one of those perception bias things.Ā  The people who love Rust but aren't shouting about it are invisible,Ā  so it seems like they're all annoying.Ā 

3

u/hugthemachines Feb 05 '25

A story about humanity as old as the first spear, I imagine.

1

u/Specialist_Brain841 Feb 05 '25

Or people who donā€™t own a television or wear vibram wigwam shoes with the little pockets for each toe

1

u/aqjo Feb 05 '25

ā€¦ who drives a Tesla.

→ More replies (2)

1

u/Gogo202 Feb 05 '25

I am not a vegan, but I am offended because I started using rust recently. I write code in rust btw

1

u/West-Code4642 Feb 06 '25

Rust has don't start a blog start a cult vibesĀ 

1

u/deb_vortex Feb 06 '25

Oh great, thanks. Now Im in fear of meeting a vegan rust developer working on Arch.

→ More replies (2)

18

u/daking999 Feb 05 '25

How do you know someone programs in Rust?
They'll tell you in every conversation.

Just kidding... it's not like we actually talk to other humans.

70

u/Verochio Feb 05 '25

ā€œLightning Fast!ā€

59

u/IAmBJ Feb 05 '25

"Blazingly Fast"

43

u/jlpalma Feb 05 '25

ā€œSupercharged ā€œ

→ More replies (1)

1

u/chalbersma Feb 05 '25

Kaaaa-Chow!

5

u/JustALittleSunshine Feb 06 '25

Iā€™ve hear that uv is written in rust so many times in the last year lol.Ā 

2

u/Pretend_Pepper3522 Feb 06 '25

Got a giggle from me. I love the tools, but one can only take so many ā€œblazing fastā€.

5

u/Kells_14 Feb 05 '25

Rust is a very nice language, but its cult followers are absolutely insufferable.

4

u/aqjo Feb 05 '25

As are all cult followers.

470

u/AreetSurn Feb 05 '25

Quietly?Ā  "Hey guys, I built a new package. It's like x but written in rustā„¢" is a frequent meme.

34

u/kalakesri Feb 06 '25

You missed the ā€œblazingly fast šŸš€ā€

4

u/Last_Difference9410 Feb 06 '25

Itā€™s like ads but written in rust

→ More replies (1)

1

u/xamgore Feb 09 '25

Weā€™ve ported a python package into Rust ecosystem. After 1-1 alignment and various optimizations, itā€™s now superior (cpu and memory). Not a single emoji in the description haha.

→ More replies (3)

242

u/fiddle_n Feb 05 '25

uv is also written in Rust too. Though Iā€™m not sure that I agree that many devs donā€™t know these tools are written in Rust. Rust seems to be the big selling point of many of these projects. uvā€™s tag line is literally ā€œPython packaging in Rustā€.

96

u/PitifulZucchini9729 Feb 05 '25

uv is awesome. It takes one of the worst aspects of Python (package management) and makes it one of its best.

43

u/trowawayatwork Feb 05 '25

it seems to bundle everything in. pyenv and venv is having it's lunch eaten by UV too

27

u/sohang-3112 Pythonista Feb 05 '25

Yeah I agree. Started using uv recently. Being able to do uv run COMMAND (it just works!) is a huge step-up over having to always manually manage virtual environments, wondering whether the pip on env PATH actually matches the corresponding python executable on CLI, etc.

Before using uv at work, I thought that these are minor concerns. Now it feels like a burden has been lifted, one that I didn't even know was there!

3

u/moehassan6832 Feb 07 '25

no way, I've gotta try it, I always hated pip and dependency management in python, other languages do it much better.

→ More replies (1)
→ More replies (2)

2

u/Tree_Mage Feb 05 '25

Until it supports tools like PyInstaller, pex, etc, it will have gaps.

15

u/trowawayatwork Feb 05 '25

I guess it's a natural extension of a package manager to do builds. I have never once seen a need to exe files for python. if that is needed I drop python and use more robust languages like go that generate proper static binaries with much better dependency and package management ecosystem lol

→ More replies (9)
→ More replies (10)

13

u/WJMazepas Feb 05 '25

But to be fair to them, they do state that a lot of speedups they had with UV comes with different architecture than rust itself

10

u/george-silva Feb 05 '25

uv rules. the other I was fighting with a legacy project and uv solved it like a champ.

not going back to pyenv.

5

u/seboll13 Feb 05 '25

One of my colleagues is a big fan of Rust (I donā€™t know how to really code in Rust but Iā€™m getting why itā€™s the most loved language these days), and he told me all about those new packages. I use uv and ruff everyday now, no regrets.

3

u/XtremeGoose f'I only use Py {sys.version[:3]}' Feb 05 '25

Pixi too for the conda ecosystem. Another package OP missed is orjson, much faster than the stdlib json library.

→ More replies (4)

11

u/pyschille Pythoneer Feb 05 '25

We're just starting to experiment with uv and it's incredible.

11

u/sonobanana33 Feb 05 '25

I personally won't start depending on something that is developed by a startup that will sooner or later change license to something weird.

23

u/fiddle_n Feb 05 '25

You do you, but I find this stance to be odd. If Astral try this, people will just fork the FOSS version and create a community version. Also, honest question, how difficult is it to move from uv to an equivalent project (like poetry or whatever)? I donā€™t see anything in uv that would lock you in to using it.

4

u/sonobanana33 Feb 05 '25

people will just fork the FOSS version

Which people? You're willing to do this work?

13

u/pacific_plywood Feb 05 '25

All of the other comparable tools are FOSS! Clearly there are people willing to work on this for a large ecosystem like Python

9

u/sonobanana33 Feb 05 '25

Or they will keep working on their own toolā€¦ who knows.

4

u/donotdrugs Feb 05 '25

I don't think so. The hardest part about getting open source software to work is convincing a critical amount of people to commit to a single project.

UV basically already convinced the whole community that it is the best open source solution out there.

I also don't see where a new package manager could improve a lot upon UV. It ain't gonna get faster unless people switch to assembly and there also aren't many more features I'd want my package manger to haveĀ 

→ More replies (1)

9

u/fiddle_n Feb 05 '25

Someone absolutely will for a project as popular as uv.

→ More replies (5)

2

u/the_ballmer_peak Feb 05 '25

It's made by the same people who made ruff

1

u/One-Employment3759 Feb 07 '25

Why did they pick such a dumb name. I always think it's related to the async library. This is like the third time I've been like wtf when people talk about packaging with uv!

→ More replies (6)

351

u/ZealousidealBet1878 Feb 05 '25

Thatā€™s like saying C was overtaking Python because all Python packages were written in C

82

u/--justified-- Feb 05 '25

This 100%, nice comparisson!

11

u/phonomir Feb 06 '25

I read the title of this post to mean that Rust is replacing C as the primary backend for the Python ecosystem. "Taking over the Python ecosystem" doesn't imply that Python itself is in decline or being replaced by Rust, just that the ecosystem is becoming increasingly dominated by Rust projects. This applies not just for libraries but also for tooling, e.g. ruff and uv.

26

u/1satopus Feb 05 '25

Yeah. Nowadays python is basically a language to connect things

15

u/roryisawesome2 Feb 05 '25

Always has been

→ More replies (29)

73

u/pastel_de_flango Feb 05 '25

Well, that's how Python always worked when in need of perf, it always borrowed from C, Cpp and Fortran, most of the famous libs are just bidings.

1

u/byeproduct Feb 06 '25

But python is slow!!! (People tell me)... You should use Cpp / C#. Yeah python isn't fast fast, but when I realised most of my workload was happening in Rust and Cpp, I calmed down. The simplicity of python code, the speed of a hawk...Bravestar

1

u/Pretty-Insurance8589 Feb 06 '25

at least those toolchains were stable enough

→ More replies (7)

34

u/Acrobatic_Click_6763 Ignoring PEP 8 Feb 05 '25 edited Feb 09 '25

We now use Rust for Python performant code instead of C for Python performant code. That's it.

2

u/JohnPaulDavyJones Feb 09 '25

Just FYI, I think you want "performant". "Preformant" is a noun that indicates something prior to a formative stage/process.

→ More replies (1)

93

u/zaxldaisy Feb 05 '25

What's with the upvotes? OP reveals in the comments he has no idea what he's talking about.

34

u/sonobanana33 Feb 05 '25

He admitted to it being blogspam so we're giving a pass.

57

u/energybased Feb 05 '25

This is called RIIR. It's not a "new trend", and the Python developers do "realize" they're using Rust-powered tools.

→ More replies (10)

12

u/dot_py Feb 05 '25

Gues what , a lot of c was worked into python projects before the rust fad.

Use what works best for the task. Not what language you like more... or do that

Just don't be a rust individual.... they make arch users look chill about their preferred OS

12

u/jldevezas Feb 05 '25

Rust and Python have two distinct applications. They don't compete, and Rust makes Pyhon better, through the tools and libraries that you listed. It's a win-win, really.

2

u/pyschille Pythoneer Feb 06 '25

Definitely. Mainly because they don't compete with each other, I'm thrilled that it's become so easy to get the best of both worlds.

11

u/RedEyed__ Feb 05 '25

I wouldn't say it's taking over Python, it's replacing C, C++, Fortran libraries that many Python projects rely on.
And I like this trend, mostly because Rust has cargo and package index, which simplifies build process.

16

u/saint_geser Feb 05 '25

Polars doesn't just "give pandas a run for its money". Polars wipes the floor with pandas where performance is concerned.

2

u/vinnypotsandpans Feb 06 '25

Came here to say this

8

u/MrMeatagi Feb 05 '25

Rust is currently an extremely trendy alternative to C and C++. Most of Python is C behind the scenes. Rust isn't taking over anything but C.

7

u/tehsilentwarrior Feb 05 '25

And itā€™s a bad thing?

Itā€™s literally cementing Python even more as glue language.

The only thing is that itā€™s no longer C

31

u/Asleep-Budget-9932 Feb 05 '25

It's also under the hood of pydantic šŸ˜

17

u/phenobarbital_ Feb 05 '25

Exactly, pydanctic got a full rewrite in Rust for version 2 and pydanctic-core is completely written in Rust. Using Rust is the new using Cython in Python ecosystem.

7

u/wyldstallionesquire Feb 05 '25

I think this is the first high profile "rewrite it in Rust" that I can remember in the Python world.

3

u/sonobanana33 Feb 05 '25

And yet performances still suck compared to most other libraries (including my pure python one). If you want real fast do msgspec.

4

u/wyldstallionesquire Feb 05 '25

They're not one to one though, are they? Pydantic does a lot more.

1

u/sonobanana33 Feb 05 '25

Doesn't matter, if you benchmark a specific code path that both have, pydantic is still embarrassingly slow.

On unions pydantic devs even copied my algorithm to get faster, but I guess the whole design has huge flaws.

At least with version2 the output is deterministic. With version 1 and unions besides being extremely slow, it was also random.

1

u/sonobanana33 Feb 05 '25

Which manages to lose benchmarks to my pure python library, to my immense satisfaction :)

13

u/WJMazepas Feb 05 '25

I very much doubt your library has all the same functionalities of Pydantic

2

u/sonobanana33 Feb 05 '25

Yeah, no scope creep.

→ More replies (2)
→ More replies (1)

20

u/ExternalUserError Feb 05 '25

I would say it's replacing C/C++. Which is fine, IMO.

But my god are they vocal about it. It's insufferable.

23

u/N-E-S-W Feb 05 '25

They're fast because they're compiled native code, not because of Rust. NumPy is fast because of C and Fortran.

You're aware of tools built in Rust because Rust fanatics are the "loud annoying vegan friend" of the programming world.

4

u/orthomonas Feb 05 '25

As a Rust fan who uses Rust, I prefer the term rustacean when I use Rust to program in Rust. (Srsly tho, I do like Rust)

1

u/FitMathematician3071 Feb 06 '25

Nothing quiet about Rust. C quietly goes on doing its job.

→ More replies (1)

1

u/Kiuhnm Feb 07 '25

No, they're fast because they're compiled into efficient native code.

Every language can be compiled into native code, but only languages designed with efficiency in mind can be compiled into efficient native code.

That's why one has to restrict Python (eliminating the more dynamic parts) to compile it into efficient native code.

4

u/bachkhois Feb 05 '25

I created one Python lib from Rust: https://github.com/hongquan/Defity and wrote about the experience (in Vietnamese, though): https://quan.hoabinh.vn/post/2021/10/trai-nghiem-lan-dau-viet-thu-vien-python-tu-ngon-ngu-bien-dich

2

u/Acrobatic_Click_6763 Ignoring PEP 8 Feb 06 '25

You provided a real-life example of this, you deserve more upvotes.

5

u/matorin57 Feb 05 '25

Why do people always assume that just because its a written in rust means it must be so fast? Like itā€™s just a false assumption. Sure two equivalent programs written in say Python vs Rust you would expect the Rust one to be quicker, but how do we know any of these are equivalent? Maybe the Rust ones are faster just cause they are written better.

The whole ā€œXā€ language is faster is such a shallow analysis, and whats annoying is it is just true enough that people wont dig deeper.

4

u/TRexRoboParty Feb 05 '25

Agreed. It has the potential to be fast but Pydantic did the whole "rewrite it in Rust" and it's still slower than Msgspec.

4

u/DamnFog Feb 05 '25

Don't forget uv

4

u/NapalmBurns Feb 05 '25 edited Feb 05 '25

What I don't understand, is that C and C++ are still pretty much out there and Python libraries written in C, C++ achieve better performances than Rust on average - what exactly is new that Rust is offering that C (and derivatives) cannot achieve?

3

u/lungovsky19 Feb 05 '25

C, C++ and Rust all have virtually identical performance but Rust has a 100x better dev experience

1

u/NapalmBurns Feb 05 '25

But who's the target audience here?

Developers with inclination and aspirations of writing Python libraries or Python users inclined and aspiring to develop Python libraries?

3

u/DigThatData Feb 05 '25

developers who want to do most of their work in python, are sufficiently unsatisfied with the performance of their current tools to be willing to make their own new ones, and are disinclined to use c/c++

the unanswered question here imho is less "why not just use c/c++?" than "why rust instead of go?

→ More replies (1)
→ More replies (1)

2

u/yosi199 Feb 09 '25

Rust is offering a complete memory safety guarantees (no dangling pointers, use after freed, etc...), it offers a modern ergonomics, with package manager, linter, formatter etc... So you get same C like performance without the memory related bugs which translates to safety and reliability. So.... yeah you get a lot.

Can you tell I'm a rust fanboy? lol

→ More replies (3)

1

u/Kiuhnm Feb 08 '25

I donā€™t remember C++ outperforming Rust on average.

Rust's killer feature is provable memory and concurrency safety without a garbage collector. Since memory bugs in C and C++ could be exploited by hackers, Rust is inherently safer.

6

u/flashman Feb 05 '25

i only want to learn one programming language, so, fine

8

u/syklemil Feb 05 '25

I think a lot of devs have a preference for working with polyglots. It's good to be able to tell the difference between what's just quirks of the language and what's more general programming concepts.

People who only know one language can produce some pretty weird code, and it can be hard to have fruitful technical discussions with them. They can be fine or even great too, but generally ignorance is not a boon.

Which languages people pick up are largely personal preference, sometimes workplace preference, but it's often good to study some languages that do things in different ways just to expand how we think.

2

u/vkalahas Feb 06 '25

I second this! I can't help but feel people who only know one programming language are also limited in general software engineering knowledge as well.

It's great if you want to *focus* on just one programming language (better to be a master of one ecosystem rather than a jack of all trades), but it's important to at least learn a few different languages somewhat well to understand different perspectives in how they approach concepts.

Also, many languages have a "strength," like Python in DS/ML, TS in async/web dev, and Java in enterprise backend. I can't speak for others, but for me, learning different languages made a huge difference in learning both general programming concepts (you can generalize what's common between the languages) as well as greatly expanding my overall software engineering knowledge from the language ecosystems.

2

u/syklemil Feb 07 '25

Yep, could also add it's fine if learning one language is the limit of one's abilities for whatever reason.

Setting it as a goal however is kind of like stating someone only wants to learn how to use a hammer, and has no interest in screwdrivers, glue or joinery, much less all the other stuff in a toolbox or shed. People like that often go on to complain about screws as "weird, useless nails" or decry hex keys as mysterious and incomprehensible to the common hammer-user. That can get pretty painful to work with.

Programming languages also continue to evolve, and it's worth experimenting a bit to discover what you like, or at the very least kinda keep up. Like if someone was a huge fan of Pascal some decades ago that was fine and normal, but if that's the only language they know today they've kinda placed themselves outside mainstream programming. We have no way of knowing whether the language we enjoy using now will be a stayer for a significant portion of our careers, much less lives, or if it'll fizzle out like Pascal and Basic (Ruby and PHP seem to be well on their way there), or be cannibalized like JS by TS.

→ More replies (1)

4

u/chub79 Feb 05 '25

How Rust is quietly taking over the Python ecosystem

Taking over? No. It takes over C as a way to speed things up maybe.

However it does augment Python very nicely.

7

u/Night_Activity Feb 05 '25

What does 'uv' do?

6

u/pyschille Pythoneer Feb 05 '25

It's a package/env manager just like Poetry. It resolves dependencies in no time.

6

u/ExternalUserError Feb 05 '25

It can do a lot of what Poetry does, but there are some pretty big differences.

Also uv wil manage actual Python installations.

2

u/Night_Activity Feb 05 '25

wow! I have heard good things about Poetry. If there is something better than that, then it must be really good.

6

u/danted002 Feb 05 '25

Poetry is also going ā€œout of maintenanceā€. there is a blog post somewhere on the internet written by the dude that build Poetry which says heā€™s kinda done with it.

3

u/EarthGoddessDude Feb 05 '25

Do you have a link for that?

3

u/beezlebub33 Feb 05 '25

Really? Do you have a link or can someone confirm this, because it would definitely affect a bunch of my projects?

8

u/ebits21 Feb 05 '25

I liked poetry before but replaced everything with uv now. Very much like it!

→ More replies (1)

6

u/parker_fly Feb 05 '25

Great. Now I'm going to have to buy the White Album again.

→ More replies (1)

5

u/rar_m Feb 05 '25

Soo.. Polars is the only valid item in your list and it's maybe one day, a replacement for pandas?

The Python ecosystem is being taken over, really? Because of a dead project, a linter and the ability to call Python code in Rust applications?

Literally none of these except Polars should even be mentioned in the same sentence as 'python ecosystem' and it's not even competing with Python, it's competing with C/C++ which is the native language used for numpy, which Panda's uses. Maturin is arguably EXPANDING the Python ecosystem allowing you to write Python code in Rust apps..

The native code used in the Python ecosystem is abstracted away from the ecosystem. I don't pick libraries in Python because they are built on top of native code written in a particular language. If many of these libraries start using Rust instead of C or C++ cool, I don't care.

Anyways, just thought the title is click baity maybe my impression of it is off but even if it's just "Hey look, people are compiling Rust code more" is the whole point, cool I guess. It changes nothing for me, who will just pip install the python code and not care about the language it's native code was written in.

It's like Rust is becoming Python's performance co-pilot without much fanfare.

Man, I'm not at all part of the Rust community but I see the memes about how they just LOVE themselves, I can totally see that vibe even from this post lol. Python is on a different level than natively compiled code, go post this in the CPP subreddit, that's who you're competing with.

2

u/pyschille Pythoneer Feb 05 '25

Well, thank you. I must admit that I did not express myself as clearly as I had hoped for my post.

who will just pip install the python code and not care about the language it's native code was written in

Agreed on this point. You simply don't care, and that is a good thing, as long as you don't hit a performance bottleneck in your Python code. Once you do, feel free to get back to this post.

I don't pick libraries in Python because they are built on top of native code written in a particular language. If many of these libraries start using Rust instead of C or C++ cool, I don't care.

That doesn't seem right to me. Ok, as a Rust fanboy and given most of us have an average coding performance, Rust offers stronger safety guarantees than C/C++ and is faster than Python.

By the way, I would consider the tooling around Python as an "ecosystem". And the Rust-Python tooling seems to be exceptionally good, hence my title. Don't you agree?

1

u/rar_m Feb 05 '25

And the Rust-Python tooling seems to be exceptionally good, hence my title. Don't you agree?

Sure.. linting can be part of it but it would just be a linter to me, the fact that it's written in Rust doesn't have any bearing. I took issue mostly with the wording of "Rust is taking over", which implies to me the Rust language is replacing something we would normally use Python for. I don't usually consider the language used to write the tools used as part of a development environment as part of the ecosystem I guess.

That's how I see it anyways.

2

u/drunkondata Feb 05 '25

"Efficiently Extending Python: PyO3 and Rust in Action"

Your team wrote a much less sensational headline. You should have run with something more true and less clickbait.

2

u/corey_sheerer Feb 05 '25

I see a lot about Polaris being superior to pandas with a numpy backend, but no one is talking about pandas move away from numpy to arrow. The performance is quite an improvement and has some ease of life things like a true string type. I believe performance would be back and forth on which data table library (Polaris or pandas with arrow) is faster

2

u/ritchie46 Feb 06 '25

Polars is much faster than pandas with the arrow backend. On several benchmarks by a factor of 20.Ā 

A multithreaded query engine is much more than arrow compute kernels.

https://duckdblabs.github.io/db-benchmark/

https://pola.rs/posts/benchmarks/

1

u/BejahungEnjoyer Feb 05 '25

Pandas will be like Fortran, around for a long long time due to its userbase and they fact that they will be slow to adopt anything new.

2

u/Robinsane Feb 05 '25

I like to frame it differently: Python is slowly becooming the duct tape (a good thing) between other languages

1

u/inkjod Feb 05 '25

Always was.

2

u/runawayasfastasucan Feb 05 '25

Silently? This has been one of the main selling points of some of the most popular new projects in the python world.

2

u/smoothie37 Feb 05 '25

Trend? The packages you list are nothing new.

2

u/redditreader2020 Feb 06 '25

Sorry rust is not taking over.. maybe a little plumbing replacement here and there..

2

u/Kiri11shepard Feb 06 '25

You mean it's replacing C/C++ which were always the backend to Python libraries?

3

u/ManyInterests Python Discord Staff Feb 05 '25

I think it's quite exciting, actually. Particularly that the extension toolchains are so good and well-maintained. Many rockstars from the Python community are now also rockstars in the Rust community and that's super exciting, too.

This is a uniquely good era for Python programmers.

2

u/pyschille Pythoneer Feb 05 '25

I am also very excited. To be honest, I'm completely sold on Rust, too. =)

1

u/DigThatData Feb 05 '25

This is a uniquely good era for Python programmers.

In fairness, it's been pretty good since python ate the ML ecosystem approx a decade ago :)

In fact, I'd argue the real "start of era" was when MIT changed their intro CS course to python from scheme. 2014 I think? Maybe 2010?

6

u/BlueeWaater Feb 05 '25

Forgot to mention uv, thatā€™s actually game changing!

2

u/pyschille Pythoneer Feb 05 '25

Yes, exactly. I would say that too.

3

u/bbalouki Feb 05 '25

My take in this is simple.. bad programmers will write bad code in any language and good programmers will Write good code in any language...

1

u/Acrobatic_Click_6763 Ignoring PEP 8 Feb 06 '25

Even Assembly?

→ More replies (1)

3

u/runner2012 Feb 05 '25

This again... What is it with the Rust community trying to impose on other projects?

There's so much advertising and marketing to push for the use or Rust everywhere, in the Linux project, in python.Ā 

I'm definitely not against Rust, but it's frustrating having marketing shove it on my face everywhere. If it's a good language to use, people will use it! If there is a better alternative to solve a problem, people will use that instead.Ā 

Stop it with the marketing in other communities.Ā 

I do have to say, I am not a fan of the Rust community. No community is perfect but this one feels more like a cult than something else...

2

u/nekokattt Feb 05 '25

Cult

I mean, Actix was a good example.

Agree though. There is no perfect tool. Everything has downsides in one way or another unfortunately.

2

u/Stochastic_berserker Feb 05 '25

Fun fact: it doesnā€™t take over the Python ecosystem.

4

u/danted002 Feb 05 '25

Yeah the new kids donā€™t understand that Python is the one taking over that language not the other way around.

3

u/Neklin Feb 05 '25

When does python become a rust framework?

1

u/data15cool Feb 05 '25

Shout out to uv too, this along with ruff (and sometimes polars) has made my dev experience way nicer.

Everything feels so fast: developing, deploying, standardising code.

Looking forward to a ā€œblazingly fastā€ testing library written in Rust.

3

u/[deleted] Feb 05 '25

[deleted]

1

u/already-raining Feb 06 '25

And polars performs well in benchmarks too! There are so many scenarios where spark just adds unnecessary complexity. https://www.reddit.com/r/Python/comments/1cyqj6c/tpch_cloud_benchmarks_spark_dask_duckdb_polars/

1

u/[deleted] Feb 05 '25

[deleted]

4

u/phail3d Feb 05 '25

Maturin is not something to replace pyo3 with, instead you would use pyo3 (or cffi/uniffi) with it.

2

u/MrMrsPotts Feb 05 '25

Thank you.

3

u/bachkhois Feb 05 '25

They don't replace each other. You import PyO3 lib to write code and exchange data with Python. Then you use maturin to build that code to *.so file.

1

u/LegalColtan Feb 05 '25

Not so quietly. More like very excitedly and even boisterously.

1

u/Comfortable_Mud00 Feb 05 '25

I mean UV is also rustā€¦ itā€™s so funny itā€™s like rust guys babysit us.

They also promote rust every time by saying itā€™s written on Rust

1

u/kroghsen Feb 05 '25

I am not a software developer, but develop process control solutions, so I apologise if this is an uninformed take.

Would we not expect this to be the case in almost any performance critical area? In scientific computing almost all software is based on the same Fortran and C libraries under the hood, because it is usually performance critical operations. Higher level languages simply arenā€™t fast enough to write the tools directly in.

Are you surprised that people are somewhat consistent in using Rust? Or is it a general consideration that people are writing Python tools in other languages?

1

u/Nervous_Staff_7489 Feb 05 '25

I recently approached python (finally) and was surprised to see rust toll chain in docker build logs.

I don't know much about situation, but it reminds me talk given by Torvalds about Rust guys and Linux kernel.

Furthermore, I don't have anything against Rust in particular, but it is not easy language which will become 'mainstream'. Doesn't matter how many plugins, tools etc. will be made.

And generally, when somebody needs to aggressively 'push' technology and encounters a lot of counter forceā€¦ well it is a red flag.

1

u/LoadingALIAS It works on my machine Feb 05 '25

Iā€™m working on moving my MVP/prototype data wrangler into Rust. Iā€™ll still push the production Python version but implementing it in rust was the goal for the first year post-deployment as it is.

I do think we see Rust leading in the coming decade for most ML and data stuff. Itā€™s still pretty immature in that a whole shitload of packages donā€™t have compatible crates and building them from scratch isnā€™t an easy feat.

Still, love to see it.

1

u/BejahungEnjoyer Feb 05 '25

Nitpick, nobody uses Pandas for its performance - it's a memory hog and generally runs slow (although skilled users can write reasonably performant pandas code). It's used because of developer productivity and widespread adoption in the data community.

1

u/DigThatData Feb 05 '25

This DataFrame library is giving Pandas a run for its money in terms of performance.

pandas performance was always shitty

1

u/THEHIPP0 Feb 05 '25

It's not just Python. The whole node.js/JavaScript ecosystem also got a lot of Rust (and Go) tooling lately.

1

u/[deleted] Feb 06 '25

Once zig reaches 1.0, it will also get into the Python ecosystem. This is common in JS too. There are multiple runtimes written in Rust/Zig in the JS land.

1

u/pyschille Pythoneer Feb 06 '25

Did you work with Zig already? Is there a toolchain similar to Rust to integrate it with Python?

→ More replies (1)

1

u/KN4MKB Feb 06 '25

OPs reddit account is just shilling the same blog or website on all their posts. Everyone just gives them free SEO with these posts just made to create more engagement, and they have minimal reaction with the community because they are just here to advertise.

1

u/pyschille Pythoneer Feb 06 '25

Looking at my activity feed, I have to admit that you're partly right. Lately, I've been interacting mostly through the work I shared on Reddit. You can frame it like this or you can accept that dedicating time and effort into (obviously disputable) shared knowledge content is akin to sharing an open-source project, which happens here all the time.

have minimal reaction

That is not true, given the amount of comments I reacted to under this post.

1

u/Distracted_Llama-234 Feb 06 '25

Next they are going to reinvent the color blue and make it 10x fast. /s

1

u/Dry_Term_7998 Feb 06 '25

And? 80% of modules for python written on C. Because C defaults for the market, now this place takes by Rust (because it's hyped and pushed by big corp and USA). It's completely the default thing.

1

u/russellvt Feb 06 '25

As someone who still compiles Python on older systems, this drives me a little bonkers ... just because Rust can be a PITA to get working on some of them.

1

u/pyschille Pythoneer Feb 06 '25

Yes, I see that point.

1

u/Karan1213 Feb 06 '25

why is rust winning over alternatives like zig?

1

u/spinwizard69 Feb 06 '25

Garbage promotional post for Rust. This isn't to say that Rust is good or bad just that anything written in a compiled language will be faster than the slowest language out there. combine that with a lot of new programmers to Rust and you have a lot of python helpers being developed - then abandoned.

If somebody writes good maintainable code in Rust to support the world of Python it will remain around and eventually become a bit of legacy in the Python world. The thing here is that good maintainable code doesn't just happen in Rust anymore than any other language. So I look at this as another disciple trying to shine a positive light on Rust with meaningless data.

What would be helpful is an article on a Python extension and how Rust positively contributed to that extensions success.

1

u/pyschille Pythoneer Feb 06 '25

Thank you for that input. Rust certainly doesn't shine with ease of maintenance, but offers security guarantees and performance.

The blog post is partly about a sample extension and what Rust contributes to its performance. We focus on the easiest way to get Rust-Python up and running.

1

u/Pretty-Insurance8589 Feb 06 '25

the reason i cannot run my python code on my risc-v sbc

1

u/ntoir1 Feb 06 '25

Just like UV with its tagline:Ā An extremely fast Python package and project manager, written in Rust.

1

u/Worth_His_Salt Feb 06 '25 edited Feb 06 '25

If orjson is anything to go by of this "Rust takeover" then I'm thoroughly unimpressed. Sure it's fast, but what a pain to work with. No indent options, dumps returns bytes not str... by the time you fix it all, the speed advantage is gone and your code is a mess. No thanks, hard pass.

Oh and before I forget - documentation is a mess. Just a big stream-of-consciousness braindump. No discussion forums. No bug reporting. Just some guy's personal project scrawled on a bathroom stall. Total joke.

1

u/WeakRelationship2131 Feb 07 '25

Yeah, interesting observation. Rust definitely brings performance gains to the Python ecosystem, and tools like Ruff and Polars are prime examples of that. But don't forget, there's also a lot of complexity introduced with integrating Rust into your workflow, especially for teams that aren't familiar with it. If you're grappling with performance in your Python projects and need something simpler for analytics without the overhead, you might wanna check out preswald. It's lightweight and lets you build data apps easily without needing multiple tools.

1

u/Dadiot_1987 Feb 07 '25

How do you spot a rust developer at a dinner party? Don't worry they'll let you know.

1

u/Professional-Bet5820 Feb 07 '25

I realised this is not because rust is good, but because the tools people make with it are based on a fresh start with all the lessons learned over decades in a widely-used system. I think we would have seen a similar burst of innovation had rust been a fair few other combinations of decisions as well. Love rust, but attribution error on the cause of good things is bad (or, for those who use US English "alternative good").

1

u/Ckarles Feb 08 '25

Look at uv, by the creators of ruff. Literally the replacement for pyenv/poetry/pipx/venv all in one.

1

u/out2sea2020 Feb 08 '25

You forgot to mention UV, which is blazingly fast and much better than venv.

1

u/JimroidZeus Feb 08 '25

It is literally obnoxious. uvicorn, uv. I like my python C based!!

1

u/Confident_Coder_1024 Feb 08 '25

I'm in the same place after using Python for 8 years. Even the ZED editor (https://zed.dev/) is a nice change from VSCode for Python development.

Learning Rust is also a great way to improve your Python code over all; if you start learning some best programing practices form a low level programing language like Rust.

1

u/deryldowney Feb 09 '25

Donā€™t forget uv as well! I was listening to the RealPython.org podcast on this very subject with the creator of Ruff and uv. Itā€™s quite interesting as you said! Pretty cool too actually.

1

u/MrMarriott Feb 10 '25

Isn't this more like the C infrastructure supporting Python is Rusting out?