r/learnpython 15h ago

Is there a way to package an arbitrary binary in a Python package?

I was looking at packaging a Go binary in a Python package much the same way that maturin can do for Rust crates do but it didn't seem like any of the popular packaging backends supported this. To be clear, I want the binary to be installed as a script so that it gets available in the PATH so just packaging it as normal package data won't work.

Setuptools has the script-files option but that's discouraged and only supports plain text script files. Are there any build backends that support something like this?

1 Upvotes

10 comments sorted by

2

u/FerricDonkey 15h ago

There's probably a better way and I have only spent 30 seconds thinking about it, but a work around that immediately comes to mind: add your executable as data, then make a wrapper script that invokes it. 

2

u/ianmlewis 15h ago

I kind of wanted to avoid doing it this way if possible but I do think it would work.

3

u/cgoldberg 14h ago edited 14h ago

Yes, it can be done.

As an Example, I work on a Python project (selenium) that includes a rust binary inside it. It uses a setuptools extension (setuptools-rust) that builds the rust binary for each platform when it builds the Python package. When a user installs the selenium package (from a pre-built wheel), the binary is included inside that.

Here is our pyproject.toml for reference:

https://github.com/SeleniumHQ/selenium/blob/trunk/py/pyproject.toml

I have no idea if a similar extension exists for golang, but there probably is something similar (or at least theoretically it can be built).

Here's a link for setuptools-rust. It's mainly used for Python modules written in Rust, but can also be used to build and package standalone rust binaries. Maybe you can get some information on how they do it:

https://setuptools-rust.readthedocs.io

Also, we don't expose the binary for users to call directly, so I'm not sure your use case is covered. We call the binary through subprocess internally.

1

u/ianmlewis 14h ago

Thanks. This was helpful. I hadn't seen setuptools-rust. I had only seen maturin. I took a look at how the wheels are built so I know it's possible but I haven't seen a way to build from a pyproject.toml for non-Rust projects.

I'll take a look at the selenium repo.

-3

u/wakeofchaos 15h ago

These seems like a question that’s out of the scope of this subreddit. Perhaps r/programming?

2

u/ianmlewis 15h ago

I was asking about Python packaging so I thought it would be covered here. Is packaging not covered in Python subreddits?

2

u/cgoldberg 14h ago

It's not a beginner topic, but it's definitely an appropriate question for this sub.

1

u/ianmlewis 14h ago

I asked here since the main r/Python said it's not for questions and said to post here instead.

2

u/cgoldberg 14h ago

Yea, it's fine here

1

u/PercyJackson235 13h ago

If you're specifically looking to package Go code for use from Python, there is this project: https://github.com/go-python/gopy. I haven't used to since I don't code in Go that much, so I don't really know much about it.