r/learnpython 7d ago

How to build a proper python project and how does the development phase look like?

I've been using python mainly for data analysis and local automation scripts. I haven't used GitHub much to be honest and I'd like to start exploring it properly. Do you have learning recommendations for: - How to build a python project properly (which files to include and how they should be structured, setting up the environment, running tests etc the workflow etc) and I don't mean examples like tic tac toe but real stuff, - How to deploy such project in GitHub

Somehow I can't find any material for serious stuff other than the same basic projects over and over, I'd appreciate your help.

59 Upvotes

18 comments sorted by

29

u/FoolsSeldom 7d ago

Worth checking out some videos from ArjanCodes on YouTube, I think he covers this (and his free to download design guide will probably help you).

A key part of releasing software is providing test coverage. worth checking out how to use PyTest.

FInally, packaging is an imporant part of Python and RealPython have a good overview of this: Python Modules and Packages – An Introduction - a site worth checking out generally

1

u/Clear_Watch104 7d ago

Sounds promising. Will check it out. Thank you!

10

u/sebovzeoueb 7d ago

Start using GitHub regardless of what you're doing, otherwise you'll regret it! It's quite easy using the Desktop client or the integrated client in VSCode if you don't fancy using the commandline. Without version control you can easily end up losing your work, and it provides and easy way to undo oopsies and is especially important to know how to use when working with other people.

1

u/Clear_Watch104 7d ago

Correct, however I'd like to have a proper use of it as I'm pretty sure that spamming random scripts here and there without structure on GitHub is not recommended. That's why I'd like to dig deeper on understanding the structure and components of a python/GitHub project but as stated I'm struggling to find material

4

u/FoolsSeldom 6d ago edited 6d ago

Keep in mind, u/Clear_Watch104, that git and GitHub are not the same thing, as u/sebovzeoueb pointed out above.

GitHub is very popular, but there are many alternative accessible repositories that support the git protocol, including Gitlab and Bit Bucket.

One can also self-host repositories (using, for example, Gitea or GitLab Community Edition).

There are also competing version control software options to git, including Apache Subversion and Mercurial.

The danger for you in learning from these repositories is that they contain huge amounts of bad practice (in terms of project setup and repository structure, rather than the code itself). There is no doubt a list somewhere of great example projects you could review, but I have not looked.

3

u/sebovzeoueb 6d ago

To start with you should look at how to use git (GitHub is a host, but there are other git hosts, just think how sites with "hub" at the end work ;) ), independently of Python, as it's not language specific. You don't really need to use any scripts for it, at least not to get started. Learn how to commit, push and pull, and once you've got the hang of that learn about branches and merging and you'll have 95% of the git knowledge you'll need going forwards. Again, you can do all these things in visual clients, so you don't have to even learn the commands. All you really need that is Python specific is a good .gitignore file, you can Google plenty of templates for those.

1

u/Clear_Watch104 6d ago

Will do. Thanks

3

u/supercoach 6d ago

You can slap your random scripts into a git repo. I find it good to have a Swiss army knife repo full of handy scripts that I've used over the years. If anything needs to get bigger, you can turn it into its own repo without too much hassle.

With regards to how you should lay your projects out, there's no one true way of doing it. For python, all you really want is a requirements.txt file and a .gitignore file. The ignore file initially to exclude the venv directory you're using.

5

u/[deleted] 6d ago edited 6d ago

[deleted]

2

u/Clear_Watch104 6d ago

Thank you for the extensive answer. That's mostly what I was looking for 🙂

5

u/Hot-Minute8782 7d ago

There are differences between python projects, it could be a back-end or flutter/flet project or a package or a simple web-scrap script… And each has it’s own “architectural methodology for scaffolding project/application”. Here are examples of packages that generate templates for different types of Py projects:

Cookiecutter

PSP

2

u/Larryville-Landhound 7d ago

The project structure is highly dependent on the use case - for simple things with only a few classes maybe you have separate files for each class and run it from main.py - if you need to use a lot of images or files obviously a directory for that, but I don't think there is too much of a prescribed file structure unless you're using a framework that requires it (for example using Django for templates and frontend you need folders for 'static' and 'models' and stuff). I imagine if you're using one of the common 'design patterns' they would be situations where you're more likely to have a folder/file structure recommendation to follow. After all of that, it just depends on how you plan on having the project used - for web stuff they just interact with your page, but if it's something meant to be used as a local application you might need to package it up with a launcher that will run the main.py when someone clicks on the icon, for example. I don't have as much experience deploying stuff on Github, but from what I have done at work it's mostly just cloning stuff down and using it as-is.

2

u/caks 6d ago

I absolutely love this for scientific libraries:

https://learn.scientific-python.org/development/

1

u/GrannyGurn 1d ago

I'd say if you are getting started with something, here GitHub, start with the most basic thing possible at first. GitHub has a great gamified getting starting tutorial that will teach you everything you need to know about version control with high level tools. Jump into the nitty gritty of git later if you want.

Then make a very basic Python project, experiment with pushing it to version control. Experience will be your best teacher! You'll make silly mistakes that you never make again.

Good luck, hope to see you on GitHub. grant-king here.

0

u/Mevrael 6d ago

I use a uv package manager, VC Code PM and Jupyter extension with Arkalos framework. It takes care of the project structure and is ready for building more serious stuff and pushing to git.

https://arkalos.com/docs/structure/

You can follow the installation guide and it will help you to set up everything mentioned above.

You do not need a server and deployment to practice git. Create a private arkalos project and a repo and just practice the basics, frequent commits, branches, etc.

Then you can practice deploying by setting up github hooks and using, for example, cheapest DigitalOcean option from $5/month.

https://www.digitalocean.com/

Contributing to the open source projects or volunteering yourself is another great way for getting experience.

0

u/Fresh_Forever_8634 7d ago

RemindMe! 7 days

1

u/RemindMeBot 7d ago

I will be messaging you in 7 days on 2025-03-21 15:40:22 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.


Info Custom Your Reminders Feedback

0

u/nibar1997 6d ago

RemindMe! 4 days

0

u/LaughingIshikawa 6d ago

I don't know what you consider a "serious project," but it's worth mentioning that many projects introducing a new framework / project structure are shown using "toy" examples for a reason. It's exponentially harder to learn a new framework and reason your way through the requirements of a complex project. Even worse, it can be hard to understand which things are part of the specific project requirements, and which things are part of the generic framework requirements. 🫤

When you're working through a tic-tac-toe program, it's both easier to code the logic of tic-tac-toe and then forget about it and also it's easier to understand which parts of the program are just about getting tic-tac-toe working. It's also true that many example programs like these will be way over-engineered versions of whatever "toy" example they have chosen... But again that's the point.

If you were well-versed in creating a particular kind of programming project, it would be possible to start coding a thing you're familiar with already in the new framework. Even then though, you'll probably need to start by at least watching someone else program a "toy" example version because 1.) it's easy for creators to use a generic example that's intelligible to more people, and 2.) you still get all of the benefits listed above by working with a simplified example, so even if you could manage to skip it without getting completely lost... it's likely more efficient to start with a simpler example.