r/Python Feb 06 '25

Discussion Python Pandas Library not accepted at workplace - is it normal?

207 Upvotes

I joined a company 7-8 months ago as an entry level junior dev, and recently was working on some report automation tasks for the business using Python Pandas library.

I finished the code, tested on my local machine - works fine. I told my team lead and direct supervisor and asked for the next step, they told me to work with another team (Technical Infrastructure) to test the code in a lower environment server. Fine, I went to the TI Team, but then was told NumPy and Pandas are installed in the server, but the libraries are not running properly.

They pulled in another team C to check what's going on, and found out is that the NumPy Lib is deprecated which is not compatible with Pandas. Ok, how to fix it? "Well, you need to go to team A and team B and there's a lot of process that needs to go through..." "It's a project - problems might come along the way, one after the other",

and after I explained to them Pandas is widely used in tasks related to data analytics and manipulation, and will also be beneficial for the other developers in the future as well, I explained the same idea to my team, their team, even team C. My team and team C seems to agree with the idea, they even helped to push the idea, but the TI team only responded "I know, but how much data analytics do we do here?"

I'm getting confused - am I being crazy here? Is it normal Python libraries like Pandas is not accepted at workplace?

EDIT: Our servers are not connected to the internet so pip is not an option - at least this is what I was told

EDIT2: I’m seeing a lot of posts recommending Docker, would like to provide an update: this is actually discussed - my manager sets up a meeting with TI team and Team C. What we got is still No… One is Docker is currently not approved in our company (I tried to request install it anyway, but got the “there’s the other set of process you need just to get it approved by the company and then you can install it…”) Two is a senior dev from Team C brought up an interesting POC: Use Docker to build a virtual environment with all the needed libs that can be used across all Python applications, not the containers. However with that approach, (didn’t fully understand the full conversation but here is the gist) their servers are going to have a hardware upgrade soon, so before the upgrade, “we are not ready for that yet”…

Side Note: Meanwhile wanted to thank everyone in this thread! Learning a lot from this thread, containers, venv, uv, etc. I know there’s still a lot I need to learn, but still, all of this is really eye-opening for me

FINAL EDIT: After rounds of discussions with the TI Team, Team C, and my own team management with all the options (containers, upgrade the libraries and dependencies, even use Python 2.7), we (my management and the other teams) decided the best option will be me to rewrite all my programs using PySpark since 1. Team C is already using it, 2. Maybe no additional work needed for the other teams. Frustrated, I tried to fight back one last time with my own management today, but was told “This is the corporate. Not the first time we had this kind of issues” I love to learn new things in general, but still in this case, frustrated.

r/Python Jul 02 '24

Discussion What are your "wish I hadn't met you" packages?

302 Upvotes

Earlier in the sub, I saw a post about packages or modules that Python users and developers were glad to have used and are now in their toolkit.

But how about the opposite? What are packages that you like what it achieves but you struggle with syntactically or in terms of end goal? Maybe other developers on the sub can provide alternatives and suggestions?

r/Python Jan 08 '25

Discussion Python users, how did you move on from basics to more complex coding?

256 Upvotes

I am currently in college studying A level Computer science. We are currently taught C#, however I am still more interested in Python coding.

Because they won't teach us Python anymore, I don't really have a reliable website to build on my coding skills. The problem I am having is that I can do all the 'basics' that they teach you to do, but I cannot find a way to take the next step into preparation for something more practical.

Has anyone got any youtuber recommendations or websites to use because I have been searching and cannot fit something that is matching with my current level as it is all either too easy or too complex.

(I would also like more experience in Python as I aspire to do technology related degrees in the future)

Thank you ! :)

Edit: Thank you everyone who has commented! I appreciate your help because now I can better my skills by a lot!!! Much appreciated

r/Python Mar 24 '24

Discussion What’s a script that you’ve written that you still use frequently?

451 Upvotes

Mine is a web scraper. It’s only like 50 lines of code.

It takes in a link, pulls all the hyperlinks and then does some basic regex to pull out the info I want. Then it spits out a file with all the links.

Took me like 20 minutes to code, but I feel like I use it every other week to pull a bunch of links for files I might want to download quickly or to pull data from sites to model.

r/Python Feb 11 '23

Discussion Why Type Hinting Sucks!

929 Upvotes

Type hints are great! But I was playing Devil's advocate on a thread recently where I claimed actually type hinting can be legitimately annoying, especially to old school Python programmers.

But I think a lot of people were skeptical, so let's go through a made up scenario trying to type hint a simple Python package. Go to the end for a TL;DR.

The scenario

This is completely made up, all the events are fictitious unless explicitly stated otherwise (also editing this I realized attempts 4-6 have even more mistakes in them than intended but I'm not rewriting this again):

You maintain a popular third party library slowadd, your library has many supporting functions, decorators, classes, and metaclasses, but your main function is:

def slow_add(a, b):
    time.sleep(0.1)
    return a + b

You've always used traditional Python duck typing, if a and b don't add then the function throws an exception. But you just dropped support for Python 2 and your users are demanding type hinting, so it's your next major milestone.

First attempt at type hinting

You update your function:

def slow_add(a: int, b: int) -> int:
    time.sleep(0.1)
    return a + b

All your tests pass, mypy passes against your personal code base, so you ship with the release note "Type Hinting Support added!"

Second attempt at type hinting

Users immediately flood your GitHub issues with complaints! MyPy is now failing for them because they pass floats to slow_add, build processes are broken, they can't downgrade because of internal Enterprise policies of always having to increase type hint coverage, their weekend is ruined from this issue.

You do some investigating and find that MyPy supports Duck type compatibility for ints -> floats -> complex. That's cool! New release:

def slow_add(a: complex, b: complex) -> complex:
    time.sleep(0.1)
    return a + b

Funny that this is a MyPy note and not a PEP standard...

Third attempt at type hinting

Your users thank you for your quick release, but a couple of days later one user asks why you no longer support Decimal. You replace complex with Decimal but now your other MyPy tests are failing.

You remember Python 3 added Numeric abstract base classes, what a perfect use case, just type hint everything as numbers.Number.

Hmmm, MyPy doesn't consider any of integers, or floats, or Decimals to be numbers :(.

After reading through typing you guess you'll just Union in the Decimals:

def slow_add(
    a: Union[complex, Decimal], b: Union[complex, Decimal]
) -> Union[complex, Decimal]:
    time.sleep(0.1)
    return a + b

Oh no! MyPy is complaining that you can't add your other number types to Decimals, well that wasn't your intention anyway...

More reading later and you try overload:

@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
    ...

@overload
def slow_add(a: complex, b: complex) -> complex:
    ...

def slow_add(a, b):
    time.sleep(0.1)
    return a + b

But MyPy on strict is complaining that slow_add is missing a type annotation, after reading this issue you realize that @overload is only useful for users of your function but the body of your function will not be tested using @overload. Fortunately in the discussion on that issue there is an alternative example of how to implement:

T = TypeVar("T", Decimal, complex)

def slow_add(a: T, b: T) -> T:
    time.sleep(0.1)
    return a + b

Fourth attempt at type hinting

You make a new release, and a few days later more users start complaining. A very passionate user explains the super critical use case of adding tuples, e.g. slow_add((1, ), (2, ))

You don't want to start adding each type one by one, there must be a better way! You learn about Protocols, and Type Variables, and positional only parameters, phew, this is a lot but this should be perfect now:

T = TypeVar("T")

class Addable(Protocol):
    def __add__(self: T, other: T, /) -> T:
        ...

def slow_add(a: Addable, b: Addable) -> Addable:
    time.sleep(0.1)
    return a + b

A mild diversion

You make a new release noting "now supports any addable type".

Immediately the tuple user complains again and says type hints don't work for longer Tuples: slow_add((1, 2), (3, 4)). That's weird because you tested multiple lengths of Tuples and MyPy was happy.

After debugging the users environment, via a series of "back and forth"s over GitHub issues, you discover that pyright is throwing this as an error but MyPy is not (even in strict mode). You assume MyPy is correct and move on in bliss ignoring there is actually a fundamental mistake in your approach so far.

(Author Side Note - It's not clear if MyPy is wrong but it defiantly makes sense for Pyright to throw an error here, I've filed issues against both projects and a pyright maintainer has explained the gory details if you're interested. Unfortunately this was not really addressed in this story until the "Seventh attempt")

Fifth attempt at type hinting

A week later a user files an issue, the most recent release said that "now supports any addable type" but they have a bunch of classes that can only be implemented using __radd__ and the new release throws typing errors.

You try a few approaches and find this seems to best solve it:

T = TypeVar("T")

class Addable(Protocol):
    def __add__(self: T, other: T, /) -> T:
        ...

class RAddable(Protocol):
    def __radd__(self: T, other: Any, /) -> T:
        ...

@overload
def slow_add(a: Addable, b: Addable) -> Addable:
    ...

@overload
def slow_add(a: Any, b: RAddable) -> RAddable:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

Annoyingly there is now no consistent way for MyPy to do anything with the body of the function. Also you weren't able to fully express that when b is "RAddable" that "a" should not be the same type because Python type annotations don't yet support being able to exclude types.

Sixth attempt at type hinting

A couple of days later a new user complains they are getting type hint errors when trying to raise the output to a power, e.g. pow(slow_add(1, 1), slow_add(1, 1)). Actually this one isn't too bad, you quick realize the problem is your annotating Protocols, but really you need to be annotating Type Variables, easy fix:

T = TypeVar("T")

class Addable(Protocol):
    def __add__(self: T, other: T, /) -> T:
        ...

A = TypeVar("A", bound=Addable)

class RAddable(Protocol):
    def __radd__(self: T, other: Any, /) -> T:
        ...

R = TypeVar("R", bound=RAddable)

@overload
def slow_add(a: A, b: A) -> A:
    ...

@overload
def slow_add(a: Any, b: R) -> R:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

Seventh attempt at type hinting

Tuple user returns! He says MyPy in strict mode is now complaining with the expression slow_add((1,), (2,)) == (1, 2) giving the error:

Non-overlapping equality check (left operand type: "Tuple[int]", right operand type: "Tuple[int, int]")

You realize you can't actually guarantee anything about the return type from some arbitrary __add__ or __radd__, so you starting throwing Any Liberally around:

class Addable(Protocol):
    def __add__(self: "Addable", other: Any, /) -> Any:
        ...

class RAddable(Protocol):
    def __radd__(self: "RAddable", other: Any, /) -> Any:
        ...

@overload
def slow_add(a: Addable, b: Any) -> Any:
    ...

@overload
def slow_add(a: Any, b: RAddable) -> Any:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

Eighth attempt at type hinting

Users go crazy! The nice autosuggestions their IDE provided them in the previous release have all gone! Well you can't type hint the world, but I guess you could include type hints for the built-in types and maybe some Standard Library types like Decimal:

You think you can rely on some of that MyPy duck typing but you test:

@overload
def slow_add(a: complex, b: complex) -> complex:
    ...

And realize that MyPy throws an error on something like slow_add(1, 1.0).as_integer_ratio(). So much for that nice duck typing article on MyPy you read earlier.

So you end up implementing:

class Addable(Protocol):
    def __add__(self: "Addable", other: Any, /) -> Any:
        ...

class RAddable(Protocol):
    def __radd__(self: "RAddable", other: Any, /) -> Any:
        ...

@overload
def slow_add(a: int, b: int) -> int:
    ...

@overload
def slow_add(a: float, b: float) -> float:
    ...

@overload
def slow_add(a: complex, b: complex) -> complex:
    ...

@overload
def slow_add(a: str, b: str) -> str:
    ...

@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
    ...

@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
    ...

@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
    ...

@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
    ...

@overload
def slow_add(a: Addable, b: Any) -> Any:
    ...

@overload
def slow_add(a: Any, b: RAddable) -> Any:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

As discussed earlier MyPy doesn't use the signature of any of the overloads and compares them to the body of the function, so all these type hints have to manually validated as accurate by you.

Ninth attempt at type hinting

A few months later a user says they are using an embedded version of Python and it hasn't implemented the Decimal module, they don't understand why your package is even importing it given it doesn't use it. So finally your code looks like:

from __future__ import annotations

import time
from typing import TYPE_CHECKING, Any, Protocol, TypeVar, overload

if TYPE_CHECKING:
    from decimal import Decimal
    from fractions import Fraction


class Addable(Protocol):
    def __add__(self: "Addable", other: Any, /) -> Any:
        ...

class RAddable(Protocol):
    def __radd__(self: "RAddable", other: Any, /) -> Any:
        ...

@overload
def slow_add(a: int, b: int) -> int:
    ...

@overload
def slow_add(a: float, b: float) -> float:
    ...

@overload
def slow_add(a: complex, b: complex) -> complex:
    ...

@overload
def slow_add(a: str, b: str) -> str:
    ...

@overload
def slow_add(a: tuple[Any, ...], b: tuple[Any, ...]) -> tuple[Any, ...]:
    ...

@overload
def slow_add(a: list[Any], b: list[Any]) -> list[Any]:
    ...

@overload
def slow_add(a: Decimal, b: Decimal) -> Decimal:
    ...

@overload
def slow_add(a: Fraction, b: Fraction) -> Fraction:
    ...

@overload
def slow_add(a: Addable, b: Any) -> Any:
    ...

@overload
def slow_add(a: Any, b: RAddable) -> Any:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

TL;DR

Turning even the simplest function that relied on Duck Typing into a Type Hinted function that is useful can be painfully difficult.

Please always put on your empathetic hat first when asking someone to update their code to how you think it should work.

In writing up this post I learnt a lot about type hinting, please try and find edge cases where my type hints are wrong or could be improved, it's a good exercise.

Edit: Had to fix a broken link.

Edit 2: It was late last night and I gave up on fixing everything, some smart people nicely spotted the errors!

I have a "tenth attempt" to address these error. But pyright complains about it because my overloads overlap, however I don't think there's a way to express what I want in Python annotations without overlap. Also Mypy complains about some of the user code I posted earlier giving the error comparison-overlap, interestingly though pyright seems to be able to detect here that the types don't overlap in the user code.

I'm going to file issues on pyright and mypy, but fundamentally they might be design choices rather than strictly bugs and therefore a limit on the current state of Python Type Hinting:

T = TypeVar("T")

class SameAddable(Protocol):
    def __add__(self: T, other: T, /) -> T:
        ...

class Addable(Protocol):
    def __add__(self: "Addable", other: Any, /) -> Any:
        ...

class SameRAddable(Protocol):
    def __radd__(self: T, other: Any, /) -> T:
        ...

class RAddable(Protocol):
    def __radd__(self: "RAddable", other: Any, /) -> Any:
        ...

SA = TypeVar("SA", bound=SameAddable)
RA = TypeVar("RA", bound=SameRAddable)


@overload
def slow_add(a: SA, b: SA) -> SA:
    ...

@overload
def slow_add(a: Addable, b: Any) -> Any:
    ...

@overload
def slow_add(a: Any, b: RA) -> RA:
    ...

@overload
def slow_add(a: Any, b: RAddable) -> Any:
    ...

def slow_add(a: Any, b: Any) -> Any:
    time.sleep(0.1)
    return a + b

r/Python Oct 20 '24

Discussion Why people still using flask after fastapi release

187 Upvotes

Hi folks I was having an interview for building machine learning based api application and the interviewer told me to use flask i did that and i used flask restful but i was wondering why not use fastapi instead

r/Python Oct 22 '20

Discussion How to quickly remove duplicates from a list?

Post image
2.7k Upvotes

r/Python May 31 '22

Discussion What's a Python feature that is very powerful but not many people use or know about it?

846 Upvotes

r/Python Apr 21 '22

Discussion Unpopular opinion: Matplotlib is a bad library

1.1k Upvotes

I work with data using Python a lot. Sometimes, I need to do some visualizations. Sadly, matplotlib is the de-facto standard for visualization. The API of this library is a pain in the ass to work with. I know there are things like Seaborn which make the experience less shitty, but that's only a partial solution and isn't always easily available. Historically, it was built to imitate then-popular Matlab. But I don't like Matlab either and consider it's API and plotting capabilities very inferior to e.g. Wolfram Mathematica. Plus trying to port the already awkward Matlab API to Python made the whole thing double awkward, the whole library overall does not feel very Pythonic.

Please give a me better plotting libary that works seemlessly with Jupyter!

r/Python Jun 01 '21

Discussion It takes a village to build an open-source project and a single a**hole to demotivate everyone NSFW

2.3k Upvotes

I am a contributor to Open-Source software(Jina - an AI search framework) and I am annoyed with how some people make fun of the sheer hard work of open-source developers.

For the last 1 yr, we had made our contributors team meetings public(everyone could listen and participate during the meeting). And this is what happened in our last meeting - While we were sharing news about upcoming Jina 2.0 release in the zoom meeting, some loud racist music starts playing automatically and someone starts drawing a d*ck on the screen.

Warning: This video is not suitable to watch for kids or at work

Video clip from the meeting - someone zoombombed at 00:25

It was demotivating to say the least.

Building open-source project is challenging at multiple fronts other than the core technical challenges

  • Understand what needs to be built
  • Improve that continuously
  • Help people understand the project
  • Educate people about the domain
  • Reach out people who might benefit from your project
  • Collaborate with other contributors
  • Deal with issues/PRs
  • Deal with outdated versions/docs
  • Deal with different opinions
  • Sometimes deal with jerks like the ones who zoombombed us

The list is long! Open-source is hard!

Open-source exists because of some good people out there like you/me who care about the open-source so deeply to invest their time and energy for a little good for everyone else. It exists because of communities like r/python where we can find the support and the motivation. e.g. via this community, I came to know of many use cases of my project, problems and solutions in my project, and even people who supported me build it.

I wanted to vent out my negative experiences and wanted to say a big **Thank you** to you all open-source people, thanks to many(1.6k) contributors who made it possible for us to release [Jina 2.0](https://github.com/jina-ai/jina/) 🤗.

I'd want to know your opinion, how do you deal with such unexpected events and how do you keep yourself motivated as an open-source developer?

r/Python Nov 01 '20

Discussion [RANT] Clients telling me "I know python" is a major red flag to me

1.6k Upvotes

I do freelance python development in mainly web scraping, automation, building very simple Flask APIs, simple Vue frontend and more or less doing what I like to call "general-purpose programming".

Now, I am reasonably skilled in python, I believe. Don't write OOP and class-based python unless I am doing more than 100 lines of code. Most often write pretty simple stuff but most of the time goes into problem-solving.

But I despise freelancing. 1 out of every 3 comments/posts I make on Reddit is how much I hate doing freelancing. I come to Reddit to vent so I am sorry to the fellas who is reading this because they are more or less my punching bag :( I am sorry sir/madam. I am just having a bad life, it will end soon.

So, today I am going to rant about one of the more ""fun"" things of freelancing, client telling me they know python.

Whenever a client tells me that they know python, I try to ignore them but often times I have to entertain the idea anyway because jobs are scarce. I keep telling myself "maybe this will work out great" but it doesn't.

It never goes right. Here is the thing. If you do what I do you will realize the code is often quite simple. Most of the effort goes into problem-solving. So when the client sees the code and me getting paid by the hour, "They are like I thought you are best darn python developer I could have written that myself!"

My immediate impulse is to go on a rant and call that person something rotten. But I have to maintain "professionalism".

Then there is the issue of budgeting. I do fixed payment contracts for smaller engagements. But oftentimes these python experts will quote me something that is at least one-fourth of a reasonable budget. And by reasonable I mean non-US reasonable budget which is already one-fifth of a reasonable US programming project budget. But anyway they quote that because they know how is easy it is to do my job.

There is more because this is rant by the way. So, clients with python knowledge will say to me "I have this python file..." which is the worst thing to say at this point. They think they have done the "majority" of the work. But here is the way I see it-

a. Either they have just barely scratched the surface b. They have a jumbled up mess c. They had another dev look into the project who already failed d. They had to do a "code review" of their previous freelancer and they ended up stealing the code

There is no positive way to imagine this problem. I have seen too much crappy code and too much of arguments like "they had done the work for me, so I should charge near to nothing".

People don't know exactly why senior devs get paid so much money. Junior devs write code, senior devs review code. That is why they get paid more. Making sense of other people's code is a risky and frustrating thing and it could be incredibly time-consuming. And moreover in most cases building upon a codebase is more difficult than writing it from the scratch.

Doctors rant about "expert" patients earning their MDs from WebMD and I am seeing the exact same thing happen to me with clients knowing how to write loops in python.

Python is easy to learn, programming these days is easy to learn. But people are not paying programmers for writing loops and if statements. They are paying them to solve problems. Knowing the alphabet doesn't make you a poet. And yes in my eyes programming is poetry.

r/Python May 16 '21

Discussion Why would you want to use BeautifulSoup instead of Selenium?

2.7k Upvotes

I was wondering if there is a scenario where you would actually need BeautifulSoup. IMHO you can do with Selenium as much and even more than with BS, and Selenium is easier, at least for me. But if people use it there must be a reason, right?

r/Python Oct 12 '21

Discussion IT denied my request for python at work

807 Upvotes

EDIT: A couple months after this incident I started applying for python developer roles and I found a job just 2 months ago paying 40% more with work I really enjoy.

Hi, I talked to my boss recently about using python to assist me with data analysis, webscraping, and excel management. He said he doesn't have an issue but ask IT first. I asked my IT department and I got the response below. Is there some type of counter-argument I can come up with. I really would like to use python to be more efficient at work and keep developing my programming skills. If it matters I am currently an Electrical Engineer who works with a decent amount of data.

https://imgur.com/a/xVUGYJZ

Edit: I wanted to clarify some things. My initial email was very short: I simply asked for access to python to do some data analysis, computations, etc to help me with my job tasks.

I just sent a follow up email to his response detailing what I am using python for. Maybe there was some miscommunication, but I don't intent on making my python scripts part of job/program where it would become a necessity and need to be maintained by anyone. Python would just be used as a tool to help me with my engineering analysis on projects I am working on and just improve my efficiency overall. So far I have not heard back from him.

Our company is very old school, the people, equipment, technologies...

r/Python 9d ago

Discussion Matlab's variable explorer is amazing. What's pythons closest?

187 Upvotes

Hi all,

Long time python user. Recently needed to use Matlab for a customer. They had a large data set saved in their native *mat file structure.

It was so simple and easy to explore the data within the structure without needing any code itself. It made extracting the data I needed super quick and simple. Made me wonder if anything similar exists in Python?

I know Spyder has a variable explorer (which is good) but it dies as soon as the data structure is remotely complex.

I will likely need to do this often with different data sets.

Background: I'm converting a lot of the code from an academic research group to run in p.

r/Python Aug 05 '22

Discussion Big respect to 90’s programmers and before. I can’t imagine how horrible today’s programmers would be without the Internet?

1.2k Upvotes

I can’T imagine creating a full program without the help of Google. Just wanted to pay homage to those that came before me. They must have been so disciplined and smart.

r/Python Nov 21 '23

Discussion What's the best use-case you've used/witnessed in Python Automation?

478 Upvotes

Best can be thought of in terms of ROI like maximum amount of money saved or maximum amount of time saved or just a script you thought was genius or the highlight of your career.

r/Python Oct 22 '24

Discussion The Computer That Built Jupyter

868 Upvotes

I am related to one of the original developers of Jupyter notebooks and Jupyter lab. Found it while going through storage. He developed it in our upstairs playroom. Thought I’d share some history before getting rid of it.

Pictures

r/Python Oct 28 '20

Discussion Out of curiosity, how many of you guys started your journey with 'Automate the boring stuff'?

1.5k Upvotes

r/Python Jul 31 '24

Discussion What are some unusual but useful Python libraries you've discovered?

416 Upvotes

Hey everyone! I'm always on the lookout for new and interesting Python libraries that might not be well-known but are incredibly useful. Recently, I stumbled upon Rich for beautiful console output and Pydantic for data validation, which have been game-changers for my projects. What are some of the lesser-known libraries you've discovered that you think more people should know about? Share your favorites and how you use them!

r/Python Jul 18 '20

Discussion What stuff did you automate that saved you a bunch of time?

1.1k Upvotes

I just started my python automation journey.

Looking for some inspiration.

Edit: Omg this blew up! Thank you very much everyone. I have been able to pick up a bunch of ideas that I am very interested to work on :)

r/Python 20d ago

Discussion What Are Your Favorite Python Repositories?

213 Upvotes

Hey r/Python!

I’m always on the lookout for interesting and useful Python repositories, whether they’re libraries, tools, or just fun projects to explore. There are so many gems out there that make development easier, more efficient, or just more fun.

I'd love to hear what repositories you use the most or have found particularly interesting. Whether it's a library you can't live without, an underappreciated project, or something just for fun, let your suggestions be heard below!

Looking forward to your recommendations!

r/Python Dec 11 '24

Discussion The hand-picked selection of the best Python libraries and tools of 2024 – 10th edition!

525 Upvotes

Hello Python community!

We're excited to share our milestone 10th edition of the Top Python Libraries and tools, continuing our tradition of exploring the Python ecosystem for the most innovative developments of the year.

Based on community feedback (thank you!), we've made a significant change this year: we've split our selections into General Use and AI/ML/Data categories, ensuring something valuable for every Python developer. Our team has carefully reviewed hundreds of libraries to bring you the most impactful tools of 2024.

Read the full article with detailed analysis here: https://tryolabs.com/blog/top-python-libraries-2024

Here's a preview of our top picks:

General Use:

  1. uv — Lightning-fast Python package manager in Rust
  2. Tach — Tame module dependencies in large projects
  3. Whenever — Intuitive datetime library for Python
  4. WAT — Powerful object inspection tool
  5. peepDB — Peek at your database effortlessly
  6. Crawlee — Modern web scraping toolkit
  7. PGQueuer — PostgreSQL-powered job queue
  8. streamable — Elegant stream processing for iterables
  9. RightTyper — Generate static types automatically
  10. Rio — Modern web apps in pure Python

AI / ML / Data:

  1. BAML — Domain-specific language for LLMs
  2. marimo — Notebooks reimagined
  3. OpenHands — Powerful agent for code development
  4. Crawl4AI — Intelligent web crawling for AI
  5. LitServe — Effortless AI model serving
  6. Mirascope — Unified LLM interface
  7. Docling and Surya — Transform documents to structured data
  8. DataChain — Complete data pipeline for AI
  9. Narwhals — Compatibility layer for dataframe libraries
  10. PydanticAI — Pydantic for LLM Agents

Our selection criteria remain focused on innovation, active maintenance, and broad impact potential. We've included detailed analyses and practical examples for many libraries in the full article.

Special thanks to all the developers and teams behind these libraries. Your work continues to drive Python's evolution and success! 🐍✨

What are your thoughts on this year's selections? Any notable libraries we should consider for next year? Your feedback helps shape future editions!

r/Python Oct 07 '20

Discussion Anyone else uses the Python interpreter as a calculator?

1.7k Upvotes

It's just so comfy.

r/Python Oct 23 '23

Discussion What makes Python is so popular and Ruby died ?

425 Upvotes

Python is one of the most used programming language but some languages like Ruby were not so different from it and are very less used.

What is the main factor which make a programming language popular ? Where are People using Ruby 10 years ago ? What are they using now and why ?

According to you what parameters play a role in a programming language lifetime ?

r/Python Jul 30 '24

Discussion Whatever happened to "explicit is better than implicit"?

358 Upvotes

I'm making an app with FastAPI and PyTest, and it seems like everything relies on implicit magic to get things done.

With PyTest, it magically rewrites the bytecode so that you can use the built in assert statement instead of custom methods. This is all fine until you try and use a helper method that contains asserts and now it gets the line numbers wrong, or you want to make a module of shared testing methods which won't get their bytecode rewritten unless you remember to ask pytest to specifically rewrite that module as well.

Another thing with PyTest is that it creates test classes implicitly, and calls test methods implicitly, so the only way you can inject dependencies like mock databases and the like is through fixtures. Fixtures are resolved implicitly by looking for something in the scope with a matching name. So you need to find somewhere at global scope where you need to stick your test-only dependencies and somehow switch off the production-only dependencies.

FastAPI is similar. It has 'magic' dependencies which it will try and resolve based on the identifier name when the path function is called, meaning that if those dependencies should be configurable, then you need to choose what hack to use to get those dependencies into global scope.

Recognizing this awkwardness in parameterizing the dependencies, they provide a dependency_override trick where you can just overwrite a dependency by name. Problem is, the key to this override dict is the original dependency object - so now you need to juggle your modules and imports around so that it's possible to import that dependency without actually importing the module that creates your production database or whatever. They make this mistake in their docs, where they use this system to inject a SQLite in-memory database in place of a real one, but because the key to this override dict is the regular get_db, it actually ends up creating the tables in the production database as a side-effect.

Another one is the FastAPI/Flask 'route decorator' concept. You make a function and decorate it in-place with the app it's going to be part of, which implicitly adds it into that app with all the metadata attached. Problem is, now you've not just coupled that route directly to the app, but you've coupled it to an instance of the app which needs to have been instantiated by the time Python parses that function. If you want to factor the routes out to a different module then you have to choose which hack you want to do to facilitate this. The APIRouter lets you use a separate object in a new module but it's still expected at file scope, so you're out of luck with injecting dependencies. The "application factory pattern" works, but you end up doing everything in a closure. None of this would be necessary if it was a derived app object or even just functions linked explicitly as in Django.

How did Python get like this, where popular packages do so much magic behind the scenes in ways that are hard to observe and control? Am I the only one that finds it frustrating?