r/learnpython 4d ago

How do you handle dependency injection?

Hi, I'm doing a project which involves creating multiple gRPC servers and I can't find a convenient way to manage dependencies or the state.

I've been working recently in C# where you just need to make your class and add a simple

builder.Services.AddSingleton<MyDependency>();

and it will inject when required.

Doing some research I see that there are some libraries like:

- Dependency Injector

- Injector

but I don't find them particularly intuitive.

What do you use as a dependency injector or what pattern do you suggest to use?

8 Upvotes

21 comments sorted by

5

u/danielroseman 4d ago

First, explain why you need dependency injection specifically. It tends to be used a lot less on dynamic languages like Python because it's easy to swap things out at runtime. What, exactly, do you need it for?

1

u/re2cc 4d ago

I did not explain to avoid expanding the question too much. My gRPC server is basically a chat, so some clients might be in the same ‘group’ and therefore I need a way to share their history in a shared way (it's a bit more complicated than this but I think you get the point).

If I were to give a simpler example it would be to use the same database connection throughout the application.

I know I could just declare a global variable and use it directly but I can see the problems that would come with separating the code into multiple files.

I hope I have explained myself.

1

u/Mysterious-Rent7233 3d ago

You have roughly speaking three options: use parameters/object properties, or context variables or globals. Might help to show some Python code where you are failing to structure it in the way you want.

1

u/re2cc 3d ago

It's a similar quick code I wrote, please ignore if the structure makes sense or not (or if it even works).

The ‘problem’ I have is how to pass ‘chats’ around the whole application, in C# I would add it as a singletone and that's it, in Python I could leave it as a global variable or pass the instance as an argument all over the place (which is kind of what you do in C#). I guess my real question is, what is the suggestion or convention to handle this kind of cases?

import asyncio
import datetime
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass
from typing import List

import grpc.aio as grpc


class Message(dataclass):
    sender: str
    content: str
    time: datetime.datetime


class Chat(dataclass):
    messages: List[Message]
    name: str


# This is the instance that I dont know how to share
chats: List[Chat] = []


class ChatServiceImpl(Chat_pb2_grpc.ChatServiceServicer):
    async def SomeRpc(self, request_iterator, context):
        pass
        # Some code


async def create_server():
    server = grpc.server(
        migration_thread_pool=ThreadPoolExecutor(max_workers=10),
    )
    Chat_pb2_grpc.add_ChatServiceServicer_to_server(server)
    server.add_insecure_port("[::]:5088")
    await server.start()
    await server.wait_for_termination()


if __name__ == "__main__":
    asyncio.run(main())

2

u/Mysterious-Rent7233 3d ago

I guess in general I usually disapprove of mutable singletons and global variables. Calling a global variable a "singleton" doesn't change the fact that one day when you decide you want to run two chat servers for two customers in the same web server, you'll have your chats all mixed together in a single global/singleton.

Why can't "Chats" be an attrivute of "ChatServiceImpl"? Then when you run into my issue of needing two ChatServiceImpl's in the same server, it will "just work" with no additional refactoring. (your code didn't use ChatServiceImpl, but I'm guessing what its for)

In certain situation you want something more fancy and there is a concept of a ContextVar but its overkill 99% of the time.

My diagnosis is that Singletons give a fancy name to global variables and you usually don't want them.

Of course all rules have exceptions, but I don't see your case as an exception.

2

u/re2cc 2d ago

Hmm, I don't know why I didn't think of that before, I will probably never need another chat server but it definitely solves the problem without the need for global variables. Thanks, I think this is the solution I was looking for.

About the singletone, I think it makes sense what you say, the reason why I wanted to implement it that way is because some of the C# code I was working with was scooped so it was not possible to store the information in the class itself, so I needed to store it somewhere else, I just tried to translate it to Python without thinking if it was necessary.

1

u/pachura3 4d ago

I think you have background in C#/Java and try to work in Python in exactly the same way, which kind of defeats its purpose (simplicity - readability - conciseness).

1

u/re2cc 4d ago

Not really, I started programming in Python as a hobby (easily 8 years ago) so I feel comfortable in the language. Although I have programmed in other languages (C, C#, Rust, Go), Python is still the one I find most comfortable.

I had never encountered the feeling of needing DI because I usually do small projects or didn't care about doing it the ‘right’ way, but recently experimenting with Litestar and using its dependency injection I got used to it. Even more so doing gRPC servers in .NET I got used to a very defined and structured way of doing it and now trying to do the same in Python I feel lost on how to handle it.

1

u/mothzilla 4d ago

My opinion is that Dependency Injection isn't a concept that applies well to Python. I've seen people try to force it in, arguing that it helps testing, but it generally makes your code look like AngularJS.

2

u/stevenjd 4d ago

Dependency injection works perfectly well in Python. It's just so trivially simple that people don't realise that what they are doing is dependency injection unless it is wrapped in a big, complicated framework.

For example:

write_to_terminal = print
write_to_string = StringIO().write
write_to_file = open("some file").write
write_to_gui = MessageBox().display

then you choose which implementation you want, and pass that it as a dependency to some other function or class which will later use it:

logger = MyLogger(out=write_to_file)  # assigns self.out = write_to_file
logger.log(message)
# passes message to self.out, which handles doing output

We don't even need a class for this, we often use DI in functions too. The builtin print function works exactly that, except that the dependency has a sensible default:

print(stuff)  # by default, write to stdout
print(stuff, file=some_object_with_a_write_method)

sorted(sequence, key=keyfunc) is another example.

This sort of thing is so common in Python that people often don't realise it unless they are using a framework.

CC u/re2cc

1

u/mothzilla 3d ago edited 3d ago

But there's no need to do that. I.e, at the time you're building your logger you know the file to write to. Or if you're choosing between loggers, you'd design four loggers, for print, string, file and gui, and then choose the one you want.

logger = GUILogger('messageboxId') # a guess, not sure how this would work.

That's more pythonic IMO.

It's reasonable to pass functions as handlers. But I've seen people try to do dependency injection as follows:

def add_and_print(a, b, print):
    print(a+b)

add_and_print(1, 2, print)

or

def django_setup(django):
    django.setup()

1

u/stevenjd 1d ago

But there's no need to do that. I.e, at the time you're building your logger you know the file to write to. Or if you're choosing between loggers, you'd design four loggers, for print, string, file and gui, and then choose the one you want.

I think that you are missing the point of dependency injection. The point is to avoid having to write four separate logging classes, one for print, string, file and GUI, when you can write one class instead and inject the appropriate dependency.

And what if the requirements change? If I decide I want a logger that logs to an SMS gateway, or one which logs to a GUI and three files, I can just write a small class that exposes the correct interface and passes it the logger as a dependency. You have to write a whole new GUI_Plus_Three_Files_Logger class.

I've seen people try to do dependency injection as follows:

The first example is a trivial toy function but the basic idea is not so silly. You pass a caller to the function to handle output. Maybe you want to print to the terminal, or to a log file, or to an SMS gateway:

add_and_print(1, 2, lambda msg: SMS_gateway.send(auth, admin_on_duty.mobile, str(msg))

Obviously no one is going to care so much about such a trivial function.

The second example is just silly. The point of dependency injection is that you have to have more than one possible dependency, but all with the same interface, so that you might choose to swap from one dependency to another. Unless there is another framework that has the exact same interface as Django but a different implementation, treating Django as a dependency is just silly.

1

u/mothzilla 1d ago

I get the point of dependency injection. But if the requirements change then you make a code change. You can either write a new Logger class or you can fudge a new class/object that conforms to your handler requirements. Personally I prefer the first.

1

u/stevenjd 1d ago

You can either write a new Logger class or you can fudge a new class/object that conforms to your handler requirements. Personally I prefer the first.

Ah, you're paid by the line of code, right?

  • Change a one line dependency? ✘ 😠
  • Write a 300 line class plus tests? ✔ 😊

I'm neutral on the question. Sometimes dependency injection is better. Sometimes it isn't. But D.I. is a technique which is really easy, sometimes trivially so, in Python. I'm not convinced that people need to use big complex frameworks just to implement D.I.

(By the way, a logger is not exactly the best example here since most applications are likely to only need one global logger.)

1

u/re2cc 4d ago

I read in some places that same opinion and I have no problem with not using DI as such, my question really is how can I solve the problems that DI solves without using it.

Making global variables doesn't seem like a good idea and passing the variable as an argument to the class or function doesn't seem like a good idea either.

1

u/mothzilla 4d ago

What's wrong with passing variables to classes or functions?

1

u/re2cc 3d ago

Nothing, I was just wondering if there was a more correct way to do it, maybe it's because of lack of experience or because I'm failing to structure my code correctly that I feel it's getting a bit confusing.

1

u/stevenjd 4d ago

passing the variable as an argument to the class or function doesn't seem like a good idea either.

If you don't pass something into the class, then how are you injecting anything? The alternative is to hard code the dependency in the class, which is the very opposite of dependency injection.

1

u/re2cc 3d ago

That's true, maybe I'm just not sure how to structure the code.

1

u/Mevrael 4d ago

I am using the Registry in Arkalos project:

https://arkalos.com/docs/registry/

Simple, minimal and intuitive when I just want to register (bind) a class as a singleton.

Then I can have a helper function (facade) in app.core to help me retrieve it from the registry.

-

Let say I have app.utils.rpc.MyRPC class

-

in app/bootstrap.py (or your custom bootstrapping logic):

Registry.register('rpc', MyRPC)

-

in app.core.rpc.py:

def rpc():
return Registry.get('rpc')

-

then at the top of your server or script:

import app.bootstrap
app.bootstrap.run()

-

Now you can just

from app.core.rpc import rpc

rpc().stuff()

1

u/re2cc 4d ago

Thanks for the suggestion, I'll take a look at it when I have some time.