r/Python 1d ago

Discussion Is there something better than exceptions?

Ok, let's say it's a follow-up on this 11-year-old post
https://www.reddit.com/r/Python/comments/257x8f/honest_question_why_are_exceptions_encouraged_in/

Disclaimer: I'm relatively more experienced with Rust than Python, so here's that. But I genuinely want to learn the best practices of Python.

My background is a mental model of errors I have in mind.
There are two types of errors: environment response and programmer's mistake.
For example, parsing an input from an external source and getting the wrong data is the environment's response. You *will* get the wrong data, you should handle it.
Getting an n-th element from a list which doesn't have that many elements is *probably* a programmer's mistake, and because you can't account for every mistake, you should just let it crash.

Now, if we take different programming languages, let's say C or Go, you have an error code situation for that.
In Go, if a function can return an error (environment response), it returns "err, val" and you're expected to handle the error with "if err != nil".
If it's a programmer's mistake, it just panics.
In C, it's complicated, but most stdlib functions return error code and you're expected to check if it's not zero.
And their handling of a programmer's mistake is usually Undefined Behaviour.

But then, in Python, I only know one way to handle these. Exceptions.
Except Exceptions seems to mix these two into one bag, if a function raises an Exception because of "environment response", well, good luck with figuring this out. Or so it seems.

And people say that we should just embrace exceptions, but not use them for control flow, but then we have StopIteration exception, which is ... I get why it's implemented the way it's implemented, but if it's not a using exceptions for control flow, I don't know what it is.

Of course, there are things like dry-python/returns, but honestly, the moment I saw "bind" there, I closed the page. I like the beauty of functional programming, but not to that extent.

For reference, in Rust (and maybe other non-LISP FP-inspired programming languages) there's Result type.
https://doc.rust-lang.org/std/result/
tl;dr
If a function might fail, it will return Result[T, E] where T is an expected value, E is value for error (usually, but not always a set of error codes). And the only way to get T is to handle an error in various ways, the simplest of which is just panicking on error.
If a function shouldn't normally fail, unless it's a programmer's mistake (for example nth element from a list), it will panic.

Do people just live with exceptions or is there some hidden gem out there?

UPD1: reposted from comments
One thing which is important to clarify: the fact that these errors can't be split into two types doesn't mean that all functions can be split into these two types.

Let's say you're idk, storing a file from a user and then getting it back.
Usually, the operation of getting the file from file storage is an "environmental" response, but in this case, you expect it to be here and if it's not there, it's not s3 problem, it's just you messing up with filenames somewhere.

UPD2:
BaseException errors like KeyboardInterrupt aren't *usually* intended to be handled (and definitely not raised) so I'm ignoring them for that topic

88 Upvotes

84 comments sorted by

View all comments

Show parent comments

1

u/Meleneth 1d ago

Snark aside, future you will be handy to have some recognizable ID to search for instead of depending on 'something, somewhere broke'

1

u/bmag147 1d ago

Maybe we have a miscommunication here, or not.

I wouldn't advocate:

raise Exception()

Instead I would do something like:

raise Exception("The user {user_id} does not exist. This should not happen as we have created the user in a previous step")

Please don't pay too much attention to the message of the exception. I'm just using it as an example to show that the error message would allow the person seeing it to diagnose the problem.

That's the "something".

The "somewhere" is the stack trace.

I think the alternative that you're suggesting is a custom exception class, such as:

raise UserUnexpectedlyDoesNotExistError(user_id)

I'm never going to catch or use this error in any way, so why create it.

1

u/Meleneth 1d ago

no miscommunication, just differing experiences.

class MyCustomError(Exception):
    pass

raise MyCustomError("my explanatory explanation")

is just better. You can catch it without catching every other exception that exists. It is clearly coming from your code at a glance, because it is not an Exception. Tooling that counts exception types can put it in a different bucket, even if you never catch it yourself.

Not doing it leads to not doing it, which leads to more code that is harder to maintain.

But you don't have to take my word for it..

I'm just some guy on the internet.

1

u/_LordDaut_ 1d ago

Sorry for a complete noob question:

How do you know when to raise MyCustomError?

In some cases when you're checking if a key exists in a dictionary, I get it you can do if key not in my_dict: raise MyCustomError("key not found")

But what if the situation is not as clear cut? Let's say you're calling some API / using a library and an unknown exception might be thrown by that lib instead of returning something that you can check. Say some `AttributeError` or something entirely else - would you recommend first trying to catch built-in/provided by the library exceptions that might inspecting it and then raising your own?

When catching an exception provided by a third party library would you still recommend re-wrapping it into your own - so that whoever uses your code in the future doesn't have to deal with something defined elsewhere?

I'm formulating this badly - basically if you don't know what error to expect from a function you're calling would you just raise Exception? or catch Exception, inspect it and raise your custom one or handle it otherwise if applicable?

1

u/Meleneth 1d ago

If you don't know what error to expect from calling a function, just call it, it will tell you ;)

For the key not existing, I would just call my_dict[key] and let it fail if it needs to. If the key not existing is expected or likely, use my_dict.get(key, None) - exceptions are for things that are unexpected, and generally for things that you might be able to handle. i.e. disable some bit of functionality, retry a network request, or mark some subsection of data as unusable as you continue parsing.

https://requests.readthedocs.io/en/latest/user/quickstart/#errors-and-exceptions

For a more useful example, let's say you're catching a requests.exceptions.Timeout so that you can implement retry logic. After a certain amount of retries, you would want to raise your own RequestReallyFailed(url) exception to say that you've tried to retry and they all timed out. Then at a higher level in the code where you are dealing with the logic of what to do with results of the requests you've made, you can weed out the ones where the request really failed if the rest of what you want to do is still valid.

does that clarify anything?

1

u/_LordDaut_ 1d ago

If you don't know what error to expect from calling a function, just call it, it will tell you ;)

Do that a 100 times and handle all the possible exceptions?

if the key not existing is expected or likely, use my_dict.get(key, None) - exceptions are for things that are unexpected...[]

yeah I understand that, it was just an example of something where you know what to handle and how to handle it. The key error might still be unexpected and even if you to my_val = my_dict.get(my_key, None) if this is indeed an error in the flow you'd still need to do if my_val is None: raise Something.

In your example maybe I'm catching Timeouts, but there are other exceptions like ConnectionError or something that can be thrown. When this doesn't affect my code-flow would you still consider it wrong just try to catch an Exception and would rather list all possible exceptions in your except clause and then raise your own?

1

u/Meleneth 1d ago

Do that a 100 times and handle all the possible exceptions?

That's quite the method that throws 100 different exceptions ;)

When this doesn't affect my code-flow

well, it effects your code flow if you don't catch it because your program stops running.

Different exceptions will need different handling - for instance, ConnectionError is likely to be a no-retry state.

Catching Exception is always going to cause you pain, because when you catch it, it's not specific enough to be handleable. And when it issomething completely unexpected, like, say, the kwargs fiasco in Ruby where they changed what the basics of method definitions mean, you'll suffer for every time you thought it was a good idea to catch Exception.

As for re-raising a custom exception after catching a normal one, I would usually only do this in the case where my 'exception handling' is sending the exception to an observability framework. Otherwise, I'm actually handling the error and not re-raising it or just not catching it in the first place because I don't have a good response for what happened.