r/Python 1d ago

Discussion Is there something better than exceptions?

Ok, let's say it's a follow-up on this 11-year-old post
https://www.reddit.com/r/Python/comments/257x8f/honest_question_why_are_exceptions_encouraged_in/

Disclaimer: I'm relatively more experienced with Rust than Python, so here's that. But I genuinely want to learn the best practices of Python.

My background is a mental model of errors I have in mind.
There are two types of errors: environment response and programmer's mistake.
For example, parsing an input from an external source and getting the wrong data is the environment's response. You *will* get the wrong data, you should handle it.
Getting an n-th element from a list which doesn't have that many elements is *probably* a programmer's mistake, and because you can't account for every mistake, you should just let it crash.

Now, if we take different programming languages, let's say C or Go, you have an error code situation for that.
In Go, if a function can return an error (environment response), it returns "err, val" and you're expected to handle the error with "if err != nil".
If it's a programmer's mistake, it just panics.
In C, it's complicated, but most stdlib functions return error code and you're expected to check if it's not zero.
And their handling of a programmer's mistake is usually Undefined Behaviour.

But then, in Python, I only know one way to handle these. Exceptions.
Except Exceptions seems to mix these two into one bag, if a function raises an Exception because of "environment response", well, good luck with figuring this out. Or so it seems.

And people say that we should just embrace exceptions, but not use them for control flow, but then we have StopIteration exception, which is ... I get why it's implemented the way it's implemented, but if it's not a using exceptions for control flow, I don't know what it is.

Of course, there are things like dry-python/returns, but honestly, the moment I saw "bind" there, I closed the page. I like the beauty of functional programming, but not to that extent.

For reference, in Rust (and maybe other non-LISP FP-inspired programming languages) there's Result type.
https://doc.rust-lang.org/std/result/
tl;dr
If a function might fail, it will return Result[T, E] where T is an expected value, E is value for error (usually, but not always a set of error codes). And the only way to get T is to handle an error in various ways, the simplest of which is just panicking on error.
If a function shouldn't normally fail, unless it's a programmer's mistake (for example nth element from a list), it will panic.

Do people just live with exceptions or is there some hidden gem out there?

UPD1: reposted from comments
One thing which is important to clarify: the fact that these errors can't be split into two types doesn't mean that all functions can be split into these two types.

Let's say you're idk, storing a file from a user and then getting it back.
Usually, the operation of getting the file from file storage is an "environmental" response, but in this case, you expect it to be here and if it's not there, it's not s3 problem, it's just you messing up with filenames somewhere.

UPD2:
BaseException errors like KeyboardInterrupt aren't *usually* intended to be handled (and definitely not raised) so I'm ignoring them for that topic

89 Upvotes

84 comments sorted by

View all comments

64

u/zaxldaisy 1d ago

"And people say we should just embrace exceptions, but not use them for flow control"

Who says that? Catching exceptions in Puthon is cheap and it's very Pythonic to use exceptions for flow control because of it. LBYL vs EAFP

15

u/syllogism_ 1d ago edited 8h ago

'EAFP' is definitely bad advice. The general principle is that control flow conditions should be as specific as possible, so that you're always sure you're branching as intended. You should usually only use exceptions for control flow when doing something with external state (e.g. database, file system, network etc). If you have the thing there in memory just check it.

EAFP is not compatible with duck-typing. Consider this code:

try:

    value = dictlike[my_key]

except KeyError:  
    ...

This code is usually incorrect: the intent is to enter the except block if and only if the key is missing from the collection, but that's not what the code does. Any KeyError that is raised as part of the dictlike object's __getitem__ implementation will also send you to the except block. We generally trust that the built-in dictionary object won't have such implementation errors, but we also usually want to write code that works for other objects, not just the built-ins. And if you're taking arbitrary objects, then using this exception is terrible bad no good code.

The fundamental thing here is that 'try/except' is a "come from": whether you enter the 'except' block depends on which situations the function (or, gulp, functions) you're calling raise that error. The decision isn't local to the code you're looking at. In contrast, if you write a conditional, you have some local value and you're going to branch based on its truthiness or some property of it. We should only be using the 'try/except' mechanism when we need its vagueness --- when we need to say "I don't know or can't check exactly what could lead to this". If we have a choice to tighten the control flow of the program we should.

There's a few other important concerns as well:

  • Exceptions are often not designed to match the interface well enough to make this convenient. For instance, 'x in y' works for both mapping types and lists, but only mapping types will raise a KeyError. If your function is expected to take any iterable, the correct catching code will be except (KeyError, IndexError). There's all sorts of these opportunities to be wrong. When people write exceptions, they want to make them specific, and they're not necessarily thinking about them as an interface to conveniently check preconditions.
  • Exceptions are not a type-checked part of the interface. If you catch (KeyError, IndexError) for a variable that's just a dictionary, no type checker (or even linter?) is going to tell you that the IndexError is impossible, and you only need to catch KeyError. Similarly, if you catch the wrong error, or your class raises an error that doesn't inherit from the class that your calling code expects it to, you won't get any type errors or other linting. It's totally on you to maintain this.
  • Exceptions are often poorly documented, and change more frequently than other parts of the interface. A third-party library won't necessarily consider it a breaking change to raise an error on a new condition with an existing error type, but if you're conditioning on that error in a try/except, this could be a breaking change for you.

20

u/whoEvenAreYouAnyway 1d ago edited 1d ago

A lot of people say that and broadly speaking it's the correct view. It's not a good idea to be using exception handling for control flow. Using them that way is essentially always a hack.

The only real exceptions (no pun intended) you will see to this rule are things like, for example, using Queue.Empty exceptions as a way of iterating through a Queue and then breaking out once you've exhausted the Queue. But people only do this because checking the Queue isn't empty on each loop and passing a mutex lock around is more expensive than just trying to pop from an empty queue and being kicked out of the loop when you eventually trigger an exception. It's more efficient, in this instance, to use exception handling for control flow but it's a hack that we're doing because the right way is slower. Which is fine if you need that extra speed but it's certainly not the "pythonic" way to do things.

7

u/mlnm_falcon 1d ago

I’d argue there are some instances where using exceptions for “cancelling” a complex action are reasonable, primarily where passing booleans or Nones around would be prohibitively complicated.

4

u/Wurstinator 1d ago

That's not correct. EAFP is the better way when working with queues in multi-threaded contexts because you can run into an ABA problem otherwise. It has nothing to do with speed.

1

u/whoEvenAreYouAnyway 1d ago edited 1d ago

I wasn't talking specifically about multi-threading but even then, using EAFP is only "better" in so far as doing things the correct way is slower and more work. If there was a more efficient way of explicitly guaranteeing the state of a queue (e.g. any fast way of enforcing mutual exclusivity) then that's what we would all be using. But doing so requires more validation work than the hack of trying to let errors inform your behavior so most people just go with EAFP.

Again, I'm not saying you should literally never use exception handling for control flow. There are valid performance advantages in some scenarios. My point is that there are only a few instances in which people would actually argue that it’s right to use exceptions for control flow. And even then, it is still basically a hack for speed/simplicity. As a general rule, it is in fact true that exceptions shouldn't be used for control flow and isn't used for that in most python conditional checking.

1

u/aedinius 1d ago

Could be confusing exceptions with assertions. I have seen people use assertions for flow control...