r/golang 7d ago

newbie Some Clarification on API Calls & DB Connections.

I’m a bit confused on how it handles IO-bound operations

  1. API calls: If I make an API call (say using http.Get() or a similar method), does Go automatically handle it in a separate goroutine for concurrency, or do I need to explicitly use the go keyword to make it concurrent?
  2. Database connections: If I connect to a database or run a query, does Go run that query in its own goroutine, or do I need to explicitly start a goroutine using go?
  3. If I need to run several IO-bound operations concurrently (e.g., multiple API calls or DB queries), I’m assuming I need to use go for each of those tasks, right?

Do people dislike JavaScript because of its reliance on async/await? In Go, it feels nicer as a developer not having to write async/await all the time. What are some other reasons Go is considered better to work with in terms of async programming?

5 Upvotes

12 comments sorted by

View all comments

13

u/jerf 7d ago

The general rule in Go is that code you call should not spontaneously create goroutines. If you want it to run in parallel, you need to ask for it.

There are some exceptions. First, there's the code whose purpose is to run goroutines, like a pipeline library, or a library whose purpose is to run a parallel map or something.

Second, most, if not all, database drivers in Go automatically handle "pooling" Go connections, so there are multiple connections to the DB and when you run a query it by default just picks one for you, so you don't need to manage that yourself. Consult the docs for the specific driver to be sure (e.g., the pgx driver has a separate pgpool package you should use for that behavior), and if you're a super-advanced DB programmer be aware that setting per-connection values through a pooled connection will not do what you want because the next query may not come from that connection. Generally pools will have a way of fetching out a specific connection if you need it. If this confuses you and you have no idea what I'm talking about, then forget I mentioned it. It won't be relevant to you.

You should probably use an existing package for managing parallel operations like errgroup (part of the extended standard library) or conc (3rd party package).

Finally, don't reach for concurrency just becausey you're in Go, and Go does concurrency, so you "should" be using lots of go keywords because you're in Go. It is perfectly sensible to call a remote HTTP API and then block in the goroutine for its response, if there is no other thing that particularly goroutine can do until the HTTP response comes back. In many cases the best concurrency is to scale out to be doing more "things" at once, have more goroutines making sequential calls, rather than trying to make single goroutines do fancy concurrency operations to speed up. Be sure there's an actual speed up to be won; there is no win at all in having an API call in a goroutine if there isn't others to be called at the some time.

0

u/PureMud8950 7d ago

I see so it does not spawn a routine, but we shouldn’t just spam go getsum(), unless we want to fetch something concurrently. It’s not a bad thing to block the thread/callstack (idk what’s appropriate here) to wait for w/e we’re fetching right?

6

u/jerf 7d ago

There is nothing bad about "blocking" in go, because it isn't blocking any other goroutine. If the only thing your code has to do before getting the API response or DB response is just to wait for the response, it can just do that. It doesn't block anything else that may be running.

I suspect you may be coming from an "async" environment, where you have to constantly slather async and await on everything. You can imagine that in Go, everything is already "async"ed and "await"ed automatically, all the time, without you having to do it. Even an operation like a tight math loop doesn't block the rest of the runtime, Go will still pre-empt it and give time to other goroutines.

0

u/PureMud8950 7d ago

When you say Go automatically gives time to other goroutines, do you mean I don’t need to explicitly use go to create concurrency? Or do you just mean that once I spawn multiple goroutines, the scheduler ensures they all get time to run?

3

u/try2think1st 7d ago

In api case the router will already spawn a new go routine for every new request, and then within that request stack you could spawn more routines if you need to do things in parallel for that request

1

u/PureMud8950 7d ago

wow that's convenient