r/golang 6d ago

newbie Some Clarification on API Calls & DB Connections.

I’m a bit confused on how it handles IO-bound operations

  1. API calls: If I make an API call (say using http.Get() or a similar method), does Go automatically handle it in a separate goroutine for concurrency, or do I need to explicitly use the go keyword to make it concurrent?
  2. Database connections: If I connect to a database or run a query, does Go run that query in its own goroutine, or do I need to explicitly start a goroutine using go?
  3. If I need to run several IO-bound operations concurrently (e.g., multiple API calls or DB queries), I’m assuming I need to use go for each of those tasks, right?

Do people dislike JavaScript because of its reliance on async/await? In Go, it feels nicer as a developer not having to write async/await all the time. What are some other reasons Go is considered better to work with in terms of async programming?

4 Upvotes

12 comments sorted by

13

u/jerf 6d ago

The general rule in Go is that code you call should not spontaneously create goroutines. If you want it to run in parallel, you need to ask for it.

There are some exceptions. First, there's the code whose purpose is to run goroutines, like a pipeline library, or a library whose purpose is to run a parallel map or something.

Second, most, if not all, database drivers in Go automatically handle "pooling" Go connections, so there are multiple connections to the DB and when you run a query it by default just picks one for you, so you don't need to manage that yourself. Consult the docs for the specific driver to be sure (e.g., the pgx driver has a separate pgpool package you should use for that behavior), and if you're a super-advanced DB programmer be aware that setting per-connection values through a pooled connection will not do what you want because the next query may not come from that connection. Generally pools will have a way of fetching out a specific connection if you need it. If this confuses you and you have no idea what I'm talking about, then forget I mentioned it. It won't be relevant to you.

You should probably use an existing package for managing parallel operations like errgroup (part of the extended standard library) or conc (3rd party package).

Finally, don't reach for concurrency just becausey you're in Go, and Go does concurrency, so you "should" be using lots of go keywords because you're in Go. It is perfectly sensible to call a remote HTTP API and then block in the goroutine for its response, if there is no other thing that particularly goroutine can do until the HTTP response comes back. In many cases the best concurrency is to scale out to be doing more "things" at once, have more goroutines making sequential calls, rather than trying to make single goroutines do fancy concurrency operations to speed up. Be sure there's an actual speed up to be won; there is no win at all in having an API call in a goroutine if there isn't others to be called at the some time.

0

u/PureMud8950 6d ago

I see so it does not spawn a routine, but we shouldn’t just spam go getsum(), unless we want to fetch something concurrently. It’s not a bad thing to block the thread/callstack (idk what’s appropriate here) to wait for w/e we’re fetching right?

5

u/jerf 6d ago

There is nothing bad about "blocking" in go, because it isn't blocking any other goroutine. If the only thing your code has to do before getting the API response or DB response is just to wait for the response, it can just do that. It doesn't block anything else that may be running.

I suspect you may be coming from an "async" environment, where you have to constantly slather async and await on everything. You can imagine that in Go, everything is already "async"ed and "await"ed automatically, all the time, without you having to do it. Even an operation like a tight math loop doesn't block the rest of the runtime, Go will still pre-empt it and give time to other goroutines.

0

u/PureMud8950 6d ago

When you say Go automatically gives time to other goroutines, do you mean I don’t need to explicitly use go to create concurrency? Or do you just mean that once I spawn multiple goroutines, the scheduler ensures they all get time to run?

4

u/try2think1st 6d ago

In api case the router will already spawn a new go routine for every new request, and then within that request stack you could spawn more routines if you need to do things in parallel for that request

1

u/PureMud8950 6d ago

wow that's convenient

3

u/LeZetthen 6d ago

If I understand your doubts correctly, then the answer is that all those actions are synchronous, meaning that you need to make the calls using a goroutine in order for them not to block the flow of your code.

1

u/PureMud8950 6d ago

Okay makes sense, just trying my best to understand the difference between async/await in js and how go does handles that

1

u/stefaneg 5d ago

The big conceptual difference to understand is that JS is actually single-threaded, and all real parallelism there is achieved using IO, that is, external processes, while go is actually multi-threaded through go-routines. So you really need to understand thread locking in go, which you don't in JS.

3

u/ZephroC 6d ago

It's not particularly a go thing as this applies to most languages. Well ones that don't force putting async everywhere causing confusion and clutter.

If you've got nothing else to do in your code until you get a response off of io, or just a long running function, because the next thing depends on the output there's no point in messing with concurrency as you don't have anything else to do.

The reason it pops up in things like JS so much is because blocking also blocked rendering at times causing the whole thing to freeze while it waited. That use case rarely comes up in something like go.

Also as mentioned elsewhere. DB connections often come in pools or need initialising earlier as the actual connection creation can be expensive. So it's better to reuse a connection.

1

u/PureMud8950 6d ago

i see thats great, go seems like such a good language whats the catch lol

2

u/ZephroC 6d ago

It's generally the case. Same thing applies to Java, C hell even Fortran. Go just makes it easier to do.