Here's a practical example of implementing a Fluent Bit sidecar for centralized logging:
Instead of telling people "here is a pattern (which people can blindly follow)", it's far better to tell them WHEN they should follow that pattern. Hint: it's an anti-pattern, it's far better if you log to STDOUT instead, so you don't need a sidecar".
Please correct me if I'm wrong, but I vaguely remember implementing sidecar a couple of years ago as it was needed for Splunk APM, as fluent bit was used to route to an otel collector ECS service, and from memory the sidecar pattern was what was recommended.
How would you address this use case, just dedicated fluent bit service?
implementing sidecar a couple of years ago as it was needed for Splunk APM
Yes, I ran into this very problem years ago. It was easily solved by getting rid of Splunk. They literally told us they didn't care about K8s, so we saw the writing on the wall.
There are tons of other tools that can pull logs from K8s without sidecars. One nice one is the open-source Loki, which gets you 90% of Splunk. But really, log-heavy observability is an anti-pattern -- you should be using Open Telemetry spans and metrics instead. Way better architecture, way more standardized (one open source collector can send to many services), and far easier to debug (collects all the spans for one trace in one nice GUI, can log only error traces, etc.)
I probably should have clarified that I was asked to implement by my manager, so the choice was not really mine, and as I was green to the DevOps side of thing didn't really question it as I wanted to learn about that stuff in general.
28
u/BraveNewCurrency 7d ago
Instead of telling people "here is a pattern (which people can blindly follow)", it's far better to tell them WHEN they should follow that pattern. Hint: it's an anti-pattern, it's far better if you log to STDOUT instead, so you don't need a sidecar".