The X-Ray daemon and SDKs are all deprecated now in favor of OTel. Things like enchrichment of resource level traces for things like the DynamoDB client in v3 of the AWS JS SDK don't work with the X-Ray SDK. And they never will now. You're now recommended to use the AWS Distro for OpenTelemetry setup and OTel SDKs. The performance overhead of this is heavy, with big cold-start penalties.

Compare this with how the Datadog layer does adaptive flushing and performs relatively much better. Rotel is also promising in this space. But right now, OTel feels immature and things are being deprecated without the replacement being fully baked.

It doesn't block, but it does consume compute/memory resources and takes forever to startup[0][1]. To be fair, Rotel is promising in this regard[2].

[0]: https://github.com/open-telemetry/opentelemetry-lambda/issue...

[1]: https://github.com/aws-observability/aws-otel-lambda/issues/...

[2]: https://github.com/rotel-dev/rotel

I don't mean to disparage anyone working on OTel. I can appreciate that it has ambitious goals and it's not an easy problem to get alignment and interop here. Especially with all the stakeholders involved. But as a user, it feels simultaeneously over-engineered and under-engineered.

Gateway collectors are unavoidable because various SaaS platforms require you to be running publicly reachable endpoints to send telemetry to.

In a runtime like Lambda, how would you avoid the need to run an edge collector? The only thing that comes to mind is to write to logs and then have a log stream processor that then writes to your gateway collector. Other than that, it seems unavoidable, no? Sure, in something like Fargate you could go app to sink. But even that has its own tradeoffs.

1. Every major vendor is still in some weird alpha/beta support for OTel even after all this time.

2. The performance hit is substantial and makes you question what the point of performance instrumentation is if you need twice as much compute/RAM to run the same workload now.

3. Serverless runtimes pay a heavy penalty for cold starts with OTel.

4. You're basically forced to run both gateway collectors and edge collectors for any realistic usage.

5. You still need to configure destination exporters in unique ways. This leaves you questioning what the value of OTel was.

6. Vendors that go beyond the scope of what OTel covers still need their own bespoke instrumentation. What was the point of any of this then?

Yes, Sentry or whoever else could build a Codeberg integration, or a Forgejo one, or a Gitlab one. But these would all be bespoke. There is no standard. So the one with the biggest market share ends up with the most integrations. This is where we are today.

I'm all for open standards. But I don't even know of a proposal for such a standard.

Sure, with AI tooling, you can vibe code your own browser extensions or other integrations in some cases. But for a lot of platforms, that just won't cut it. That's before we even get into the maintenance burden of that.

What makes Github special is the ecosystem and integrations. Sentry, Linear, and so many other tools have special UX around Github. You need centralization (or at the very least, a standard) for that.

It sucks to see Github's fall from grace, but I just can't bring myself to use a service like this under the custody of SpaceX. I've been saying this for a while, but I really wish Linear would swoop in and become the unified platform for software development.

Comments URL: https://news.ycombinator.com/item?id=48243293

Points: 197

# Comments: 63