The authors mention that NCCL and Ray initialization were too slow (see quote below), but from the description it sounds like they’ve reimplemented a layer that’s increasingly being standardized by frameworks like nixl and uccl.

> Distributed executor: Inference engines support model parallelism via distributed executors (e.g., Ray [32] and NCCL [9]), whose initialization takes tens of seconds.

From there, what’s helped me most is a cycle of reading new material, building prototypes and exploring how an open source system solves similar problems. I've definitely hit that wall as systems programming can get confusing fast.

I’ve also noticed that I sometimes get stuck trying to make something perfect before I’ve even started experimenting. Forcing myself to build the lowest-effort version of an idea has been surprisingly productive. Debugging things that don’t work is frustrating, but that failure often reveals insights I wouldn’t have discovered if I were overanalyzing.

You’ve probably seen some of these resources already, but just sharing in case any of it’s useful. I work with eBPF full-time and had many similar challenges along the way, but recommend jumping back in when you have the time.

Curious to hear what hurdles you ran into.

I work on CNCF Pixie, which uses DWARF for eBPF uprobes and dynamic instrumentation. While I understood how our project uses DWARF, the book made many details much clearer for me.

- Pixie (https://px.dev) -- which I contribute to

- Beyla (https://github.com/grafana/beyla)

- Coroot (https://github.com/coroot/coroot)

If you are interested in the details and how the strategy for this tracing has evolved, you can learn more in this blog (https://blog.px.dev/ebpf-tls-tracing-past-present-future/).

Pixie [1] is a similar project and offers the self hosted model you are looking for.

We also support 11 application protocols [2] with TLS handshake tracing and MQTT support coming soon (encrypted traffic tracing has been supported for a long time).

[1] https://px.dev

[2] https://docs.px.dev/reference/datatables/

Pixie (https://px.dev) can be installed in under 5 mins and gives this level of visibility across all applications. No need to change your application (wrap in `subtrace run`) to get instant visibility.

We also support 11 application protocols (https://docs.px.dev/reference/datatables/) with TLS handshake tracing and MQTT support coming soon (encrypted traffic tracing has been supported for a long time).

Comments URL: https://news.ycombinator.com/item?id=41245214

Points: 5

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=41045802

Points: 1

# Comments: 0