There's no IB OR Ethernet underneath- I could have implemented this properly as it's own distinct transport kind, but seemed easier just to pretend to be something that is already known.

The 'the chip that understands both TB and IB and translate RDMA requests between the two' in this instance is your CPU, so orders-of-magnitude worse latency than an ASIC, but still better than anything on top of IP/Ethernet. I think there's also potential to do device-initiiated RDMA, where e.g GPU itself can write to some mailbox and have message appear across the abstracted transport in another GPUs mailbox. Even if the CPU is involved in shuffling pointers across mailboxes it doesn't necessarily mean it'll be a bottleneck

- On a single cable with two rails available, the thunderbolt-net grabs one and uses that. Without patching the kernel, there's no way to make it present a second interface using the remaining pair.

- If you had a second cable between the machines (for 4 total rails), thunderbolt-net will still only grab one rail, because the abstraction across which it's making the links sees an identical peer at the end of both links and so falls into the same trap as above. There is no LRO/GRO anyway (or it's buggy- I forget) on the linux version.

- Why you only get 10G rather than 20G on single pair- actually, this might be something specific to the Strix Halo SoC that I was testing on- on a different (still AMD) chipset and an Apple TB5 Mac I did see closer to 22G in one direction, but still 8 in the other. The Strix Halo NHI seems to be 'stripped down' (as expected, for mobile) in ways I don't really understand.

- Intuition on why- I can't point you to the line number, but I think it has to do with a fixed 4kb page size when communicating with the NHI that ends up becoming a bottleneck, perhaps 16kb pages on aarch64 apple help here?

Good q about 'bridging into actual InfiniBand', I don't know the answer there either. My naive understanding would be that: since this is host-initiated RDMA (it's still the host cpu invoking into dma buffers, though they may be device-memory mapped), actually it should work fine, at least between two machines? I'm curious enough to try- I have a couple of machines with thunderbolt AND RoCE-capable NICs- the experiment is to see if we can use this across diverse transports simultaneously? I think this is what it does already (since the MacOS FA57 vs linux native are already 'different transports'), but say if you have a better scenario to demonstrate what 'bridging into actual infiniband' would look like!

Comments URL: https://news.ycombinator.com/item?id=48334042

Points: 4

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=45515338

Points: 1

# Comments: 0

As to whether the cause is Tory ban of onshore wind- I'm skeptical. I don't think wind is any cheaper than gas once you include the necessary energy storage capacity- especially when there is huge amounts of gas sitting under the north sea which the new Labour government is planning to ban pumping: https://www.forbes.com/sites/gauravsharma/2024/07/12/sudden-...

lspci | grep ASP c2:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 52)

[1] https://www.aliexpress.com/item/1005007369816019.html [2] https://github.com/sipeed/NanoKVM

I put it in a fanless case (https://streacom.com/products/fc9-fanless-micro-atx-case/) with mellanox connectx-4 nic, now I can route/NAT my internet connection in silence at line rate (25Gbps)

I'm building a product which will help with this and would be really grateful if you could answer some questions. Some of the usual ones:

- What problems to do you currently have? - Does cost affect how you chose your current provider, and if so, what sort of discount would make you consider switching?

Domain-specific: - What sort of models are you running, at what scale? - How frequently do you change/update the model?

and some more esoteric:

- Does censorship matter to you? - Does open-source matter to you?

Email/telegram in profile if you're happy to chat, also grateful for any replies below if not

Comments URL: https://news.ycombinator.com/item?id=39747305

Points: 3

# Comments: 0