Edit: Their benchmarks are also run using ZRAM, which compresses pages before writing to swap. Not sure what the performance overhead of that is, but it's probably quite a bit.

First of all, it's a userspace program hooking the nbd driver, which is known for being slow. It also uses a bounce buffer in userspace before transferring to the GPU. So when the kernel needs to swap a page, it has to first copy it into a userspace facing buffer. The userspace program that has to wake back up and issue the cuda operation to copy the page into device memory.

nbd also doesn't really do a good job of supporting high queue depth or merging adjacent accesses. So if the kernel is issuing a bunch of 4K page swaps without any coalescing, you're going to end up with at least million kernel/userspace context switches per second just to handle 4 GB/s (4 GB / 4K page), let alone 64 GB/s. And that's just the NBD portion, forget the mess that is the NVIDIA driver. PCIe can move a lot of data, but in order to get anything even resembling the full bandwidth, you have to have use DMA engines with long page lists. Having to set up a transfer for every 4K page over PCIe will not reach full saturation of the bus.

Swapping to NVMe is a very optimized path -> the swapper can submit lists of pages directly to the NVMe driver and the controller can DMA them directly out of RAM, no copies or context switches CPU side at all.

This could probably be improved by migrating to the ublk driver as it might let you avoid the userspace bounce buffer. It'd also be able to have multiple write queues to at least set up CUDA copies in parallel.

The problem with putting (system) RAM on a PCIe card is that PCIe is not a cache-coherent interconnect. If you have a cache line that resides on your GPU sitting inside your processor's cache a remote modification to that memory by either the GPU, another CPU core or some other PCIe device with NOT invalidate the CPU cache line. You also have the fun situation that if it's modified on both ends simultaneously the resulting state will be non-deterministic.

Device drivers have to be very careful about synchronization when accessing memory-like areas on PCIe. CXL adds a cache coherency protocol among other things, so that invalidations and snoops can be exchanged over the interconnect.

Based on my experience, I think this will prove more true than not in the long run, unfortunately.

Professionally, I see people largely falling into two camps: those that augment their reasoning with AI, and those that replace their reasoning with AI. I’m not too worried about the former, it’s the latter for whom I’m worried.

My mom is a (US public school) high school teacher, and she vents to me about the number of students who just take “Google AI overview” as an absolute source of truth. Maybe it’s just the new “you can’t cite Wikipedia”, but she feels that since the pandemic, there’s a notable decline in the critical thinking skills of children coming through her classes.

We have a whole generation (or two) of kids that have grown up being told what to like, hate, believe, etc. by influencers and anonymous people on the internet. They’d already outsourced their reasoning before LLMs were a thing. Most of them don’t appear to be ready to constructively engage with a system that is designed to make them believe they are getting what they want with dubious quality.

On one hand, it seems rather strange, because back in the early days of C (and later C++) there were far more CPU architectures in play. Every big Unix hardware vendor had their own CPU architecture, whereas today we only have about six. (In my mind: x86, arm, mips, risc-v, ppc, and s390x)

But it might be that in the early days of C/C++, development involved connecting to large shared Unix environments where the machine you developed on what always the machine (or at least the same type of machine) the program would run on, and also that those vendors weren't exactly incentivized to make developing for competitor's architectures easy.

I have an even stronger sentiment regarding cross compilation though - In any build system, I think the distinction between “cross” and “non-cross” compilation is an anti-pattern.

Always design build systems assuming cross compilation. It hurts nothing if it just so happens that your host and target platform/architecture end up being the same, and saves you everything down the line if you need to also build binaries for something else.

If I were more money motivated I’d probably be building CRUD apps too. I just like weird puzzles XD.

It really opened my eyes to just how food centric society is.

I am a big guy (6’4, 330 lbs), but I was amazed that Ozempic just seemed to do nothing. I was having the gastric side effects, but I could still eat 3000 calories a day if I cheated without feeling anything.

But I feel you on sugar. Took me a long time to cut sugar cravings. A decade ago I cut regular soda out of my diet, which a few years later led to me cutting out pretty much anything sweetened. Realistically it wasn’t the sweetness for me, it was the “mouthfeel” or doing something with your mouth. Just straight sparkling water satisfied the entire craving for me.

The hardest thing for me to give up / heavily cut back on was fried things. Maybe that’s the result of my parents using french fries as the reward food when I was a kid…

Decided to try Ozempic and was on it for about 6 months. Didn’t do a single thing for my appetite unfortunately, even on the max dose.

Sample size of one here, but if you’ve got mental health struggles that feed into your eating patterns, GLP-1s might not help with your weight problems.

Sure we’ve had the ability to keep the pipeline on GPU for awhile, but it usually required platform specific API bindings to convert to a platform specific descriptor (handles on windows, IOSurface on macOS, dmabuf on Linux), which you then had to pull into a platform specific decoder/encoder API (DXGI, WMF, AVFoundation, VAAPI, etc.), and then all of that again but in reverse to get the surface back into your 3D API.

This whole thing just makes life easier for everyone.

JACK doesn’t support device hotplug (ya know, connecting and disconnecting a headset, something most of us do) and it also doesn’t support multiple applications generating audio without the user having to configure how audio is mixed.

JACK is designed for low latency in environments like Digital Audio Workstations (DAW) where you know 1) what audio hardware is present at all times, and 2) what applications are going to generate audio.

Many people who use/used JACK ran a PulseAudio bridge on top of it for every application that wasn’t the one or two applications that needed ultra low latency audio.

PulseAudio had some major warts, but JACK wasn’t some panacea that did everything better.

Regardless of how you feel about Wayland, its creation set off _massive_ improvements across the entire Linux graphics stack.

For those of us who were using Linux on the desktop in decades past, remember when you couldn't use a GPU without X running? Remember the days when you needed an X session running in order to use CUDA or OpenCL? Remember the days when the entire graphics driver lived inside of X? When display server issues caused kernel panics? Remember the days when you couldn't share a hardware graphics surface between processes? When it was impossible to get hardware acceleration to work offscreen?

Wayland's aggressive stance on "it doesn't work on platforms that don't fix all of that" is one of the only things that pushed the stability and flexibility of the graphics stack on Linux forward.

I don't really think anything less than saying "We the X developers are going to stop X development and X is going away" would have been enough to push graphics card vendors to actually rework the drivers.

Makes doing things larger than 8/16 bit computers very complicated and usually very slow :(

The problem is not that gaming GPUs are in demand, it’s that selling silicon to AI center buildouts is so absurdly profitable right now they just aren’t making many gaming GPUs.

If you can only get so many mm^2 of dies from TSMC, might as well make 50x selling to AI providers.