You may also want to look at other sorting algorithms - common CPU sorting algorithms are hard to maximize GPU hardware with - a network sort like bitonic sorting involves more work (and you have to pad to a power of 2) but often runs much faster on parallel hardware.

I had a fairly naive implementation that would sort 10M in around 10ms on an H100. I'm sure with more work they can get quite a bit faster, but they need to be fairly big to make up for the kernel launch overhead.

At ~$600/kWh for capacity, the ROI isn't great. I have a pretty big differential on my rates because I have an EV, and even then I'd need over a decade to make the $1,000 back assuming I fully discharged it every day.

Their paper [1] only mentions using PTX in a few areas to optimize data transfer operations so they don't blow up the L2 cache. This makes intuitive sense to me, since the main limitation of the H800 vs H100 is reduced nvlink bandwidth, which would necessitate doing stuff like this that may not be a common thing for others who have access to H100s.

1. https://arxiv.org/abs/2412.19437

I'd think things like optimizing for occupancy/memory throughput, ensuring coalesced memory accesses, tuning block sizes, using fast math alternatives, writing parallel algorithms, working with profiling tools like nsight, and things like that are fairly transferable?

The register file size makes sense, I didn't think they were that much of the die on those processors but I guess they had to be pretty aggressive to meet power goals?

Optimizing something for SIMD execution isn't often straightforward and it isn't something a lot of developers encounter outside a few small areas. There are also a lot of hardware architecture considerations you have to work with (memory transfer speed is a big one) to even come close to saturating the compute units.

Being able to redact a few values would be really important for us (I just wrote Python scripts to clean the har files up), but I’m going to play around with it this weekend.

Two questions -

1) Any way to customize which headers/cookies get scrubbed?

2) Is there a way to get something like the lower-level export you can get by going to chrome://net-export/?

I've actually had some success with getting ChatGPT to create Redshift queries based on user text and then I can run them and render results, which has some interesting use-cases.

Max calls out the biggest problem with using something like ChatGPT in a search flow - it is way too slow. I've talked to a lot of people wondering if we can just shove a catalog at ChatGPT and have it magically do a really good job of search, and token limits + latency are two pretty hard stops there (plus I think it would be generally a worse experience in many cases).

What I'm trying to look at now is how LLMs can be used to make documents better suited for search by pulling out useful metadata, summarizing related content, etc. Things that can be done at index time instead of search time, so the latency requirements are less of an issue.

Around the time that change was made to Go, Andrew and I were looking at this and wondering how big of a performance hit it was and if there were a better way to structure that. I had a hunch that the compiler should be smart enough to not compile that as a switch/giant if block, and a quick trip to a disassembler showed it using binary search. This commit: https://github.com/golang/go/commit/1ba96d8c0909eca59e28c048... added the jump table and has some nice analysis on where it makes sense to do binary search vs jump tables.

As far as I can tell, with certain restrictions (that are fine in this case), it is pretty reliable at optimizing giant if/else blocks and switches.

> E.g. Haskell has an awesome concurrent GC that’d work like crap for Java, because it assumes tons of really short-lived, really small garbage and almost no mutation. The other way around is also bound to be problematic—I don’t know how the Scala people do it

I don't know a ton about Haskell's GC, but at surface level it seems very similar to several of the JVM GC implementations - a generational GC with a concept of a nursery. Java GC is very heavily designed around the weak generational hypothesis (ie, most objects don't live long) and very much optimizes for short-lived object lifecycles, so most GC implementations have at least a few nursery-type areas before anything gets to the main heap where GC is incredibly cheap, plus some stuff ends up getting allocated on the stack in some cases.

The only big difference is that in Haskell there are probably some optimizations you can do if most of your structures are immutable since nothing in an older generation can refer to something in the nursery. But it isn't super clear to me that alone makes a big enough difference?

One of the things I really like about Go is that a lot of the designs and decisions, along with their rationales, are pretty well documented. Here are the GC docs, for example - https://go.dev/doc/gc-guide.

For example, Go doesn't move data on the heap around so to combat fragmentation, it breaks the heap up into a bunch of different arenas based on fixed sizes. So a 900 byte object goes into the 1K arena, etc. This wastes some heap space, but saves the overhead and complexity with moving data around.

In our case, it wasn't a single hard-coded number - the input data gave us the upper bound, and the difference between the upper bound and the median case was so small that going with the upper bound worked out best.

One in particular - I was profiling an application with low-latency needs and GC was taking up a ton of time. Mission control showed tons of allocations of arrays - at one point it was creating a bunch of lists in a loop and adding stuff to them, which triggered creating a new underlying array. We found that a) Many of the arrays were just over the first resizing size, and b) There was a good heuristic that we could use to give them an initial size that would never have to be expanded and wouldn't result in huge amounts of waste.

This had a pretty dramatic effect on our GC times and the overall latency. I think this is where the JVM really shines - tons of tooling to help you profile and observe these kinds of details to help you figure out when you actually need to care about stuff like the initial array capacity.

It has been a fun project to play around with for someone like me who thinks this kind of stuff is fun and interesting but will probably never get a chance to work on it full time. Cool to see it noticed, though!