https://go-review.googlesource.com/c/go/+/729920

    func Encrypt(message []byte) ([]byte, error) {
        var ciphertext []byte
        var encErr error
    
        secret.Do(func() {
            // ...
        })
        
        return ciphertext, encErr
    }

I get that the example is simplified (because what it should actually do is protect the long-term key, not the ephemeral one)... so, yeah, it's just a bad example.

yes, that's quite literally what I meant by "GOGC has always been there". 1.5 was released 10 years ago, just 3 years after 1.0.

So to summarize: there is one knob (that has been there from basically the beginning), plus a second one (if you squint hard enough: GOMAXMEM), and absolutely no plans to add further ones, or to add alternative GCs.

The older GC algorithm won't stay, IIRC the plan is for it to be removed in 1.27 (it's kept for now just to give a fallback in case of bugs in the first release).

If you really store just references to the same arena, better to use an offset from the start of the arena. Then it does not matter whether allocations are moved around.

A word of caution. If you do this and then you store pointers into that slice, the GC will likely not see them (as if you were just storing them as `uintptr`s)

Somehow concluding that "By killing Memory Arenas, Go effectively capped its performance ceiling" seems quite misguided.

This is probably much less of a concern today, as NVMe drives - beside having many orders of magnitude higher IOPS capacity - also have (at least on paper) much better hardware support for high I/O concurrency. It may still make sense, even today, if your hardware (or stack) limits IOPS.

Comments URL: https://news.ycombinator.com/item?id=38716034

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=38677815

Points: 2

# Comments: 0

I know my example is not exactly what is happening here, but the two sound pretty affine to me and there seem to be a fairly blurry line dividing the two... so I would argue that where "this must be included in a open source release" ends and "this does not need to be included in a open source release" starts is not always so cut and dry.

(A variant of this, that happens fairly frequently, is when you find a commit that says something along the lines of "this change was made because it made an internal, non-public workload X% faster"; if the data that measurement is based upon did not exist, or if the workload itself didn't exist, that change wouldn't have been made, or maybe it would have been made differently... so again you end up with logic due to data that is not in the open source release)

If we want to go one step further, we could even ask: what about static assets (e.g. images, photographs, other datasets, etc.) included in a open-source release... maybe I'm dead wrong here, but I have never heard that such assets must themselves be "reproducible from source" (what even is, in this context, the "source" of a photograph?).

That being said, I sure wish the training data used for all of these models was available to everyone...

Another simple way, if we don't like the idea of triggering GCs manually,is to allocate the same buffer both off-heap and on-heap: use the off-heap one for actual key storage, and the on-heap one just to generate heap memory pressure.

ByteBuffer.allocateDirect should do that IIRC. This allows you to use the standard ConcurrentHashMap while being able to get a stable pointer for use by the rust logic.