I’m curious for references or citations to this. When I was going over double descent I tried to find citations like this (just in a couple places like ML/stats textbooks).

Comments URL: https://news.ycombinator.com/item?id=27767328

Points: 11

# Comments: 1

This evidence seems weak at best.

I want to be charitable, but hard to imagine this is anything other than an incredible stretching to make a topic to generate lots of clicks/views.

Comments URL: https://news.ycombinator.com/item?id=22242001

Points: 2

# Comments: 0

Small chance, but in case anyone on HN is in a place google is blocked, would be an interesting test to run.

Comments URL: https://news.ycombinator.com/item?id=22221899

Points: 2

# Comments: 0

I haven't thought about this before on HN, but it makes a lot of sense. I'm curious if you or others have written about this intuition and your experience with it -- I'd like to understand it more.

Environment Generation: https://github.com/openai/multi-agent-emergence-environments

Worldgen: https://github.com/openai/mujoco-worldgen

Is this what you were looking for?

Some things that helped me scale ptrace-interception up:

- SECCOMP_BPF filter (getting these right matters a lot)

- moving all of your intercept work to a single side (enter or exit)

- ensure affinity between the traced and tracing processes

- nuke vdso

- remove vdso from the aux vector (otherwise good libc's will find it again)

At the end of the day unfortunately the better solution would have been to write kernel support for what I wanted to do, but it's a fun exercise in learning about system calls.

After I read this last year it changed the way I interacted and observed things around me for a while. Poking/prodding more, and asking more "what/how" questions.

Possibly mis-copied link?

VDSO is a small set of (3) calls which are not syscalls but direct calls (for speed/efficiency). Our goal is to remove this functionality to force libs to call through the (slower) syscall route instead.

I mention in another comment how EHDR censoring is needed for robust VDSO removal.

I've not run into a libc where censoring EHDR breaks time calls (i.e. it doesn't fallback to syscalls) but possibly golang has this.

In this case it's straightforward to setup a fake VDSO and then instead of EHDR censoring you just replace it with your fake VDSO address and you're golden!

Ironically, because the folks working on containers/VMs are _really_ good at what they do, time access calls in particular have been really optimized (they get called a lot). This makes it very hard to intercept time calls at this layer! e.g. KVM and LXC both essentially hand time calls straight to the host.

This means time intercepts at the VM/container layer need fundamental support (I mentioned affine time transformation in the linux kernel in another comment) which doesn't work for people who need to deploy on current hosted container.

If they're inspecting EHDR and calling VDSO directly, though, or they've statically compiled in their libc, then it won't help though.

I've also had a lot of issues getting it into tightly sandboxed contexts (e.g. the flash runtime in chrome).

Some issues off the top of my head (that I ran into): VDSO censoring is a lot harder than just symbol overriding, it has to actually be removed from the aux vector (third thing on the process stack when the process launches after arguments and environment variables. The EHDR entry is what you need to remove.

Gist for censoring EHDR: https://gist.github.com/machinaut/a08b581c921775263cf0e20ccc...

Some libc's (notably glibc) are really good at finding/using EHDR even if you do that symbol overriding, so dumping EHDR is the most assured way of making sure it's gone.

ptrace overhead is HUGE -- because you're debugging a userspace program with another program every time call now results in 4 context switches (to/from your debugging program at every time call entry/exit), even pinning both to the same CPU this is not fast.

This is where my least favorite part of the linux kernel comes in handy: SECCOMP-BPF. Instead of firing _every_ syscall, you can write a syscall packet filter rules list that only matches certain time-based syscalls with certain arguments. This greatly improves the performance (but for me, still not fast enough to play video games live).

At the end of the day I ended up reviving a >10 year old patch someone sent to the linux kernel to add these parameters (time offset and time warp) to thread structs and do the warping in the kernel (much faster -- dont pay the context overhead, etc). Sadly even this didn't work because our end application needed to run on multiple clouds in docker, and we'd need to have access to the host kernel to do these operations.

I'd like to have an affine time warp as part of the cgroups, and then maybe extend it through runc so anyone can run time-warped docker containers, but maybe that's wishful thinking.

Overall I think this is great work, and super happy you posted it. I'd love to chat about it sometime.

(P.S. most ironic to me was my version of this was called 'timelord' :)