We've carefully ran some numbers before doing this, and this affects a few hundreds to a few thousand people (hard to say, ballpark), and most of those people are on 64bits CPUs, but are using a 32bits Firefox or 32bits userspace.

The comparatively high ratio of 32bits users on Windows is not naively applicable to the Linux Desktop population, that has migrated ages ago.

Other systems go further, such as Web Audio Modules (that builds on top of AudioWorklet) implement sample-accurate parameter change from within the rendering thread, using wait-free ring-buffers. That requires `SharedArrayBuffer` but works great, and is the lowest latency possible (since it uses atomic loads and stores from e.g. the main thread to the rendering thread).

https://bugzilla.mozilla.org/show_bug.cgi?id=1308431

Article in French: https://www.franceinfo.fr/les-jeux-olympiques/paris-2024/bai...

You're out of luck on Safari because it seems that the important APIs aren't implemented yet: https://developer.mozilla.org/en-US/docs/Web/API/AudioContex... (scroll down). This means this cannot be made correct, e.g. there will always be a desynchronization when using high latency audio output devices such as anything wireless. Their handhelds (on which non Safari is not allowed) and their MacBooks, in wired/built-in speaker mode have excellent latency figures and we can get away with not doing anything explicit there, granted the audio is not happening somehow before the visuals, that is jarring for a human. A bit late is a lot more natural if it can't be exact.

lmk if you need further details on tight synchronization of real time audio and visuals, padenot@mozilla.com, happy to help, and congratulations on the delightful websites!

As one could imagine it's a bit (read: a lot) more complicated than just pausing the AudioContext after some time of silence, but we'll get it fixed regardless, it's possible because others did it. There are tradeoffs we're willing to do.

Source: Firefox implementer of a lot of things around this, editor of the Web Audio API standard.

Producing, broadcasting, or any sort of scenario that require reliable low latency can benefit from a real time kernel, but not merely listening to music.

The Wake Lock API is for other use cases, such as recipe websites, or other document for which you don't want the screen to go away / dim, the kind where you happen to need to look at the screen for long period of time without touching it/interacting w/ mouse and keyboard.

Prior to this API being introduced, websites used to play an inaudible/almost invisible looping media file to keep the screen awake. This has power usage implication, a small single digit number of watts (1 to 3.5 depending on os, hardware, mobile or not) is required to keep audio running (because of high priority threads, frequent wakeups, and simply because the audio hardware itself needs power).

One of those libraries source, for illustration: https://github.com/richtr/NoSleep.js/blob/master/src/index.j...

Within a single process, or tree of processes that can cooperate, this is straightforward (modulo the actual audio signal processing which isn't) to do: keep what you're playing for a few hundreds milliseconds around, compare to what you're getting in the microphone, find correlations, cancel.

If the process aren't related there are multiple ways to do this. Either the OS provides a capture API that does the cancellation, this is what happens e.g. on macOS for Firefox and Safari, you can use this. The OS knows what is being output. This is often available on mobile as well.

Sometimes (Linux desktop, Windows) the OS provides a loopback stream: a way to capture the audio that is being played back, and that can similarly be used for cancellation.

If none of this is available, you mix the audio output and perform cancellation yourself, and the behaviour your observe happens.

Source: I do that, but at Mozilla and we unsurprisingly have the same problems and solutions.

Web Codecs can take a compressed media packet and get you the decoded image or audio buffer it corresponds to. Conversely it can take audio or images with timestamps and get you a series of encoded media packets you can then containerize (we say mux) and get you e.g. an mp4 file.

https://w3c.github.io/webcodecs/

So I spoofed the user-agent in a nightly build here on my Linux desktop workstation, then had to alias one method that we should have implemented years ago but only have with a `moz` prefix (`HTMLMediaElement.mozCaptureStream`). This is on us to fix.

Then it looks like a worker script is served with the `Content-Type` `text/html` instead of `application/javascript` or something like that. We also have a pref flip to bypass that check, so I did that, but this is on the dev to fix.

When you do this it works, I've loaded project demos containing videos, audio, various things composited on top, scrubbed the timeline aggressively in a debug build, moved things around in various bits of the interface and also in the rendering frame, etc., things seem to work as they should, perf is as I'd expect it to be (and again, I'm running it in a debug build with optimizations disabled for anything media related, enabled for other parts of the browser).

What's missing is `window.showSaveFilePicker` and file system related stuff. It's possible to use https://developer.mozilla.org/en-US/docs/Web/API/File_System... instead (that we ship, e.g. Photoshop on the Web uses it). We think that it's much less scary than giving access to the file system to a content process of a Web browser. Maybe because videos can sometimes be extremely big files, direct access to the FS could be of use there. Thankfully, we also ship extremely modern video encoders to make them tiny instead, but that's currently a limitation Firefox has, for better or worse.

https://paul.cx/public/pikimov-firefox-nightly.webm

Then we'll gradually optimize (e.g. enable the use of more hardware encoders, decoding being generally in hardware at launch of supported) release after release, but generally almost everything will be supported at launch.