So the comparison is not only "built with and without LLM" but "would you even build this if you didn't have the LLM?". The gap in productivity in this case is much more wide.

However, I see the appeal of this. Kind of surprised it hasn't happened yet to be honest.

I’m building an AI gaming companion. Essentially an AI NPC (a speaking dog, think dogmeat of fallout on steroids) that is grounded in the game world, can perceive what’s happening, perform actions, follow complex multi-step instructions, and strategize with the player. I want it to feel natural, so I 'm aiming for real audio conversationality (no push to talk & wait), so audio is important, but also... every millisecond trimmed from audio can be dedicated to heavier AI systems (caching helps heavily there, but extracting player's intent before it even reaches an LLM is probably the biggest trick).

Budgets are pretty tight, ~400ms for everything. I have optimized qwen-3-asr and added streaming detection to it, as well as pocket-tts (this) and qwen-3-tts, with a goal of ~70ms for audio for both detection/creation and then ~350ms for the planning.

It's interesting that gpt-live was just announced today, planning to make more things open source as well as the learnings while building this.

As I was using it I noticed that it didn't solve the latency issues fully due to leading silence and artifacts generation (quite common with all these systems). So I started poking under the hood with the help of agents to try and improve it.

On m4 max, the new browser build runs around ~14x realtime on longer utterances, and around ~3.5x realtime on an iPhone 16 Pro. The native CPU path is faster still: roughly 32–33x realtime warm, with first useful audio around 30ms on my benchmarks. Other systems will vary; my Windows browser results were slower but still usable. This does not introduce a new model or claim better voice quality. The goal was to preserve the intended PocketTTS quality while making it fast enough for low-latency interactive scenarios.

I decided to open source it in case it's useful for anyone else, or at least as a demo of what things that are finally possible now that the barrier of trying stuff has been lowered.

EDIT: Keep in mind it downloads 67mb of data on load.

Comments URL: https://news.ycombinator.com/item?id=48834069

Points: 4

# Comments: 4

- https://pantel.is/projects/css3d/

- https://keithclark.co.uk/labs/css-fps/ (the original)

but quake and doom took it to the next level :)

Audiomass is an audio editor, meaning that it works with already existing files that might need to be cut, edited, cleaned, or filtered, or in case where one has multiple stems put together to be mixed.

Think of it a bit like vector vs pixel editing. OpenDAW is vectors (music info), audiomass is pixels (music data)

I 'll be heading off soon, so decided to write some features included in this release that might not be apparent right away (still working on improving the UX).

- Drag n drop multiple audio files into multitrack = multiple channels automatically created

- Double click on a waveform box in multitrack, opens it in the original audio editor (for more precise editing)

- Copying (command+c, or shift+c) works between multitrack and regular editor. So for example, you can open a file in original editor tweak it, and then copy all or part of it and paste it in multitrack in a specific channel.

- Most effects have already pre-made presets to make them easier to use

- You can make your own effect presets by clicking the 3 dots after having made a modification in an effect (stored in localstorage)

- Zero crossing selection is under "Edit"

- There is a tempo tools section in View. You can automatically detect tempo of a track, tap to guess a tempo and play with metronome

- There is also an id3 tag viewer (for mp3 files) there as well

- You can right click or press M to add markers (makes it easier to highlight sections of a track, especially when working with longer audio tracks)

- Seamless Loop tool (under Edit): crossfade preview, silence trim, repeat loop, open loop in a new editor tab.

- Offline/PWA support: Help > store offline version, will open a new page that will trigger a service worker that will make the site work offline as well.

- Session export as .amss file for multitrack. If you aren't done working on a session you can export all of its audiofiles and configuration in an LZMA compressed container file (will still be pretty big though). For single audio mode, mp3/wav/flac export is available.

- Pressing X in multitrack when 2 waveform boxes overlap, makes them crossfade smoothly (denoted by an "XF" label at the top right - undocumented behavior but can be quite useful, will keep improving it)

- You can open the menu by pressing the ~ key and then navigate it with arrow keys (left/right for sections, bottom up for selections, enter to trigger an action and esc to close) for a tiny bump in speed of getting things done. Similarly the time controls are clickable and open a mini menu where a time can be specified to jump quickly to it (arrow keys to go between minutes/s/ms)

And finally my favorite feature of them all (though not a new one per se),

- In "View" select "Frequency analyzer" or "Multitrack mixer" and then press the dock button. Audiomass supports the ability to extract elements of it into new pop up windows. So you could have parts of the application on a different monitor keeping the main app in the main tab. It's a very old trick, but I find it kinda cool :)

Thanks again and hope you enjoyed the sample music! (edit: formatting)

Did you load it into multitrack, or the regular editor? (in multitrack it does not scale currently, but working on it). On regular editor it should in theory try to zoom.

There is a pyramid cache mechanism for long files, basically it tries to optimize with simple heuristics how many peak-lines to show for every zoom level. The renderer is pretty dumb right now - just old-school 2d canvas "ctx.lineTo" calls - no gpu, so enormous files can really make it slow, this is the reason for the drops (to ease load). So it might be dropping way too many samples in this case and then not switching properly to the next cache level because the zoom to duration/length ratio is enormous.

I 'll look into it. Thanks again

When I started developing I was a little frustrated with how bloated the web felt back then so I took that direction, it's much better today though and it's no longer an issue, but I still find it fun to impose these constraints and try to work within or around them (there's this fascinating concept of constrained creativity)