Hacker News: toebee

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 23:37:59 +0000

we haven't tried so can't say for sure, but if optimized for a scenario where the batch size is 1 and max concurrency is 1, it seems possible to get something pretty fast. i'm guessing mobile hardware specific optimizations will be important but we are not experts in that field.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 23:29:51 +0000

we continuously compared output to qwen's original implementation and do not see differences in output quality. let us know if it works well for you!

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 23:26:59 +0000

Yep, might need some changes.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 23:02:59 +0000

We do graph capture etc at startup (same as vLLM) but this model variant doesn’t require prefix caching - the prefix is just 10 tokens.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 22:59:45 +0000

Qwen3 TTS has input streaming mode: you can stream LLM output into the speech model. So don’t need to wait for a full sentence. We also implement this websocket variant, and it also runs at sub 50 ms.

LLM TTFT is still a big issue, and we might tackle that problem as well.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 22:56:09 +0000

We got a rtx 4090 handling around 10 concurrent requests at 50 ms TTFA after some config changes / adjustment as it doesn’t have FP8. So this 50 ms TTFA thing is very much possible on consumer hardware.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 22:54:47 +0000

Hey thank you for your kind words! Yes, we’ve shifted to inference but will also continue doing finetuning etc. on top of open models. Don’t have plans to do pretraining though.

New comment by toebee in "How we made a text-to-speech model respond in sub-50 ms"

toebee — Fri, 21 Aug 2026 15:51:10 +0000

time-to-first-audio (TTFA) is critical for realtime voice applications. open source implementations (e.g. vLLM-Omni, SGLang-Omni) are often too slow for production and can have issues with realtime playback if you push for lower latency. we wanted to fix that.

we optimized qwen3-tts, a popular OSS TTS model, to achieve 34 ms p95 TTFA at 10 requests per second on 1 x H100. we open source the implementation and benchmark, as well as a breakdown of how it was done.

github: https://github.com/nari-labs/nari-qwen3-tts

How we made a text-to-speech model respond in sub-50 ms

toebee — Fri, 21 Aug 2026 15:51:10 +0000

Article URL: https://nari-labs.com/blog/qwen3-tts-speed-cost-frontier/

Comments URL: https://news.ycombinator.com/item?id=49389952

Points: 92

# Comments: 24

Show HN: Dia2, open-weights TTS model for realtime speech to speech

toebee — Wed, 19 Nov 2025 18:59:54 +0000

Dia2 is an open-weights, streaming dialogue TTS model. It is capable of generating speech without a full sentence, making it suitable for low-latency speech-to-speech systems. It can generate up to 2 minutes of English audio, and supports audio prefixing.

The inference code and weights (1B / 2B variants) are uploaded to Github and Hugging Face with Apache 2.0 license, to accelerate research. This work was heavily influenced by KyutaiTTS, Mimi, and Sesame. We thank the TPU research cloud for providing computational resources.

Comments URL: https://news.ycombinator.com/item?id=45983574

Points: 3

# Comments: 2

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 03:08:14 +0000

You're absolutely right. We used Jordan's Whisper-D, and he was generous enough to offer some guidance along the way.

It's also a valid criticism that we haven’t yet audited the dataset for existing list of tags. That’s something we’ll be improving soon.

As for Dia’s architecture, we largely followed existing models to build the 1.6B version. Since we only started learning about speech AI three months ago, we chose not to innovate too aggressively early on. That said, we're planning to introduce MoE and Sliding Window Attention in our larger models, so we're excited to push the frontier in future iterations.

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 02:57:22 +0000

not part of the license!

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 01:22:28 +0000

Thank you for the kind words! We only support English at the moment.. Hope to add more languages in the future.

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 01:21:53 +0000

Thanks you!! We personally used Quickpod and Runpod the most. But you can try it now on HF Spaces without spinning up GPUs yourself!

https://huggingface.co/spaces/nari-labs/Dia-1.6B

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 01:20:35 +0000

Thanks for the interest! We also enjoyed using E5-F2 :) You can try it now on HF Spaces: https://huggingface.co/spaces/nari-labs/Dia-1.6B

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 01:19:08 +0000

Thank you so much for the kind words :) We only support English at the moment, hopefully can do more languages in the future. We are planning to release a technical report on some of the details, so stay tuned for that!

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 01:17:45 +0000

We will try to make it work, but not sure if will be an easy task. For now, you can try with https://huggingface.co/spaces/nari-labs/Dia-1.6B

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 00:59:13 +0000

Thank you! You can add audio prompts of calm voices to make them a bit smoother. https://huggingface.co/spaces/nari-labs/Dia-1.6B you can try it here!

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 00:58:36 +0000

Thank you!! Works for English only unfortunately :((

New comment by toebee in "Show HN: Dia, an open-weights TTS model for generating realistic dialogue"

toebee — Tue, 22 Apr 2025 00:58:12 +0000

Thanks for the kind words! We're just following our interests and staying upwind.