https://github.com/hamzaq2000/wolf3d-reimpl-rs

Senko has two clustering types, (1) spectral for audio < 20 mins in length, and (2) UMAP+HDBSCAN for >= 20 mins. In the clustering code, spectral actually already supports orcale/min/max speakers, but UMAP+HDBSCAN doesn't. However, someone forked Senko and added min/max speakers to that here (for oracle, I guess min = max): https://github.com/DedZago/senko/commit/c33812ae185a5cd420f2...

So I think all that's required is basically just testing this thoroughly to make sure it doesn't introduce any regressions in clustering quality. And then just wiring the oracle/min/max parameters to the Diarizer class, or diarize() func.

Yeah unfortunately, since the diarization is acoustic features based, it really does require high recorded voice fidelity/quality to get the best results. However, I just added another knob to the Diarizer class called mer_cos, which controls the speaker merging threshold. The default is 0.875, so perhaps try lowering to 0.8. That should help.

I'll also get around to adding a oracle/min/max speakers feature at some point, for cases where you know the exact number of speakers ahead of time, or wanna set upper/lower bounds. Gotten busy with another project, so haven't done it yet. PR's welcome though! haha

Also https://zanshin.sh, if you'd like speaker diarization when watching YouTube videos

https://github.com/FluidInference/FluidAudio

https://github.com/FluidInference/eddy-audio

Remote: Yes

Willing to relocate: Yes

Technologies: diarization, Voice AI, PyTorch, CoreML,

Svelte/SvelteKit, Flask, SQLite, Tauri

Résumé/CV: https://hamzaq.com/Hamza_Qayyum_Resume_Public.pdf

Email: mhamzaqayyum [at] icloud [dot] com

---------

Projects:

- Senko: very fast, accurate, speaker diarization (https://senko.sh)

- Zanshin: novel media player that allows you to navigate by speaker (https://zanshin.sh)

I didn't care for those; just told Claude Code to add in the feature directly. So they probably wouldn't accept the PR if I made one.

Comments URL: https://news.ycombinator.com/item?id=45701223

Points: 4

# Comments: 2

Comments URL: https://news.ycombinator.com/item?id=45518843

Points: 6

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=45369869

Points: 3

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=45324612

Points: 2

# Comments: 2

On M3 MacBook Air, 1 hour in 23.5 seconds (~14x faster).

This is a custom speaker diarization pipeline I've developed; it's a modified version of the pipeline found in the excellent 3D-Speaker project by Alibaba Research.

My optimizations/modifications were the following:

- changed VAD model

- multi-threaded Fbank feature extraction

- batched inference of CAM++ embeddings model

- clustering is accelerated by RAPIDS, when NVIDIA GPU available

Optimizations aside, massive credit needs to be given to the CAM++ speaker embeddings model, whose efficiency is where the majority of the speed comes from.

This pipeline powers the Zanshin media player, which is an attempt at a usable integration of diarization in a media player. Check it out here: https://zanshin.sh And discuss here: https://news.ycombinator.com/item?id=45104866

Let me know what you think! Were you also frustrated by how slow speaker diarization is? Does Senko's speed unlock new use cases for you? Cheers, everyone.

Comments URL: https://news.ycombinator.com/item?id=45106441

Points: 2

# Comments: 1

  - Visualize who speaks when & for how long
  - Jump/skip speaker segments
  - Remove/disable speakers (auto-skip)
  - Set different playback speeds for each speaker

It has first-class support for YouTube videos; just drop in a URL. Also supports your local media files. All processing runs on-device.

Download today for macOS. Also works on Linux and WSL, but currently without packaging. You can get it running though with just a few terminal commands. Check out the repo for instructions: https://github.com/narcotic-sh/zanshin

Zanshin is powered by Senko, a new, very fast, speaker diarization pipeline I've developed.

On an M3 MacBook Air, it takes over 5 minutes to process 1 hour of audio using Pyannote 3.1, the leading open-source diarization pipeline. With Senko, it only takes ~24 seconds, a ~14x speed improvement. And on an RTX 4090 + Ryzen 9 7950X machine, processing 1 hour of audio takes just 5 seconds with Senko, a ~17x speed improvement.

Senko's speed is what make's Zanshin possible. Senko is a modified version of the speaker diarization pipeline found in the excellent 3D-Speaker project. Check out Senko here: https://github.com/narcotic-sh/senko

Cheers, everyone; enjoy 残心/Zanshin and Senko. I hope you find them useful. Let me know what you think!

~

Side note: I am looking for a job. If you like my work and have an opportunity for me, I'm all ears :) You can contact me at mhamzaqayyum [at] icloud.com

Comments URL: https://news.ycombinator.com/item?id=45104866

Points: 2

# Comments: 0