Hacker News: mtone

New comment by mtone in "The state of open source AI"

mtone — Fri, 17 Jul 2026 16:18:47 +0000

Firefox's AI Chatbot feature only proposes one open model provider (Mistral) and zero local option. They're not walking the talk.

New comment by mtone in "Cohere's First Model for Developers"

mtone — Tue, 16 Jun 2026 05:07:05 +0000

It was trained from scratch by Cohere. They're the only Canadian AI lab - I'm glad they're releasing open weights and I wish them luck catching up!

New comment by mtone in "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"

mtone — Tue, 16 Jun 2026 04:23:25 +0000

I am using the `voipmonitor/vllm:lucifer` docker from the RTX6K discord community discussed at the same link the other commenter posted. It is based around this PR https://github.com/vllm-project/vllm/pull/43477

New comment by mtone in "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"

mtone — Mon, 15 Jun 2026 21:38:31 +0000

Here's a DeepSeek-V4-Flash benchmark on 2X RTX Pro 6000:

  - Prefill: ~10K tok/s
  - Decode: 190 | 375 | 980 tok/s (for 1 | 4 | 16 concurrent requests)
  - GPU power draw during benchmark: Average: 585W | Max: 849W | Limit: 1200W with undervolt. Idle PC is 125W.

I've asked it to calculate the following considering a realistic blend of cached prompts and decode for agentic dev scenario.

Electricity-only (@ USD $0.08/kWh)

  Usage          | IN price  | OUT price | Monthly cost
  Concurrency=1  | $0.040/M  | $0.080/M  | $8.65 to $38.88 (5% to 100% active)
  Concurrency=4  | $0.024/M  | $0.044/M  | up to $48.67 (cheaper per token but higher power draw)

Total cost of ownership over 3 years is electricity + USD $20K (pre-hike pricing). In a production scenario, how much would I have to charge my users to break even, aiming for 4 concurrent requests 24/7?

A) Breakeven API pricing (est. 2B IN + 1B OUT throughput/month):

                        IN price    OUT price
  Self-hosted           $0.121/M    $0.363/M
  OpenRouter (budget)   $0.098/M    $0.196/M
  OpenRouter (DeepSeek) $0.140/M    $0.280/M

B) Breakeven subscription (users active ~1.5h/day):

    1 user: $563/mo (oh, hai)
    25 users: $23/mo
    100 users: $6/mo

New comment by mtone in "Local AI needs to be the norm"

mtone — Mon, 11 May 2026 03:59:13 +0000

You're forgetting a critical factor: concurrency. If a given hardware serves a single request at 150 tokens/s, it can also serve 20-30 requests at 100 tokens/s. Suddenly your $5K becomes $100K/month, enough to recoup the cost of the hardware in a year or so.

The reason it works: each time you read the model (memory bound) to calculate the next token, you can also update multiple requests (compute bound) while at it. It's also much more energy-efficient per token.

[1] https://aimultiple.com/gpu-benchmark

New comment by mtone in "Artemis II crew take 'spectacular' image of Earth"

mtone — Fri, 03 Apr 2026 21:34:26 +0000

Thanks!

There's a heading control to include rotation in link: https://earth.google.com/web/@3.63731074,-23.1618975,-2690.7...

New comment by mtone in "Large-Scale Online Deanonymization with LLMs"

mtone — Thu, 26 Feb 2026 08:39:24 +0000

> Does privacy of Netflix ratings matter? The issue is not “Does the average Netflix subscriber care about the privacy of his movie viewing history?,” but “Are there any Netflix subscribers whose privacy can be compromised by analyzing the Netflix Prize dataset?”

Well said.

New comment by mtone in "The path to ubiquitous AI (17k tokens/sec)"

mtone — Fri, 20 Feb 2026 17:22:37 +0000

For this type of repetitive application I think it's common to "fine-tune" a model trained on your business problem to reach higher quality/reliability metrics. That might not be possible with this chip.

New comment by mtone in "On privacy and control"

mtone — Thu, 01 Jan 2026 02:19:16 +0000

Good question. That was for my MS account/licenses and some Azure stuff. I use Google Authenticator for most things.

Thanks for the link, I'll take a look. I might just move it to a secondary device first.

New comment by mtone in "On privacy and control"

mtone — Thu, 01 Jan 2026 00:59:10 +0000

Just looked - Microsoft Authenticator doesn't appear to work. I might be able to get off of it but it will take some prep. My banks are supported so that's good.

New comment by mtone in "Reflections on AI at the End of 2025"

mtone — Sat, 20 Dec 2025 21:56:08 +0000

> if you could exclude all of the R&D and training costs

LLMs have a short shelf-life. They don't know anything past the day they're trained. It's possible to feed or fine-tune them a bit of updated data but its world knowledge and views are firmly stuck in the past. It's not just news - they'll also trip up on new syntax introduced in the latest version of a programming language.

They could save on R&D but I expect training costs will be recurring regardless of advancements in capability.

New comment by mtone in "New coding models and integrations"

mtone — Thu, 16 Oct 2025 19:12:10 +0000

Recently llama.cpp made a few common parameters default (-ngl 999, -fa on) so it got simpler: --model and --context-size and --jinja generally does it to start.

We end up fiddling with other parameters because it provides better performance for a particular setup so it's well worth it. One example is the recent --n-cpu-moe switch to offload experts to CPU while filling all available VRAM that can give a 50% boost on models like gpt-oss-120b.

After tasting this, not using it is a no-go. Meanwhile on Ollama there's an open issue asking for this: https://github.com/ollama/ollama/issues/11772

Finally, llama-swap separately provides the auto-loading/unloading feature for multiple models.

New comment by mtone in "Qwen3-Next"

mtone — Fri, 12 Sep 2025 15:51:55 +0000

Do you really need a H200 for this? Seems like something a consumer GPU could do. Smaller models might be ideal [0] as they don't require extensive world knowledge and are much more cost efficient/faster.

Why can't you build this today?

[0]: https://arxiv.org/pdf/2506.02153 Small Language Models are the Future of Agentic AI (Nvidia)

New comment by mtone in "The surreal joy of having an overprovisioned homelab"

mtone — Wed, 26 Mar 2025 04:06:03 +0000

Related discussion on Anubis: https://news.ycombinator.com/item?id=43427679

New comment by mtone in "Google Scholar PDF Reader"

mtone — Thu, 21 Mar 2024 03:39:15 +0000

I use Readdle Documents to sync PDF folders with my server PC via FTP. Free version supports PDF highlighting & simple annotations, basic file management, and automatically syncs back everything.

New comment by mtone in "Disabled at 22 million commits"

mtone — Sun, 02 Jul 2023 23:02:14 +0000

Assuming it's correct, I think this answer explains it well: https://stackoverflow.com/a/31996121/283879

Basically, yes it may write snapshots per-file (never per-commit) locally but there is a separate routine to transparently repack the whole thing with deltas.

New comment by mtone in "TS to JSDoc Conversion"

mtone — Thu, 11 May 2023 07:51:47 +0000

There was talk about introducing type syntax as valid but ignored in the JS language, making TS valid JS.

It would take forever to become mainstream but if node and major browsers started to support this tomorrow, along with ESM modules we could drop TS compilation and bundling entirely during development, safely publish npm packages as TS (even a bundled TS) and simplify tooling for monorepos, IDEs, etc.

Unfortunately that wouldn't solve dealing with templates like JSX/TSX or future language syntax/features.

https://devblogs.microsoft.com/typescript/a-proposal-for-typ...

New comment by mtone in "Use databases without putting domain logic in them"

mtone — Fri, 21 Apr 2023 15:57:01 +0000

They're turing-complete and modular so it's not really about what they can or cannot do.

Testability, tooling and the open-source ecosystem and either bad or non-existent. Writing PL/SQL is the worst environment I've worked in. That database sent emails, processed CSVs scheduled jobs, etc. yet there was still a web app to maintain next to it.

They're OK for certain things like essential triggers or performance-sensitive functions, but I would never deliberately put app logic in there. Major red flag.

New comment by mtone in "Windows needs to stop showing tabloid news"

mtone — Mon, 27 Mar 2023 17:22:09 +0000

Just upgraded to Windows 11 (for its HDR features) and weather is now part of the "Widgets" bombarding me with ads and poor news sources. I genuinely tried to customize my "feed" but it's all junk -no reputable sources whatsoever- and I didn't find a way to remove them.

So the only sane course of action was to disable widgets altogether while I still can. And now I don't have the weather anymore.

New comment by mtone in "TypeScript 5.0"

mtone — Thu, 16 Mar 2023 20:30:22 +0000

In a monorepo with project references where each symlinked (not paths) package has a src/ and output lib/, I've always had issues with VsCode incorrectly proposing to import from src/ over lib/ from other packages. Never got it to work just right.

Does anyone know if the new package.json resolution settings can help resolve these issues?