Cost is a big factor - i really want to make models that can run on average CPU only machines so most of the world can benefit, rather than needing expensive GPUs or an internet connection + subscriptions. Another big factor is privacy (you don't need to trust a 3rd party with your inputs).

As for the hyperparameters, pure bruteforce trial and error. It feels more like a dark art than a science. You roll the dice and then start tweaking things until the loss looks like it's dropping nicely and consistently, and the checkpoints are starting to output things resembling what we want. I sometimes do inference using checkpoints just to get a feel of if the model is learning (regardless of loss)

I wanted to process raw transcripts locally without OpenRouter. Llama 3.2 3B with a prompt was decent but incomplete, so I tried SFT. I fine-tuned Llama 3.2 3B to clean/analyze dictation and emit structured JSON (title, tags, entities, dates, actions).

Data: 13 real memos → Kimi K2 gold JSON → ~40k synthetic + gold; keys canonicalized. Chutes.ai (5k req/day).

Training: RTX 4090 24GB, ~4h, LoRA (r=128, α=128, dropout=0.05), max seq 2048, bs=16, lr=5e-5, cosine, Unsloth. On 2070 Super 8GB it was ~8h.

Inference: merged to GGUF, Q4_K_M (llama.cpp), runs in LM Studio.

Evals (100-sample, scored by GLM 4.5 FP8): overall 5.35 (base 3B) → 8.55 (fine-tuned); completeness 4.12 → 7.62; factual 5.24 → 8.57.

Head-to-head (10 samples): ~8.40 vs Hermes-70B 8.18, Mistral-Small-24B 7.90, Gemma-3-12B 7.76, Qwen3-14B 7.62. Teacher Kimi K2 ~8.82.

Why: task specialization + JSON canonicalization reduces variance; the model learns the exact structure/fields.

Lessons: train on completions only; synthetic is fine for narrow tasks; Llama is straightforward to train. Dataset pipeline + training script + evals: https://github.com/bilawalriaz/local-notes-transcribe-llm

Comments URL: https://news.ycombinator.com/item?id=45095353

Points: 31

# Comments: 8

https://www.nytimes.com/2025/07/15/opinion/israel-gaza-holoc...

And let’s not pretend history started on October 7th, that region has been in turmoil for almost a century now, so trying to excuse the sniping of children in the head, mass starvation, subjugation of a whole population, straight up ethnic cleansing in this day and age is indefensible no matter what happened. You say you’re not excusing but only explaining but I don’t want to hear any explanation that equates babies as threats that are okay to kill in war. Hamas could’ve dropped a nuke on Tel Aviv and it still wouldn’t excuse sniping innocent children in the head. Did it become okay to liquidate the ghettoised Jews because they struggled for their freedom? They tried to rise up too? They weren’t prepared to lay down without a fight. And where is the outrage against the Israeli government for firing on their own citizens at the festival?