Hacker News: ModelForge

New comment by ModelForge in "Kimi K3 Architecture Overview and Notes"

ModelForge — Tue, 28 Jul 2026 22:56:24 +0000

And adding to that, there is also the recurrent state in the Kimi Delta Attention. I wouldn't call it position information but more sth like "position sensitivity"

Kimi K3 Architecture Overview and Notes

ModelForge — Tue, 28 Jul 2026 15:48:34 +0000

Article URL: https://sebastianraschka.com/blog/2026/kimi-k3-architecture-notes.html

Comments URL: https://news.ycombinator.com/item?id=49085698

Points: 352

# Comments: 56

Inkling: A New Open-Weight 975B Moe with a Few Surprises

ModelForge — Thu, 16 Jul 2026 14:03:05 +0000

Article URL: https://sebastianraschka.com/blog/2026/inkling-architecture-benchmark-notes.html

Comments URL: https://news.ycombinator.com/item?id=48934761

Points: 3

# Comments: 0

Claude Code's Real Secret Sauce Isn't the Model

ModelForge — Tue, 31 Mar 2026 14:43:51 +0000

Article URL: https://sebastianraschka.com/blog/2026/claude-code-secret-sauce.html

Comments URL: https://news.ycombinator.com/item?id=47588089

Points: 6

# Comments: 0

The State of LLMs 2025: Progress, Problems, and Predictions

ModelForge — Wed, 31 Dec 2025 15:40:41 +0000

Article URL: https://magazine.sebastianraschka.com/p/state-of-llms-2025

Comments URL: https://news.ycombinator.com/item?id=46445076

Points: 3

# Comments: 0

A Researcher's Field Guide to Non-Standard LLM Architectures

ModelForge — Tue, 04 Nov 2025 15:00:57 +0000

Article URL: https://magazine.sebastianraschka.com/p/beyond-standard-llms

Comments URL: https://news.ycombinator.com/item?id=45811756

Points: 2

# Comments: 0

Explanation of Gated DeltaNet (Qwen3-Next and Kimi Linear)

ModelForge — Mon, 03 Nov 2025 16:59:49 +0000

Article URL: https://github.com/rasbt/LLMs-from-scratch/blob/main/ch04/08_deltanet/README.md

Comments URL: https://news.ycombinator.com/item?id=45801296

Points: 3

# Comments: 0

The Core Components of Modern LLMs and the Models Beyond Transformers [video]

ModelForge — Mon, 27 Oct 2025 15:40:25 +0000

Article URL: https://www.youtube.com/watch?v=lONyteDR4XE

Comments URL: https://news.ycombinator.com/item?id=45722231

Points: 3

# Comments: 0

Popular Attention Alternatives: GQA, MLA, SWA

ModelForge — Wed, 15 Oct 2025 14:17:52 +0000

Article URL: https://sebastianraschka.com/llms-from-scratch/ch04/

Comments URL: https://news.ycombinator.com/item?id=45592961

Points: 4

# Comments: 0

Multi-Head Latent Attention

ModelForge — Mon, 13 Oct 2025 18:24:28 +0000

Article URL: https://sebastianraschka.com/llms-from-scratch/ch04/05_mla/

Comments URL: https://news.ycombinator.com/item?id=45571699

Points: 4

# Comments: 0

Thinking Machines Lab Co-Founder Departs for Meta

ModelForge — Sat, 11 Oct 2025 19:57:45 +0000

Article URL: https://www.wsj.com/tech/ai/thinking-machines-lab-co-founder-departs-for-meta-442d7461

Comments URL: https://news.ycombinator.com/item?id=45552181

Points: 7

# Comments: 0

OpenAI's internal Slack messages could cost it billions in copyright suit

ModelForge — Fri, 10 Oct 2025 20:41:08 +0000

Article URL: https://sherwood.news/power/openais-internal-slack-messages-could-cost-them-billions-in-copyright-suit/

Comments URL: https://news.ycombinator.com/item?id=45543544

Points: 8

# Comments: 1

LLM Evaluation from Scratch: Multiple Choice, Verifiers, Leaderboards, LLM Judge

ModelForge — Sun, 05 Oct 2025 15:55:26 +0000

Article URL: https://magazine.sebastianraschka.com/p/llm-evaluation-4-approaches

Comments URL: https://news.ycombinator.com/item?id=45482523

Points: 4

# Comments: 0

New comment by ModelForge in "Gemma 3 270M re-implemented in pure PyTorch for local tinkering"

ModelForge — Wed, 20 Aug 2025 20:48:43 +0000

No the compiled version is actually faster.

From that table, the A100 tok/sec (larger is faster) numbers are:

- Eager: 28

- Compiled: 128

And

- KV cache eager: 26

- KV cache compiled: 99

The reason that the KV cache is slower is likely because it's not GPU-optimized code. On CPU the KV cache is faster. To make it faster on GPU, you would pre-allocate the tensors on the device for example instead of `torch.cat`ting them on the fly

New comment by ModelForge in "Gemma 3 270M re-implemented in pure PyTorch for local tinkering"

ModelForge — Wed, 20 Aug 2025 20:44:55 +0000

Could be an artifact of the small size not fully taking advantage of the GPU. For example, for the slightly larger Qwen3 0.6B model the A100 is faster (you can see it when scrolling to the bottom here: https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/11...)

New comment by ModelForge in "Gemma 3 270M re-implemented in pure PyTorch for local tinkering"

ModelForge — Wed, 20 Aug 2025 20:39:59 +0000

I'd say the common ones (besides educational) are

- private, on-device models (possibly with lower latency than models via web API); also edge devices

- algorithm research (faster and cheaper to prototype new ideas)

- cheap tasks, like classification/categorization; sure, you don't need a decoder-style LLM for that, but it has the advantage of being more free-form, which is useful in many scenarios; or maybe a sanity checker for grammar; or even a router to other model (GPT-5 style)

Gemma 3 270M re-implemented in pure PyTorch for local tinkering

ModelForge — Wed, 20 Aug 2025 14:01:26 +0000

Article URL: https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/12_gemma3

Comments URL: https://news.ycombinator.com/item?id=44962059

Points: 417

# Comments: 57

New comment by ModelForge in "GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2"

ModelForge — Sun, 10 Aug 2025 21:52:15 +0000

I think GPT-4.5 was potentially the original GPT-5 model that was larger and pre-trained on more data. Too bad it was too expensive to deploy at scale so that we never saw the RL-ed version

New comment by ModelForge in "GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2"

ModelForge — Sun, 10 Aug 2025 21:50:47 +0000

The ollama one uses even less (around 13 GB), which is nice. Apparently the gpt-oss team also shared the mxfp4 optimizations for metal

New comment by ModelForge in "GPT-OSS vs. Qwen3 and a detailed look how things evolved since GPT-2"

ModelForge — Sun, 10 Aug 2025 19:04:09 +0000

Good point. LLMs lower the barrier to entry if someone has enough resources because those architectures are more robust to tweaks given one throws enough compute and data at them. You can even violate scaling laws and still get a good model (like Llama 3 showed back then)