Hacker News: croqaz

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 19:20:05 +0000

Thank you very much! It is humbling and motivating to see other people interested in this.

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 19:18:57 +0000

Do share! I read all the blog posts where people share their experiences of building small scale LLMs "from scratch".

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 19:17:14 +0000

That's a fair point TBH. I said in my post that this LLM is first of all a learning project and I skipped an important step: the training loop. But on the other hand, how many data scientists are writing their own training loops? Is it even worth it? And how much learning do you want for one project, I mean, where do you stop? Why use "Huggingface Transformers" when you can write it from scratch, for learning? Why use Torch when you can write it from scratch, for learning? Why use Python when you can write in C, etc. It's cheating, right? In my case, I decided to skip the training loop and focus on the data processing and the hyper params and the rest of the higher level steps that took a ton of time anyway, and I reduced the friction. I do get your point tho. Now that I know how to train an LLM, maybe I'll write a training loop from scratch as a project, to learn how to do it.

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 13:50:36 +0000

"A more granular followup would be cool too"

Do you mind expanding this question? More granular in what way? what would you like to know that is missing from the post?

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 13:49:04 +0000

That's exactly what I had in mind. When I started this, I was jumping back and forth between this thought: "Can this model size actually generate logical English text?" and I played with a few different models of the same size and I was really really depressed when seeing how bad they are.... but then I discovered more and more tiny models and LaMini-125M, LaMini-256M, and nanowhale-100m, and SmolLM2-135M-Instruct are very very decent. So I decided to give it a try.

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Fri, 12 Jun 2026 10:56:19 +0000

It looks like ROT13 text to me, I hope it's not Welsh. Don't want to offend anyone if that's their actual language :)

New comment by croqaz in "Making a vintage LLM from scratch"

croqaz — Thu, 11 Jun 2026 08:38:00 +0000

I am creating my tiny Llama 340M base model from scratch. If you're curious about the steps, challenges and cost, read on. I am still working on the instruct model.

Making a vintage LLM from scratch

croqaz — Thu, 11 Jun 2026 08:38:00 +0000

Article URL: https://crlf.link/log/entries/260525-1/

Comments URL: https://news.ycombinator.com/item?id=48487829

Points: 103

# Comments: 29

New comment by croqaz in "TwoFold (2f), CLI text expander/template engine"

croqaz — Sun, 11 May 2025 18:28:46 +0000

TwoFold is a small command line app that allows plain text files to behave like dynamic files. It is a hybrid between a text expande, a template engine and a mini programming language.

TwoFold (2f), CLI text expander/template engine

croqaz — Sun, 11 May 2025 18:28:46 +0000

Article URL: https://github.com/ShinyTrinkets/twofold.ts

Comments URL: https://news.ycombinator.com/item?id=43955834

Points: 17

# Comments: 1

New comment by croqaz in "[dead]"

croqaz — Fri, 12 Aug 2022 14:31:51 +0000

What are web snapshots; comparison of the most popular methods: WARC, HTML, rrWeb and a better alternative called "recorded".

New comment by croqaz in "Blocking adware, malware, and tracking sites"

croqaz — Tue, 21 Dec 2021 10:39:11 +0000

DYI blocking, a few methods

Blocking adware, malware, and tracking sites

croqaz — Tue, 21 Dec 2021 10:39:11 +0000

Article URL: https://crlf.link/log/entries/211220-1/

Comments URL: https://news.ycombinator.com/item?id=29636013

Points: 1

# Comments: 2

Harden and secure browsers in containers, with GUI

croqaz — Fri, 08 Oct 2021 16:28:52 +0000

Article URL: https://crlf.link/log/entries/211008-1/

Comments URL: https://news.ycombinator.com/item?id=28801119

Points: 58

# Comments: 31

New comment by croqaz in "Tomb: File Encryption on GNU/Linux"

croqaz — Tue, 10 Aug 2021 20:29:06 +0000

Tomb consists of a simple shell script (Zsh) using standard filesystem tools (GNU) and the cryptographic API of the Linux kernel (cryptsetup and LUKS).

Tomb: File Encryption on GNU/Linux

croqaz — Tue, 10 Aug 2021 20:29:06 +0000

Article URL: https://www.dyne.org/software/tomb/

Comments URL: https://news.ycombinator.com/item?id=28133987

Points: 6

# Comments: 2

Things that only someone who has been programming 20 years would know

croqaz — Wed, 28 Jul 2021 22:09:01 +0000

Article URL: https://www.quora.com/What-are-some-things-that-only-someone-who-has-been-programming-20-50-years-would-know/answer/John-Byrd-2?share=1

Comments URL: https://news.ycombinator.com/item?id=27990400

Points: 23

# Comments: 7

New comment by croqaz in "Fountain pen ink properties"

croqaz — Sat, 13 Jun 2020 21:10:08 +0000

The same in Romania. In the 90s, everyone I know had to learn how to write with a fountain pen. For me at least, I write horribly when I use a ballpoint pen, vs a fountain pen. Because the ballpoint slides on the paper much too easy, like on ice, and you need much more discipline to write nicely. A fountain pen nib is generally more scratchy and feels easier to control. My 2 cents anyway...

New comment by croqaz in "Fountain pen ink properties"

croqaz — Sat, 13 Jun 2020 21:06:03 +0000

I love my Plaisir too! I have a green color and of course, it's always inked with green :)

New comment by croqaz in "Fountain pen ink properties"

croqaz — Sat, 13 Jun 2020 21:04:02 +0000

Wow, I had no idea my crappy post ended on Hacker news! I just collected a bunch of info and links for myself :) I guess I have to update the post with some pictures, at least...