Hacker News: markusheimerl

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 15:48:29 +0000

https://github.com/markusheimerl/gpt/blob/main/train.c - in this file, search for the line "const int batch_size = 15;" - reduce this number

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 14:13:44 +0000

Sure it could be extended to support LoRA finetuning but this implementation has the goal to be as lean and efficient as possible for a pre-training stack as you can be.

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 14:12:23 +0000

the data gets downloaded via curl from huggingface - sure you can make your own data, simply dump all text you want the model to be trained on into "corpus.txt" and skip "make data".

As the tokenizer adds substantial complexity, this implementation does not include any tokenziation logic and works on raw bytes. Feel free to add your own tokenzier with the help of the coding model of your choice.

You can stop the training using CTRL+C You can train on as little memory as you have. Simply reduce batch size and/or model dimensions in train.c You can change the context window size in train.c via the "seq_len" variable.

Regarding Ruby, LORA and quantization I'll have to refer you to the coding agent of your choice.

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 14:06:13 +0000

I did run it as a test on the NVIDIA Jetson Orin Nano Super Dev. Kit once - so yea it works on arm like a charm ;)

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 14:04:07 +0000

Reduce batch size in train.c

New comment by markusheimerl in "Tiny hackable CUDA language model implementation"

markusheimerl — Mon, 08 Jun 2026 08:12:14 +0000

I deleted the numerical checks a while back after confirming the backward pass is correct to keep the code base lean - running https://github.com/markusheimerl/gpt/blob/main/transformer/a... is also somewhat of a confirmation that the backward pass is correct, since an analytically incorrect backward pass cant fit perfectly to synthetic data.

Tiny hackable CUDA language model implementation

markusheimerl — Fri, 05 Jun 2026 17:41:58 +0000

Article URL: https://github.com/markusheimerl/gpt

Comments URL: https://news.ycombinator.com/item?id=48415828

Points: 81

# Comments: 13

Stick in Bike Wheel OOP

markusheimerl — Wed, 31 Jul 2024 10:53:14 +0000

Article URL: https://imgflip.com/i/8yu4c5

Comments URL: https://news.ycombinator.com/item?id=41117998

Points: 1

# Comments: 0