Hacker News: behohippy

New comment by behohippy in "Kimi Work"

behohippy — Tue, 21 Jul 2026 02:13:56 +0000

It's pretty simple nowadays if you know conceptually how they work. Running the LLM calls in a loop with tools is an agent. You only need 10 or so basic tools to accomplish nearly anything, and you can build a dynamic skill system from that. Look at https://github.com/patw/pengy, ignore the app look at the spec.md file, feed that to your current agent of choice and make your own version. Use whatever tech stack or UI you're comfortable with. Change some of the choices in how it works, so it fits what you want to work.

New comment by behohippy in "ASML's Best Selling Product Isn't What You Think It Is"

behohippy — Mon, 04 May 2026 11:32:40 +0000

You might have a business idea there. I wouldn't mind a twinscan plushie for sitting on top of the workstation.

New comment by behohippy in "Benchmark Framework Desktop Mainboard and 4-node cluster"

behohippy — Mon, 11 Aug 2025 15:44:27 +0000

Yeah 48g, sub 200W seems like a sweet spot for a single card setup. Then you can stack as deep as you want to get the size of model you want for whatever you want to pay for the power bill.

New comment by behohippy in "Ask HN: Do you think differently about working on open source these days?"

behohippy — Mon, 11 Aug 2025 15:42:43 +0000

Sure, all the slop code projects I produce get MIT licensed on public repos. It wasn't mine to begin with, so I wouldn't prevent anyone from using it.

New comment by behohippy in "Benchmark Framework Desktop Mainboard and 4-node cluster"

behohippy — Fri, 08 Aug 2025 01:08:44 +0000

Used 3090s have been getting expensive in some markets. Another option is dual 5060ti 16 gig. Mine are lower powered, single 8 pin power, so they max out around 180W. With that I'm getting 80t/s on the new qwen 3 30b a3b models, and around 21t/s on Gemma 27b with vision. Cheap and cheerful setup if you can find the cards at MSRP.

New comment by behohippy in "Deepseek R1-0528"

behohippy — Wed, 28 May 2025 18:31:13 +0000

About 768 gigs of ddr5 RAM in a dual socket server board with 12 channel memory and an extra 16 gig or better GPU for prompt processing. It's a few grand just to run this thing at 8-10 tokens/s

New comment by behohippy in "DeepSeek-V3 Technical Report"

behohippy — Thu, 27 Mar 2025 11:37:49 +0000

These articles are gold, thank you. I used your gemma one from a few weeks back to get gemma 3 performing properly. I know you guys are all GPU but do you do any testing on CPU/GPU mixes? I'd like to see the pp and t/s on pure 12 channel epyc and the same with using a 24 gig gpu to accelerate the pp.

New comment by behohippy in "Building a personal, private AI computer on a budget"

behohippy — Thu, 13 Feb 2025 18:37:16 +0000

I run the KV cache at Q8 even on that model. Is it not working well for you?

New comment by behohippy in "Building a personal, private AI computer on a budget"

behohippy — Wed, 12 Feb 2025 10:49:22 +0000

Qwen is a little fussy about the sampler settings, but it does run well quantized. If you were getting infinite repetition loops, try dropping the top_p a bit. I think qwen likes lower temps too

New comment by behohippy in "Building a personal, private AI computer on a budget"

behohippy — Tue, 11 Feb 2025 12:38:57 +0000

You probably won't be running fp16 anything locally. We typically run Q5 or Q6 quants to maximize the size of the model and context length we can run with the VRAM we have available. The quality loss is negligable at Q6.

New comment by behohippy in "Ask HN: Is anyone doing anything cool with tiny language models?"

behohippy — Wed, 22 Jan 2025 12:41:02 +0000

Just this pic: https://imgur.com/ip8GWIh

New comment by behohippy in "Ask HN: Is anyone doing anything cool with tiny language models?"

behohippy — Wed, 22 Jan 2025 12:40:41 +0000

I don't have a video but here's a pic of the output: https://imgur.com/ip8GWIh

New comment by behohippy in "Ask HN: Is anyone doing anything cool with tiny language models?"

behohippy — Wed, 22 Jan 2025 12:40:00 +0000

It's a 3b model so the creativity is pretty limited. What helped for me was prompting for specific stories in specific styles. I have a python script that randomizes the prompt and the writing style, including asking for specific author styles.

New comment by behohippy in "Ask HN: Is anyone doing anything cool with tiny language models?"

behohippy — Tue, 21 Jan 2025 20:57:37 +0000

I have a mini PC with an n100 CPU connected to a small 7" monitor sitting on my desk, under the regular PC. I have llama 3b (q4) generating endless stories in different genres and styles. It's fun to glance over at it and read whatever it's in the middle of making. I gave llama.cpp one CPU core and it generates slow enough to just read at a normal pace, and the CPU fans don't go nuts. Totally not productive or really useful but I like it.

New comment by behohippy in "Phi-3 Technical Report"

behohippy — Tue, 23 Apr 2024 15:35:19 +0000

I had this same issue with incomplete answers on longer summarization tasks. If you ask it to "go on" it will produce a better completion, but I haven't seen this behaviour in any other model.

New comment by behohippy in "Microsoft Phi-2 model changes licence to MIT"

behohippy — Sat, 06 Jan 2024 12:16:04 +0000

It's probably an evolution of the phi-1/1.5 "Textbooks are all you Need" training method: https://arxiv.org/abs/2309.05463

New comment by behohippy in "Amazon-Unveils-Q"

behohippy — Fri, 01 Dec 2023 15:24:09 +0000

No joke, that would be an awesome LLM project name!

New comment by behohippy in "Amazon-Unveils-Q"

behohippy — Thu, 30 Nov 2023 14:33:21 +0000

Top_p and top_k are pretty important concepts for LLMs same as temperature so P,K,C and F are underutilized

New comment by behohippy in "OpenLLaMA 13B Released"

behohippy — Sun, 18 Jun 2023 18:04:45 +0000

Hey emad, thanks for SD and this! What's the plan if Meta does Apache 2.0 for LLaMA? Just keep going and making the 30b and 65b or build different models?

New comment by behohippy in "RedPajama 7B (an Apache 2.0-licensed LLaMa) is now available"

behohippy — Tue, 06 Jun 2023 19:04:51 +0000

Vicuna-13b (4bit) got the answer right, the first time as well.