Hacker News: jmorgan

New comment by jmorgan in "Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?"

jmorgan — Mon, 15 Jun 2026 18:38:12 +0000

The larger models are available on Ollama's cloud as most folks don't have the hardware to run 500B-1T parameter models.

New comment by jmorgan in "Pi – A minimal terminal coding harness"

jmorgan — Wed, 25 Feb 2026 06:51:39 +0000

For local models I've been trying it with GLM-4.7-Flash and the new LFM2 24B model. I'm excited to try it with the new Qwen3.5 models that came out today as well.

New comment by jmorgan in "Pi – A minimal terminal coding harness"

jmorgan — Tue, 24 Feb 2026 22:33:52 +0000

I've been using Pi day to day recently for simple, smaller tasks. It's a great harness for use with smaller parameter size models given the system prompt is quite a bit shorter vs Claude or Codex (and it uses a nice small set of tools by default).

New comment by jmorgan in "ollama launch"

jmorgan — Sun, 25 Jan 2026 21:05:04 +0000

That's not good, sorry. I work on Ollama - shoot me an email (jeff@ollama.com) and we can help debug

New comment by jmorgan in "GLM-4.7-Flash"

jmorgan — Mon, 19 Jan 2026 22:28:31 +0000

It's available (with tool parsing, etc.): https://ollama.com/library/glm-4.7-flash but requires 0.14.3 which is in pre-release (and available on Ollama's GitHub repo)

New comment by jmorgan in "A guide to local coding models"

jmorgan — Mon, 22 Dec 2025 03:59:33 +0000

The source is available here: https://github.com/ollama/ollama/tree/main/app

New comment by jmorgan in "Claude Is Down"

jmorgan — Sat, 08 Nov 2025 04:13:47 +0000

The gpt-oss weights on Ollama are native mxfp4 (the same weights provided by OpenAI). No additional quantization is applied, so let me know if you're seeing any strange results with Ollama.

Most gpt-oss GGUF files online have parts of their weights quantized to q8_0, and we've seen folks get some strange results from these models. If you're importing these to Ollama to run, the output quality may decrease.

New comment by jmorgan in "Ollama Web Search"

jmorgan — Thu, 25 Sep 2025 20:18:04 +0000

We did consider building functionality into Ollama that would go fetch search results and website contents using a headless browser or similar. However we had a lot of worries about result quality and also IP blocking from Ollama creating crawler-like behavior. Having a hosted API felt like a fast path to get results into users' context window, but we are still exploring the local option. Ideally you'd be able to stay fully local if you want to (even when using capabilities like search)

Ollama Web Search

jmorgan — Thu, 25 Sep 2025 19:21:52 +0000

Article URL: https://ollama.com/blog/web-search

Comments URL: https://news.ycombinator.com/item?id=45377641

Points: 348

# Comments: 176

New comment by jmorgan in "Gemma 3 270M: Compact model for hyper-efficient AI"

jmorgan — Thu, 14 Aug 2025 18:32:09 +0000

Amazing work. This model feels really good at one-off tasks like summarization and autocomplete. I really love that you released a quantized aware training version on launch day as well, making it even smaller!

New comment by jmorgan in "Ollama Turbo"

jmorgan — Wed, 06 Aug 2025 05:42:39 +0000

It should open ollama.com/connect – sorry about that. Feel free to message me jeff@ollama.com if you keep seeing issues

New comment by jmorgan in "Open models by OpenAI"

jmorgan — Tue, 05 Aug 2025 19:25:10 +0000

Sorry about this. Re-downloading Ollama should fix the error

New comment by jmorgan in "Magistral — the first reasoning model by Mistral AI"

jmorgan — Wed, 11 Jun 2025 03:50:08 +0000

Working on adding tool calling support to Magistral in Ollama. It requires a tokenizer change and also uses a new tool calling format. Excited to see the results of combining thinking + tool calling!

UI-TARS-1.5: open-source computer use vision-language model

jmorgan — Thu, 17 Apr 2025 16:44:40 +0000

Article URL: https://seed-tars.com/1.5/

Comments URL: https://news.ycombinator.com/item?id=43719311

Points: 4

# Comments: 1

New comment by jmorgan in "Deepseek R1 Distill 8B Q40 on 4 x Raspberry Pi 5"

jmorgan — Sun, 16 Feb 2025 22:38:05 +0000

This is a great point. apt-get would definitely be a better install experience and upgrade experience (that's what I would want too). Tailscale does this amazing well: https://tailscale.com/download/linux

The main issue for the maintainer team would be the work in hosting and maintaining all the package repos for apt, yum, etc, and making sure the we handle the case where nvidia/amd drivers aren't installed (quite common on cloud VMs). Mostly a matter of time and putting in the work.

For now every release of Ollama includes a minimal archive with the ollama binary and required dynamic libraries: https://github.com/ollama/ollama/blob/main/docs/linux.md#man.... But we could definitely do better

New comment by jmorgan in "Qwen2.5-1M: Deploy your own Qwen with context length up to 1M tokens"

jmorgan — Sun, 26 Jan 2025 20:38:28 +0000

Sorry this isn't more obvious. Ideally VRAM usage for the context window (the KV cache) becomes dynamic, starting small and growing with token usage, whereas right now Ollama defaults to a size of 2K which can be overridden at runtime. A great example of this is vLLM's PagedAttention implementation [1] or Microsoft's vAttention [2] which is CUDA-specific (and there are quite a few others).

1M tokens will definitely require a lot of KV cache memory. One way to reduce the memory footprint is to use KV cache quantization, which has recently been added behind a flag [3] and will 1/4 the memory footprint if 4-bit KV cache quantization is used (OLLAMA_KV_CACHE_TYPE=q4_0 ollama serve)

[1] https://arxiv.org/pdf/2309.06180

[2] https://github.com/microsoft/vattention

[3] https://smcleod.net/2024/12/bringing-k/v-context-quantisatio...

New comment by jmorgan in "Phi 4 available on Ollama"

jmorgan — Sun, 12 Jan 2025 04:16:20 +0000

Phi-4's architecture changed slightly from Phi-3.5 (it no longer uses a sliding window of 2,048 tokens [1]), causing a change in the hyperparameters (and ultimately an error at inference time for some published GGUF files on Hugging Face, since the same architecture name/identifier was re-used between the two models).

For the Phi-4 uploaded to Ollama, the hyperparameters were set to avoid the error. The error should stop occurring in the next version of Ollama [2] for imported GGUF files as well

In retrospect, a new architecture name should probably have been used entirely, instead of re-using "phi3".

[1] https://arxiv.org/html/2412.08905v1

[2] https://github.com/ollama/ollama/releases/tag/v0.5.5

New comment by jmorgan in "Fast LLM Inference From Scratch (using CUDA)"

jmorgan — Mon, 16 Dec 2024 04:17:21 +0000

Thank you for writing this!

Diffusion models are real-time game engines

jmorgan — Wed, 28 Aug 2024 02:59:40 +0000

Article URL: https://gamengen.github.io

Comments URL: https://news.ycombinator.com/item?id=41375548

Points: 1149

# Comments: 409

New comment by jmorgan in "Gemma 2: Improving Open Language Models at a Practical Size [pdf]"

jmorgan — Fri, 28 Jun 2024 07:39:11 +0000

Currently when the context limit is hit, there's a halving of the context window (or a "context shift") to allow inference to continue – this is helpful for smaller (e.g. 1-2k) context windows.

However, not all models (especially newer ones) respond well to this, which makes sense. We're working on changing the behavior in Ollama's API to be more similar to OpenAI, Anthropic and similar APIs so that when the context limit is hit, the API returns a "limit" finish/done reason. Hope this is helpful!