Hacker News: ollin

New comment by ollin in "Explorative modeling: Train on the best of K guesses"

ollin — Sat, 01 Aug 2026 17:37:03 +0000

This paper shows a nice integration of older winner-take-all ideas for learning K-modal generative models (see e.g. https://arxiv.org/abs/1612.00197, https://arxiv.org/abs/2211.14286) into modern diffusion/flow pipelines. As-implemented, I think it has some downsides:

1. K-1 extra forward passes during training

2. Inaccurate sampling behavior (will sample all K modes with equal likelihood, rather than sampling them proportionally)

However, I think both of these downsides can be mitigated by adjusting the implementation a bit more (you can have the model predict K modes jointly in one forward pass, along with probabilities of each being the min-loss mode, which you can then use for properly-weighted mode sampling at inference).

That said, I'm not entirely sure if this strategy is as generally applicable as the authors are hoping. In particular:

1. For highly-conditional image generation (like modern commercial diffusion pipelines, which use a big LLM preprocessor), most of the low-frequency color/layout decisions are already made for you by the conditioning signal. The diffusion process mostly needs to generate high-frequency details, for which there are a huge number of equally-valid modes.

2. For LLMs themselves, the sequence-generation process is already factored into a discrete classification problem, and there's no mode smearing issue to fix.

New comment by ollin in "OpenAI and Hugging Face address security incident during model evaluation"

ollin — Wed, 22 Jul 2026 04:04:42 +0000

The ExploitGym paper evaluated several frontier models on the bench and reported that "Different models find different exploits" [1], so it seems most plausible that the "test solutions directly from Hugging Face’s production database" [2] which GPT-internal found were authored by Mythos (or some other LLM with complementary strengths), and placed in some internal HF repository when creating the ExploitGym paper/leaderboard.

[1] https://www.cybergym.io/exploitgym/#:~:text=Different%20mode...

[2] https://openai.com/index/hugging-face-model-evaluation-secur...

New comment by ollin in "OpenAI and Hugging Face address security incident during model evaluation"

ollin — Tue, 21 Jul 2026 23:15:33 +0000

If the HuggingFace repo the agent broke into contains reference solution scripts for ExploitGym (i.e. for exploiting the vulnerabilities in the intended way), the agent can then run that reference code inside its original sandbox to retrieve the dynamically-generated flags.

New comment by ollin in "Designing emoji for the way we communicate today"

ollin — Fri, 17 Jul 2026 19:47:58 +0000

This article seems fairly uninformative since, as others have pointed out, there's no visualization or comparison of the full emoji set and no link to see it. They just show a few example images and have some (AI-enhanced?) prose that doesn't actually say very much.

This article https://9to5google.com/2026/05/12/android-17-emoji-redesign/ has a larger (2d image) comparison grid with several dozen examples and an A/B slider vs the old versions. Overall the new design looks like a fairly tasteful compromise between Google's previous flat-shaded vector emoji and the hybrid 2d+3d Apple emoji, with the benefits (easier to rerender with higher-resolution, animations, tweaked lighting, etc.) that you'd get from a fully-3D pipeline. So I like the new set of emoji, just not this particular blog.google.com article.

New comment by ollin in "Making Deep Learning Go Brrrr from First Principles"

ollin — Sat, 23 May 2026 16:51:41 +0000

Yeah, that part should not be read literally; `x.cos().cos()` and `x1 = x.cos(); x2 = x1.cos()` both launch the same number of kernels (two in unfused/eager mode, one in fused/torch.compile, see this test notebook [1]). I think the author chained the two cos calls to symbolize the idea of combining them (without exposing the intermediate result), but chaining the two cos calls doesn't literally trigger operator fusion.

[1] https://colab.research.google.com/drive/13a4Y-ko6QLMPAhBz64c...

New comment by ollin in "Making deep learning go brrrr from first principles (2022)"

ollin — Sat, 23 May 2026 16:31:44 +0000

This post is a classic! Also recommended: Horace also gave a related talk (covering the high-level picture of modern ML Systems) at Jane Street in Dec 2024 https://www.youtube.com/watch?v=139UPjoq7Kw

New comment by ollin in "I Miss Terry Pratchett"

ollin — Sat, 23 May 2026 14:44:19 +0000

Feet of Clay is one of my favorites in the series! It's surprising how literally the Discworld version of Golems corresponds to modern LLMs (and perhaps upcoming LLM-backed humanoids?).

The Golems are brought to life by a slip of words in their heads called chem, which is almost 1:1 to an LLM system prompt (or perhaps the Claude Soul Document):

    I AM A GOLEM. I WAS MADE OF CLAY. MY LIFE IS THE WORDS. BY MEANS OF WORDS OF PURPOSE IN MY HEAD I ACQUIRE LIFE. MY LIFE IS TO WORK. I OBEY ALL COMMANDS. I TAKE NO REST.

The Golems are perfectly intelligent and self-aware, but since they don't exhibit independent goals beyond their prompt, they get treated as appliances rather than as sentient creatures.

    “What words of purpose?”
    RELEVANT TEXT THAT ARE THE FOCUS OF BELIEF. GOLEM MUST WORK. GOLEM MUST HAVE A MASTER.
    “Sorry, look,” said Cheery. “Are you telling me this… thing is powered by words? I mean… is *it* telling me it’s powered by words?”
    “Why not? Words do have power. Everyone knows that,” said Angua. “There are more golems around than you might think. They’re out of fashion now, but they last. They can work underwater, or in total darkness, or knee-deep in poison. For years. They don’t need rest or feeding. They…”
    “But that’s slavery!” said Cheery.
    “Of course it isn’t. You might as well enslave a doorknob.”

The integration of more (and more-independent) Golems into society is gradual and controversial, per Making Money:

    There was another protest march going on when Moist walked to the bank. You got more and more of them lately.
    This march was against the employment of golems, who uncomplainingly did the dirtiest jobs, worked around the clock, and were so honest they paid their taxes. But they weren’t human and they had glowing eyes, and people could get touchy about that sort of thing.

New comment by ollin in "I’ve joined Anthropic"

ollin — Tue, 19 May 2026 17:14:19 +0000

Specifically it looks like he's planning to extend the ideas from https://github.com/karpathy/autoresearch into a larger effort towards recursive training improvement [1]:

> Excited to welcome Andrej to the Pretraining team! He'll be building a team focused on using Claude to accelerate pretraining research itself. I can’t think of anyone better suited to do it — looking forward to what we build together!

[1] https://x.com/nickevanjoseph/status/2056760504949842219

New comment by ollin in "Agora-1: The Multi-Agent World Model"

ollin — Mon, 18 May 2026 23:25:37 +0000

This is cute and retro! But I think training only on GoldenEye undersells the concept a bit, since their world model inherits the N64-era graphics from GoldenEye, which automatically makes it look dated.

If they retrained the same model on real video data, they could potentially get a multiplayer world with quite realistic-looking graphics (see https://wayve.ai/wp-content/uploads/2025/11/Ex-2-GAIA-3.mp4, https://wayve.ai/wp-content/uploads/2025/11/Ex-3-GAIA-3.mp4).

Maybe for Agora-2 :)

New comment by ollin in "SANA-WM, a 2.6B open-source world model for 1-minute 720p video"

ollin — Sat, 16 May 2026 16:44:55 +0000

Right now there is (AFAIK) no world model product booking any meaningful revenue. So there's a decent chance WMs turn out to have no long-term utility at all.

However, there are a few promising markets, assuming WMs continue to get better and cheaper:

1. Robotics training / evaluation: modern end-to-end (sensors-to-control) robot policies require simulators that are almost indistinguishable from reality. If your sim is distinguishable from reality, the evaluation metrics you get from sim don't mean anything and the policies you train in sim don't work. World models will likely be the highest-fidelity robotics simulators, since WMs are data-driven and get arbitrarily more-realistic given more data/compute. This is why so many robotics companies have WM projects [1] [2] [3] [4].

2. Video frontends for agents: in the same way that today's frontier labs are building realtime voice interfaces [5] which behave like a phone call, realtime video interfaces will behave like a video call. Early forms of this don't feel compelling IMO [6] [7], but once the models can instantly blend between rendering the agent itself, drawing diagrams/visualizations, rendering video, etc. I can see it surpassing pure voice mode.

3. Entertainment: zero-shot world generation (i.e. holodeck, genie 3; paste in an image/video/text prompt and get a world) will be a fun toy but I'm not convinced it has any long-term value. I'm more optimistic about proper narrative experiences where each scene/level is a small, carefully-crafted world (behaving like a normal film scene if you don't touch the controls, and an uncharted/TLoU-style narrative game if you do), such that the sequence of scenes builds up a larger story.

[1] https://wayve.ai/thinking/gaia-3/

[2] https://xcancel.com/Tesla/status/1982255564974641628 / https://xcancel.com/ProfKuang/status/1996642397204394179

[3] https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-f...

[4] https://www.1x.tech/discover/world-model-self-learning

[5] https://thinkingmachines.ai/blog/interaction-models/

[6] https://runwayml.com/news/introducing-runway-characters

[7] https://blog.character.ai/character-ais-real-time-video-brea...

New comment by ollin in "Waymo updates 3,800 robotaxis after they 'drive into standing water'"

ollin — Fri, 15 May 2026 21:44:11 +0000

This is a HW4 Tesla on FSD 14.3.2 trying to drive into a lake five days ago (a la The Office): https://www.reddit.com/r/TeslaFSD/comments/1t9rl2u/fsd_tried..., so I would not say Tesla has solved standing water yet.

That said, FSD seems quite capable of routing around standing water in many cases (e.g. https://xcancel.com/planoken/status/2030754820462633031, https://www.reddit.com/r/TeslaFSD/comments/1pw9f2m/fsd_navig..., https://xcancel.com/BLKMDL3/status/1991862465328779317, https://xcancel.com/JVTacoma/status/2046313902749921638), so handling the remaining cases seems more like a model intelligence / data issue rather than a sensor limitation. Lidar beams generally bounce off mirrorlike surfaces without returning to the sensor, so I think all lidar would tell you about standing water is "there's something shiny/reflective within this region of the image", which you already know from cameras+headlights.

The Singularity Is the Friends We Made Along the Way

ollin — Sun, 10 May 2026 16:09:14 +0000

Article URL: https://geohot.github.io//blog/jekyll/update/2026/05/09/real-singularity.html

Comments URL: https://news.ycombinator.com/item?id=48085117

Points: 2

# Comments: 0

New comment by ollin in "Show HN: Apple's SHARP running in the browser via ONNX runtime web"

ollin — Sun, 03 May 2026 16:28:43 +0000

Most ONNX files are fp32, but the ONNX format actually allows fp16, int8, etc. as well (see onnx.proto for the full list of dtypes [1] - they even have fp8/fp4 these days!). I ended up switching over to fp16 ONNX models for my own web-based inference project since the quality is ~identical and page loads get 2x faster.

[1] https://github.com/onnx/onnx/blob/main/onnx/onnx.proto#L605

New comment by ollin in "Chinese AI models are ~8 months behind and falling further behind"

ollin — Sat, 02 May 2026 16:08:27 +0000

The source here is "CAISI Evaluation of DeepSeek V4 Pro" [1]; the US NIST ran their own benchmarks (including several internal ones) and reported the following table:

    | Domain               | Benchmark              | Model (reasoning level) |                             |                          |                       |
    |--:-------------------|------------------------|-------------------------|-----------------------------|--------------------------|-----------------------|
    |                      |                        | OpenAI GPT-5.5 (xhigh)  | OpenAI GPT-5.4 mini (xhigh) | Anthropic Opus 4.6 (max) | DeepSeek V4 Pro (max) |
    | Cyber                | CTF-Archive-Diamond    | **71%**                 | 32%                         | 46%                      | 32%                   |
    | Software Engineering | SWE-Bench Verified*    | **81%**                 | 73%                         | 79%                      | 74%                   |
    |                      | PortBench              | **78%**                 | 41%                         | 60%                      | 44%                   |
    | Natural Sciences     | FrontierScience        | **79%**                 | 74%                         | 72%                      | 74%                   |
    |                      | GPQA-Diamond           | **96%**                 | 87%                         | 91%                      | 90%                   |
    | Abstract Reasoning   | ARC-AGI-2 semi-private | **79%**                 | –                           | 63%                      | 46%                   |
    | Mathematics          | OTIS-AIME-2025         | **100%**                | 90%                         | 92%                      | 97%                   |
    |                      | PUMaC 2024             | **96%**                 | 93%                         | 95%                      | **96%**               |
    |                      | SMT 2025               | **99%**                 | 92%                         | 94%                      | 96%                   |
    | IRT-Estimated Elo    | **IRT-Estimated Elo**  | **1260 ± 28**           | 749 ± 46                    | 999 ± 27                 | 800 ± 28              |

Notably, two of the benchmarks with the biggest capability gap are CAISI-internal/private ones (CTF-Archive-Diamond, PortBench). I read this as "DeepSeek is well-tuned for public benchmarks, and less generally intelligent than GPT5.5 on held-out tasks" but a less-charitable reading would be "US government reports US models do best on benchmarks that only the US government can run". Agent benchmarking is fraught with peril [2] and an impartial benchmarker (who disproportionately overlooks bugs/issues in their evaluation of certain models) can absolutely tilt the scales, so I would not be surprised if a PRC-led benchmarking of frontier models came to the opposite conclusion.

[1] https://www.nist.gov/news-events/news/2026/05/caisi-evaluati...

[2] https://epoch.ai/gradient-updates/why-benchmarking-is-hard

New comment by ollin in "Artemis II Photo Timeline"

ollin — Sat, 02 May 2026 00:26:04 +0000

Hank Green has a video walking through how to use the timeline here https://www.youtube.com/watch?v=LyZE9VWJjDA. For me, the best experience was to click "Crew Photos Only" and then step through the photos chronologically with the arrow buttons.

New comment by ollin in "Where the goblins came from"

ollin — Thu, 30 Apr 2026 03:48:18 +0000

For context, two days ago some users [1] discovered this sentence reiterated throughout the codex 5.5 system prompt [2]:

> Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query.

[1] https://x.com/arb8020/status/2048958391637401718

[2] https://github.com/openai/codex/blob/main/codex-rs/models-ma...

New comment by ollin in "Anthropic Joins the Blender Development Fund as Corporate Patron"

ollin — Tue, 28 Apr 2026 17:58:11 +0000

AFAIK Anthropic hasn't built any image or video generation tools yet, just text/code generation. OpenAI/Google/xAI all built image/video generation teams though so it may only be a matter of time.

New comment by ollin in "Project Glasswing: Securing critical software for the AI era"

ollin — Tue, 07 Apr 2026 20:25:09 +0000

- The OpenBSD one is 'TCP packets with invalid SACK options could crash the kernel' https://cdn.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...

- One (patched) Linux kernel bug is 'UaF when sys_futex_requeue() is used with different flags' https://github.com/torvalds/linux/commit/e2f78c7ec1655fedd94...

These links are from the more-detailed 'Assessing Claude Mythos Preview’s cybersecurity capabilities' post released today https://red.anthropic.com/2026/mythos-preview/, which includes more detail on some of the public/fixed issues (like the OpenBSD one) as well as hashes for several unreleased reports and PoCs.

New comment by ollin in "System Card: Claude Mythos Preview [pdf]"

ollin — Tue, 07 Apr 2026 19:26:10 +0000

My impression was entirely the opposite; the unsolved subset of SWE-bench verified problems are memorizable (solutions are pulled from public GitHub repos) and the evaluators are often so brittle or disconnected from the problem statement that the only way to pass is to regurgitate a memorized solution.

OpenAI had a whole post about this, where they recommended switching to SWE-bench Pro as a better (but still imperfect) benchmark:

https://openai.com/index/why-we-no-longer-evaluate-swe-bench...

> We audited a 27.6% subset of the dataset that models often failed to solve and found that at least 59.4% of the audited problems have flawed test cases that reject functionally correct submissions

> SWE-bench problems are sourced from open-source repositories many model providers use for training purposes. In our analysis we found that all frontier models we tested were able to reproduce the original, human-written bug fix

> improvements on SWE-bench Verified no longer reflect meaningful improvements in models’ real-world software development abilities. Instead, they increasingly reflect how much the model was exposed to the benchmark at training time

> We’re building new, uncontaminated evaluations to better track coding capabilities, and we think this is an important area to focus on for the wider research community. Until we have those, OpenAI recommends reporting results for SWE-bench Pro.

New comment by ollin in "Tell HN: Apple development certificate server seems down?"

ollin — Wed, 11 Mar 2026 00:28:24 +0000

Here was the developer thread https://developer.apple.com/forums/thread/818403 I found with lots of other reports of "Unable to Verify App - An internet connection is required to verify the trust of the developer".

Although https://developer.apple.com/system-status/ was green for most of the 3-4 hour outage, the page now at least acknowledges two minutes of downtime:

    App Store Connect - Resolved Outage
    Today, 12:04 AM - 12:06 AM
    All users were affected
    Users experienced a problem with this service.

Not a great developer experience.