On an RTX 3090 power-limited to 220W: - 411 tok/s vs 229 tok/s on M5 Max (1.8x) - 1.87 tok/J, beating M5 Max efficiency - 1.55x faster decode than llama.cpp on the same GPU - 3.4x faster prefill

The RTX 3090 launched in 2020. Everyone calls it power-hungry. It isn't, the software is. The conventional wisdom NVIDIA is fast but thirsty. Apple Silicon is slow but sips power. Pick a side.

With stock frameworks, the numbers back that up: Setup | tok/s | Power | tok/J RTX 3090 (llama.cpp) | 267 | 350W | 0.76 M5 Max (LM Studio) | 229 | ~130W | 1.76

Case closed. Except the 3090 has 936 GB/s of bandwidth and 142 TFLOPS of FP16 compute, and llama.cpp extracts 267 tok/s out of it. That ratio is absurd.

Traditional inference dispatches one kernel per operation. For 24 layers, that's roughly 100 launches per token. Every boundary means: - Return control to the CPU - Dispatch the next kernel - Re-fetch weights from global memory - Synchronize threads

Why nobody had done this yet? Qwen3.5-0.8B isn't a vanilla transformer. It alternates: - 18 DeltaNet layers: linear attention with a learned recurrence - 6 Full Attention layers: standard MHA

This hybrid pattern is where frontier models are heading: Qwen3-Next, Kimi Linear, all of them. DeltaNet scales linearly with context length instead of quadratically.

It's new, and nobody has shipped a fused kernel for it. MLX doesn't have DeltaNet kernels at all. llama.cpp supports it generically. Everyone else is waiting. The 267 tok/s wasn't a hardware ceiling, it was the software ceiling for a brand-new architecture.

We wrote a single CUDA kernel that runs the entire forward pass in one dispatch. Data stays in registers and shared memory as it flows through the network. Zero CPU round-trips, zero redundant memory fetches.

- 82 blocks x 512 threads, all SMs occupied - BF16 weights and activations, FP32 accumulation DeltaNet recurrence runs in warp-cooperative F32 registers - Full attention fuses QKV, RoPE, causal softmax, and output projection - Cooperative grid sync replaces kernel launches between layers

Results on the same RTX 3090, same model, same weights: Setup | Prefill (pp520) | Decode (tg128) Megakernel | 37,800 tok/s | 413 tok/s llama.cpp BF16 | 11,247 tok/s | 267 tok/s PyTorch + HF | 7,578 tok/s | 108 tok/s

Then we turned the power down Fewer wasted cycles means less heat, so we swept nvidia-smi -pl: Power limit | Clock | Draw | tok/s | tok/J | Notes 420W (stock) | 1980 MHz | 314W | 433 | 1.38 | baseline 300W | 1935 MHz | 299W | 432 | 1.44 | -5% power, 99.8% speed 220W | 1635 MHz | 220W | 411 | 1.87 | -30% power, 95% speed 150W | 405 MHz | 150W | 194 | 1.29 | clock cliff, too aggressive

At 220W we hit the sweet spot: 95% of the throughput for 70% of the power. Tighter execution converts almost directly into saved watts. Measurement: NVML energy counters for NVIDIA, powermetrics for Apple Silicon, matching Hazy Research's Intelligence Per Watt methodology. Accelerator power only, not wall draw.

Without the megakernel the 3090 barely edges out a laptop chip. With it, a five-year-old GPU beats Apple's latest on throughput, matches it on efficiency, and costs a quarter as much. The NVIDIA vs Apple efficiency gap isn't silicon. It's software.

Try it git clone https://github.com/Luce-Org/luce-megakernel.git cd luce-megakernel pip install -e . python bench_pp_tg.py

Requires: NVIDIA Ampere+ (tested on 3090), CUDA 12+, PyTorch 2.0+, ~1.5GB VRAM.

Code is open source (MIT): https://github.com/Luce-Org/luce-megakernel

Let us know if you have any feedback

Comments URL: https://news.ycombinator.com/item?id=47691182

Points: 5

# Comments: 1

Use cases:

- Run multiple agents in parallel with isolated app views

- Automate your iPhone using the iPhone Mirroring app

- Improve agent task precision and reduce VLM distractions

Works only on macOS (Sequoia+) and requires experiments=["app-use"]. No extra processes, just clever compositing.

More details: https://www.trycua.com/blog/app-use

Feedback and experiments welcome!

Comments URL: https://news.ycombinator.com/item?id=44300812

Points: 2

# Comments: 1

When building virtualized environments for AI agents, we needed a reproducible way to package and distribute macOS VMs. Inspired by projects like dockur/windows (https://github.com/dockur/windows) that pioneered running Windows in Docker, we wanted to create something similar but optimized for Apple Silicon. The existing solutions either didn't support M-series chips or relied on KVM/Intel emulation, which was slow and cumbersome. We realized we could leverage Apple's Virtualization Framework to create a much better experience.

Lumier takes a different approach: it uses Docker as a delivery mechanism (not for isolation) and connects to a lightweight virtualization service (lume) running on your Mac. This creates true hardware-accelerated VMs using Apple's native virtualization capabilities.

With Lumier, you can: - Launch a ready-to-use macOS VM in minutes with zero manual setup - Access your VM through any web browser via VNC - Share files between your host and VM effortlessly - Use persistent storage or ephemeral mode for quick tests - Automate VM startup with custom scripts

All of this works natively on Apple Silicon (M1/M2/M3/M4) - no emulation required.

To get started:

1. Install Docker for Apple Silicon: https://desktop.docker.com/mac/main/arm64/Docker.dmg

2. Install lume background service with our one-liner:

  /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"

  docker run -it --rm \
  --name lumier-vm \
    -p 8006:8006 \
    -e VM_NAME=lumier-vm \
    -e VERSION=ghcr.io/trycua/macos-sequoia-cua:latest \
    -e CPU_CORES=4 \
    -e RAM_SIZE=8192 \
    trycua/lumier:latest

For persistent storage (so your changes survive container restarts):

mkdir -p storage docker run -it --rm \ --name lumier-vm \ -p 8006:8006 \ -v $(pwd)/storage:/storage \ -e VM_NAME=lumier-vm \ -e HOST_STORAGE_PATH=$(pwd)/storage \ trycua/lumier:latest

Want to share files with your VM? Just add another volume:

mkdir -p shared docker run ... -v $(pwd)/shared:/shared -e HOST_SHARED_PATH=$(pwd)/shared ...

You can even automate VM startup by placing an on-logon.sh script in shared/lifecycle/.

We're seeing people use Lumier for: - Development and testing environments that need macOS - CI/CD pipelines for Apple platform apps - Disposable macOS instances for security research - Automated UI testing across macOS versions - Running AI agents in isolated environments

Lumier is 100% open-source under the MIT license. We're actively developing it as part of our work on C/ua (https://github.com/trycua/cua), and we'd love your feedback, bug reports, or feature ideas.

We'll be here to answer any technical questions and look forward to your comments!

Comments URL: https://news.ycombinator.com/item?id=43985624

Points: 159

# Comments: 52

Comments URL: https://news.ycombinator.com/item?id=43975620

Points: 13

# Comments: 2

Comments URL: https://news.ycombinator.com/item?id=43968685

Points: 3

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=43968575

Points: 3

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=43955582

Points: 3

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=43940630

Points: 6

# Comments: 0

tlume will:

- Locate your Tart VM in ~/.tart/vms/

- Create a Lume VM in ~/.lume/

- Copy all VM files with optimized buffering

- Convert the Tart config to Lume format

- Preserve all your VM hardware settings, network config, and display settings

This new bridge interface will greatly simplify adoption for those with existing Tart images and help commoditize Apple Virtualization.framework solutions across the ecosystem!

Lume repo: https://github.com/trycua/cua/tree/main/libs/lume

Comments URL: https://news.ycombinator.com/item?id=43445122

Points: 5

# Comments: 2

Why we built Lume: - Run native macOS VMs in 1 command, using Apple Virtualization.Framework: `lume run macos-sequoia-vanilla:latest`

- Prebuilt images on https://ghcr.io/trycua (macOS, Ubuntu on ARM)

- API server to manage VMs programmatically `POST /lume/vms`

- A python SDK on github.com/trycua/pylume

Run prebuilt macOS images in just 1 step: lume run macos-sequoia-vanilla:latest

How to Install:

brew tap trycua/lume

brew install lume

You can also download the `lume.pkg.tar.gz` archive from the latest release https://github.com/trycua/lume/releases, extract it, and install the package manually.

Local API Server: `lume` exposes a local HTTP API server that listens on `http://localhost:3000/lume`, enabling automated management of VMs.

lume serve

For detailed API documentation, please refer to API Reference(https://github.com/trycua/lume/blob/main/docs/API-Reference....).

HN devs - would love raw feedback on the API design and whether this solves your Apple Silicon VM pain points. What would make you replace UTM/Multipass/Docker Desktop with this?

Repo: https://github.com/trycua/lume Python SDK: github.com/trycua/pylume Discord for direct feedback: https://discord.gg/8p56E2KJ

Comments URL: https://news.ycombinator.com/item?id=42908061

Points: 309

# Comments: 75

Comments URL: https://news.ycombinator.com/item?id=42725367

Points: 4

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=42697911

Points: 1

# Comments: 1