Hacker News - Newest: "LLM"

My self-hosted local LLM server setup

onthemarkdata — Sat, 20 Jun 2026 18:48:49 +0000

Article URL: https://old.reddit.com/r/LocalLLM/comments/1ub1iu2/my_selfhosted_llm_server_setup_to_access_open

Comments URL: https://news.ycombinator.com/item?id=48611821

Points: 2

# Comments: 1

Hand-powered LLM (YouTube) [video]

mcchen51 — Sat, 20 Jun 2026 16:52:17 +0000

Article URL: https://www.youtube.com/watch?v=HSapdLYpmWY

Comments URL: https://news.ycombinator.com/item?id=48610738

Points: 3

# Comments: 0

Changes that cut our LLM pipeline costs more than model-switching did

Abbas_Maka — Sat, 20 Jun 2026 13:05:31 +0000

I have been building multiple LLM systems and for our Organization biggest cost savings weren't from prompt-wordsmithing or model switchings. Sharing useful to anyone watching their token bill :

1) JSON → TOON for structured output: JSON was not made for LLMs. well you can implement your own verison that fits for your needs that reduce tokens usage but what worked for us was TOON. TOON cut output our tokens by ~30% same information, way less syntax tax.

2) Full markdown/HTML → condensed markdown: Using markdown for writing your prompts, getting intermediate results or communication between your Agents eats a lot of tokens. We swithced to condesed markdown and short system prompts that replicate Caveman. this alone cut just on input token costs ~50% on calls that pass prior context forward which can be implemented between Agent Calls.

3) Long Do/Don't instruction lists → 2-3 multi-shot examples: Counterintuitive one - replacing a large lists of DO's and Don'ts for agents rules don't help. rather couple of concrete examples that convers major and all cases actually improved output quality more reliably and it's usually fewer tokens once the instruction list gets long enough to cover real edge cases.

I have seen most people on this sub reddit talk about using open-source or cheaper models. Like we were spending thousands of dollar's but this all changes alone helped reduce cost by 60%.

edit: Open to Discussion, anyone whether something similar would help their setup.

Comments URL: https://news.ycombinator.com/item?id=48608978

Points: 3

# Comments: 0

Show HN: Local automation runner with built-in LLM steps – YAML pipelines

rorlikowski — Sat, 20 Jun 2026 09:21:46 +0000

Article URL: https://rorlikowski.github.io/stepyard/

Comments URL: https://news.ycombinator.com/item?id=48607721

Points: 5

# Comments: 2

Show HN: FERNme – agent memory that updates with ~zero LLM calls

mirkofr — Sat, 20 Jun 2026 08:55:51 +0000

Hi HN, FERNme is a memory layer for AI agents. Most systems (Mem0, etc.) call an LLM on every turn to extract/summarize memory, or dump everything into a vector DB. I wanted to see how far a cheaper, more brain-like approach goes. Each user is a node in a fuzzy preference graph; edges (0–9, plus explicit negative edges for dislikes) update with a Hebbian co-occurrence rule — no LLM on the write path — decay ACT-R-style, and compile to a flat, ~40-token "memory card." Retrieval is spreading activation, not vector search. Some early results (all reproducible in the repo): ingesting 86 free-form diary entries about one person ran with 0 write-time LLM calls, kept a flat ~40-token card, and on a LoCoMo-style QA set the context-seeded retrieval answered ~4× more questions than frequency/recency baselines at equal budget — and was the only LLM-free method that handled preference drift. Honest about what this is not: the Hebbian + spreading-activation mechanism isn't novel — HippoRAG, Ori-Mnemos, and HeLa-Mem all use versions of it. My bet is on the combination: near-zero-cost writes, multi-tenant + a privacy-preserving population prior, user-owned cross-surface memory, and evaluating on actions rather than QA. Benchmarks so far are synthetic or single-person; a real LLM-memory (Mem0) head-to-head needs API keys and isn't run yet. It's MCP-compatible, and the repo has a paper draft (PAPER.md) plus a reproducible demo (demo/elena/ — builds memory from 86 free-form diary entries and runs the QA benchmark). Code: github.com/mirkofr/FERNme I'd love feedback on: the memory representation, whether the action-coupled eval is meaningful, failure modes you'd expect, and benchmarks/competitors I should test against — especially a real Mem0 head-to-head. Criticism very welcome.

Comments URL: https://news.ycombinator.com/item?id=48607574

Points: 1

# Comments: 0

Ask HN: Will we start seeing tools for LLM use?

bonigv — Sat, 20 Jun 2026 06:58:33 +0000

Many projects and addons exists to reduce the verbosity on standard bash / git /npm etc. Commands that agents pass regularly as tools to LLMs. (eg. rtk, headroom, lean-ctx). Tool output compression does yield good token savings. Though it can lead to increased turns - effectively nullifying the per turn token savings. This is the current topology. Are we going to see a class of products and libraries that structure o/p to what models want to see?

Comments URL: https://news.ycombinator.com/item?id=48606997

Points: 2

# Comments: 1

Compress tool outputs, logs, files, RAG chunks before LLM for 60-95% less tokens

gmays — Sat, 20 Jun 2026 04:45:42 +0000

Article URL: https://github.com/chopratejas/headroom

Comments URL: https://news.ycombinator.com/item?id=48606411

Points: 4

# Comments: 0

I am dreading our LLM-written incident report future

azhenley — Sat, 20 Jun 2026 00:50:52 +0000

Article URL: https://surfingcomplexity.blog/2026/06/19/i-am-dreading-our-llm-written-incident-report-future/

Comments URL: https://news.ycombinator.com/item?id=48605136

Points: 6

# Comments: 0

Show HN: slash-agent – Native LLM copilot for your terminal

akatzmann — Fri, 19 Jun 2026 20:07:55 +0000

Article URL: https://github.com/akatzmann/slash-agent

Comments URL: https://news.ycombinator.com/item?id=48602690

Points: 2

# Comments: 0

Pipeline-parallel LLM inference across GPUs on separate machines

ngaut — Fri, 19 Jun 2026 19:14:34 +0000

Article URL: https://github.com/leyten/shard

Comments URL: https://news.ycombinator.com/item?id=48602121

Points: 5

# Comments: 0

LLM Quantization Project Part 1: What Even Is an LLM?

LabsLucas — Fri, 19 Jun 2026 17:36:17 +0000

Article URL: https://www.lttlabs.com/articles/2026/06/19/llm-quantization-part-1-what-even-is-an-llm

Comments URL: https://news.ycombinator.com/item?id=48600968

Points: 4

# Comments: 2

AI-gateway product that cuts LLM API TOKEN costs by 40-70%

arnab777 — Fri, 19 Jun 2026 16:44:47 +0000

Comments URL: https://news.ycombinator.com/item?id=48600462

Points: 2

# Comments: 2

Show HN: Wyolet Relay – high throughput, open source LLM router

aaliboyev — Fri, 19 Jun 2026 13:36:00 +0000

Article URL: https://github.com/wyolet/relay

Comments URL: https://news.ycombinator.com/item?id=48598419

Points: 3

# Comments: 0

The LLM industry must keep the RAM prices at absurd levels

ndr42 — Fri, 19 Jun 2026 11:02:04 +0000

Article URL: https://infosec.exchange/@masek/116775772309957886

Comments URL: https://news.ycombinator.com/item?id=48597189

Points: 4

# Comments: 0

Self-adapting and mutating LLM based viruses/worms

rozumbrada — Fri, 19 Jun 2026 10:58:21 +0000

I am thinking about a future of malware and cyber worms. I bet it's gonna be self-mutating and adapting to local environment using local models (once they are built-in to all devices and performant enough in future years). Basically almost a real organism resembling real biological viruses. In this case the non-determinism of LLMs is a feature. Every infection could take its own development path - and half might die, half might survive. Think genetic programming but autonomous and on steroids. For some non tech (even tech?) people this reminds Skynet and it's fascinating that we are in a trajectory that this suddenly imaginable and theoretically soon possible.

Why is not happening now? Inference is still expensive and local models are not there yet, so there's no ROI in making this at scale. But once inference is local and cheap as electricity or running water, this is the natural development. How do we stop the spreading then?

Are there already some documented experiments?

Comments URL: https://news.ycombinator.com/item?id=48597160

Points: 3

# Comments: 4