The problem: Python data pipelines become the bottleneck when fine-tuning large models. Your GPUs sit idle waiting for data.

The solution: Drop-in Rust acceleration. One import line, zero config changes.

Results on 50k rows: - Streaming data loading: 0.009s vs 0.724s (77x faster) - Parallel SHA256 hashing: 0.027s vs 0.052s (1.9x faster)

Works with Parquet, Arrow, JSON, JSONL, CSV. Supports compression. Cross-platform.

Usage:

import fast_axolotl import axolotl # now accelerated pip install fast-axolotl

Built with PyO3 and maturin. MIT licensed. Happy to answer questions about the Rust/Python interop or benchmark methodology.

Comments URL: https://news.ycombinator.com/item?id=47338327

Points: 1

# Comments: 0

The problem: Solana has mass dev interest (17k+ active developers in 2025), but the Rust learning curve remains a 3-6 month barrier. Anchor helps, but you still need to grok ownership, lifetimes, and borrowing. Meanwhile, there are 30k+ Solidity developers who already know how to write smart contracts.

SolScript bridges that gap. You write this:

  contract Token {
      mapping(address => uint256) public balanceOf;

      function transfer(address to, uint256 amount) public {
          balanceOf[msg.sender] -= amount;
          balanceOf[to] += amount;
          emit Transfer(msg.sender, to, amount);
      }
  }

- Parser turns Solidity-like source into an AST - Type checker validates and annotates

- Two codegen backends: (1) Anchor/Rust output that goes through cargo build-sbf, or (2) direct LLVM-to-BPF compilation - Mappings become PDAs automatically, account structs are derived from your type system

What's supported:

- State variables, structs, arrays, nested mappings

- Events and custom errors

- Modifiers (inlined)

- Cross-program invocation (CPI)

- SPL Token operations - msg.sender, block.timestamp equivalents

Current limitations:

- No msg.value for incoming SOL (use wrapped SOL or explicit transfers) - No Token 2022 support yet (planned for v0.4) - Modifiers are inlined, so keep them small

The output is standard Anchor/Rust code. You can eject anytime and continue in pure Rust. It's a launchpad, not a lock-in.

Written in Rust. Ships with a VS Code extension (LSP, syntax highlighting, go-to-definition, autocomplete).

Install: cargo install solscript-cli

I'd love feedback on the language design, the compilation approach, or use cases I haven't thought of. Happy to answer questions about the internals.

Comments URL: https://news.ycombinator.com/item?id=46693924

Points: 1

# Comments: 0

I built SolScript, a compiler that lets you write smart contracts in Solidity syntax and deploy them to Solana.

The problem: Solana has mass dev interest (17k+ active developers in 2025), but the Rust learning curve remains a 3-6 month barrier. Anchor helps, but you still need to grok ownership, lifetimes, and borrowing. Meanwhile, there are 30k+ Solidity developers who already know how to write smart contracts.

SolScript bridges that gap. You write this:

  contract Token {
      mapping(address => uint256) public balanceOf;

      function transfer(address to, uint256 amount) public {
          balanceOf[msg.sender] -= amount;
          balanceOf[to] += amount;
          emit Transfer(msg.sender, to, amount);
      }
  }

How it works:

- Parser turns Solidity-like source into an AST - Type checker validates and annotates - Two codegen backends: (1) Anchor/Rust output that goes through cargo build-sbf, or (2) direct LLVM-to-BPF compilation - Mappings become PDAs automatically, account structs are derived from your type system

What's supported:

- State variables, structs, arrays, nested mappings - Events and custom errors - Modifiers (inlined) - Cross-program invocation (CPI) - SPL Token operations - msg.sender, block.timestamp equivalents

Current limitations:

- No msg.value for incoming SOL (use wrapped SOL or explicit transfers) - No Token 2022 support yet (planned for v0.4) - Modifiers are inlined, so keep them small

The output is standard Anchor/Rust code. You can eject anytime and continue in pure Rust. It's a launchpad, not a lock-in.

Written in Rust. Ships with a VS Code extension (LSP, syntax highlighting, go-to-definition, autocomplete).

Install: cargo install solscript-cli

Repo: https://github.com/cryptuon/solscript

I'd love feedback on the language design, the compilation approach, or use cases I haven't thought of. Happy to answer questions about the internals.

Comments URL: https://news.ycombinator.com/item?id=46692514

Points: 1

# Comments: 0

What does BJH stand for though?

THE PROBLEM

Profiled my LangGraph agents. 50-100ms per invocation, most of it not the LLM. Found two culprits:

1. ThreadPoolExecutor created fresh every invoke() — 20ms overhead

2. Checkpointing uses deepcopy() — 52ms for 35KB state, 206ms for 250KB

THE FIX

Rewrote hot paths in Rust via PyO3:

Checkpoint serialization (serde vs deepcopy):

35KB state: 0.29ms vs 52ms = 178x faster

250KB state: 0.28ms vs 206ms = 737x faster

E2E with checkpointing: 2-3x faster

Drop-in usage:

export FAST_LANGGRAPH_AUTO_PATCH=1

# or explicit from fast_langgraph import RustSQLiteCheckpointer

checkpointer = RustSQLiteCheckpointer("state.db")

KEY INSIGHT

PyO3 boundary costs ~1-2μs per call. Rust only wins when you:

- Avoid intermediate Python objects (checkpoint serialization)

- Batch operations (channel updates)

- Handle large data (state > 10KB)

For simple dict ops, Python's C-dict still wins.

Architecture: Python orchestration (compatibility) + Rust hot paths (performance).

Runs regular compatibility checks!

MIT licensed. Feedback welcome.

Comments URL: https://news.ycombinator.com/item?id=46216644

Points: 2

# Comments: 0

The LLM docgen in this case Claude has been over enthusiastic due to my incessant prodding :D.

The interesting bit was that the bindings overheads dominate, and makes this shim not that much of a performance bump.

My assumption was that LiteLLM, being a Python library, would have plenty of low-hanging fruit for optimization. I set out to create a Rust layer using PyO3 to accelerate the performance-critical parts: token counting, routing, rate limiting, and connection pooling.

The Approach

- Built Rust implementations for token counting using tiktoken-rs

- Added lock-free data structures with DashMap for concurrent operations

- Implemented async-friendly rate limiting

- Created monkeypatch shims to replace Python functions transparently

- Added comprehensive feature flags for safe, gradual rollouts

- Developed performance monitoring to track improvements in real-time

After building out all the Rust acceleration, I ran my comprehensive benchmark comparing baseline LiteLLM vs. the shimmed version:

Function Baseline Time Shimmed Time Speedup Improvement Status

token_counter 0.000035s 0.000036s 0.99x -0.6%

count_tokens_batch 0.000001s 0.000001s 1.10x +9.1%

router 0.001309s 0.001299s 1.01x +0.7%

rate_limiter 0.000000s 0.000000s 1.85x +45.9%

connection_pool 0.000000s 0.000000s 1.63x +38.7%

Turns out LiteLLM is already quite well-optimized! The core token counting was essentially unchanged (0.6% slower, likely within measurement noise), and the most significant gains came from the more complex operations like rate limiting and connection pooling where Rust's concurrent primitives made a real difference.

Key Takeaways

1. Don't assume existing libraries are under-optimized - The maintainers likely know their domain well 2. Focus on algorithmic improvements over reimplementation - Sometimes a better approach beats a faster language 3. Micro-benchmarks can be misleading - Real-world performance impact varies significantly 4. The most gains often come from the complex parts, not the simple operations 5. Even "modest" improvements can matter at scale - 45% improvements in rate limiting are meaningful for high-throughput applications

While the core token counting saw minimal improvement, the rate limiting and connection pooling gains still provide value for high-volume use cases. The infrastructure I built (feature flags, performance monitoring, safe fallbacks) creates a solid foundation for future optimizations.

The project continues as Fast LiteLLM on GitHub for anyone interested in the Rust-Python integration patterns, even if the performance gains were humbling.

Edit: To clarify - the negative performance for token_counter is likely in the noise range of measurement, suggesting that LiteLLM's token counting is already well-optimized. The 45%+ gains in rate limiting and connection pooling still provide value for high-throughput applications.

Comments URL: https://news.ycombinator.com/item?id=45968461

Points: 27

# Comments: 9

FastWorker is a brokerless task queue requiring only Python processes. No Redis, no RabbitMQ – just 2-3 Python services instead of 4-6+.

---

Quick example:

# tasks.py

from fastworker import task

@task

def send_email(to: str, subject: str):

    return {"sent": True}

from fastworker import Client

client = Client()

@app.post("/send/") async def send_notification(email: str):

    task_id = await client.delay("send_email", email, "Welcome!")

    return {"task_id": task_id}

fastworker control-plane --task-modules tasks

fastworker subworker --task-modules tasks # optional

---

Architecture: Uses NNG messaging for direct peer-to-peer communication. Control plane coordinates task distribution via priority heap and tracks worker load. Workers auto-discover via discovery socket. Results cached in-memory with LRU/TTL.

Designed for: Moderate-scale Python apps (1K-10K tasks/min) doing background processing – image resizing, report generation, emails, webhooks. Great for FastAPI/Flask/Django.

NOT for: Extreme scale (100K+ tasks/min), multi-language stacks, or systems requiring persistent task storage. For those, use Celery/RabbitMQ/Kafka.

Try it:

pip install fastworker

Repo: https://github.com/neul-labs/fastworker

FastAPI integration docs: https://github.com/neul-labs/fastworker/blob/main/docs/fasta...

Would love feedback on whether this fills a useful niche or if the limitations make it too narrow.

Comments URL: https://news.ycombinator.com/item?id=45952809

Points: 1

# Comments: 0

We are looking to integrate more data connectors.

Do you have any suggestions? We are focused on the google ecosystem to begin with.

So we built Zyler – an AI agent that connects to all your marketing channels (Google Analytics, Ads, SEO, YouTube) and generates unified insights through natural language, without the hallucination problem that plagues most AI analytics tools.

The core problem: Marketing teams are drowning in fragmented data across multiple platforms right as Google deprecates cookies in 2025. You need one dashboard for GA, another for Google Ads, a third for SEO metrics, YouTube analytics somewhere else – and none of them talk to each other. Most AI solutions hallucinate when trying to connect these dots.

What makes this technically interesting:

Zero-hallucination architecture: We've focused heavily on accuracy over creativity – the AI only makes claims it can back up with your actual data across platforms

Natural language to analytics translation: No more switching between 4+ dashboards to understand campaign performance

Mobile-first analytics: First mobile-compatible multi-platform analytics AI agent

Real-time processing: Instant insights from large datasets across GA, Ads, SEO, and YouTube without the usual loading screens

Cross-platform insights: Ask "Why did my YouTube ads perform better than Google Ads last month?" and get unified analysis across all your channels

The drag-and-drop interface makes anyone a "marketing data expert" in about 5 minutes. We're seeing customers reduce reporting time by 80% while getting unified insights across all their marketing channels.

Early traction: Hit Product Hunt top 10, users across 6 continents, and some customers seeing 300% ROI increases. All organic growth so far – no funding raised yet, just Suryansh and me bootstrapping this.

Pricing: $50/month (down from our original $99 pricing) vs the $1000+/month enterprise alternatives, which makes unified marketing analytics accessible for startups and small teams.

It's still rough around the edges, and we're working on expanding to Meta Ads and LinkedIn next.

I'd love feedback, especially on: What other marketing platforms should we integrate next? (Meta, LinkedIn, TikTok?) How do you currently handle cross-platform marketing attribution? Any interest in an API for custom integrations?

Try it out at https://www.zyler.ai – there's a free tier to test with your marketing data.

Happy to answer any technical questions about the AI architecture or product direction – Suryansh can speak to the marketing side and I can dive deep on the engineering!

Comments URL: https://news.ycombinator.com/item?id=44364508

Points: 7

# Comments: 4

Thanks!

It not well designed for those cases for sure.

Would you be up for a chat if i drop you an email? More than happy to learn your general views :)

To that end, I just did a show HN on a couple of my projects

What if google did not focus on hyper commercialisation? 10 Blue links, but sorted by how they answer your query- https://www.unlob.com

Can we answer questions with lesser hallucinations? A snippet cited answer engine which only picks links that focus on answering your query - https://www.unzoi.com

https://www.unzoi.com/query/set-up-model-context-protocol-se...

Current research tools often struggle with accurately representing source material. They frequently introduce factual errors, misattribute quotes, present opinions as facts, and fail to provide sufficient context.

Many AI assistants also add their own interpretations or editorializing that wasn't present in the original sources.

Unzoi takes a different approach by:

- Extracting only information that directly answers the query

- Maintaining critical context to prevent misunderstandings

- Clearly differentiating between facts and opinions from sources

- Avoiding the introduction of unsourced claims or commentary

- Preserving the integrity of quotes without alteration

The tool is particularly useful for researching complex topics where accuracy is essential and misrepresentation could be harmful.

Some example queries:

- Video games with highest learning curves for new players: https://www.unzoi.com/query/video-games-have-highest-learnin...

- Setting up model context protocol servers with PostgreSQL: https://www.unzoi.com/query/set-up-model-context-protocol-se...

- Eligibility criteria for assisted dying in the UK: https://www.unzoi.com/query/eligibility-criteria-assisted-dy...

I'd appreciate feedback from the HN community on both the concept and implementation.

How do you currently handle these challenges in your research workflows?

Comments URL: https://news.ycombinator.com/item?id=43536101

Points: 2

# Comments: 2

Thanks for trying it out!

Instead of: - Wading through pages of SEO-optimized garbage

- Getting vague AI-generated text with no citations

- Opening 10+ tabs to find one piece of information

Unlob gives you precise answers with direct attribution to real sources. No AI hallucinations, no sponsored results pushing products, just factual information extracted from the web.

Try these examples:

- When is John Wick 5 coming out - https://www.unlob.com/lib/john-wick-5

- What's the best telescope under $500? - https://www.unlob.com/lib/whats-best-telescope-under-500

- Best low-cost robotic arms available today - https://www.unlob.com/lib/best-low-cost-robotic-arms-availab...

I'd love your feedback, especially on search accuracy and the types of queries where this approach works best.

What kind of searches would you try with a tool like this?

Comments URL: https://news.ycombinator.com/item?id=43528648

Points: 3

# Comments: 5