Liftbridge adds Kafka-style durability to NATS: commit log, partitions, ISR replication. Single Go binary, no JVM or ZooKeeper.

Architecture: Raft for metadata consensus, ISR (like Kafka) for data replication. Memory-mapped segment files with offset/timestamp indexes.

We're using it for Industrial IoT (streaming layer for Arc, our time-series database), but it works standalone.

Roadmap: Go 1.25+, dependency updates, security audit, CI/CD fixes (Done) and we fixed a few panic errors / race conditions bugs. First release mid January 2026.

Happy to answer questions.

Comments URL: https://news.ycombinator.com/item?id=46329715

Points: 1

# Comments: 1

I’ve posted here before about Arc, and wanted to share a quick update. We’ve been heads down the last few weeks, and things are moving fast, performance is up, features are landing, and the community response has been awesome.

Arc uses DuckDB as the SQL engine, with MessagePack, Arrow, and Parquet under the hood. Right now we’re seeing around 2.45M RPS for metrics, and about 1M RPS for logs, events, and traces. It now supports retention policies, deletes, and continuous queries that aggregate data as it comes in.

We also built a VS Code extension to explore and query Arc, added Apache Superset integration, and are working on Telegraf and a Grafana datasource next.

Feedback is always welcome. good, bad, or brutal. It all helps.

Happy Thursday!

Comments URL: https://news.ycombinator.com/item?id=45758142

Points: 3

# Comments: 1

Today, the results are in, Arc is now the fastest time-series database on ClickBench, outperforming QuestDB and TimescaleDB across nearly every hardware configuration. It also ranks among the top 5 fastest systems overall, alongside DuckDB, ClickHouse, and Salesforce Hyper.

Arc ingests 2.42 million records per second, tops the Cold Run charts for all system sizes, and does it all in roughly 5,500 lines of code.

It’s built on DuckDB for SQL, MessagePack for ingestion, Parquet + Arrow for columnar storage, and a lock-free multi-threaded core for compaction and WAL.

Benchmarks aren’t everything—I agree—but performance matters. It’s what lets smaller infrastructure handle serious workloads without ballooning costs.

This enables massive IoT ingestion, real-time logistics tracking, aerospace telemetry, observability at scale, and market data analytics—all without a huge footprint.

13 days since launch: 252 GitHub stars, 100+ community conversations shaping the roadmap.

Full technical breakdown and benchmarks: https://basekick.net/blog/arc-fastest-timeseries-database-cl...

Happy to answer questions about the architecture, design decisions, or benchmarking methodology.

https://news.ycombinator.com/item?id=45505407

This community pushed Arc forward faster than I imagined, from architecture ideas to performance discussions, it’s been awesome to geek out with so many of you.

We’ve now crossed 210 stars on GitHub and added fresh ClickBench results (36.43 s cold / 35.12 s hot) on AWS c6a.4xlarge, plus 2.01 M records/sec write throughput using MessagePack.

Landing > basekick.net Blog > basekick.net/blog Repo > github.com/Basekick-Labs/arc (Star it if you feel it)

Docs are coming later today, and more benchmarks soon.

Appreciate everyone who tested, challenged, or just said “this looks cool.”

Comments URL: https://news.ycombinator.com/item?id=45579188

Points: 1

# Comments: 0

The benchmark measures fully written time-series records, not bytes. Each record typically includes 1–4 fields, tags, and timestamps, similar to InfluxDB’s Line Protocol structure.

For comparison, the same hardware (AWS c6a.4xlarge) handles around 240K RPS using Line Protocol, while Arc reaches 1.88M RPS with MessagePack, about 7.8× faster on ingestion throughput.

You can see the full ClickBench and ingestion benchmarks are in the repo.

TL;DR: Arc’s strength isn’t massive single records, it’s sustained high-throughput ingestion of structured time-series data while still staying analytical-query friendly.

Schema inference: yes, Arc infers the schema automatically from incoming data (both for MessagePack and Line Protocol). Each measurement becomes a table, and fields/tags map to columns.

Schema evolution: supported. New fields can appear at any time, they’re added to the Parquet schema automatically without migration or downtime.

Custom partitions: currently partitioning is time-based (hour-level by default), but custom partitioning by tag or host or whatever is planned. The idea is to allow you to group by any tag (e.g. device, region) in the storage path for large-scale IoT data.

Roadmap: absolutely. Grafana data source, Prometheus remote write, retention policies, gRPC streaming, and distributed query execution are all in the works.

We are going to start to blogging about it, so, stay tune.

Would love any feedback on what you’d prioritize or what would make adoption easier for your use case.

I’d say both roles are possible, though the original intent of Arc was indeed to act as an offload / long-term store for systems like TimescaleDB, InfluxDB, Kafka, etc. The idea: you send data into Arc to reduce storage and query load on your primary database for ML, deep analysis, etc.

But as we built it, we discovered that Arc is really good not just at storage but at actively answering queries, so it’s kind of hybrid: somewhat “warehouse-like,” but still retaining database qualities in performance. I feel that saying a database its too much, but we are going on that direction.

IoT is absolutely one of the core use cases. You’re often ingesting tens or hundreds of thousands of events per second from edge devices, and you need a system that doesn’t choke. Our binary MessagePack ingestion helps shrink the payload size and reduce parsing overhead, that allows higher throughput for writes, which is crucial in IoT scenarios.

Let me know if you want to explore this a little more, not for selling you anything, at least not yet, I would love to understand your use case. Let me know if you are open: ignacio[at]basekick[dot]net

Small Parquet files: we batch writes by measurement before flushing, typically every 10 K records or 60 seconds. That keeps file counts manageable while maintaining near-real-time visibility. Compaction jobs (optional) can later merge smaller Parquet files for long-term optimization.

Updates/deletes: today Arc is append-only (like most time-series systems). Updates/deletes are planned via “rewrite on retention”, meaning you’ll be able to apply corrections or retention windows by rewriting affected partitions.

The current focus is on predictable write throughput and analytical query performance, but schema evolution and partial rewrites are definitely on the roadmap.

The benchmarks weren’t run on the same machine as MinIO, but on the same network, connected over a 1 Gbps switch, so there’s a bit of real network latency, though still close to local-disk performance.

We’ve also tried a true remote setup before (compute around ~160 ms away from AWS S3). I plan to rerun that scenario soon and publish the updated results for transparency.

Regarding “hot vs. cold” data, Arc doesn’t maintain separate tiers in the traditional sense. All data lives in the S3-compatible storage (MinIO or AWS S3), and we rely on caching for repeated query patterns instead of a separate local tier.

In practice, Arc performs better than ClickHouse when using S3 as the primary storage layer. ClickHouse can scan faster in pure analytical workloads, but Arc tends to outperform it on time-range–based queries (typical in observability and IoT).

I’ll post the new benchmark numbers in the next few days, they should give a clearer picture of the trade-offs.

If it’s not too much trouble, feel free to share feedback at ignacio [at] basekick [dot] net.

I didn’t really worry about confusion since this isn’t a browser, it’s a completely different animal.

The name actually came from “Ark”, as in something that stores and carries, but I decided to go with Arc to avoid sounding too biblical.

The deeper reason is that Arc isn’t just about ingestion; it’s designed to store data long-term for other databases like InfluxDB, Timescale, or Kafka using Parquet and S3-style backends that scale economically while still letting you query everything with SQL.

Over the past months I’ve been building Arc, a time-series data platform designed to combine very fast ingestion with strong analytical queries.

What Arc does? Ingest via a binary MessagePack API (fast path), Compatible with Line Protocol for existing tools (Like InfluxDB, I'm ex Influxer), Store data as Parquet with hourly partitions, Query via DuckDB engine using SQL

Why I built it:

Many systems force you to trade retention, throughput, or complexity. I wanted something where ingestion performance doesn’t kill your analytics.

Performance & benchmarks that I have so far.

Write throughput: ~1.88M records/sec (MessagePack, untuned) in my M3 Pro Max (14 cores, 36gb RAM) ClickBench on AWS c6a.4xlarge: 35.18 s cold, ~0.81 s hot (43/43 queries succeeded) In those runs, caching was disabled to match benchmark rules; enabling cache in production gives ~20% faster repeated queries

I’ve open-sourced the Arc repo so you can dive into implementation, benchmarks, and code. Would love your thoughts, critiques, and use-case ideas.

Thanks!

Comments URL: https://news.ycombinator.com/item?id=45505407

Points: 30

# Comments: 19

Comments URL: https://news.ycombinator.com/item?id=45213546

Points: 1

# Comments: 0

I’m Ignacio (most people call me Nacho). After years in DevOps and SRE, I kept running into the same story: founders building great products, only to hit a wall when it came to infrastructure.

Sometimes it was CI/CD pipelines that hadn’t worked reliably in months. Other times, a GPT-generated Dockerfile or Terraform config had quietly pushed their AWS bill through the roof. A few had no monitoring, no alerts, just vibes and PagerDuty at 3AM.

So I started helping. At first freelance. Then more structured. That work became Basekick, a kind of “infra rescue” service for startups who’ve gone too far without an infra person, or trusted AI a bit too much.

It’s not a tool or platform, it’s a flat-fee service where I jump into your stack, fix what’s fragile, stabilize your deployments, kill the waste, and help you ship without fear. No vendor lock-in, no bullshit phases, just experience applied fast.

Pricing is flat and public: $1.5k for rescue, $3.5k for foundational rebuilds, and $2k/mo if you want me around long term.

If your infra is a mess, your cloud bill gives you anxiety, or you just want to ship safely again, check it out. Would love your feedback. AMA.

Nacho

Comments URL: https://news.ycombinator.com/item?id=45103816

Points: 1

# Comments: 0

We built ExyData Historian to fix that.

What it does?

- Automatically exports old InfluxDB 1.x/2.x data to compressed Parquet in S3 or MinIO - Keep recent data hot in InfluxDB, move the rest to cheap storage - Run fast SQL on archived data via Apache Arrow + DuckDB - Query it all through one interface and / API. No hot/cold boundary for the user

Why it matters - 70–80% lower storage costs - Historical queries that are as fast (or faster) than InfluxDB itself - No manual exports, no query rewrites, no downtime

Who’s using it right now? InfluxDB Enterprise Customers and Huge instances of OSS, telcos and logistics companies are trying this right now.

We help you to reduce your Enterprise licensing cost, cause you are going to shrink your InfluxDB cluster.

You keep your existing InfluxDB running, Historian works alongside it, moving history to cheap storage while giving you more analytics power.

We’d love feedback from anyone managing large InfluxDB deployments.

Comments URL: https://news.ycombinator.com/item?id=44842265

Points: 4

# Comments: 0

But in practice, large or long-running environments rarely have clean cert inventories. You get:

- Internal CA sprawl (and no single source of truth)

Certs embedded in keystores, containers, or staging systems that nobody owns anymore

- “Temporary” certs that live on for years - People leaving without handing off cert responsibilities

We’re not automating monitoring because it’s hard, we’re doing it because teams forget.

And forgetting is what causes outages, broken mTLS, and failed compliance audits, even in air-gapped setups. I have a few horror story on PCI environments.

Automation helps catch the edge cases before they become fire drills.

But in practice, it tends to break down when:

- Certificates multiply across internal services, vendors, CDNs, load balancers, staging/prod, and even IoT or embedded devices.

Ownership is unclear, people leave, and calendar invites don’t update themselves.

No visibility, unless someone’s checking regularly, you won’t know if a cert was rotated early, replaced with a shorter one, or removed entirely.

No alerting or state tracking, a calendar doesn’t notify you if someone messed up the renewal or if the cert is already expired.

No integration with the rest of your monitoring/incident tooling.

SSL Guardian gives you a real-time dashboard of every cert, expiration tracking, and actual notifications, so you’re not relying on tribal knowledge, calendar hygiene, or someone noticing too late.

We’re not trying to over-engineer a reminder, we’re replacing a fragile human process with something that scales across orgs and environments.

Dont ask me more how I know that a calendar doesn't work lol!