Its dbt inspired stream ETL tool (or maybe just the TL?), it currently just has a dev mode that does RabbitMQ to local Parque files while I'm getting the core of it to a place I'm happy with.

It runs SQL models against the incoming messages and outputs the results to one or more output tables. Has a local WAL so you can tune it to have sensible sized output files (or not, if you need regular updates but at the expense of query perf.)

Planning on adding Protobuf messages, Kafka as a source and S3 and Iceberg tables as sinks this week.

Lightly inspired by a some projects at work where a lot of time and effort was spent doing this and resulted in something not very reusable without a lot of refactor work. Feel like the stream -> data lake pattern should be something that is just SQL + Config, same way dbt is for transformations within a data warehouse.

No plans on adding any cross message joins or aggregations as that would require cross worker communications and I explicitly want to keep the workers stateless (minus the WAL of course)

Would really appreciate any feedback on the core concept, esp. if this is something you'd actually use in prod (if it were finished!) Not sure if there is something that does this already that I don't know about, or if this genuinely fills some sort of hole in the exisitng tooling

TPUv5e with 16 tensor cores for 2 days for the 200M param model.

Claude reckons this is 60 hours on a 8xA100 rig, so very accessibile compared to LLMs for smaller labs

An LLM observability SDK that let's you store pre and post request metadata with every call in as lightweight an SDK as possible.

Stores to S3 in batched JSON files, so can easily plug into existing tooling like DuckDB for analysis.

It's designed to answer questions like; "how do different user tiers of my services rate this two different models and three different systems prompts?". You can capture all the information required to answer this in the SDK and do some queries over the data to get the answers.

It can also gather custom metadata about a call, this can be any KV pairs you want, both pre and post request.

```python

import caliper

import anthropic

caliper.init(target="s3") # This is all that's required for basic observability, no changes needed to LLM calls for basic metrics

client = anthropic.Anthropic()

response = client.messages.create(

  model="claude-sonnet-4-20250514",

  messages=[{"role": "user", "content": "What is 2 + 2?"}],

  caliper_metadata={"campaign": "q4"}, # Pre request metadata

print(response.content[0].text)

caliper.annotate(sentiment="positive") # Post request metadata

```

You can use this to track effectiveness of model changes, tracking them against difference user tiers. Maybe your free tier users don't notice if you use a cheaper model but you paying users do? How do you know if a recent system prompt change was effective? You can track the version of the prompt in metadata and compare post request rating annotations between prompt versions.

It has a dev mode which logs locally, it can also send files to S3. The SDK has a background queue and worker which flushes in batches that are configurable in size and time between flushes. It exports to S3 as batched JSON files to readily to integrate into most data engineering pipelines or you can just query directly with a tool like DuckDB.

PyPi: https://pypi.org/project/caliper-sdk/

Edits: formatting and PyPi Link

Comments URL: https://news.ycombinator.com/item?id=47296746

Points: 2

# Comments: 0

And it's quite a trivial feature at that.

The cost of a work days worth of downtime is rarely enough to justify the expense of trying to deploy across multiple regions or clouds.

Esp if you are public facing and not internal. You just go 'well everyone else was down to because of aws' and your customers just go 'ah okay fair enough'

Often Cisco/etc will refuse support cases if you aren't using their optics, if the switches/routers even work with them in the first case, which isn't a given as often they'll refuse to work with non branded optics.

Really just a money grab by the big network vendors.

This box allows you to flash the firmware on the optic to say its from whatever brand you want (Cisco, Dell, Aruba, Juniper etc) so that you can get it to work in that companies switch/router.

For most SMEs, the brand of optics makes no difference. Maybe keep a few legit branded ones around for debugging and when you need to raise a support case. But otherwise, the generic ones flashed to look like branded ones work just fine.

Could I write code that runs on Spark and effortlessly run it on a big GB300 system with no code changes?

Plus, the TCO is already way under the cloud equiv. so might as well spend a little more to get something much newer and more reliable