Comments URL: https://news.ycombinator.com/item?id=48851678

Points: 2

# Comments: 0

I have experienced similar behavior between opus and haiku when benchmarking Dara engineering tasks. The “cheaper” model takes many more turns to figure out the task and this is without taking into account other important factors.

Another interesting behavior that I observed is that Haiku tended to cheat more maybe because it was having a harder time to find the root cause of the problem.

Benchmarking and evaluation of agentic systems is very interesting and if there’s one thing that someone should keep from the Databricks post is how important is for everyone to build and run their own.

A deterministic layer like a compiler or generator of code with some kind of IR that the LLM generates and feeds it with.

I feel we will be seeing this more and more in the near future.

What this gets you is the ability to work with structured and unstructured data in the same context. Most importantly, the LLMs aren't integrates as opaque UDF black boxes. They're exposed as "semantic" operators that the planner can reason about alongside the classic ones.

(There are examples and code snippets on the repo to see how everything works together)

Why build this? I'm a data infra / systems person. When LLMs showed up, what I saw was a new type of compute that changes the characteristics of the workloads we deal with. I wanted to experiment with how our current systems can absorb these new workloads and compute types, and what it would take to make the DX as seamless as possible, that's where the UDF + arbitrary prompt was feeling too problematic.

To support this properly, we had to introduce a few really cool things:

New plan operators. You don't just send prompts at an LLM. You use operators like semantic join, semantic map and reduce, and semantic filter, among others. They mix with the classic operators, and because the planner sees them as real operators rather than black boxes, it can reorder work around them.

Typed outputs. There's ergonomics to turn the output of a semantic operator straight into a typed dataframe column. A Pydantic schema for the LLM output becomes a typed struct column you can unnest, explode, and so on.

New data types like a markdown data type. Markdown became an important way to share information with LLMs, even though it started life as a way to format text for presentation. It carries structure, and being able to access that structure the way you would a struct or JSON type adds to the developer experience I mentioned.

Async UDFs. One of the more interesting shifts in workloads from the LLM explosion is the need to put heavily I/O-bound steps in your pipeline: fetching a response from an API, crawling a website, and so on. Async UDFs fill that gap, and the implementation handles the nuances for you: concurrency, retries, and the rest.

An LLM-inference-aware planner and runtime. This is one of the parts I'm most excited about, and there's a lot still to do. Today: identical prompts within a batch collapse to a single model call, so duplicates cost zero tokens; requests are dispatched concurrently under per-provider rpm/tpm limits with retries and backoff; null and empty cells skip the model entirely; and you get token and cost metrics per operator. There's also an optional persistent response cache so re-runs skip the model.

MCP as a new catalog primitive. Much like a registered view, you can register a dataframe pipeline as an MCP tool in the catalog. fenic then serves an MCP server with that pipeline as the tool's logic, executed over your data.

These are just some of what's gone into fenic while experimenting with how LLMs can become part of our compute infrastructure. There's more, and plenty more to polish on what's already there.

I've been using fenic for all sorts of things. On the small/personal end, I use it to take my podcast audio recordings and turn them into nicely structured tables of metadata I can research. On the heavier end, I use it as tooling for agents to analyze agent traces exported from Pydantic Logfire, to discover evals and turn them into reproducible artifacts in the form of dataframe pipelines.

  pip install fenic
  Repo: https://github.com/typedef-ai/fenic
  Docs: https://docs.fenic.ai

I'd love to hear your thoughts, criticism, and anything else that comes to mind.

I'm here to answer questions.

Comments URL: https://news.ycombinator.com/item?id=48735301

Points: 3

# Comments: 0

Priests were interpreting the oracles (at least at a place like Delphi) according to the context of the people asking the questions aka participating in politics of that ancient times.

Subjectivity was a feature and I’m not sure that fits to mathematics though.

I wonder if mathematics as a science field moves more into engineering or if a different branch will emerge that is closer to that because to the point of the article, science is about understanding not just results.

Comments URL: https://news.ycombinator.com/item?id=48605517

Points: 1

# Comments: 0

Instead they were focusing more on optimizations of the existing algorithm that has been implemented. Maybe it's an artifact of the problem I was throwing to them (I was asking to optimize the implementation of select_k in Arrow, which is currently using a max-heap streaming algorithm).

I've started documenting my journey with this here: https://www.kostasp.net/posts/16-ai-experiments-apache-arrow in case you want to take a look. Any advice would be highly appreciated, I'm looking for more inspiration on how to torture myself with that stuff.

My feeling is that the research is converging to what the paper claims, that the combination of two is the right way to do it and it's a matter of how you combine the two as part of the harness you built that makes the difference.

At the AID-Wild / ACM CAIS 2026 workshop that happened recently, there are plenty of examples in the accepted papers on that.

A great example is AI-PROPELLER: Warehouse-Scale Interprocedural Code Layout Optimization with AlphaEvolve. It uses AlphaEvolve and Vizier to evolve compiler code-layout heuristics. (https://arxiv.org/abs/2606.00131)

What questions I'd like to answer with the diffing is more like: will the grain go from one-row-per-user to one-row-per-user-per-day, will a key stop being unique, will a join start fanning out and quietly double a measure, will something additive become non-additive.

This diff is over structure but this structure is latent in the transformation that produces it and to make things harder, if we are talking about some declarative language being used (e.g. SQL) the code doesn't even describe how things are getting done, but what the output would be.

What I've ended up doing is recovering the structure from the code by analyzing it and then using * cheap * profiling than a full row compare.

As an example, my equivalent impact sub-command output would be something like this: "this change makes account_id non-unique three models downstream"

Or maybe what works in math doesn’t work with chemistry?

Now someone can argue that lines of code are not a good proxy of engineering productivity, but I wouldn’t be surprised if the audience they target with this content is not the HN commenters of this thread.

What is a bi-temporal record? I don’t think I’ve heard the term before and I’d love to learn more.

This is the equivalent of Claude Design but for security.

Different harness, different packaging and obviously different distribution because the persona is different.

It’s funny because from all the posts I’ve read from companies reporting on Mythos, everyone is building their own harness for it.

Cisco even published a specification for one.

But Anthropic is the one who has figured out how to package and distribute this. Great GTM!

I wonder though, is such a dialect better for agents? Have you tried to measure if an agent performs better expressing queries in such a language instead of SQL?

What you describe has been pitched by many different products for different parts of the data platform. Fivetran for example claims to do that for the extraction and loading part, good old Informatica was offering the ETL in a graphical interface etc.

The problem that many teams ended up having is the explosion of the tooling needed by data teams.