Comments URL: https://news.ycombinator.com/item?id=47522367

Points: 3

# Comments: 0

Compare https://docs.firetiger.com with https://docs.firetiger.com/llms.txt and https://docs.firetiger.com/llms-full.txt for a realy example.

I think some of the reason is that there's so much coverage of writing SQL in its training set.

One thing that works very well is just giving it one or two example valid programs/statements in the custom language. It usually picks up what you're getting at very quickly.

When it slips up, you get good signal you can capture for improving the language. If you're doing things in a standard agent-y loop, a good error message also helps it course-correct.

It might better in the system throughput, and those benefits may even outweigh the misery put on that one person. But in the US, we largely sort that out by using cool-down times, hearings, and "community input."

Net result, according to my friend at least, is that bus stops feel _very_ sticky and hard to change.

Comments URL: https://news.ycombinator.com/item?id=47131370

Points: 2

# Comments: 0

I am always reluctant to jump on with these independent ambitious projects. The first version is understandably rough, and the company seems to fold before they get to a second or third version.

But maybe advances in manufacturing in China are making high-quality, small-batch products like this more tractable?

This isn't really true. Your mock inplementation can embed the interface, but only implement the one required method. Calling the unimplemented methods will panic, but that's not unreasonable for mocks.

That is:

    type mockS3 struct {
        S3Client
    }

    func (m mockS3) PutObject(...) {
        ...
    }

Defining a zillion interfaces, all the permutations of methods in use, makes it hard to cone up with good names, and thus hard to read.

I use DuckDB today to query Iceberg tables. In some particularly gnarly queries (huge DISTINCTs, big sorts, even just selects that touch extremely heavy columns) I have sometimes run out of memory in that DuckDB instance.

I run on hosts without much memory because they are cheap, and easy to launch, giving me isolated query parallism, which is hard to achieve on a single giant host.

To the extent that its possible, I dream of being able to spread those gnarly OOMing queries across multiple hosts; perhaps the DISTINCTs can be merged for example. But this seems like a pretty complicated system that needs to be deeply aware of Iceberg partitioning ("hidden" in pg_lake's language), right?

Is there some component in the postgres world that can help here? I am happy to continue over email, if you prefer, by the way.

The models are one part of the story. But the software around it matters at least as much: what tools does the model have access to, like bash or just file reading or (as in your example!) just a cache of files visited by the IDE (!). How does the software decide what extra context to provide to the model, how does it record past learnings from conversations and failed test runs (if at all!) and how are those fed in. And of course, what are the system prompts.

None of this is about the model; its all "plain old" software, and is the stuff around the model. Increasingly, that's where the quality differences lie.

I am sorry to say but Copilot is just sort of shoddy in this regard. I like Claude, some people like Codex, there are a bunch of options.

But my main point is - its probably not about the model, but about the products built on the models, which can vary wildly in quality.