> If you do need a short-lived, signed token for something, there is a better spec called PASETO which is designed to be secure

Suggesting to non security people (like myself) something for auth that isn't a mainstream idea seems like a bad idea? Not to mention that it doesn't refer any reasons why it's better

I wish they'd just make ~ pixelbook with ubuntu... it'd be such a powermove, and they if anyone could pull it off

I was excited at first by this, but the "designed for gemini intelligence"... like what does that even mean

As agents become more widely adopted, more software engineering and product people have start building them. But I’ve noticed that many teams are not yet fluent in systematic evaluation, or in the processes needed to keep agent quality high over time.

For large organizations, that gap is rarely the bottleneck due to dedicated teams. But after speaking with a number of startups, it became clear that building strong, up-to-date evals is much harder in a fast startup, especially when the team does not have a data science background.

So I tried to condense as much of my experience as possible into a Claude Skill: a practical starting point for evaluating your agent.

The idea is simple: tell Claude you need evals, and it will set up a solid baseline directly in your codebase - that's it! The evals will follow patterns I've seen many times before, and will get you a summary of what your agent does well and what it doesnt.

Looking forward to your feedback!

Comments URL: https://news.ycombinator.com/item?id=48013746

Points: 9

# Comments: 1

And in my experience it's quite hard to figure that out by quickly looking at it.

Not to mention that contributions on github (almost?) never include the prompt chain anyway, so the status quo is even worse

So what are you arguing for? I genuinely cannot tell.

It's just bad/wrong/context lacking decisions and mental models it introduces, that if not carefull will just create a massive mess of a codebase. (I know, because I've tried, and had to deal with it)

And if someone vibecodes a PR and it works, why dont they just share the prompt so a repo owner could vibecode it themselves?

To your later comment, the devices are provided. You dont need to buy them.

Also that's not actual price. the tuition fees are that, doesn't mean that's the price. It's just heavily subsizied by the government. Hard to find sources, but the actual price/student in Germany seems to be ~10k Euro/student/year.

Hard to find

I understand how appealing it is to build an AI coding agent and all that, but shouldn't they - above everything else - make sure they remain THE platform for code distribution, collaboration and alike? And it doesnt need to be humans, that can be agents as well.

They should serve the AI agent world first and foremost. Cause if they dont pull that off, and dont pull off building one of the best coding agents - whcih so far they didnt - there isn't much left.

There's so many new features needed in this new world. Really unclear why we hear so little about it, while maintainers smack the alarm bell that they're drowning in slop.

1. There needs to be reframing of the engineer role in the team, by the leadership. It's not just to build things, but to help the customer. The job of the engineer is to build a good product for the customer, not just to "build", fix Jiras, ...

2. At the same time, get some engineers on customer calls. It doesnt need to be everyone, but you need to shorten the feedback cycle that engineers directly get signal from their users. A lot of information gets lost through Jira's, intermediates, ...

They dont have need talk, just listen. Will they pay attention? That comes back to the reframing of their job being to create a good product for users. If everyone understands that's the goal, they will.

If you have engineers building custom extensions for your database, because your CRM product needs some low level performance optimizations - it'll still be good but less interesting.

The goal is to get some engineers talking to customers, cause they'll talk to other engineers. And engineers know how to talk to engineers. It'll shorten the feedback cycle, and that's good for both speed + signal.

Now I don't know how much of that you already do, and they might even do all these things already, and what you see is the output of the worst issues already taken care of. If the latter is the case, there's some more cross team comms needed

(Motivated) people at small companies "care", and what I mean with that is they are responsible and can see a large enough portion of the customer experience that - if something is broken - they'll see the pain and try to address it.

At a big company no one cares. They of course care about their job, but their job is such a small fraction of the overall customer experience, that seeing their work having an impact on their customer is exceptionally difficult.

That's why large companies need to encode customer feedback into a system to imitate feedback cycles. Mostly in metrics. That's a very lossy way to capture signal, and leaves a lot to be desired, but so far it doesnt seem like anyone has come up with a better system.

It's fully open, fairly minimal, very extensible and (while getting very frequent updates) never has broken on me so far.

Been using it more and more in the last two months, switching more and more from codex to it now.

We'll need a bunch of steps like that, to get closer to the efficiencies we're hoping for.

I do think it's more painful to distribute files when you're a distributed as a single binary vs scripts, since the latter has to figure out bundling of files anyway.

But still - it does exist

Can't tell if you're ragebaiting here, but I'm very confused by this question because they support an entirely different set of features, and if you use both it's painfully obvious how they're different.

Docker is built for running services, distribution is part of that, but it's core is that you can pull an image and run random service on your machine packaged with all the right libraries, network them into your machine in the way you like so it can access the right things, constrain it's resources, and create our own image based on it.

Snap/Flatpak is built to distribute applications, sandboxing being a core part of it, with applications wanting to integrate into distribution mechanics such as audio, URL handling, taking screenshots, ...

GCI itself is just a command line tool, just layering opinions and convenience on top to make the experience more smooth