When prod breaks, it lets Cursor, Claude, and others drop virtual breakpoints or probes safely in your running code, and extract the exact variable values that logs don’t have.

All this saves time and effort for engineers who’d otherwise dig through logs and traces or redeploy with console.logs or print statements until they find the root cause.

Here is the link to the video that explains this: https://www.youtube.com/watch?v=ivV7I--ta5c

Agents write most of our code now. This shrinks the useful context engineers need to debug AI written code, a problem not helped by the limited telemetry added in the same code by AI.

So when something breaks in prod, the first instinct for an engineer is to open logs or throw them to your agents. But if the line you are looking for is not there, agents will start guessing the root cause on non-existent data, forcing you to add a log, and redeploy.

This analysis-inference loop of agents with existing data does not come cheap, burning a lot of tokens. And the add log, redeploy cycle is so slow and painful that it makes engineers hate on-call.

Our approach lets agents capture telemetry on-demand at the exact moment and point of failure, killing the log-redeploy cycle and getting the most accurate RCA while burning fewer tokens.

The obvious problem is making it work on a running service. You can't pause a live service the way you'd pause a debugger on your laptop. Getting the value out of a running process safely, without pausing a thread or slowing the host is the challenge.We are making this happen.

Before this I ran engineering at a 100 member team. Then Karan and I spent three years on HyperTest which was a testing tool.

At HyperTest, we turned production traffic into integration tests using OpenTelemetry. That was production instrumentation too. The hard parts of pulling real runtime state out of a running service without breaking it, were the hard parts we learnt to put together.

We learnt some other lessons the hard way too. HyperTest tried to prevent bugs with better tests, and adoption was a fight every time. Calls kept getting cancelled because teams were firefighting production. Testing was hygiene. Broken prod was hair on fire. This made us see where priorities lie.

This seeded the idea of building a truly autonomous on-call agent i.e. one that takes an alert, probes, diagnoses and fixes it in a few minutes. But this is how it works as of now:

You talk to your coding agent the way you already do. Tell it what's wrong: "checkout returns 200 but some users are seeing their order fail, find out why." It locates the line in your local code, connects to us over MCP, and drops a probe on that line in the running service. The probe is read-only and sits dormant until real traffic hits. When hit, it captures the local variables at every frame of the call stack at that exact moment. It hands them to the agent, which diagnoses with real data.

There are two pieces. An SDK that runs inside your service, and an MCP server your coding agent talks to. The SDK is what makes setting probes (virtual breakpoints, log or metric) possible without a redeploy. In Node and Python it hooks in-process. In Java it attaches as a JVM agent, instrumenting at the bytecode level. Either way the service keeps running and serving traffic. Nothing pauses.

When your agent wants to look at a line, it calls the MCP server, which tells the SDK to place a probe there. When a request hits the line, the SDK captures what the probe asked for, sanitizes it in-process, and streams it back to the agent via the MCP.

This can run inside prod, so a probe can read any value sitting in that variable. We ensure redaction happens in-process, inside your own container's memory. This is before anything goes on the wire. Keys like password, token, authorization, ssn and credit card are redacted by default and you add your own.

Also the probes read but never write, and if you want no captured state to ever leave your network you can self host the server, broker, and even the database in your infra.

On overhead: when idle, the SDK adds negligible memory and effectively nothing to throughput and response time. Probes only cost anything while actively capturing. Also captures are bounded. A separate monitor watches in real time and pulls every active probe if overhead ever spikes.

Every log-and-trace tool hands the agent data that already exists and asks it to reason backward to what probably happened. We think it is more useful to give agents eyes and ears into the running code, so they capture what they need when they need it, right at the point of failure.

This seems like the simplest and fastest way to debug prod incidents.

We’d love the community to try this in any environment to debug any known or unknown issue by just chatting with your coding agent. And let us know what more features you need to make this a truly autonomous on-call agent

Supported platforms: NodeJs, Java, Python.

Comments URL: https://news.ycombinator.com/item?id=49185389

Points: 45

# Comments: 31

Comments URL: https://news.ycombinator.com/item?id=43609935

Points: 2

# Comments: 1

We have plans to pick up Python and Go next. Will keep everyone posted here

The project is available to set-up for node services and if you are interested you can go ahead and read our detailed set-up docs - https://docs-v2.hypertest.co/setup-guide/readme/integrate-sd...

We are trying to make integration testing easy for developers. A lot of other teams and tools have taken a stab at this problem and having seen them we believe we have improvised the approach to help developers achieve this with minimum effort and pain.

How it works: Developers set-up our SDK (2-lines) in the source code their (backend) services and configure it to record traffic from any environment. When HyperTest works in RECORD mode it collects end to end trace of every incoming request i.e. the request, response and outbound calls.

These requests (tests) can be replayed on a new build later (pre-push or at CI) to check for regressions in API responses and outbound calls. In the REPLAY mode HyperTest uses mocked responses of all dependent systems to keep tests non-flakey and results deterministic and consistent. 3-min demo - https://www.youtube.com/watch?v=x6hmDUNFGW4

What does it do: HyperTest SDK auto-instruments all key functions and methods across all libraries you use to make outbound calls. This helps HyperTest mock these calls in REPLAY without asking developers to make any change in their source code.

How is this better: 1. Set up is just like how you will set up an APM, i.e., 5 mins adding 2-lines of the SDK.

2. Support all protocols like http, graphQL, gRPC, Kafka and AMQP to cater to more use cases. Adding more as we speak

3. Test can be generated from any environment can be run anywhere even locally.

4. Active de-duplication to reduce the number of requests run on REPLAY. Optimise for code coverage & filter requests that don't cover additional lines of code

5. Distributed tracing to help developers debug root cause faster

6. Auto-updates mocks as dependencies change to keep test results trustworthy.

HyperTest is currently available only for node projects. We work the teams with 5 or more services at the moment and have 50+ teams using it actively.

If this seems valuable can set-up a quick intro and explain how to get started here -https://calendly.com/shailendra-hypertest/30min

Would love feedback!

Comments URL: https://news.ycombinator.com/item?id=41479555

Points: 5

# Comments: 4