They build good stuff and care.

Other than that, yes, durable execution does all of this for you.

TLDR on Durable Endpoints: you can automatically use steps in API endpoints which checkpoint state in the BG, and then retry on failure. This means you can run jobs in the background _somewhat_ transactionally (somewhat because there's delay between checkpointing) to minimize any tradeoffs here. And, if you want full transactionality, don't buffer checkpoints in the BG and instead do it synchronously.

Also, Redis is good for medium scale load. We're hitting millions of RPS (aggregated) on our services (I work at Inngest) and it doesn't scale so well at this load, at all. We had to invest in other infra.

Wanted to show you something we've been working on: Durable Endpoints. It brings durability and steps directly into your APIs. Each step persists the state in the background. The happy path means no additional latency for your users, and you get tracing & metrics out of the box, and failures resume from the last successful step checkpoint rather than restarting from scratch.

The problem we kept hitting: durable execution systems (Temporal, our own workflows, etc.) are designed for async background work. But with AI workloads, you often need durability and low latency in a synchronous request: a user is waiting on a chat response that involves multiple LLM calls, any of which can fail or timeout.

The typical solution is splitting into two systems: a streaming frontend for responsiveness, plus a background job system for reliability. Durable Endpoints let you keep it as one cohesive implementation (here's a Next example):

    import { Inngest } from "inngest";
    import { endpointAdapter } from "inngest/next";
    
    const inngest = new Inngest({ id: "my-app", endpointAdapter });
    
    export const handler = inngest.endpoint(async (req: Request) => {
      const { input } = parseParams(req);
    
      // Each step.run() checkpoints on completion
      const processed = await step.run("process-input", async () => {
        return await expensiveOperation(input);
      });
    
      // If this fails, we resume here — not from the beginning
      const result = await step.run("generate-output", async () => {
        return await anotherExpensiveOperation(processed);
      });
    
      return Response.json({ result });
    });

- Steps are identified by their string ID, not execution order. This makes the system tolerant to code changes... you can refactor, add/remove steps, and existing in-flight executions continue correctly.

- We use checkpointing (https://www.inngest.com/blog/introducing-checkpointing) to async checkpoint steps. State is persisted to our execution store rather than held in memory.

- If endpoints turn async (step errors or waitForEvent), we redirect the user to a new URL which uses keepalives and waits for the API result. Inngest re-enters the endpoint and picks up where it left off, then sends the result to the redirected user.

- Similarly, on failure, our executor re-invokes the endpoint. Completed steps are memoized from the checkpoint... their functions don't re-run, we just return the stored result.

- Works with streaming responses via ReadableStream.

The same step primitives we use in durable workflows (`step.run`, `step.sleep`, `step.waitForEvent`) work here, so you get retries, timeouts, and human-in-the-loop patterns in synchronous handlers.

Beyond AI, this is useful anywhere failure mid-request is expensive: payment flows, webhook processing, third-party API orchestration.

It's in public beta. Docs: https://www.inngest.com/docs/learn/durable-endpoints. Code is in our TypeScript SDK.

Happy to dig into the execution model, how we handle determinism, or the checkpointing implementation.

Comments URL: https://news.ycombinator.com/item?id=47064354

Points: 8

# Comments: 0

IOW, doesnt look as bad as the title suggests?

Yes... but people can only focus on one thing at a time. We don't have 360 vision. We have blind spots! We don't even know the exact speed of our car without looking away from the road momentarily! Vision based cars obviously don't have these issues. Just because some cars are 100% vision doesn't mean that it has to share all of the faults we have when driving.

That's not me in favour of one vs the other. I'm ambivalent and don't actually care. They can clearly both work.

If you are changing the interface, though, that would mean a contract change. And if you're changing the contract, surely you wouldn't be able to even use the old tests?

This isn't really a go problem at all. Any contract change means changing tests.

StepKit is an open source SDK and framework for building and iterating on durable workflows that run on any platform (self-hosted, Inngest, Cloudflare, Netlify, etc.) without requiring any provider or bundler-specific code.

Here’s the repo: https://github.com/inngest/stepkit.

StepKit extracts the core execution loop that we built in Inngest and makes it fully open, Apache 2, and hackable/pluggable to different backends. We don’t want people to have to go through multiple major versions in SDKs to learn the lessons we’ve already learned in production:

It needs to work anywhere: This SDK is push-based and receives tasks via API endpoints, so it works anywhere (servers, serverless, k8s, etc). That's also customizable. It doesn’t need any specific runtime or bundler support.

It needs to be complete: StepKit includes the entire execution engine: step discovery, memoization, a core event loop to turn async steps into generators without relying on `try/catch` based control, and middleware for extensibility (eg, end-to-end encryption, Sentry integrations, and so on).

It needs a simple API: StepKit APIs are explicit `step.*()` functions, which we designed in our original Inngest SDK back in 2022. They’re easy to read, understand, implement, and use. They contain all of the primitives for durable execution: steps, suspend/resume, human-in-the-loop, and observability. These same APIs have also been adopted by Cloudflare, Netlify, Convex, and others, and support billions of runs every month.

It needs to be resilient: Steps tolerate changes when refactoring, and provide the building blocks of durable execution without complex abstractions.

We’re starting with in-memory, filesystem, Inngest, and Cloudflare (WIP) drivers, with more coming soon.

The roadmap includes middleware, compatible SDKs in different languages, as well as extensions like concurrency controls and idempotency: https://github.com/inngest/stepkit/discussions/52.

Excited to see what you think: https://github.com/inngest/stepkit.

Comments URL: https://news.ycombinator.com/item?id=46048322

Points: 36

# Comments: 16

Comments URL: https://news.ycombinator.com/item?id=45998047

Points: 88

# Comments: 33

Comments URL: https://news.ycombinator.com/item?id=45699554

Points: 3

# Comments: 0

I strongly believe that being obvious about steps with `step.run` is important: it improves o11y, makes things explicit, and you can see transactional boundaries.