Comments URL: https://news.ycombinator.com/item?id=46424789

Points: 31

# Comments: 8

Edit: Also, the example indicates financial transactions. Can you explain why you need serializability but not durability?

Comments URL: https://news.ycombinator.com/item?id=45749591

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=45572184

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=45273388

Points: 9

# Comments: 0

The actual order in which child workflows finish and their results hit the persistence layer is indeed nondeterministic in real-time execution. Trying to force deterministic completion order would likely be complex and defeat the purpose of parallelism, as you noted.

However, this external nondeterminism is outside the scope of the workflow execution's determinism required for replay.

When the workflow replays, it doesn't re-run the race. It consumes events from the log. The `-await-next` operation during replay simply reads the next recorded result, based on the fixed order. Since the log provides the same sequence of results every time, the workflow's internal logic proceeds identically, making the same decisions based on that recorded history.

Determinism is maintained within the replay context by reading the persisted, ordered outcomes of those nondeterministic races.

All responses and completed delays are stored in a table with an auto-incremented id, so the `-await-next` will always resolve to the same value.

As you mention, putting a persistent sleep and a child execution into the same join set is not yet implemented.

Although I don't expect to be an issue practically speaking, Obelisk checks that the replay is deterministic and fails the workflow when an unexpected event is triggered. It should be also be possible to add an automatic replay of each finished execution to verify the determinism e.g. while testing.

[1] https://docs.rs/wasmtime/latest/wasmtime/struct.Config.html#...

Edit: Enabling the flags here: https://github.com/obeli-sk/obelisk/pull/67

Obelisk has a concept called join sets [3], where child executions are submitted and awaited. In the future I plan on adding cancellation and allow custom cleanup functions.

[1] https://github.com/WebAssembly/component-model/blob/main/des...

[2] https://github.com/WebAssembly/component-model/blob/main/des...

[3] https://obeli.sk/docs/latest/concepts/workflows/join-sets/

I believe the biggest difference is that Obelisk relies on the WASM Component Model:

Obelisk aims to avoid vendor lock-in. It is possible to write activities, workflows and webhooks with no obelisk SDK. Activities and webhooks are WASI 0.2 components that can be run on any compatible runtime like wasmtime e.g. for testing. This should also help with the adoption as any runtime will need a ton of integrations.

Ah understood. I have no plans supporting native workflow executors.

> Out of curiosity, are you working full-time on this?

Yes, currently I work on it full time.

> I'm working part-time on the same problem, looking to go full time soon, and it's interesting to see how the same ideas are popping up somewhat independently across different projects :)

Nice website! I also see the ideas of determinism, replayability etc more and more.

> let me know if you're interested in chatting!

Sure, my email is visible in the Git commit history.

Yes, currently all activities must conform to the WASI 0.2 standard. This is the simplest for deployment, as you only need the obelisk executable, toml config file. The webhooks, workflows and activities pulled from a OCI registry on startup.

To support native code I plan to add external activities as well, with an interface similar to what Netflix Conductor uses for its workers.

> Finally, do you see the component's interface as the right layer for capturing IO?

An activity must encapsulate something much higher level than a single IO operation. So something like "Configure BGP on a router", "Start a VM" etc. It needs to be able to handle retries and thus be idempotent.

Regarding performance, a workflow execution can call 500-700 child executions serially, or around 1400 child executions concurrently per second.

Either the fix does not break the determinism, meaning the the execution did not hit the fix yet. In this case the execution can be replayed and continue on the patched WASM component.

Otherwise, the execution replay causes "Non determinism detected" error. In this case you need to handle the situation manually. Since the execution log is in a sqlite file, you can select all execution affected by the bug and perform a custom cleanup. Also you can create a "forked" execution just by copying the execution log + child responses into a new execution, however there is no API for it yet.

> Can I update the workflow while it has running instances without interfering the running instances?

If you mean keep the in-progress executions on the old version of the code, you can do that by introducing a new version in the WIT file and/or change the new function name.

There is always this chicken and egg problem on a new platform, but I am hoping that LLMs can solve it partially - the activities are just HTTP clients with no complex logic.

Regarding the restrictions required for determinism, they only apply to workflows, not activities. Workflows should be describing just the business logic. All the complexities of retries, failure recovery, replay after server crash etc. are handled by the runtime. The WASM sandbox makes it impossible to introduce non-determinism - it would cause a compile error so no need for runtime checks.

1: https://github.com/obeli-sk/demo-stargazers

Comments URL: https://news.ycombinator.com/item?id=43636568

Points: 103

# Comments: 27