We built a GraphQL / SQL hybrid expression language that does just this here: https://docs.gadget.dev/guides/data-access/gelly#gelly , curious what yall think.

Comments URL: https://news.ycombinator.com/item?id=44904456

Points: 2

# Comments: 0

Boo!

I was just talking with a Temporal solutions engineer this week and this metric is their recommended one for autoscaling on. Instead of autoscaling on queue depth, you scale on queue latency! Specifically for them they split up the time from enqueue to start, and then the time from start to done, and you scale on the former, not the total ("ScheduleToStart" in their terms).

There's always an autoscaling delay, but Rails itself (and the community) don't seem to fit into the serverless paradigm well such that these questions around how to design your queues come up.

I think a lot of Lambda developers or Cloud Run developers would instead say "well my max instances is set to 500, I am pretty sure I'm going to break something else before I hit that", you know? Especially when using the cloud's nice integrations between their queues and their event-driven serverless products its super easy to get exactly as much compute as you need to keep your latency really low.

We built a solution to the same problem with a similar approach[1], but that just snapshots any old files instead of doing nix derivations. Nix couples the build process to the content-addressability of the output, which works great if you want to put all the effort in to deterministic builds. We just read files like git does which works great for non-deterministic processes like npm install (tragically).

I like the idea of the Big Disk style of attaching a content addressable cache, but in our experiments we still found the network latency to the attached disk too high when reading file by file, like when booting a node app, so we’re caching a much smaller amount on a local SSD for each prod server. Maybe replit isn’t as sensitive to read perf from the cache layer, or they have fancy local per-node read through caching within the overlay setup? Regardless, cool!!

[1]: https://github.com/gadget-inc/dateilager

We think that so much of software development is still the same stuff repeated over and over: auth, hosting, CRUD, search, tables, forms, etc etc. Each app always has some juicy special something about it, but that core is wrapped in layers of stuff you don’t need to redo every time. Our mission is to make the first and only lines of code you write super pertinent to the specific problem you’re solving.

We’re starting with Shopify apps cause we can give developers a one click, fully managed, code-extensible API integration which is a lot of work with the Shopify API otherwise. Would love to know what y’all think!

And finally, ReadySet's (Noria's) big innovation is that the result set can be only partial, storing only the elements of the result set (and underlying data flow graph) that are frequently accessed, instead of the whole result set like a materialized view would.

I can’t stand that you need to pay for a pro license to avoid completely avoidable data loss. Sidekiq doesn’t use RPOPLPUSH for moving jobs around until you start paying — doing the exact same thing with a different, more correct command. No new infrastructure or configuration is needed. To me this feels sketchy and like a betrayal of the pro-version business model — willfully introducing data loss in order to drive upgrades instead of just removing capabilities or features.

Comments URL: https://news.ycombinator.com/item?id=27383279

Points: 7

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=25899286

Points: 583

# Comments: 123

If we continued to play in just the known parts of the problem space, we'd see it melt away further and further into Brooks' model of the world. Instead, we built distributed systems, we work in absolutely giant teams, we deploy to the strange heterogeneous runtime we call the web, etc etc, and all of that generates new complexities to abstract away before we get back to just the essential stuff.