The report actually seems to confirm this - it was indeed a crash on ingesting the bad config. However I'm actually surprised that the long duration didn't come from "it takes a long time to restart the fleet manually" or "tooling to restart the fleet was bad".

The problem mostly seems to have been "we didn't knew whats going on". Some look into the proxy logs would hopefully have shown the stacktrace/unwrap, and metrics about the incoming requests would hopefully have shown that there's no abnormal amount of requests coming in.

I fully agree that this and the cancellation issues discussed before can lead to surprising issues even to seasoned Rust experts. But I’m not sure what really can be improved under the main operating model of async rust (every future can be dropped).

But compared to working with callbacks the amount of surprising things is still rather low :)

- Proposal from 2020 about async functions which are forced to run to completion (and thereby would use graceful cancellation if necessary). Quite old, but I still feel that no better idea has come up so far. https://github.com/Matthias247/rfcs/pull/1

- Proposal for unified cancellation between sync and async Rust ("A case for CancellationTokens" - https://gist.github.com/Matthias247/354941ebcc4d2270d07ff0c6...)

- Exploration of an implementation of the above: https://github.com/Matthias247/min_cancel_token

> When a NOTIFY query is issued during a transaction, it acquires a global lock on the entire database (ref) during the commit phase of the transaction, effectively serializing all commits.

It only serializes commits where NOTIFY was issued as part of the transaction, right? Transactions which did not call NOTIFY should not be affected?

Does Whatsapp expose these messages via an API? If yes, then it seems like this is not only on Google.

If no: Are they reading data from raw UI widgets? Are they intercepting input controls? Are they intercepting network traffic? That seems unlikely, given its probably end to end encrypted and the decryption happens within the scope of the Whatsapp process.

In general this seems to work only if there's a single offline client that accepts the writes.

With limitations to the data scheme (e.g. have distinct tables per client), it might work with multiple clients. However those would need to be documented and I couldn't see anything in this blog post.

But maybe its intentional to avoid one player scoring too many points at once?

WebTransport is a bit different - it offers raw QUIC streams that are running concurrently with the requests/streams that carry the HTTP/3 requests on shared underlying HTTP/3 connections and it also offers a datagram API.

There is not even a fixed order between 2) and 3). The server can start sending headers or body data before the client sent any body byte.

What is correct is that a lot of servers and clients (including javascript in browsers) don't support this and make stricter assumptions regarding how HTTP requests are used - e.g. that the request bytes are fully sent before the response happens. I think ReadableStream/WritableStream APIs on browsers were supposed to change that, but I haven't followed the progress in the last few years.

NGINX falls into the same category. It's HTTP/2 support (and gRPC support) had been built with a very limited use-case in mind. That's also why various CDNs and service meshes use different kinds of HTTP proxies - so that various streaming workloads don't break in case way the protocol is used is not strictly request->response.

Is the main point that things like `slice_freeze_mut` could also be used for slices of e.g. `struct Coordinate { x: u32, y: u32, z: u32 }`?

It would obviously not work for f64 things, since there also not all bit-patterns are valid.

At least for housing I see the average home prices in each category (condos, townhomes, detached) still higher in Vancouver than Toronto (when googling a bit, I found https://wowa.ca/reports/canada-housing-market with some data. But there's probably lots of real estate related sites with more).

If your send/receive calls spend most CPU time in going through routing/fragmentation/filter/BPF/etc path in the networking stack, then uring (or other APIs which just reduce the system call overhead, like SendMmsg/Recvmmsg for UDP) might only make a small difference. Source: Lots of profiling while implementing QUIC libraries.

An alternative to DPDK that allows to bypass the kernel networking stack would be AF_XDP.

4k/5k 27" screens with appropriate font scaling make text so much easier to read and are thereby much easier on the eyes. And they still have plenty screen estate (having 2-3 files open next to each other is no problem).

Anyhow, the nice thing about the system is that one can get anything that is preferable as long as the logs are annotated correctly (with pod and container id).

Translating a UDP packet into a HTTP request and back is reasonably easy. Yes, maybe one can do that in a coding interview with some pseudo code. But scaling it and making it reliable is yet another dimension.

Any candidate would need to understand that a single UDP socket itself would probably already a bottleneck for just running this on single machine, and figure out how to increase concurrency there. That's not easy - the amount of engineers knowing how SO_REUSEPORT works and when it doesn't work is low.

After that you start to dig into how you can actually spread load between hosts. Would an answer like I "I hope my cloud provider solves this for me" be good enough? Probably not. If it actually is, do candidates have to both understand the cloud providers native APIs and Terraform (mentioned in the blog post). Seems pretty unnecessary, terraform is just one tool out of the myriad of tools which can be used to configure cloud services. Not everyone will have used it before. Or would it even expect candidates to do a long writeup about the pro's and con's of client-side load balancing?

Are applicants required to talk about upstream connection pooling? Describe and implement a CDN like multi-tier architecture?

Last but far from least is that the requested architecture is very easy to misuse for denial of service and amplification attacks. Just being able to describe how to monitor and mitigate that is already a very very hard task, that very few specialists have worked on so far.

It's very fuzzy what would be good enough if this is a "homework task". At least in a synchronous interview the interviewer could give feedback that they are satisfied. So I think in a synchronous interview the question might be ok - but there will probably just be time to either talk about coding or about system architecture.

You can also do something like

select client_ip from requests where elapsed_ms > 10000

which is incredibly powerful