But the articles point is that to the people - it’s not the number of requests that matters - it’s how often they are waiting for them. The question is “how much time am I sitting here waiting?” In that case the 10s request is 5x worse than a 2s request - it takes 5x of the “time spent”.

So you can change the weighing to 1 / 2 2s 1 / 2 10s to 1 * 2/12 & 1 * 10/12 - that gives us 17% and 83%. And the average there is 9.04s.

The difference is the question. If you think about it as road segments, let’s say you have a group of road segments with different lengths. You can ask the “average length of segment” - or you can ask “if you pick a random point among all the segments, how long is the segment that I landed in?” You’re picking very differently there - the second is proportional to the length!

Technically IMO the blog is slightly off - you want to use “mean residual life”. Ie if I pick a random TIME how long do I have to wait for my request to finish. But it’s reasonably close.

I'm pretty sure what the author is saying is:

E(X) =:= \sum_t(t * P(X = t)) is the definition

another important note is P(X^2 = t^2) = P(X = t) - because it's the same distribution.

E_a(X) is a bit sloppy, but consider X_a aka Alice's latency "experience" distribution. The argument is:

P(X_a = t) = t * P(X = t) / \sum_u(u * P(X = u)) - i.e. scale the probability up by t but make it sum to 1.

Then

E(X_a) = \sum_t(t * P(X_a = t)) = \sum_t(t * t * P(X = t) / \sum_u(u * P(X = u))

aka

E(X^2) / E(X)

Then (from wikipedia)

Var(X) = E(X^2) - (E(X))^2

And we get

E(X_a) = (Var(X) + (E(X))^2) / E(X) = E(X) + Var(X) / E(X)

- There’s an almost wysiwig editor for mermaid at https://www.mermaidchart.com/play . It’s very convenient and appropriately changes the layout as you draw arrows!

- Notion supports inline mermaid charts in code blocks (with preview!) It’s awesome for putting some architecture diagrams in Eng docs.

Explicit dynamic bindings are better if you need something like this since those are thread local.

  sigterm_handler() {
    make_healthcheck_fail();
    sleep(20);
    stop_web_server();
    exit(0);
  }

That said trying to enforce overhead limits with RSS limits also won’t end well. Java doesn’t make guarantees around max allocated but unused heap space. You need something like this: https://github.com/bazelbuild/bazel/blob/10060cd638027975480... - but I have rarely seen something like that in production.

If you have an actual memory leak in a JVM app what you want is an exception called java.lang.OutOfMemoryError . This means the heap is full and has no space for new objects even after a GC run.

An OOMKilled means the JVM attempted to allocate memory from the OS but the OS doesn't have any memory available. The kernel then immediately kills the process. The problem is that the JVM at the time thinks that _it should be able to allocate memory_ - i.e. it's not trying to garbage collect old objects - it's just calling malloc for some unrelated reason. It never gets a chance to say "man I should clear up some space cause I'm running out". The JVM doesn't know the cgroup memory limit.

So how do you convince the JVM that it really shouldn't be using that much memory? It's...complicated. The big answer is -Xmx but there's a ton more flags that matter (-Xss, -XX:MaxMetaspaceSize, etc). Folks think that -XX:+UseContainerSupport fixes this whole thing, but it doesn't; there's no magic bullet. See https://ihor-mutel.medium.com/tracking-jvm-memory-issues-on-... for a good discussion.

So about an 8% chance. You’re significantly more likely (ie 92% chance) to see _something else_. And that’s _the most_ likely outcome.

So tldr - it’s not so much that “you never see an all tails sequence in practice” - you’re actually unlikely to see any particular sequence. All the probabilities get astonishingly low very quickly.