Remote: OK

Willing to relocate: No

Technologies: go, python, postgres, ClickHouse, kafka, K8S, and various other things tied to high throughput backend systems. Also have an MSc in stats from LSHTM.

Résumé/CV: https://drive.google.com/file/d/1BQUA3tVHzEZKyUQjOBIZKf453m9...

Email: hblanks@artifex.org

I'm a career backend engineer who's worked most lately as a technical lead/manager delivering reliable, high throughput, containerized job scheduling to enterprise customers. Ex Twilio, Cloudflare, Segment. Looking for a long term opportunity in anything tied to high performance backend services or data infra, be that in a lead, EM, or IC role.

I've done a lot of early stage work and am a generalist at heart, though these days I'm most interested in post series A (including public) companies.

Anyway. Cloudflare's always been pretty cost efficient machine wise, so it was a natural choice given the performance needs we had. In my time in the data team there, Cap'n Proto was always pretty easy to work with, and sharing proto definitions from a central schema repo worked pretty well, too. Thanks for your work, Kenton!

https://en.wikipedia.org/wiki/Age_of_Anger , his book from 2017, isn't perfect but it's a good read of what he's about.

Comments URL: https://news.ycombinator.com/item?id=31415569

Points: 2

# Comments: 0

The 1st issue is that ZMQ implementations generally have an IO thread handling all socket IO. REQ/REP patterns tend to work out just fine, because the client is going to wait for a reply from the server. But things like PUSH/PULL become...harder to follow. You can push things in the client, but how do you know when they've flushed out of the IO thread and made it to the server? Those sorts of things matter when shutting down a process, running integration tests, etc. In contrast, with an HTTP request, you're basically always doing REQ/REP, and so you know when the data has been pushed.

The 2nd issue is that ZMQ implementations tend to be harder to observe / operate than a more common path like HTTP requests. (HTTP requests can go through a load balancer, have standard response codes, use headers for authorization, etc.) For these reasons, I've tended to avoid ZMQ ever after -- it seems like often, you're best off either using REST or GRPC if by HTTP, or raw TCP if it's purely a data push kind of operation (e.g. forwarding structured logs, with framing, to a remote TLS endpoint).

Comments URL: https://news.ycombinator.com/item?id=24433480

Points: 1

# Comments: 0

I think the most surprising insight is his commentary on Heart of Darkness as a novel about bureaucracy -- it's worth a read.

Just say no and hit send.

> In February 2018 Nicholas Thompson and Fred Vogelstein of Wired wrote a deeply reported piece that mentioned the 2016 meeting. It was called so that the company could “make a show of apologizing for its sins.” A Facebook employee who helped plan it said part of its goal—they are clever at Facebook and knew their mark!—was to get the conservatives fighting with each other. “They made sure to have libertarians who wouldn’t want to regulate the platform and partisans who would.” Another goal was to leave attendees “bored to death” by a technical presentation after Mr. Zuckerberg spoke.

(http://peggynoonan.com/overthrow-the-prince-of-facebook/)

There's some physiology to explain this, but one key point is that once you're in a coordinated turn, even a descending one, your weight is still going straight down the vertical axis of the seat. So, you'll think you're doing fine, and possibly even pull the turn tighter (the graveyard spiral), and fixate on things other than the attitude indicator and your plummeting altitude.

> Today when a periodical asks its readers a question, it does so in order to collect opinions on some subject about which everyone has an opinion already; there is not much likelihood of learning anything new. In the eighteenth century, editors preferred to question the public on problems that did not yet have solutions. I don't know whether or not that practice was more effective; it was unquestionably more entertaining.

> In any event, in line with this custom, in November 1784 a German periodical, Berlinische Monatschrift published a response to the question: Was ist Aufklärung? And the respondent was Kant.

(https://www.libarts.colostate.edu/leap/wp-content/uploads/si...)

* Hour restrictions for passenger transport are significantly more stringent than for freight.

* Southern Air's fleet consists 737s and 777s -- not small planes.

* These plans have to fly over and into major metropolitan areas.

* Many parts of flying can be automated, but in the end, and for aircraft this size, you probably want a human operator in the plane for the sheer reason that they are better equipped to handle unplanned emergencies.

It's a tough call whether flying freight or regional airlines pays worse -- but in either case, you're anywhere from a quarter to half of what a software engineer of similar years experience in the bay area would make. The hours are in a lot of ways worse -- mostly because you're away from home a lot of the time, and you have really limited control of your schedule. (As someone who's worked as an on-call engineer for ten years, and who also flies, I'd take the on-call responsibilities over flying freight any day of the week.)

I'd agree that the article lacks for details. But, a relatively quick reading through things like airliners.net will give you an idea of the kind of hours and pay these guys typically work -- it's not an easy job. If we care about their safety -- and our own, given the number of freight aircraft flying over us every night -- we shouldn't write them off as just asking for money they don't deserve.

It's been good so far. Good support from the maintainers, too.

It doesn't end well.

Peachum [The Beggar King]: "Because they don't know that we need them."

--Die Dreigroschenoper (1931 Film)

Your queries tend to break down into ones where you're hitting a small number of shards (such as when we serve data for the analytics page), or else ones where you tend to aggregate up into temporary, non-distributed tables for further analysis (such as you'd do for infrequent business reporting).

All told, it's been one of the easier tools for us to operationalize, largely thanks to the fact that it's "just PostgreSQL." The one thing I wish we had was better documentation of what kind of consistency guarantees to expect, although for an append-only store like our current use case, that's less of a concern. And to be fair, Ozgun and the other guys at Citus have always been really happy to answer any questions we have.

With this code now going open source, it should be pretty easy to look into these sorts of internals.

It's a good product, and it was even fairly easy to do a major version upgrade / cluster relocation. At least as easy as such a thing can be. :-)