No, that is controversial because the time saved per run (even 1000s of times per run with multiple daily runs) is never going to come close to amortizing the upfront sunk cost of that migration and future maintenance.

I’m specifically saying in the exact case you highlighted, people will short-sightedly think it’s a clear case to migrate out of the easy-but-slow interpreted language or never start with it to begin with, and they would be quantitatively wrong, missing the forest for the trees.

This is backwards. It costs extra developer overhead and code overhead to write those invocations in an AOT compiled language. The trade off is usually that occasional minor slowness from the interpreted language pales in comparison to the develop-time slowness, fights with the compiler, and long term maintenance of more total code, so even though every run is a few milliseconds slower, adding up to hours of slowness over hundreds of thousands of runs, that speed savings would never realistically amortize the 20-40 hours of extra lost developer labor time up front, plus additional larger lost time to maintenance.

People who say otherwise usually have a personal, parochial attachment to some specific “systems” language and always feel they personally could code it up just as fast (or, more laughably, even faster thanks to the compiler’s help) and they naively see it as frustration that other programmers don’t have the same level of command to render the develop-time trade off moot. Except that’s just hubris and ignores tons of factors that take “skill with particular systems language” out of the equation, ranging from “well good luck hiring only people who want to work like that” to “yeah, zero of the required domain specific libraries for this use case exist in anything besides Python.”

This is a case where this speed optimization actually wastes time overall.

I saw this in one company where it was the CTO running amok with an aggressive culture of yelling in meetings, slamming doors, and emphasizing arbitrary deadlines.

I saw it in another org where it was the CPO instead, trying to install Dilberty consultant snake oil with constant reorgs and zero accountability for product managers.

I know you will not like this comment and you will want to knee-jerk reply to “refute” it, but you need to resist that urge and admit you have really significant and really troubling prejudices that come through like a megaphone to others observing you, and spend time just dealing with that.

I think my earlier comment which you described as a “no u” comeback was actually really apt and crystal clearly accurate based on your follow-ups.

Neural nets typically don’t benefit much from it because you can use batch normalization, dropout and clever activation functions to achieve the same results, by having the network learn diminished sensitivity to outliers that produce neurons which saturate the low end of an activation function.

This is preferable because many of the robust potential functions involve absolute values, order statistics and other non-differentiable quantities that are hard to put into backpropagation-based optimizers. You almost always would need to relax the loss function to something that trades off smoothness against outlier robustness, where convergence will be slower and slower as you crank the trade off closer to outlier robustness.

Clearly the goal (no matter what anyone’s separate normative opinion is) is to balance some complex tradeoff between costs borne by tax payers, costs borne by corporations (through taxes and through employer based healthcare for working age adults and their dependents), and a high level of access for all people.

Nothing about this censored data effect can say anything about the morality of different regions of that trade off space.

As to your second comment, this article is not massive news and it seems laughable to say it is. It’s just a blip in the news cycle, using some data artifact to drum up attention to something that is already well-known for any econometrician or health policy analyst.

Full disclosure: I personally favor nationalized medicine and welcome higher taxes across the board. Nonetheless I don’t find your comment to be accurate or valuable.

- https://www.routledge.com/The-Ecological-Approach-to-Visual-...

- http://fmdb.cs.ucla.edu/Treports/soatto_extended_v18.pdf

I love roasting poor government policy decision making as much as the next person, but that would be a bridge too far.

Comments URL: https://news.ycombinator.com/item?id=26640827

Points: 3

# Comments: 0

Given that it’s (a) more difficult, (b) more business critical, (c) more stressful and requiring an incident alerting on-call rotation, then “data work” should be much better paid and offer job security and career growth.

Yet no company I know of pays expert modelers & researchers less than expert data platform engineers.

So either the companies know something you don’t (e.g. that data platform work is more commodity and easier to replace than rarer modeling talent) or there’s a free lunch you can get by exploiting the arbitrage opportunity to pay data platform experts more and consume correspondingly higher business value that other orgs are missing out on by putting modeler / researcher higher on the status hierarchy than data platform engineer.

My perspective after many years of experience managing machine learning teams (both platform/infra and research/modeling) is that data platforming is just a worse job. It’s unpleasant and stressful and business stakeholders who are removed from backend engineering complexity and just want the report or just want the model couldn’t care less about organizational structures and workflows that support healthier lives for intermediate data platform teams. Because of this, the pay and bonuses for data platform roles should be much higher, but politically speaking it’s impossible to advocate for that, so it becomes a turnover mill where everyone burns out to keep the existing shitty system running, with comparatively low pay and low autonomy, and so nobody ends up wanting to join that team or do that work.

How will you keep a job that supports the mortgage? Companies are exceedingly disloyal and looking to cut labor costs whenever possible, whole industries go through huge labor contraction, not to mention accumulating effects of automation.

Even if I saved a downpayment, I would not feel comfortable entering into a mortgage like this at all. It just adds stress to your life and you get none of the independence or peace of mind you expect to get from private property ownership.