The market for tech recruiting has been hit as hard and in some ways harder than the market for tech workers. Most folks are finding or placing jobs through in-network referrals and recruiters are finding much lower demand for their services after a long period of having it good.

I know some actually good recruiters and really feel for those folks right now, as well as early career folks like yourself. It took me 8+ months to find a job myself in 2023.

Wishing you the best of luck in your search! Sincerely rooting for you and hope your luck will turn around.

One way of thinking about explainability is that it deals with determining for some input data how much each feature is contributing to the final outcome (e.g. variable 1 and 2 contributed x% and y% to the final inferred value).

You're correct to suggest that when you backpropogate residual error there are also non-linear interactions between features and that will affect how much each variable is contributing to the updated weight value (in fact that's kind of the point of a deep network ;).

In fairness I haven't given this article a thorough read but my initial impression is that I'm finding myself frustrated by the FUD this article is attempting to spread. As my boss would often remark to remind us all: model explainability is an under-constrained optimization problem. By definition there isn't a unique explanation decomposition unless you further constrain the problem.

Therefore, I personally find that hand-wringing around there not being 100% agreement around different explanations for a model inference, while definitely thought provoking and worth considering, should at least account for this reality. For some reason a lot of folks in the ML community seem to have come the opinion that because the problem is under-constrained that means that explanations shouldn't be calculated or have no utility.

Would you prefer a model that examines which features are driving the model to deny a disproportionate number of folks of a particular race or ethnicity or not, all things being equal? My point is even if there are limitations to explainability I think there are a lot of very real, critical scenarios where applying SHAP can be of actual, real world utility.

Furthermore, it's not clear that LIME or other explainability methods will provide better or more robust explanations than Shapley values. As someone that has looked at this pretty extensively in credit underwriting I'd personally feel most comfortable computing SHAP values while acknowledging some of the limitations and risks this article calls out.

Axioms such as completeness are also pretty reasonable and I think there is a fair amount of real world utility to explainability algorithms that derive from such an axiomatic basis.

One of my family members literally maintains a software platform for test bedding production NASA spacecraft at JPL and it is all on top of Jupyter notebooks.

People are literally on course to build AGI and there’s a good chance that large portions of that work will be done within Jupyter notebooks.

To suggest real work doesn’t get done in notebooks is simply asinine.

As I have embedded further into data science and machine learning teams within my career I have come to learn that a lot of the things software engineers care about around maintainability, reusability etc. are often completely antithetical to data science productivity. In many cases practitioners would consider some of the listed cons of notebooks as features rather than bugs.

There is obviously a time and a place for pulling out those practices in such settings but blindly applying software engineering standards to this kind of work is both all too common and often rather counter productive.

If notebook style dev doesn’t jive with you personally I get that, and god speed. I’ve worked on CV/ML teams with a lot of CS degree grads and they don’t always want Jupyter notebooks. However, I’ve noticed a tendency for folks to extrapolate that to blanket statements about notebooks in general. As someone who has inherited and had to clean up many jumbled/bad notebooks I’m empathetic but I would challenge folks to lean into considering more about the circumstances the previous notebook author was working in and what kinds of productivity outcomes they were responsible and prioritizing rather than casting judgement on said author or their tools of choice.

It seems to me that Hans Reiser's write up still has a lot of leaps and lurches, both emotionally and in terms of reasoning. I felt there was a lot of instances where he expresses remorse for key hiring decisions or meetings between stakeholders in the open source world, but with tinges of belief that had he done things differently then that would've made the difference in a successful outcome.

In a lot of ways it seems like Hans Reiser had a pretty grandiose vision to rearchitect filesystems to become more like databases- which then demanded huge changes both in terms of the physical layout of data stored on disk as well as the implications around how operating systems would leverage and use such a technology capability. He sees himself in a lineage with the plan9 folks... and also sort of implies that this was an idea ahead of its time and somewhat downplaying what a large amount of change would be required (sort of reminiscent to me of the criticisms ppl wage against systemd).

He's now in jail for murdering his wife.

Richard Hipp, on the other hand, literally has a values page on his website. His software is in the public domain. He and his team have been steadily and methodically building what is now the most widely deployed database of all time and it powers a wide range of critical applications and has been approved for FAA use cases, used in missile targeting and powers literally all of the apps on our phones.

He's a deeply Christian human (and I say this as a lifelong atheist who wasn't raised on these values) and has approached evangalism of his technology with heaping amounts of humility and hardcore praxis (SQLite is arguably one of the more comprehensively tested libraries out there). Hipp also is rather opinionated as a technologist- he wrote his own SCM on top of SQLite! But he doesn't come across as a zealot whatsoever but rather a seasoned and mature technologist who is methodically executing on a radical vision to the benefit of all.

In the process I feel that in a lot of ways his accomplishments have achieved the vision that Hans Reiser wanted around advancing new ideas in databases and filesystems. However, instead of doing it at the filesystem layer Hipp instead achieved this vision in process with a library that is extremely easy to include in a huge variety of projects within userspace. In the process the revolution that Reiser wanted was achieved in many ways and with a lot less churn and violence in the process (figuratively and sadly literally).

I could not think of a more opposite and extreme contrasting examples of technologists and approaches, and for me it teaches a lot about how to approach socio-technological endeavors successfully as well as providing a good illustration of the way in which ethics and morals play into said endeavors.

(linked to barlow's reddit AMA as it was the only primary source link I could find quickly/easily)

The Python ecosystem once fell apart not too long ago because it was supporting two versions where one moved from “print “ to “print(“ and these were incompatible and broke things such as doc tests.

There’s a reason that ppl strongly started advocating to hard pivot to 3: there was a very real chance that Python 2 could fork the ecosystem.

Incompatible concurrency semantics would be a much worse can of worms.

However, I would speculate part of PSF's hesitancy is likely specifically around the perceived violence that gross' "GIL-less" changes may incur to the runtime semantics' backwards compatibility.

PSF in particular has a responsibility here as well I feel in that CPython is arguably the working spec or standard from which these other implementations work and are defined.

Do you also strongly prefer languages with different underlying concurrency semantics? While stackless and pypy etc. are around and available and this could suggest the answer could be "yes" we've been lucky that they haven't fundamentally changed the experience of writing Python.

The possibility that a ton of libraries might now be able to use efficient multi-threaded execution where they were previously constrained to multiprocessing will be a landslide of changes on its own, and likely reminiscent of python 2 -> 3 compatibility if we have to preserve "two ways of doing things."