It would be interesting to know if the stronger results on Python are not just an artefact of the Python-specific evaluation, if they carry over to other common general-purpose languages, and if they are driven by something specific in the training processes.

Abstract:

"Incremental view maintenance has been for a long time a central problem in database theory. Many solutions have been proposed for restricted classes of database languages, such as the relational algebra, or Datalog. These techniques do not naturally generalize to richer languages. In this paper we give a general solution to this problem in 3 steps: (1) we describe a simple but expressive language called DBSP for describing computations over data streams; (2) we give a general algorithm for solving the incremental view maintenance problem for arbitrary DBSP programs, and (3) we show how to model many rich database query languages (including the full relational queries, grouping and aggregation, monotonic and non-monotonic recursion, and streaming aggregation) using DBSP. As a consequence, we obtain efficient incremental view maintenance techniques for all these rich languages."

I agree this is a very interesting area to consider Ray for. There are lots of projects/products that provide core components that could be used but there’s no widely used library. It feels like one is overdue.

>Imagine it’s 2021, peak MDS, and you meet the CDO of a large bank. “Oh cool,” she says, “you’re the CEO of a tech company. What does your product do?” What do you say?

>“We build a tool that leverages the power of the cloud to apply standard SQL and software engineering best practices to the historically mundane (but critical!) job of data transformation.”

>“We’re the standard for data transformation in the modern data stack.”

>I will tell you that, empirically, option #2 is more effective.

This tallies with what I've seen from a lot of enterprise CxOs and their teams as technology hype moved from big data and block chain and onto data science/machine learning.

There is so much to write about this, but I'll just recommend "Life Cycle of a Silver Bullet" http://freyr.websages.com/Life_Cycle_of_a_Silver_Bullet.pdf, which deserves more attention than it's had on HN.

These all help to lower the cognitive barrier to learning and maintaining the code base effectively. For developers new to the code base they help with learning and for those more experienced they help with ongoing design and maintenance.

Most long-lived code bases I've seen have adopted or built such tooling at some point, often with tools customized to the code base. For example in one large code base (c. 250 devs) we built tooling that simulated and helped optimize the changes to implement a major refactor of the overall module structure.

Mathematical Physics :)

It seems that negligence is not in short supply.

Likewise when Frank argues in favor of his working for the banking industry, this raises conflicts of interests for legislators and potential for undue and hard-to-detect influence in a way that e.g. taking an academic position concerning banking or lobbying for gay rights would not.

Although Python is not going to match a full Lisp, Haskell or ML in all their strengths, using a functional style can be useful and expressive. The toolz docs give some relevant background at https://toolz.readthedocs.io/en/latest/heritage.html .

At a language level, Peter Norvig gave a lengthy comparison of Python and Lisp at https://norvig.com/python-lisp.html in 2000.

For comparison, the more recent approach of increased pedestrianisation and traffic calming has been a significant improvement to life in the New York areas where it's been applied.

To me it’s one of the high points of the web and Paul Ginsparg is an absolute hero for setting it up; he reviews some of its history at https://arxiv.org/abs/1108.2700.

I think it’s an interesting social question why other fields have taken so much longer to adopt the arxiv model.

Do you have any plans to add a graphical visualization of top/central papers?