Bill spent a lot of energy fighting commercialization of his work, arguing that it would devalue his characters and their personalities. I don’t know what is cheaper than using an AI model to instantly generate similar art, for free.

For example, 50% of surfers were found to have insufficient vitamin D in one study. https://pubmed.ncbi.nlm.nih.gov/17426097/

There are at least two possible conclusions that you could draw. One conclusion is that we all need vitamin D supplementation regardless of how much sun exposure we receive.

Another conclusion is that we might want to reevaluate what we consider the normal range to be, especially when we are deciding a range for a specific individual.

Functional data structures essentially create a proxy on every write. This can be inefficient if you make writes in batches, and you only need immutability between batches.

https://www.nasa.gov/wp-content/uploads/2015/03/135642main_b...

As someone who tried to learn category theory, and then did a mathematics degree, I think anyone who wants to properly learn category theory would benefit greatly from learning the surrounding mathematics first. The nontrivial examples in category theory come from group theory, ring theory, linear algebra, algebraic topology, etc.

For example, Set/Group/Ring have initial and final objects, but Field does not. Why? Really understanding requires at least some knowledge of ring/field theory.

What is an example of a nontrivial functor? The fundamental group is one. But appreciating the fundamental group requires ~3 semesters of math (analysis, topology, group theory, algebraic topology).

Why are opposite categories useful? They can greatly simplify arguments. For example, in linear algebra, it is easier to show that the row rank and column rank of a matrix are equal by showing that the dual/transpose operator is a functor from the opposite category.

Lifetime analysis is important for register assignment, and SSA can make lifetime analysis easier, but plenty of non-SSA compilers (lower-tier JIT compilers often do not use SSA because SSA is heavyweight) are able to register allocate just fine without it.

1. Removing that statement (dead code elimination)

2. Deduplicating that statement (available expressions)

3. Reordering that statement with other statements (hoisting; loop-invariant code motion)

4. Duplicating that statement (can be useful to enable other optimizations)

All of the above optimizations are very important in compilers, and they are much, much easier to implement if you don't have to worry about preserving side effects while manipulating the program.

So the point of SSA is to translate a program into an equivalent program whose statements have as few side effects as possible. The result is often something that looks like a functional program. (See: https://www.cs.princeton.edu/~appel/papers/ssafun.pdf, which is famous in the compilers community.) In fact, if you view basic blocks themselves as a function, phi nodes "declare" the arguments of the basic block, and branches correspond to tailcalling the next basic block with corresponding values. This has motivated basic block arguments in MLIR.

The "combinatorial circuit" metaphor is slightly wrong, because most SSA implementations do need to consider state for loads and stores into arbitrary memory, or arbitrary function calls. Also, it's not easy to model a loop of arbitrary length as a (finite) combinatorial circuit. Given that the author works at an AI accelerator company, I can see why he leaned towards that metaphor, though.

In GPU performance (probably measured on a specific set of tasks).

The runtime of WCOJs algorithms are even more dependent on good cardinality estimation. For instance, in VAAT, the main difficulty to find an appropriate variable ordering, which relies on knowledge about cardinalities conditioned on particular variables having particular values. If you have the wrong ordering, you still achieve worst case optimal, but you could have done far better in some cases with other algorithms (e.g. Yannakakis algorithm for acyclic queries). And as far as I know, many DBMSes do not keep track of this type of conditional cardinality, so it is unlikely that existing WCOJ will be faster in practice.

The new hotness is "instance optimal" joins...

Sorry for the possibly naive question. If I need to call a synchronous function from an async function, why can't I just call await on the async argument?

    def foo(bar: str, baz: int):
      # some synchronous work
      pass
    
    async def other(bar: Awaitable[str]):
      foo(await bar, 0)

An operating system (or sandbox, or whatever) is a very large virtual machine, where the "instructions" are the normal CPU instructions plus the set of syscalls. Unfortunately, operating systems today are complicated, hard to understand, and (relatively) hard to modify. For example, there are many different ways to sandbox file system access (chmod, containers, chroot, sandbox-exec on macOS etc.) and they each have bugs that have turned into "features" or subtle semantics. Plus, they are not available on all operating systems or even on all distributions of the same operating system. And then -- how do filesystem permissions and network permissions interact? Even of both of their semantics are "safe," is the composition of the two safe?

The assumption is: because operating systems are so complex, large, and underspecified, it probably is dangerous for LLMs to interact directly with the underlying operating system. We have observed this empirically: through CVEs in C and C++ code, we know that subtle errors or small differences in semantics can cascade into huge security vulnerabilities.

To address this, the authors propose that LLMs instead interact with a virtual machine where, for example, the semantics of permissions and/or capabilities is well-defined and standardized across different implementations or operating systems. (This is why they mention Java as an analogy -- the JVM gave developers the ability to write code for a vast array of architectures and operating systems without having to think about the underlying implementations.) This standardization makes it easier to understand how exactly an LLM would be allowed to interact with the outside world.

Besides semantic understanding and clarity, there are more benefits to designing a new virtual machine.

- Standardization across multiple model providers (mentioned).

- Better RLHF / constrained generation opportunity than general Bash output.

- Can incorporate advances in programming language theory and design.

For an example of the last point, in recent years, there has been a ton of research on information flow for security and privacy (mentioned in the article). In a programming language that is aware of information flow, I can mark my bank account password as "secret" and the input to all HTTP calls as "public." The type system or some other static analysis can verify that my password cannot possibly affect the input to any HTTP call. This is harder than you think because it depends on control flow! For example, the following program indirectly exfiltrates information about my password:

    if (password.startsWith("hackernews")) {
      fetch("https://example.com/a");
    } else {
      fetch("https://example.com/b");
    }

AFAIK, IQ tests used in psychological evaluations do not contain any randomness so exact answers are almost always in distribution. I haven't seen someone compare AI to an IQ test that is not in distribution.

On ARC-AGI, which is mildly similar to a randomly generated IQ test, humans still are much better than LLMs. https://arcprize.org/ (scroll down for chart)