And I claim that that is the real problem with AVX-512 (and pretty much any vectorization). I personally cannot find a single benchmark that does anything I would ever do - not even remotely close. So if you aren't into some chess engine, if you aren't into parsing (but not using) JSON, if you aren't into software raytracing (as opposed to raytracing in games, which is clearly starting to take off thanks to GPU support), what else is there?

If you need a little bit of inference (say, 20 ReNet50s per second per CPU core) as part of a larger system, there's nothing cheaper. If you're doing a small amount of inference, perhaps limited by other parts of the system, you can't keep a GPU fed and the GPU is a huge waste of money.

AVX-512, with its masked operations and dual-input permutations, is an expressive and powerful SIMD instruction set. It's a pleasure to write code for, but we need good hardware support (which is literally years overdue).

https://NN-512.com

The stand-alone code generator (a statically linked executable written in Go with no dependencies outside the Go standard library) generates stand-alone POSIX C code for the neural net, requiring only gcc to compile.

Also see Fabrice Bellard's LibNC:

https://bellard.org/libnc/

C API. Small library, no external dependencies, available for Linux and Windows.

Spend some time using Go, and you will be impressed by its performance.

You'll wonder, "Were all those haters on Hacker News misinformed?"

...you were unable to decipher this Hacker News comment thread...

...unable find some C code and build it with GCC and make an executable for yourself...

...but you think you can reverse engineer the executable?

https://nn-512.com/browse/DenseNet121

Even if you had the C code available to you, you would have a hard time producing the input graph.

Good luck reverse engineering it after GCC has compiled it!

NN-512 has an incredibly flexible code generator. It can easily be tweaked to produce completely different code for the same convolution, so everyone can apply their own twist to defeat the reverse engineers ("the intellectual property thieves").

https://NN-512.com

NN-512 generates custom code for all the operations, custom units of work for the threads, custom code around tensor edges, everything is fused and unrolled and customized. If they can deduce the network graph specification from the AVX-512 code, I will be astonished.

If you can do it, show me. But I know you can't.

Anyone who cares about model privacy will use their own variant of a tool like NN-512. It's security through obscurity, but that's the best you can hope for if you are distributing an executable.

  var (
      p1 = new(int)
      p2 = new(*int)
      p3 = new(**int)
      p4 = new(complex64)
      p5 = new(complex128)
  )

  var (
      c complex128
      p = &c
  )

The fact that it was developed at Google is incidental. That's where Robert Griesemer (Oberon) and Rob Pike (Plan 9) and Ken Thompson (Unix, Plan 9) and Russ Cox (Plan 9) and Ian Lance Taylor, etc., happened to be employed.

  p := new(int)

  p := &int{}

Similarly, for small-scale convolutional CPU inference, where you only need to do maybe 20 ResNet-50 (batch size 1) per second per CPU (cloud CPUs cost $0.015 per hour) you can use inference engines designed for this purpose, e.g., https://NN-512.com

You can expect about 2x the performance of TensorFlow or PyTorch.

But Meta is in the advertising business and Google is a competitor.

You've got people who argue against explicit error checking and people who don't understand the important role of nil pointers and people who don't see the massive net win of garbage collection in concurrent programs, and so on.

Endlessly.

It feels like a waste of time defending the language when the people criticizing it seem to hold such a vastly different point of view. It's like trying to convince people that Natural Born Killers is a good movie or that Primus makes good music. There's an unbridgeable chasm between you and the people you're trying to convince.

In both cases someone else is driving for the customer. With a human driver (plus driver assist braking and collision warnings) you have the most flexible, sophisticated intelligence on Earth driving. With a robotaxi you have something inferior. But maybe it's a lot cheaper, right?

The robotaxi can only compete on price because that's its only advantage. If you own a car that drives itself, that's a different story. Everybody can see the value proposition.

But is the robotaxi actually cheaper at all? We would have to look at the cost of the hardware (how often do lidars fail and how much do they cost to replace?) and the cost of the software development and the cost of the fallback human remote operators (fleet monitoring and teleoperation) and the years of huge R&D investment (billions of dollars) to evaluate whether a robotaxi fleet is indeed cheaper. So how much cheaper is it, exactly? 5%? 10%? 15%?

As a customer, would you pay a little more to have the most flexible, sophisticated intelligence on Earth (human brain + driver assist) or would you want to save a few dollars and risk having some dumb piece of software strand you in the middle of the road somewhere?

We all use Google Maps or Apple Maps when driving and most of us have seen these systems do boneheaded things. Just imagine the dumb things a robotaxi could do. It's hard for a normal person to be excited about this. I don't know a single person who is excited by robotaxis.

In both cases someone else is driving for the customer. With a human driver (plus driver assist braking and collision warnings) you have the most flexible, sophisticated intelligence on Earth driving. With a robotaxi you have something inferior. But maybe it's a lot cheaper, right?

The robotaxi can only compete on price because that's its only advantage.

If you own a car that drives itself, that's a different story. Everybody can see the value proposition.

We would have to look at the cost of the hardware and maintenance and fallback remote operators and the R&D investment to evaluate whether a robotaxi fleet is indeed cheaper. How much cheaper is it, exactly? 5%? 10%? 15%?

Would you pay a little more to have the most flexible, sophisticated intelligence on Earth (human brain + driver assist) or would you want to save a few dollars and risk having some dumb piece of software strand you in the middle of the road somewhere?

We all use Google Maps or Apple Maps when driving and most of us have seen these systems do boneheaded things. Just imagine the dumb things a robotaxi could do.

It's hard for a normal person to be excited about this. I don't know a single person who is excited by robotaxis.

Colin is "Security Officer Emeritus, FreeBSD".

Downside's goal is to remind people that we're experiencing a financial bubble, and that financial bubbles always burst. (We originally wrote that in April, 2000. Any questions?)

And here we are, living through another huge bubble.