Comments URL: https://news.ycombinator.com/item?id=47091578

Points: 2

# Comments: 0

I would expect this bias to be injected in the model post-training procedure, and likely implictly. Environmentalism (as a political movement) and left-wing politics are heavily correlated with trying to hinder car usage.

Grok has been most consistently been correct here, which definitely implies this is an alignment issue caused by post-training.

The reason why highly developed economies have become so service driven is because they have become sort of bimodal: The cost of labor is such that only jobs that are productive enough (profitability per hour) are done in these countries, and jobs that absolutely have to be done there to sustain the population. Jobs in the middle, everything that is not highly profitable or location-dependent, is offshored to lower-cost countries due to the cost of labor. This results in these developed countries having issues: Cost of living is high due to labor cost and there's high economic inequality due to wildly differing productivity.

The solution would be to bring these "mid-productivity" jobs back to developed countries. However, the main roadblocks still remain: The cost of labor is too expensive for most of these jobs to be competitive globally. However, I think there might be a way to do this in the near future: Advancements in robotics would mean a higher level of automation for industrial work, meaning more industrial jobs would become viable in high-cost countries. Each worker would be productive enough that the cost of labor is not critical anymore.

To make this happen, I believe it's important to ensure that the country is viable for this kind of manufacturing: Energy supply needs to be abundant and cheap, workforce needs to be educated, outside the "elite" students, and there needs to be low trade barriers. Low trade barriers are needed, because virtually all manufacturing is part of a global supply chain where parts cross many borders before the product is sold (and (high-value) products are sold globally). Additionally, the viability of automation will vary between different parts of the supply chain, and so you likely cannot automate everything.

However, this act would, in my understanding, give much more power to Finland and Estonia to detain these ships, and charge the crew for the crimes they have committed. Right now there seems to be a loophole in the legislation that Russia is actively exploiting for hybrid warfare purposes. If the strait rules would give Russia more ways to cause harm, some other way of dissuading Russia from making these acts should be done.

In general though, it feels stupid that we have to play by these rules, when the enemy makes a mockery of them and actively tries to exploit them to cause as much harm as possible. But that's the reality when bordering Russia.

Russia clearly hasn't acted in such way that they should enjoy these kinds of acts of benevolence. Finland and Estonia should seriously consider retreating from this agreement.

By far the worst in this aspect has been Scala, where every codebase seems to use a completely different dialect of the language, completely different constructs etc. There seems to have very little agreement on how the language should be used. Much, much less than C++.

Only if you're ignoring mobile entirely. One of the things Vulkan did which would be a shame to lose is it unified desktop and mobile GPU APIs.

In this context, both old Switch and Switch 2 have full desktop-class GPUs. They don't need to care about the API problems that mobile vendors imposed to Vulkan.

Also, I'd say if you buy for example a Macbook with an M4 Pro chip, it is already is a big GPU attached to a small CPU.

AI training is a bandwidth problem, not a latency problem. GPUs need to be fed at 10GB/s+. Making millions of small HTTP requests introduces massive overhead (headers, SSL handshakes, TTFB) that kills bandwidth. Even if the storage engine has 0ms latency, the network stack does not.

Agree that throughput is more of an issue than latency, as you can queue data to CPU memory. Small object throughput is definitely an issue though, which is what I was talking about. Also, there's no need to use HTTP for your requests, so HTTP or TLS overheads are more of self-induced problems of the storage system itself.

You can fetch specific byte ranges from a large blob without "coupling" the storage layout significantly.

This has exact same throughput problems as small objects though.

There are solutions for this, but the added complexity is big. In any case, your training code and data storage become tightly coupled. If you can avoid it by having a faster storage solution, at least I would be highly appreciative of it.

Matrix multiplication didn't "win" because HW was designed for it. It won because matrix multiplication is a fundamental part of linear algebra and is very effective in deep learning (most kinds of functions you might want to write for deep learning can be expressed as a matmul). Acceleration of it became later. Additionally, matrix multiplication is a good fit for physics, as you can design the HW so that data movement is minimized, and most of the chip area and power are spent in actual computation, and not moving data around.

Fundamentally speaking, you also want to make your algorithm compatible with real-world physics. The need for heavy parallelism is required by the fact that you cannot physically make a fast chip that processes dependent operations. It's just not possible to propagate signals through transistors fast enough to make it possible. Even CPUs, even if they present a non-parallel programming environment, have to rely on expensive tricks like speculative out-of-order execution to make "sequential" code parallel to make it fast.

In general though, I personally would wish that chips would be made with taking programmability in mind. A fixed-function matrix multiplier might be slightly more efficient than a parallel computing chip with smaller matrix multipliers. But it would be significantly more programmable, and you can design much more interesting (and potentially more efficient) algorithms for it.