Hacker News: maciejzj

New comment by maciejzj in "The gap in shared understanding of LLM capability is widening"

maciejzj — Mon, 12 Oct 2026 08:02:57 +0000

I would agree that a decent EE major is equipped with knowledge sufficient to grasp both Transformer architecture and gradient descent training. However, these elements do not fully cover the key parts of modern AI systems and also are not enough to predict the AI capabilities. You have to account for reinforcement learning from human feedback (RLHF) and with verifiable rewards (RLVR) which are crucial elements of the wider model creation/training process. These do not have obvious analogues in other engineering disciplines, but are still within your grasp if you are decent CS/EE/adjacent major. The issue is that you still can't deduce end result AI capabilities from first principles. You may have prefect understand of y = softmax(QK^T/sqrt(d_k))*V and it will give you no clue of the effectiveness of a given AI model. The range and complexity of training RLVR tasks may hint at the model capabilities, but the final results are very much emergent.

New comment by maciejzj in "Transformers Explained Visually"

maciejzj — Tue, 22 Sep 2026 07:06:31 +0000

One of the better visualisations I've seen with the exception of Q/K/V weights and how they are presented. I believe they should be put more upfront since they are the core learnable parameter of attention. IMO, they should also be part of the "Head N of M" block since each head has its own weights (although they all can be collapsed into one huge matrix computation).

New comment by maciejzj in "We must pace the frontier"

maciejzj — Sun, 13 Sep 2026 09:20:38 +0000

Had they issued an initiative on slowing down when they had the best model in the game, it would have been both more (anyhow?) effective and less dubious.

New comment by maciejzj in "The Navier–Stokes Millennium Prize Problem"

maciejzj — Wed, 09 Sep 2026 07:25:24 +0000

I know that this may be somewhat dramatised and even infantile, but my reflection is that in the world run by these reckless AI companies everyone looses. Navier-Stokes is solved but it feels like no one has won anything, controversy prevails, there is no glory in the math breakthrough. There is hardly anything to cherish, and even the guys at the top of it in OA who sit on the (supposedly) superhuman intelligence come across as massive losers and frauds.

New comment by maciejzj in "I were 17, I'd learn how to build LLMs from scratch"

maciejzj — Mon, 24 Aug 2026 10:23:19 +0000

I would advise any somewhat ambitious 17 year old to avoid tech and get into healthcare if they can stomach human interactions and bodily fluids. Sure, it is not all sunshine and rainbows, but there will still be plenty of work helping people who are ill or elderly. Even in the worst-case economic scenario, medicine will be a more socially rewarding and stable life path.

New comment by maciejzj in "I'm Tired of Talking to AI"

maciejzj — Wed, 27 May 2026 11:30:09 +0000

I've recently been connecting some machines to a new switch and my colleague has been monitoring web logs at the same time using Claude. He send me a Claude-generated observation that the machines that I was able to put my hands on simultaneously must be in different buildings due to high pings. Surreal experience.

New comment by maciejzj in "OpenAI Privacy Filter"

maciejzj — Sun, 26 Apr 2026 07:31:25 +0000

On a side note, when I click the link it redirects me to machine-translated version of OpenAI website with completely botched meaning - the word “redacted” is translated to a false friend “redagować” which means to edit/refine text, not anonymize.

New comment by maciejzj in "Lambda Calculus Benchmark for AI"

maciejzj — Sat, 25 Apr 2026 14:18:05 +0000

Can anyone more familiar with lambda calculus speculate why all models fail to implement fft? There are gazzilion fft implementations in various languages over the web and the actual cooley-tukey algorithm is rather short.

New comment by maciejzj in "Show HN: How LLMs Work – Interactive visual guide based on Karpathy's lecture"

maciejzj — Fri, 24 Apr 2026 10:13:58 +0000

AFAIK – the input is (at most basic level) a matrix with L tokens (rows) and d embedding length (cols). The input tokens are initially coded into discrete IDs but they are turned into embeddings by something like `torch.nn.Embedding`. The embedding layer can be thought of as a "lookup table" but it is matrix multiplication learned through gradient descent (adjusted at train time, fixed values at inference time). The length of embedding (d) is also fixed, L is not. If you check out the matrix multiplication formulas for both embedding layer and attention you will notice that they work for any number of rows/tokens/L (linear algebra and rules of matrix multiplication). The context limit is imposed by auxiliary factors – positional encoding and overall model ability to produce coherent output for very long input.

When it comes down to the meaning of "bank" embedding, it cannot be interpreted directly, however, you can run statistical analysis on embeddings (like PCA). If we were to say, the embedding for "bank" contains all possible meaning of this word, the particular one is inferred not by the embedding layer, but via later attention operations that associate this particular token with the other tokens in the sequence (e.g. self attention).

New comment by maciejzj in "John Ternus to become Apple CEO"

maciejzj — Tue, 21 Apr 2026 07:42:30 +0000

Anecdotal evidence, but I do use Apple Maps in Poland and they work just fine for me, I guess the mileage may vary.

New comment by maciejzj in "NASA Force"

maciejzj — Fri, 17 Apr 2026 16:36:50 +0000

Is this gig-workification of the space industry?

New comment by maciejzj in "Learnings from 4 months of Image-Video VAE experiments"

maciejzj — Thu, 26 Feb 2026 09:54:53 +0000

Happy to see this architecture stil being handy for other people. Right now, I am using a heavily modified VAE for generation of synthetic satellite imagery. I am working with very limited dataset sizes, so I have chosen VAE over diffusion as a starting point.

New comment by maciejzj in "OpenClaw is changing my life"

maciejzj — Sun, 08 Feb 2026 16:01:51 +0000

Mind you, that regardless of your sentiment towards OpenClaw, not everyone is able to afford a sparse Mac Mini (especially given ram prices) and a ton of Claude tokens/super beefy GPU for local models to run this stuff. That's to the supposed "democratisation of knowledge and technology".

New comment by maciejzj in "Europe’s next-generation weather satellite sends back first images"

maciejzj — Mon, 02 Feb 2026 11:48:05 +0000

I don't mean "demo-like" in terms of poor technology. I meant that this technology doesn't yield products or services with global scales to an extent it happens in the US. Google Maps successfully uses both LANDSAT and Sentinel imagery. This is the wider problem of European failure to build companies/systems on top of technology.

New comment by maciejzj in "Europe’s next-generation weather satellite sends back first images"

maciejzj — Thu, 29 Jan 2026 14:12:39 +0000

ESA has done a lot of good for public benefit with the Sentinel-1/2 missions. I happen to work with remote sensing and Sentinel data has been my entry point to the field.

I hope that ESA keeps pushing forward even more. I am afraid that although Sentinel missions are great, ESA projects are a bit demo-like and limited in scope. Europe should focus on scaling up and applying the tech, not just proving that ambitious projects are possible for their own sake.

New comment by maciejzj in "The tech market is fundamentally fucked up and AI is just a scapegoat"

maciejzj — Thu, 29 Jan 2026 14:03:51 +0000

Mind you that the IT over investment sucked money out of other industries. My friends who chose different career paths do not seem to be particularly content either.

New comment by maciejzj in "New books aren't worth reading?"

maciejzj — Wed, 28 Jan 2026 21:11:12 +0000

> Unfortunately, reading books for entertainment is ridiculous. You do not live in a log cabin on the prairie. You have Netflix, you have video games, you have TikTok, you have Twitter (you really spend too much time on Twitter anon). No one reads books for entertainment anymore, because paper is an inferior entertainment platform.

I see that this has already been criticised here, but to add my two cents, I believe that reading books has become one of the very last widely available (perhaps free?) entertainment media that is not anxiety inducing and mentally draining. 10 years ago reading books may have been a kind of a snobbish activity, but now it is one of very few things to engage in before you go to bed that will help you relax and get good sleep.

New comment by maciejzj in "ASML staffing changes could result in a net reduction of around 1700 positions"

maciejzj — Wed, 28 Jan 2026 09:58:51 +0000

Yeah, irony put asisde, I believe that the reality is that people who already have power will consolidate it even more. Regardless if you were an average engineer, mid-level manager, consultant, whatever, you will be thrown under the bus any day. People can make up any identity and philosophy of entrepreneur/vibe engineer/manager/hardcore engineer/guy who run away to trades/anything but it doesn't change the fact that an average Joe is economically not viable in the current setup and he can't do anything particular to turn the tides.

New comment by maciejzj in "ASML staffing changes could result in a net reduction of around 1700 positions"

maciejzj — Wed, 28 Jan 2026 08:59:54 +0000

Guys, is the current narrative that, due to AI, pure engineering is gone and we’re all supposed to be “managers”, or is it the other way around? I kinda lost the plot here.

New comment by maciejzj in "The next two years of software engineering"

maciejzj — Tue, 13 Jan 2026 08:23:40 +0000

If majority of housing is owned for profit by REITs or landlords they have such a leverage over ordinary person, that they can indefinitely hold the prices/rents at a level where they extract maximum of available resources from owning land while making sure people have enough processed food and cheap internet-provided entertainment that they don't rebel.

The prices will adapt, but the equilibrium will always be elite-oriented economy where accommodation of the masses is a second-tier goal.