Hacker News: craigacp

New comment by craigacp in "Why we need SIMD"

craigacp — Thu, 09 Oct 2025 01:43:07 +0000

SIMT - Single Instruction, Multiple Threads

New comment by craigacp in "When good pseudorandom numbers go bad"

craigacp — Fri, 23 May 2025 02:36:58 +0000

The same operations in the same order is a tough constraint in an environment where core count is increasing and clock speeds/IPC are not. It's hard to rewrite some of these algorithms to use a parallel decomposition that's the same as the serial one.

I've done a lot of work on reproducibility in machine learning systems, and its really, really hard. Even the JVM got me by changing some functions in `java.lang.Math` between versions & platforms (while keeping to their documented 2ulp error bounds).

New comment by craigacp in "Show HN: Txeo – A Modern C++ Wrapper for TensorFlow"

craigacp — Sat, 22 Feb 2025 02:14:19 +0000

Some of the gradients are present in the TF C API (and thus the C++ API), but it's hit & miss which are in C and which are in Python. There was an attempt years ago to port more gradients to C, but this petered out in the way that most TF related efforts seem to do at Google.

We use the C API to generate gradients for TF-Java, and have some success training models with it. Replicating the Python bits in another language is a huge effort though that we haven't completed.

New comment by craigacp in "A Replacement for BERT"

craigacp — Fri, 20 Dec 2024 02:39:29 +0000

The encoder's embedding is contextual, it depends on all the tokens. If you pull out the embedding layer from a decoder only model then that is a fixed embedding where each token's representation doesn't depend on the other tokens in the sequence. The bi-directionality is also important for getting a proper representation of the sequence, though you can train decoder only models to emit a single embedding vector once they have processed the whole sequence left to right.

Fundamentally it's basically a difference between bidirectional attention in the encoder and a triangular (or "causal") attention mask in the decoder.

New comment by craigacp in "The super effectiveness of Pokémon embeddings using only raw JSON and images"

craigacp — Mon, 01 Jul 2024 01:20:25 +0000

There are a bunch of these things in a word2vec space. I had a blog post years ago on my group's blog which trained word2vec on a bunch of wikias so we could find out who is the Han Solo of Doctor Who (which I think somewhat inexplicably was Rory Williams). You need to carefully implement word2vec, and then the similarity search, but there are plenty of vaguely interesting things in there once you do.

New comment by craigacp in "Java 22 Released"

craigacp — Wed, 20 Mar 2024 01:09:00 +0000

Java has had the ability to run a single code file directly by running `java ` since Java 11 - https://openjdk.org/jeps/330. And it's had a Java REPL since Java 9.

New comment by craigacp in "Latest GPT-4 training data updated to December 2023"

craigacp — Tue, 20 Feb 2024 02:23:48 +0000

There's usually a two or three step training procedure, first training to predict the next word on a huge corpus of text (billions or trillions of words), then possibly some instruction tuning (giving the model question & answer pairs and training on the answer) and then finally RLHF (or RLAIF, DPO etc) where the model is trained to match human preferences. It's this last step that is used to increase the helpfulness & harmlessness of the model, training it to not respond to certain topics.

New comment by craigacp in "SD4J – Stable Diffusion pipeline in Java using ONNX Runtime"

craigacp — Mon, 01 Jan 2024 17:17:00 +0000

I did. It depends what you want, for an overview of how ONNX Runtime works then Microsoft have a bunch of things on https://onnxruntime.ai, but the Java content is a bit lacking on there as I've not had time to write much. Eventually I'll probably write something similar to the C# SD tutorial they have on there but for the Java API.

For writing ONNX models from Java we added an ONNX export system to Tribuo in 2022 which can be used by anything on the JVM to export ONNX models in an easier way than writing a protobuf directly. Tribuo doesn't have full coverage of the ONNX spec, but we're happy to accept PRs to expand it, otherwise it'll fill out as we need it.

New comment by craigacp in "SD4J – Stable Diffusion pipeline in Java using ONNX Runtime"

craigacp — Mon, 01 Jan 2024 16:34:43 +0000

Both GPU support and SIMD CPU support are being worked on in OpenJDK projects, there are talks on them from last year's JVM language summit - https://youtube.com/playlist?list=PLX8CzqL3ArzW90jKUCf4H6xCK...

New comment by craigacp in "SD4J – Stable Diffusion pipeline in Java using ONNX Runtime"

craigacp — Mon, 01 Jan 2024 16:19:09 +0000

ONNX Runtime needed a good demo application in Java and this was fun to do (I maintain the Java API for ONNX Runtime and wrote this SD implementation). I've added SDv2 and SDXL support (and the turbo variants thereof) after the initial release, and I'll upgrade it to the latest ONNX Runtime when that comes out to get FP16 support among other things.

New comment by craigacp in "Persimmon-8B"

craigacp — Fri, 08 Sep 2023 00:30:32 +0000

There's a correction to that tweet, larger vocab means fewer tokens for any given sequence (usually, assuming it's not to add other languages or character sets).

New comment by craigacp in "ONNX runtime: Cross-platform accelerated machine learning"

craigacp — Wed, 26 Jul 2023 00:19:12 +0000

As ONNX models are protobufs you can edit them at a Python or Java REPL (or other language but I've personally used those two). Dumping them out as text seems like a lot more work, and a lot less typesafe.

New comment by craigacp in "Stochastic gradient descent written in SQL"

craigacp — Wed, 08 Mar 2023 02:13:41 +0000

We built model & data provenance into our open source ML library, though it's admittedly not the W3C PROV standard. There were a few gaps in it until we built an automated reproducibility system on top of it, but now it's pretty solid for all the algorithms we implement. Unfortunately some of the things we wrap (notably TensorFlow) aren't reproducible enough due to some unfixed bugs. There's an overview of the provenance system in this reprise of the JavaOne talk I gave here https://www.youtube.com/watch?v=GXOMjq2OS_c. The library is on GitHub - https://github.com/oracle/tribuo.

New comment by craigacp in "Understanding and coding the self-attention mechanism of large language models"

craigacp — Sat, 11 Feb 2023 01:32:33 +0000

A conventional neural network, i.e. one using a stack of dense layers, can't unroll across a sequence in the way the transformer does. So while it could compute the relative importance and interaction of the features it sees it wouldn't be able to compute that across arbitrary length sequences without a mechanism for the sequence elements to interact, which is what self attention provides.

New comment by craigacp in "JDK 20 will introduce float16 conversion methods"

craigacp — Thu, 13 Oct 2022 01:17:54 +0000

Exactly so, there are a bunch of Java interfaces to native ML libraries that have some messy bit hacking to convert between fp16 and Java floats. This API along with the intrinsification (https://github.com/openjdk/jdk/pull/10500) will mean I can throw away that code and replace it with something that performs correctly and is much faster.

New comment by craigacp in "An Introduction to Probabilistic Programming"

craigacp — Fri, 22 Oct 2021 22:07:36 +0000

Probabilistic programming can be done via MCMC approaches, but you can also infer the necessary quantities by using variational inference (which approximates the distribution described by your program with something that's simpler and easier to estimate).

Basically probabilistic programming is a way of describing a distribution, and then MCMC is one way of inferring the quantities in that distribution.

New comment by craigacp in "A categorized list of all Java and JVM features since JDK 8 to 16"

craigacp — Thu, 01 Apr 2021 20:15:48 +0000

There is extensive discussion of that topic in the valhalla-dev and valhalla-spec-experts mailing lists, but I think the details are still being worked out.

New comment by craigacp in "A categorized list of all Java and JVM features since JDK 8 to 16"

craigacp — Thu, 01 Apr 2021 18:02:35 +0000

There are JEPs out for parts of it now - Primitive Objects (which is the new name for value types) https://openjdk.java.net/jeps/401 and Unify the basic primitives with objects https://openjdk.java.net/jeps/402

New comment by craigacp in "Exploiting machine learning Pickle files"

craigacp — Fri, 19 Mar 2021 01:29:27 +0000

XGBoost also has a binary file format which is neither pickle nor json. It's shared across their Python, Java and R frontends and parsed by the C++ library.

New comment by craigacp in "The NeurIPS 2020 broader impacts experiment"

craigacp — Fri, 25 Dec 2020 04:17:56 +0000

The reviews for published NeurIPS papers for at least the past 5 years are available from each paper's page on http://papers.neurips.cc.