Hacker News: ngoldbaum

New comment by ngoldbaum in "Scaling NumPy on Free-Threaded Python"

ngoldbaum — Fri, 07 Aug 2026 13:07:42 +0000

I’m saying that it’s a new issue on the free-threaded build.

New comment by ngoldbaum in "Scaling NumPy on Free-Threaded Python"

ngoldbaum — Thu, 06 Aug 2026 11:14:34 +0000

There’s also the fact that INCREF and DECREF on shared objects is a lot more expensive than plain integer addition, so stuff becomes a bottleneck that was never a bottleneck. Kumar also fixed a bottleneck caused by a lock added only for safety on the free-threaded build. It’s hard to tell in advance than a fancy lock-free data structure is needed for something.

New comment by ngoldbaum in "Scaling NumPy on Free-Threaded Python"

ngoldbaum — Thu, 06 Aug 2026 11:11:30 +0000

I’m not sure why it took me, a NumPy developer, looking at the benchmark numbers and saying “hmm, this is a bug”. But that is what it took. People are sometimes slow to treat behavior in dependencies like NumPy as bugs.

Scaling NumPy on Free-Threaded Python

ngoldbaum — Thu, 30 Jul 2026 16:09:39 +0000

Article URL: https://labs.quansight.org/blog/scaling-numpy-on-free-threaded-python

Comments URL: https://news.ycombinator.com/item?id=49112023

Points: 114

# Comments: 24

New comment by ngoldbaum in "What Every Python Developer Should Know About the CPython ABI"

ngoldbaum — Sun, 26 Jul 2026 17:22:08 +0000

I’m glad you enjoyed the post. It really helped me to crystallize my understanding. I was also confused about the API/ABI distinction for a long time.

New comment by ngoldbaum in "What Every Python Developer Should Know About the CPython ABI"

ngoldbaum — Sun, 26 Jul 2026 17:17:03 +0000

Thanks for pointing that out, I’ll fix that.

What Every Python Developer Should Know About the CPython ABI

ngoldbaum — Fri, 10 Jul 2026 17:18:48 +0000

Article URL: https://labs.quansight.org/blog/python-abi-abi3t

Comments URL: https://news.ycombinator.com/item?id=48862673

Points: 2

# Comments: 0

Scaling Asyncio on Free-Threaded Python

ngoldbaum — Thu, 11 Sep 2025 13:02:58 +0000

Article URL: https://labs.quansight.org/blog/scaling-asyncio-on-free-threaded-python

Comments URL: https://news.ycombinator.com/item?id=45211102

Points: 2

# Comments: 0

New comment by ngoldbaum in "GitHub was having issues"

ngoldbaum — Tue, 12 Aug 2025 15:27:35 +0000

I wonder why the github status page has an atlassian cookie request pop-up.

Python Free-Threading Guide

ngoldbaum — Wed, 23 Jul 2025 12:14:50 +0000

Article URL: https://py-free-threading.github.io/

Comments URL: https://news.ycombinator.com/item?id=44658334

Points: 2

# Comments: 0

The first year of free-threaded Python

ngoldbaum — Tue, 13 May 2025 19:47:35 +0000

Article URL: https://labs.quansight.org/blog/free-threaded-one-year-recap

Comments URL: https://news.ycombinator.com/item?id=43976886

Points: 16

# Comments: 1

New comment by ngoldbaum in "Ty: A fast Python type checker and language server"

ngoldbaum — Thu, 08 May 2025 14:46:57 +0000

Either something about beanie babies or something riffing on "thank you". Couldn't ever make up my mind then basically forgot about it.

New comment by ngoldbaum in "Ty: A fast Python type checker and language server"

ngoldbaum — Wed, 07 May 2025 20:10:23 +0000

I gave away the “ty” project name on pypi to Astral a week or so ago. I wanted to use it for a joke a few years ago but this is a much better use for a two letter project name. They agreed to make a donation to the PSF to demonstrate their gratefulness.

New comment by ngoldbaum in "Branchless UTF-8 Encoding"

ngoldbaum — Fri, 17 Jan 2025 21:47:19 +0000

You could do two passes over the string, first get the total length in bytes, then fill it in codepoint by codepoint.

You could also pessimistically over-allocate assuming four bytes per character and then resize afterwards.

With the API in the linked blog post it's up to the user to decide how they want to use the output [u8;4] array.

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Thu, 24 Oct 2024 13:43:12 +0000

The main difference is the strings are stored in a single contiguous arena buffer (with some minor caveats if you mutate the array in-place). With object strings each string has its own heap allocation.

More details in NEP 55: https://numpy.org/neps/nep-0055-string_dtype.html

This post is based on the content of a 25 minute talk and it’s hard to explain everything fully…

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Thu, 24 Oct 2024 13:40:34 +0000

Ah I could see how that’s confusing. I was trying to indicate that the size stored for the string in the example is 28, but it’s stored in a 64 bit uint.

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Thu, 24 Oct 2024 13:37:30 +0000

They’re stored on the DType instance. This requires that there’s only one DType instance per “owned” array buffer, which I figured out how to do along with Sebastian Berg and others using the new DType system.

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Thu, 24 Oct 2024 00:23:06 +0000

That is something I’d like to see but I don’t want to wade into the already very complicated discussion around arrow strings in pandas. If a Pandas developer wanted to take this on I think that would make things easier since there’s so much complexity around strings in Pandas.

That said there is a branch that gets most of the way there: https://github.com/pandas-dev/pandas/pull/58578. The remaining challenges are mostly around getting consensus around how to introduce this change.

If NumPy had StringDType in 2019 instead of 2024 I think Pandas might have had an easier time. Sadly the timing didn’t quite work out.

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Wed, 23 Oct 2024 20:41:49 +0000

This was a case of convergent evolution, both projects ended up working simultaneously on similar ideas.

One issue with using Arrow directly in NumPy is PyArrow exposes an immutable 1D array, while NumPy exposes a mutable ND array.

New comment by ngoldbaum in "My NumPy year: Creating a DType for the next generation of scientific computing"

ngoldbaum — Wed, 23 Oct 2024 20:39:52 +0000

Well, there was no concept of sidecar storage. Now we have the hack we came up with for StringDType to store data on the DType instance and also make it so StringDType arrays don't share StringDType instances, unless the array is a view.

EDIT: looking back at the NEP, I'm not sure it does a great job explaining exactly how the per-array descriptor works. Ultimately it's powered by a hook in the DType API: https://github.com/numpy/numpy/pull/24988. There is only one spot in NumPy where array buffers are allocated, so we hooked there and made sure any arrays with newly allocated buffers get a new DType instance.