https://sqlite.org/rsync.html

This explains the final 2 sentences of the original Linux announcement:

> PS. Yes - it's free of any minix code, and it has a multi-threaded fs. It is NOT portable (uses 386 task switching etc), and it probably never will support anything other than AT-harddisks, as that's all I have :-(.

The book publisher is blamed for preventing Minix from being freely distributed: https://en.wikipedia.org/wiki/Minix#Licensing

It is notable that ARM designed explicit co-processor instructions, allowing for 16 co-processors. They must have taken the 8086/8087 approach into account when doing that.

I used to use Nikola, but gave up on that for two reasons. One was it was adding every possible feature which meant an ever growing number of dependencies which gets scary. And that is hard to use - eg how do you embed a youtube video becomes a trawl through documentation, plugins, arbitrary new syntax etc.

But the biggest problem and one that can affect all the SSG is that they try to do incremental builds by default. Nikola of the time was especially bad at that - it didn't realise some changes mattered such as changes to config files, themes, templates etc, and was woolly on timestamps of source files versus output files.

This meant it committed the cardinal sin: clean builds produce different output from incremental builds

Pelican has kept it simple.

For those not familiar with TCL, the C API is flavoured like main. Callbacks take a list of strings argv style and an argc count. TCL is stringly typed which sounds bad, but the data comes from strings in the HTML and script blocks, and the page HTML is also text, so it fits nicely and the C callbacks are easy to write.

[1] Mosaic Netscape 0.9 was released the week before

And it also allows async functions, since state is held off the C stack, so frames can be easily switched when returning to the event loop.

The other thing made easy is C extension authoring. You compile CPython without free lists and an address sanitizer, and getting reference counting wrong shows up.

1) Use regular GIL Python and you get the highest levels of integrity and correctness of operation of my package

2) Use a free threaded Python, the GIL will be enabled at load time, and you get the highest levels of integrity and correctness

3) Use a free threaded Python, and set $PYTHON_GIL=0 or start with -Xgil=0 to keep the GIL disabled, and providing you do not do concurrent mutation of data provided to my package, you get the highest levels of integrity and correctness

BTW I did not randomly choose to provide the free threaded builds. I specifically asked the setuptools maintainers (under the Python Packaging Authority) how to prevent free threaded builds for PyPI. They encouraged me to do the free threaded builds so that a user doesn't to have maintain parallel regular and free threaded Python installations. And it allows option 3 above.

1) Saying your package supports free threading, but it isn't safe - ie concurrent mutation can result in corruption and crashes

2) Allowing the package to be loaded into a free threaded Python, which immediately enables the GIL. Concurrent mutation does not result in corruption and crashes because of the GIL. The user doesn't have to maintain two Python installations. They can set the environment variable PYTHON_GIL=0 or start Python with -Xgil=0 which will keep the GIL disabled, and they will be fine if they avoid concurrent mutation.

I chose 2. The stdlib json package (along with many others) picked 1. Heck I'll guarantee that most that picked 1 aren't 100% safe either, because doing the changes is hard work, *every* case has to be covered, and tools like thread sanitizers don't work.

The reason I chose 2 is because I care about data integrity. I will eventually reach 1, but only once I can be certain the code is correct.

Is it madness that other packages claim they support running without the GIL, yet it is possible to cause corruption and crashes just by writing concurrent Python code? That is the case with the standard library. Compiler thread sanitizers don't work with free threaded Python. Diligent code inspection by humans is the only way to update C code so far.

Free threading is at the beginning of the project. It works. You get slow downs in single threaded performance due to extra locking, and speedups in concurrent performance due to threading. But it is possible to cause corruption and crashes via Python code. Don't expose it to untrusted data and code.

But do investigate it and see what works well and what doesn't. See what code patterns are now possible. Help improve the tools and documentation. It had to start somewhere, and the current state is somewhere.

> so that you don't have to keep separate GIL full and free threaded interpreters around

It means the user doesn't have to keep two Pythons around, install packages in both of them, etc.

It is also possible with the free threaded Python to keep the GIL disabled even if a package such as mine says it needs the GIL. And my package will indeed work just fine, until you supply it with mutable data and concurrently modify it in another thread.

This is the case with my own package which is on the hugovk list (apsw) which will cause the GIL to be re-enabled if you load it into a free threaded Python. The reason I provide a binary wheel is so that you don't have to keep separate GIL full and free threaded interpreters around. They have a different ABI so you can't use extensions compiled against one with the other.

Free threading is at the beginning of its journey. There is a *lot* of work to on all C code that works with Python objects, and the current documentation and tools are immature. It is especially the case that anyone doing Python concurrent object mutation can cause corruption and crashes if they try, and that more auditing and locking need to be done in the C code. Even modules in the standard library have only been partially updated.

You can see a lot details and discussion in the comments at https://news.ycombinator.com/item?id=45633311

The locking is all about reading and writing Python objects. It is not applicable to outside things like external libraries. Python objects are implemented in C code, but Python users do not need to know or care about that.

As a Python user you cannot corrupt or crash things by code you write no matter how hard you try with mutation and concurrency. The locking ensures that. Another way of looking at Python is that it is a friendly syntax for calling code written in C, and that is why people use it - the C code can be where all the performance is, while retaining the ergonomic access.

C code has to opt in to free threading - see my response to this comment

https://news.ycombinator.com/item?id=45706331

It is true that more fine grained locking can end up being done than is strictly necessary, but user's code is loaded at runtime, so you don't know in advance what could be omitted. And this is the beginning of the project - things will get better.

Aside: Yes you can use ctypes to crash things, other compiled languages can be used, concurrency is hard

* Point out using APIs that return borrowed references

* Suggest assertions that critical sections are held when operating on objects

* Suggest alternate APIs

* Recognise code patterns that are similar to those done during the stdlib auditing work

The compiler thread sanitizers didn't work the last time I checked - so get them working.

Edit: A good example of what can be done is Coccinelle used in the Linux kernel which can detect problematic code (locking is way more complex!) as well as apply source transformations. https://www.kernel.org/doc/html/v6.17/dev-tools/coccinelle.h...

The free threaded implementation adds what amounts to individual object locks at the C level (critical sections). This still means developers writing Python code can do whatever they want, and they will not experience corruption or crashes. The base objects have all been updated.

Python is popular because of many extensions written in C, including many in the standard library. Every single piece of that code must be updated to operate correctly in free threaded mode. That is a lot of work and is still in progress in the standard library. But in order to make the free threaded interpreter useful at this point, some have been marked as free thread safe, when that is not the case.

You have to explicitly compile the extension against a free threaded interpreter in order to get that ABI tag in your extension and even be able to load the extension. The extension then has to opt-in to free threading in its initialization.

If it does not opt-in then a message appears saying the GIL has been enabled, and the interpreter continues to run with the GIL.

This may seem a little strange but is helpful. It means the person running Python doesn't have to keep regular and free threaded Python around, and duplicate sets of extensions etc. They can just have the free threaded one, anything loaded that requires the GIL gives you the normal Python behaviour.

What is a little more problematic is that some of the standard library is marked as supporting free threading, even though they still have the audit and update work outstanding.

Also the last time I checked, the compiler thread sanitizers can't work with free threaded Python.

https://docs.python.org/3/howto/free-threading-extensions.ht...

And a dedicated web site:

https://py-free-threading.github.io/

But as an example neither include PySequence_Fast which is in the json.c changes I pointed to. The folks doing the auditing of stdlib do have an idea of what they are looking for, and so would be best suited to keep a list (and tool) up to date with what is needed.