These kind of papers often talk the world, but often lack a proper baseline model. They only compare against very simple (naive forecast), or non tuned models. In my experience a gradient boosting model will probably solve 95% of your forecasting problems, and trying to get fancy with a transformer (or even just a simple neural net) is more trouble then it is worth.

The Netherlands actually is second place in terms of solar generation per capita in the world (only Australia has more).

[1] https://pytorch-geometric.readthedocs.io/en/latest/

If you are looking for an actual timeseries method I would checkout either darts [0] or statsforecast [1]. They are currently the most mature timeseries packages.

[0] https://unit8co.github.io/darts/ [1] https://github.com/Nixtla/statsforecast

For instance if I have some body of text that can't be found elsewhere on the internet, if the reply of the model references the information in that text in some way you may be fairly certain it was used in training.

The hard part is probably finding such a body of text.

I am currently trying to put together a small analog synthesizer.

My take on why they have build the output layer like it is, is that next to feeling more human, it also forces you to be a bit more thoughtfull with your requests, and thus spam the system less. In the end it is still really expensive to run these models..