https://github.com/python/cpython/issues/135824#issuecomment...

`x in range(n)` is already optimized, but that was easier since the `__contains__` method already existed, but an equivalent `__min__` or `__max__` does not.

- s[i:j] is O(j - i) because it creates a copy instead of a view

- max(range(n)) is O(n)

- substring search is O(n), which is good, but rfind is O(n m)

- iterative string concatenation (for c in ...: s += c) can be O(n^2) due to string immutability according to footnote 10, although it is O(n) in most cases due to an implementation detail of CPython: https://stackoverflow.com/a/34008199

For more LLM-specific stuff, you can pick some agent trace dataset on https://huggingface.co/datasets?format=format%3Aagent-traces and check out what people are doing with it (usually linked on the right when you click on a dataset).

And of course https://scholar.google.com/ for research papers.

I am not sure how they are using the data for post-training, but there probably are ways to get signal out of it, e.g. sentiment analysis when the user begins cursing at the agent, or checking whether the user continued another session with the generated code, or started a new session with the same starting point as before, i.e. they git-stashed.

Generally, you can train on data that is quite bad (e.g. the entire internet). It will still work, but take much longer compared to clean data.

I think the margins of DeepSeek may be a bit better than with this vibe-coded framework here, since they had the liberty of optimizing their models for their own hardware.

For DeepSeek V3, they claimed a cost profit margin of 545%: https://github.com/deepseek-ai/open-infra-index/blob/main/20...

At the time, open frameworks were not anywhere close to achieving that number. Not sure whether they caught up. The software wizards at DeepSeek are quite skilled.

Lets assume that you've got 2 million input tokens, 1 million output tokens and 98.5 million cached tokens to process. That would cost 2 * $0.14 + 1 * $0.28 + 98.5 * $0.0028 = $0.8358 with DeepSeek API pricing.

For comparison, it would take 2M / 8000 + 1M / 800 = 1500 seconds to process this amount of tokens with the linked framework, which is about $0.83 when we assume $2/hr for one MI300X.

However, other inference providers have 10 times higher prices for cached tokens, which results in a comfortable margin.

And we should not discount that DeepSeek also gets paid in data, which is probably more valuable to them.

And I believe that this framework still has some room for optimization for generation with high batch sizes.

I guess they were using ollama, which does not tell you where it puts the models it downloads.

    Traceback (most recent call last):
      File "/app.py", line 1, in 
        import spider
    ModuleNotFoundError: No module named 'spider'

1. Visit https://www.codembark.com/

2. Click on "Start Coding Right Now"

3. Write "import spider" into app.py in the right panel

4. Click the Play button in the bottom middle panel.

(Also, I know Tom Jobbins (TheBloke), and have personally contributed to increase the adoption of GGUF, e.g. in the transformers and ktransformers libraries, so I find the personal dig quite amusing.)

Unfortunately, electricity prices did not fall by the same factor, so I fear that training a frontier model will still cause a an unsustainable dent in my monthly budget.

I sure wish I had a few 100M of disposable income to train a frontier model.

> or in the future it could be useful when training is cheaper.

I do not think that physics will allow hardware getting that much faster. But maybe we will have different, cheaper architectures by then.

(Kimi K3 tech report section 4.1.1 https://arxiv.org/pdf/2607.24653)

A few quotes from the transcript:

> Our current computing capacity is approximately 20,000 H-equivalent units, most of which have just arrived within the past month or two

> Regarding the Huawei 950, Huawei currently provides us with 16,000 SIM cards

> A Huawei 950 [cluster] with 16,000 cards is equivalent to only a B-series card [cluster] with 4,000 cards.

> because we added the new deepseek which we do not yet have a ZDR with we cannot blanket say we offer ZDR

I wonder how the website can make the statement that data will not be used for training.