It's a single git project at my $USER home, that is referenced in global memory. It contains as much information about work things, as possible, to be productive.

I found that, if I allowed Claude to create the notes, it actually became more and more useful, but without the guideline, I just could barely get through reading it manually.

I'd never publish anything with such origin.

I'm asking to scan projects on gitlab, go through some docs to find more grounding material, write a subarticle (in the same style), scan logs on the test env, issue some curls, etc.; until the whole article is digestible - in the "backing knowledge graph" department.

I used "CD era MSDN reference and Raymond Chen blogging style" as a starting prompt for the styleguide and my work ability to digest AI plans raised a lot.

Couldn't recommend it more. Humble, insightful and respecting the reader

One for lay-offs, because it was the best move at the time with the knowledge they had.

Second for quick correction, ability to pivot and execute quickly.

It's been always like that

I'm pretty sure it's mostly due to the training data quality. No idea, why this never gets mentioned in those discussions.

It was obvious right from the get go, that the scaling law just enabled some abilities, that were described by the underlying data and allowing the ANN to abstract it in the latent space.

https://github.com/noonghunna/club-3090

You can find info about running a patched version of vllm for 1x24gb, 2x and 4x. There's also quite a few "blackwell" subreddits, where people seem to share a lot of substantial information, if you're going the 6000 route.

later:

> My latest experiment was setting up vLLM (the gold standard for production and concurrent serving) and even with an NVLink (175GBP) and tensor parallelism turned on, it was 3 tokens/second slower than llama.cpp during generation for an equivalent setup.

In all my tests, getting vllm to run is worth it. It was the single biggest thing, that helped for looping issues, agents going whack and losing focus on the task, long context being essentially useless.

FP8 model, unquantized cache in vllm an you have a league better overall experience, with any other stack I tested. Then, you can actually focus on using the model for other things and stop tinkering with settings.

Something is up

I'm years after that strict diet needs, but that itch of fixing or easing some parts of the process stayed.

The moment I'm trying something open-ended or ambitious, Claude/ChatGPT clearly take you to the goal quicker.

For things, where there's a way to build a knowledgebase though, the local llm definitely can be a true contender. Plus, having a big context and no worries about filling it over and over - you can get quite far.

I'm writing this, literally in between cooking a pasta, that the local llm ordered products for me online. I've built a grocery shopping skill, so that it roughly knows what I have in fridge (losely), my last 10 representative orders (general preferences plus rich info about shops and skus around me) and actual real-time in stock info. The last part has been my personal pet peeve for every product that promised cooking ingredient delivery (that is not packaged specifically for that).

This is what has been promised to us by every big tech company with an agent, and now a local llms actually solved that for me fully.

The biggest observable improvement in my post and pre ai development is the ability to tackle two projects at once, if the agent is on track. If not, and I have to do a deep dive to debug, it basically regresses to plain old everything like before.