The point of Fine-tuning small models is for specific downstream tasks, which SOTA models can do, but at higher costs. It is purely an economic play, not an attempt at pushing boundaries of SOTA.

I want more blogs/discussion from the community about the existing tools.

In 3/6 months, how many skills have you written? How many times have you used each skill? Did you have edit skills later due to unseen corner cases, or did they generalize? Are skills being used predominantly at the individual level, or are entire teams/orgs able to use a skill as is? What are the usecases where skills are not good at? What are the shortcomings?

(You being the metaphorical HN reader here of course.)

HN has always been a place of greater technical depth than other internet sites and I would like to see more of this sort of thing on the front page along with day one calls.

Would strongly encourage you to open-source/write blog posts on some concrete examples from your experience to bridge this gap.

I would encourage you to write up a blog post of your experience and share a version of the skill you have built. And then follow up with a blog post after 3 months with analysis like how well the skill generalized for your daily use, whether you had to make some changes, what didn't work etc. This is the sort of content we need more of here.

3 months ago, Anthropic and Simon claimed that Skills were the next big thing and going to completely change the game. So far, from my exploration, I don't see any good examples out there, nor is a there a big growing/active community of users.

Today, we are talking about Cowork. My prediction is that 3 months from now, there will be yet another new Anthropic positioning, followed up with a detailed blog from Simon, followed by HN discussing possibilities. Rinse and Repeat.

This is something I have experienced first hand participating in the Vim/Emacs/Ricing communities. The newbie spends hours installing and tuning workflows with the mental justification of long-term savings, only to throw it all away in a few weeks when they see a new, shinier thing. I have been there and done that. For many, many years.

The mature user configures and installs 1 or 2 shiny new things, possibly spending several hours even. Then he goes back to work. 6 months later, he reviews his workflow and decides what has worked well, what hasn't and looks for the new shiny things in the market. Because, you need to use your tools in anger, in the ups and downs, to truly evaluate them in various real scenarios. Scenarios that won't show up until serious use.

My point is that Anthropic is incentivized in continuously moving goalposts. Simon is incentivized in writing new blogs every other day. But none of that is healthy for you and me.

I love org for all its bells and whistles and use them in various ways. But most of the time I need a small subset of org in a form-factor that allows ease of use.

[1] https://github.com/ChanderG/toodoo.el

I had been using dwm (4000 lines of C) for many years and wished to write my own for a long time, but what made me take the leap was really steveWM [2] and TinyWM [3] which are both super small.

[1] - https://github.com/ChanderG/cellwm [2] - https://github.com/stodd1031/steveWM [3] - https://github.com/mackstann/tinywm

When I got into BQN recently, I had the same thought and tried my hand at recreating LLM building blocks and saw the compactness APL-likes afford first-hand: https://www.chandergovind.org/blog/5126/aipasi/Array-program...

Comments URL: https://news.ycombinator.com/item?id=42057276

Points: 3

# Comments: 0

1. Software: this is all Pytorch/HF, so completely open-source. This is total parity between what corporates have and what the public has.

2. Model weights: Meta and a few other orgs release open models - as opposed to OpenAI's closed models. So, ok, we have something to work with.

3. Data: to actually do anything useful you need tons of data. This is beyond the reach of the ordinary man, setting aside the legality issues.

4. Hardware: GPUs, which are extremely expensive. Not just that, even if you have the top dollars, you have to go stand in a queue and wait for O(months), since mega-corporates have gotten there before you.

For Inference, you need 1,2 and 4. For training (or fine-tuning), you need all of these. With newer and larger models like the latest Llama, 4 is truly beyond the reach of ordinary entities.

This is NOTHING like open-source, where a random guy can edit/recompile/deploy software on a commodity computer. Wrt LLMs, Data/Hardware are in the equation, the playing field is complete stacked. This thread has a bunch of people discussing nuances of 1 and 2, but this bike-shedding only hides the basic point: Control of LLMs are for mega-corps, not for individuals.

Stories/memes/narratives are the most easy/potent form of mental ingestion - so much so that I think humans cannot ingest facts or ideas at all, only stories. And this puts a collective responsibility on all of us to be very careful about the stories we create.