I love the style of this blog-post, you can really tell that Luke has been deep down in the rabbit hole, encountered the Balrog and lived to tell the tale.

I use a keyboard shortcut to start and stop recording and it will put the transcription into the clipboard so I can paste into any app.

It's a huge productivity boost - OP is correct about not overthinking trying to be that coherent - the models are very good at knowing what you mean (Opus 4.5 with Claude Code in my case)

I used to pride myself of knowing all the little ins and outs of the tech stack, especially when it comes to ops type stuff. This is still required, the difference is you don't need to spend 4 hours writing code - you can use the experience to get to the same result in 4 minutes.

I can see how "ask it for what you want and hope for the best" might not end well but personally - I am very much enjoying the process of distilling what I know we need to do next into a voice dictated prompt and then watching the AI just solve it based on what I said.

It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else.

Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum).

His answer was "a few days".

What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.

We used https://github.com/pgvector/pgvector under the hood and found it extremely easy to integrate with our database schema - being able to just specify the structure of a table and have metadata fields alongside the embeddings made the code very easy to reason about.

> Finetune Mistral, Llama 2-5x faster with 70% less memory!

Could be very useful for us!

Disclaimer: I work on Helix

What we found was the IO latency for loading model weights into VRAM will kill responsiveness if you don't "re-use" sessions (i.e. where the model weights remain loaded and you run multiple inference sessions over the same loaded weights).

Obviously projects like https://github.com/vllm-project/vllm exist but we needed to build out a scheduler that can run a fleet of GPUs for a matrix of text/image vs inference/finetune sessions.

disclaimer: I work on Helix

We are also adding function calling so the model would know to reach out to an external API to fetch some data before generating a response.

disclaimer: I work on Helix

> Also, I don’t get the comparison of rag vs finetuning in articles like this - why not do both

It's interesting you say this because we are very close to adding RAG support to Helix sessions and it will be "both at the same time" not an "either or" setup. You can choose to do either or but we are interested in seeing if doing both at the same time yields better results than either or - watch this space!

disclaimer: I work on Helix

> I often wonder how you'd go about organizing training data for a full historic github repo in a way that makes sense for training (or RAG)?

This is the hard part :-) But you are right - it would be intriguing to see what the output of a fune-tuned & RAG model would look like for this use-case. We are currently experimenting with adding RAG alongside the fine tuned model (so it's both, not either or) to see if it produces better results.

I will make sure we take a look at the gihub repo use case because it feels like that would be an interesting experiment to do!

disclaimer: I work on Helix

It would be possible to include some parts of the new documents in the prompt so you can answer questions about new facts in the style and tone of your old documents, which we feel is useful. We are also experimenting with adding Retrieval Augmented Generation alongside fine tuning to see if the results are better than either or.

disclaimer: I work on Helix

So yes, I also thought my Internet was broken :-)

Comments URL: https://news.ycombinator.com/item?id=27771257

Points: 3

# Comments: 2

You are quite right that if you want to quickly publish a single page - using Google docs directly would be faster :-)

Thanks for giving it a test!

Yes - dealing with access to a users entire drive is something we are looking at to try and get working for exactly that reason - the words `this app can delete all your files` is scary :-)

There isn't an option to remove the footer but should be +1 added to backlog - thanks again!

We do have a template system though so it would be possible for a frontend dev to design their own wrapper for the content.

We think it's perfect for documentation for non-technical companies who don't know what Markdown is.