So why did I write yet another LLM prompting library?

I found the existing Ruby ones either too abstract — hiding the LLM’s capabilities behind unseen prompts, too low-level — leaving my classes hard to follow and littered with boilerplate managing prompts and responses, or they used class level abstractions — forcing me to create classes when I didn’t want to.

After reading an early version of Patterns of Application Development Using AI by Obie Fernandez and using Obie’s library raix, I felt inspired. The book has many great patterns, and raix’s transcript management and tool management were the first I’d used that felt ruby-ish. At the same time libraries in the python community such as guidance, DSPy, LangSmith, and TEXTGRAD had caught my eye. I also liked what the cross-platform BAML was doing too. I didn’t love the code generation and freemium aspects.

So, with motivation high, I set out to build an opinionated library of gems that improves my Ruby (and Rails) LLM developer experience.

The first gem (this one) is instruct. It is the flexible foundation that the other gems will build on. While the API is similar to guidance, it has a different architecture based around attributed strings and middleware which enables some unique features (like async guard rails, content filters, self-healing, auto-continuation, and native multi-modal support).

I’m currently working on a hopefully elegant API that makes requesting and handling streaming structured output easy (taking inspiration from BAML, but with automatic upgrades to json schema if the API supports it). Along with that, I’ve been working on a conversational memory middleware that automatically prunes historic irrelevant bits of the conversation transcript. I hope this keeps the model more steerable, but without loss of crucial details.

Thanks in advance for taking a look and providing any constructive feedback or ideas. Lastly, if you’re interested in contributing, please message me.

Comments URL: https://news.ycombinator.com/item?id=42470719

Points: 4

# Comments: 0

The saddest thing is that this project has been once of the most demoralizing I've ever taken on. Day to day we see so many students being failed by teachers and school leadership who care more about "adapting to AI" than real student outcomes today.

In practice, we've found teachers don't generally want to have the difficult conversations with students when the hard evidence of cheating is given to them.

And generally school/university/college leadership have no real tactics to implement their "AI strategy" other than train their own chat bots (wtf) and "adapting assessment to use AI".

Unfortunately, a simple non AI fix to the problem is definitely not as good for their careers.

IMHO without a change, we're creating a pretty bleak future for students of the next few years.