At my day job we have experimented with fine tuned transformers for our receipt processing workflow. We take images of receipts, run them through OCR (this step might not even be necessary, but we do it at scale already anyways), and then take the OCR output text blobs and "transform" them into structured receipts with retailer, details like zip code, transaction timestamps, line items, sales taxes, sales, etc.

I trained a small LLM (mistral-7b) via SFT with 1000 (maybe 10,000? I don't remember) examples from receipts in our database from 2019. When I tested the model on receipts from 2020 it hit something like 98% accuracy.

The key that made this work so well is that we had a ton of data (potentially billions of example input/output pairs) and we could easily evaluate the correctness by unpacking the json output and comparing with our source tables.

Note that this isn't running in production, it was an experiment. There are edge cases I didn't consider, and there's a lot more to it in terms of accurately evaling, when to re-train, dealing with net new receipt types, retailers, new languages (we're doing global expansion RN so it's top of mind), general diversity of edge cases in your training data, etc.

I did an experiment where I did very simple SFT on Mistral 7b and it was extremely good at converting receipt images into structured json outputs and I only used 1,000 examples. The difficulty is trying to get a diverse enough set of examples, evaling, etc.

If you have great data with simple input output pairs, you should really give it a shot.

Interesting snippet towards the end. I wonder if they were using claude.ai or claude code. Sounds like they ran out of context and entered the "dumb zone."

https://duckdb.org/2025/05/19/the-lost-decade-of-small-data

Also, thanks for choosing a technical blog post for presenting this information.

The app is beautiful - much better than I could build - what tech is it using if you don't mind me asking? Is it flutter, react native, something else? Just want to get better at mobile dev.

> Prometheus stole fire from the gods and gave it to man. For this he was chained to a rock and tortured for eternity.

Any internal webapps I need to build like this will 100% be set up with django in the future due to this. I don't need it to be pretty, I just want the UI, database migrations, users, roles, groups, etc for free

You have to tell it about the backstory. It does not know unless you write about it somewhere and give it as input to the model.

There was probably initial excitement about not having to manually break the earth, then stories spread about farmers ruining entire crops with one tractor, some farms begin touting 10x more efficiency by running multiple tractors at once, some farmers saying the maintenance burden of a tractor is not worth it compared to feeding/watering their mule, etc.

Fast forward and now gigantic remote controlled combines are dominating thousands of acres of land with the efficiency greater than 100 men with 100 early tractors.

I didn't make it past the tldr lol is this some kind of poisoned data for GPT 6?

It was a linux user's dream. I swapped out the spinny HDD with an SSD. It had hotswappable batteries, a spill resistant keyboard with a drain system, and I ran every linux distribution I could get my hands on, eventually settling on the xfce flavor of Ubuntu.

I still have that machine, and will keep it for my son when he's older. I plan to replace the battery and SSD with something more reliable. These days I use an m2 macbook air for portability... but that thinkpad is still something I will treasure for years.

I'm also not building webapps. I work in data engineering on a large legacy airflow project, internal python libraries, infrastructure with terraform, etc.

Assume you're writing code manually, and you personally make a mistake. It's often worthwhile to create a mechanism that prevents that class of mistake from cropping up again. Adding better LSP or refactoring support to your editor, better syntax highlighting, better type checking, etc.

That same exact game of whack a mole now has to be done for you and whatever agent you're building with. Some questions to ask: What caused the hallucination? Did you have the agent lay out its plan before it writes any code? Ask you questions and iterate on a spec before implementation? Have you given it all of the necessary tools, test harnesses, and context it needs to complete a request that you've made to it? How do you automate this so that it's impossible for these pieces to be missing for the next request? Are you using the right model for the task at hand?