Some of my use cases are very latency sensitive. What sort of overhead are you seeing?

So much content is just straight copy/pasted from the LLM now. Articles, blog posts, linked in posts, reddit comments, etc. Even just using the LLM for 'editing' tends to shift the voice to an obvious LLM voice when used naively. It is getting worse too. Last week a co-worker sent me a screenshot of Claude for me to review their "work", which was just whatever Claude made up.

Usually, if something is very obviously unfiltered LLM output, I just stop reading.

I do use LLMs for writing myself. They are useful, but are poor authors.

When I was in school, decades ago now, very few people went into CS compared to other majors. Everyone I knew going into it did it because they loved it. I would have done it regardless of the career opportunities because I want to build stuff.

Interviewing candidates over the years since then, my experience has been there are still very few of those passionate nerds and a lot of people who did it for other reasons, like the money or similar. There is nothing inherently wrong with this. I don’t fault people for it.

Maybe if we get very lucky, it will go back to a relatively few passionate people building stuff because it is cool?

Something like this was always inevitable. I just hope it doesn’t ruin a good thing.

My model for skills is similar to this, but I extended it to have explicit use when and don’t use when examples and counter examples. This helped the small model which tended to not get the nuances of a free form text description.

Not the only reason of course.

We have done a similar operation routinely on databases under pretty write intensive workloads (like 10s of thousands of inserts per second). It is so routine we have automation to adjust to planned changes in volume and do so a dozen times a month or so. It has been very robust for us. Our apps are designed for it and use AWS’s JDBC wrapper.

Just one more thing to worry about I guess…

The biggest miss on our side is that, although we designed a multi-region capable application, we could not run the failover process because our security org migrated us to Identity Center and only put it in us-east-1, hard locking the entire company out of the AWS control plane. By the time we'd gotten the root credentials out of the vault, things were coming back up.

Good reminder that you are only as strong as your weakest link.

At my company, we only hire in India now and the executives are intentionally causing "attrition" in the US by running people into the ground with demands that amount to 996 style work.

The default install only binds to loopback, so I am sure it is pretty common to just slap OLLAMA_HOST=0.0.0.0 and move on to other things. I know I did at first, but my host isn't publicly routable and I went back the same night and added IPAddressDeny/Allow rules (among other standard/easy hardening).

On topic: I have watched every episode on TNG more than once and never noticed this. How embarrassing!