Try using a LLM model for RAG embeddings and get back to us on that.

I have never had that issue, and LLMs are used for more than just code generation.

> And 3090 is $1500 used

Second hand Macs are also cheaper.

Not sure what your point is, but you just seem to arguing because of your hatred of Macs.

Back in early 2000's I was commissioned to build an application that should also run on the Mac.

I bought a second hand old Mac Mini. Within a couple of months I bought a brand new one, and since then I have been on Mac when it comes to non-gaming (even though it can do gaming).

The main reason was I could just get shit done. The operating system is transparent to what I needed to do. With Windows you were forever dealing with the operating system to solve things.

Windows eventually got better in this regards, and more recently worse again.

I own/use windows as well. But mainly for gaming or a VM on an internal server.

Often when people give a comparison of how much PC is better, they compare a PC desktop to a laptop. Windows laptops for the same features is same or more expensive, while feeling cheap/heavy/noisy.

So form factor is one part of it, and then how the whole ecosystem works together with my phone/iPad and other devices.

At the end of the day, if I am doing any serious inferencing then I use my companies server, or Claude.

...

What has amazed me over the years is the sheer hate I get from some people for even owning a Mac. I was probably the same, but I was younger then.

Its died down since ARM came in, but the irrational hate seems wasted. Someone prefers Linux, Android, Windows, etc. Good for them.

I went for a job in the US back in the late 80's (Specialist on hardware).

California it was exceptionally strict. While I was more than qualified it took months of the company advertising in the US, and at the end of it all an American was hired.

The second job for another company in California, same deal except they only advertised in the local newspaper (which felt off), and when I finally got an interview they said they couldn't hire me but recommended places in Mexico to jump the border if I ever wanted a job. Noped out of that fast.

Did eventually get to a spend a couple of years in the US in the 90's. It was the same hoops to jump through as the first company. Wasn't easy or fast.

No idea what it is like now. But I had seen the racism of "They are an immigrant, so they were only hired because we can pay them cheap", not as bad as my Indian co-workers though.

Volume of documents, size of documents, versioning, frequency of update, documents similar or overlapping information, how much or exactly what you need for the LLM to understand, AI friendly documents, who has access and at what level, blue teaming, red teaming, multi-lingual, does the LLM know the domain language of the user and documents.

I probably missed a few things even with that.

But this is an old argument that seems to be thrown at Macs. Normally everything is cherry picked to show it's better than the Mac while ignoring all the features.

Although I would hope if you pay $8K you can get something that outperforms the Mac in every way.

You mean someone who uses a Mac? I find the term "Apple fan" is used as a way to attack the person.

> that can run full agentic loops, you may as well just use cloud inference for the price.

You can run full agentic loops fine with quantised models. In fact that is likely what you are doing with a 32GB PC Graphics card. Or what model are you using?

You use the right model for the right job. For example granite4.2 is optimised for agentic work and only needs 5GB of memory.

Gemma4 MLX runs fine with the larger model needing 19GB.

Prior to that I had OpenClaw (in Parallels VM) create an application with local models that worked the exact same way as created by Claude. It was more an experiment in how OpenClaw works, hence the VM.

> And 57 gb is split across 3 cards quite easily,

I assume you are talking about a good graphics card. A good 32GB will run you $2K a card, so that $6K to beat out a laptop of similar price, and where the difference doesn't matter.

[edit]

Anyway my main point is Macs work fine for local models. I have a 6 year old machine that proves that.

Tend to be better than Apple AI.

I have an old M1 Max with 32GB that runs inference models fine, especially those using MLX.

Dedicated graphics cards are faster but not to the point that it matters.

That’s a six year old machine.

My newer machine M5 max 128GB will far outperform your typical 32GB gaming card once the model exceeds memory.

To even get close to that on a dedicated graphics card you are paying upwards of $20K.

That said, models are getting smaller and faster which allows me to run multiple different models with ease.

[edit] Just checked the 5 models I use take up 57GB when all loaded at the same time.

Now the senior can use the AI instead of dealing with a junior.

Plus the biggest danger I’ve seen numerous times is the junior engineer can’t easily tell if the AI is wrong.

They either accept it as correct or spend more time trying to fix AI output with an AI in an area they don’t have the skills for.

Related patent: https://image-ppubs.uspto.gov/dirsearch-public/print/downloa...

As long as you are using red squiggles instead of blue you should be fine though.

IMHO though, they should say GenAI as AI is too broad.

But Claude can do this with little prompting.

[edit] Not sure why the downvote, but Claude Design can do this, its just a matter of knowing the exact prompt.

This was created with just the prompt on from their front page and nothing else.

https://imgur.com/2pGvVcl

That's literally a cut+paste. If I had spent time on it, you can get it close to what's shown.

Claude also created that as a Blender file with assets for each of the generated items.

It's a broad statement and can mean different things depending where you are in that chain.

You have design and compliance. Compliance is what you are allowed to do (laws). Design is how you construct your applications to understand how it will impact the people directly or indirectly.

Then you have the accountability, transparency, auditing and reproducing. Understanding why the model worked the way it did. Models can go wrong, but if you have the details of how it went wrong and who is at fault, it can protect people who use it.

Then there is alignment, which is a higher longer goal.

Your comment about security questions is a matter of ethics as well.

If you say apply it to medical, is it ethical to allow a model to give medical advice, knowing that it can be wrong. Most people would say no, but by doing so you are denying people who can't afford medical advice, so there has to be a balance. Most companies err on the side of not getting sued.

I agree with you, but then we have HR for a reason.

Is this your opinion, or do you have something to back it up?

My comment is anecdotal, but everyone I know who works in AI Ethics are not marketing people.

They are data scientists who have moved into the role, as ethics in data has been around long before GenAI. They have to be familiar with the laws around AI and how its going to impact their products. One of them is closer to sort of "HR for AI", in that the focus is not getting the company sued for ethical breaches.

If any of them left, it would be because the company in question is ignoring their direction and they don't want to be there when the shit hits the fan. Plus very few skilled people want to be in a role where they have no control (even if the pay is good).

If OpenAI are hiring marketing people for the role (which I strongly doubt) that would be more troubling IMHO.