what model can i run on 1TB and how many tokens per second ?

for instance Nvidia Nemotron Llama 3.1 quantized at what speed ? ill get a GPU too but not sure how much VRAM I need for the best value for your buck

What should I do now? What stuff should I run?

I have a hello world Flask app running but obviously its not enough to use the full potential.

I'm thinking of running KVM and selling a few VDS to friends or companies.

Also thought of running thousands of Selenium browser tests but I do this maybe once a year, not enough to fully utilize the server 24/7

Help! I might have gone overboard with server capacity, I will never have to pay for AWS again, I can literally run every single project, APIs, database I want and still have space left over.

Comments URL: https://news.ycombinator.com/item?id=41871843

Points: 24

# Comments: 38

Piggybacking on OpenAI or any GPT is not at all the same as SQL

VDS seems to go up in cost when you go beyond 32GB with a decent CPU

amen to the depreciation but methinks 7950x + 512gb of DDR5 RAM should be fairly future proof no ?

This was never going to work out because OpenAI is literally watching everybody and how they are using it including prompts. AI wrapper startups serve these functions:

- wrappers prop up NVIDIA which are owned by the people giving VCs the money to invest in wrappers who are long Nvidia too.

- wrappers give OpenAI RSU holders a fuzzy warm feeling and they are able to see what the market is doing with their product so they can pull "Amazon Basics"

- IRS, attorneys, accountants, politicians love them all because they all get to squeeze milk from the fume

If I were to buy the same spec it would cost $200~300/month forever

Does it make sense to buy my own server and colocate? What are the 1U rack fees now in 2024?

Are there any tips or gotchas for beginners like me?

When I shopped around for VPS/VDS I found that they were all sharing something between different clients whether its i/o or cores, which gave me an icky feeling as I need strong consistent throughput from cores for video stream mixing (im launching something similar to Twitch)

Comments URL: https://news.ycombinator.com/item?id=41850953

Points: 2

# Comments: 2

I've seen a startup that was bootstrapped 10 years ago but took on too much money and ended up getting acquired for "undisclosed sum" but at the same time everybody knew the investors were just recouping their money while founders got nothing (they had sold their equity for raising more money as their costs were way up but nobody was buying their product). Cue local news reporters don't know the nuances so they'll just announce "Startup X acquired by Y, wow!"

Founders place a lot of emphasis on getting "acquired" but there is roughly 95% chance of you selling at breakeven where the terms leave you with a year's worth of salary as a junior SWE

Get ready to see a lot more of these "acquired" news coming out in the near future. something around less than 1% chance of getting more money than you put in.

Startup has horrible odds especially if you are building a simple GPT wrappers

starving artists are going to famish now, not sure how to feel about it

https://i.postimg.cc/vT6SV7pq/replicate-prediction-6ap8z1jv5...

https://i.postimg.cc/vZzMTM71/replicate-prediction-7r4b4p6sj...

https://i.postimg.cc/rs6wM5LJ/replicate-prediction-d8s4c93v5...

I DEMAND TO KNOW HOW RUN LOCAL SAAR

dont know why all the critical comments about flux are being downvoted or flag sure is weird

crazy not even a year has past since Emad's downfall a local open source and superior model drops

which just shows how little moat these companies have and are just lighting cash on fire which we benefit from

All I wanna know is the prompt that was used to generate the art speaking of which i wanna know how to create cartoony images like that OP

we really need something intelligent (no, o1 doesn't count) and its unclear what that will look like. Perhaps it will be some RNN with neurosymbolism