I am not in U.S., but my govt. prefers to fund low hanging fruit w.r.t rare-disease i.e. those disease in which mechanistic details are clear and that has better probability of successful treatment; this ignores all those rare diseases which doesn't even have clear mechanistic details like GNE Myopathy.

After a long struggle, with help of corporate donors and a research institute we're now conducting a natural history study ourselves[1].

[1] https://www.gne-myopathy.org/

If the work by Cody Trangbarger pays off and some how helps find a cure or at least halt the muscle wasting of GNE patients, then I will start to believe that AI is truly revolutionary and not a hyped up normal technology.

Rare disease research have always suffered from lack of funding as pharma companies don't have incentive to find a cure for small numer of patients when there are easier ways to make money from diseases which affect larger populations.

Interestingly AI companies seem to be desperate to prove the usefulness of their product and so for the first time rare diseases are getting the spotlight, I hope this works as I live with a rare disease myself.

Use Vulkan build of llama.cpp to run low parameter MoE models in your IGPU and CUDA llama.cpp to run dense models in your DGPU.

Ofcourse you need to set your expectations right on what problem you want to solve, I've not been happy with using quantized models for coding even on a decent laptop; where as for NLP, ASR, STT etc. work great with this setup.

I might write more about this setup soon.

I was lost and didn't see any meaning to life, until;

I met this women, with a progressive muscle wasting rare-disease without cure which would eventually make her bed-ridden. Apart from regular work, she works day and night to help people with same disease as hers.

She's always jovial and never seen her complain. Watching her, my problems became minuscule. We fell in love and got married.

I have found my new meaning for life.

P.S. My username is my name, feel free to reach out.

But then again considering there's no barrier for entry to develop runtimes now, I guess these companies are just going to develop their own.

Congratulations to the Deno employees.

4090 16gb, 96 gb RAM.

But I like this overall approach of neutering experts and loading them from ssd to make the model run in low VRAM systems.

I run small models on various kind of devices including 1.7B model on the original Jetson Nano (4B) abandoned by Nvidia, I had to upscale the software (OS, Lllama.cpp etc.) to run the model but the model runs at 17t/s.

Small models are great at NLP stuff like classification (e.g. bookmarking), ASR etc. I have built custom browser extensions to save time with bookmarking and categorizing to use with local LLMs.

Shotcut is my default video editor, if anyone's trying it on Linux please use their AppImage as it solved most of my headaches when I was using Flatpak version.

But more importantly for business, if HF fails to use the upcoming ASIC inference chips for their ZeroGPU after the acquisition, their competitors using them might have an advantage.

Local models for coding, is just not worth the effort IMO; unless of course you have the hardware to fit it unquantized in your VRAM.

I myself have an accessibility related plugin to make clicking easier [2] built using their Logi Actions SDK. It finds the clickable elements near the cursor and displays them in the Actions Ring and the LCD keypad. If you have the Logi Dialpad then you can have a complete mouse free workflow with Access Ring.

[1] https://marketplace.logi.com/plugins/

[2] https://abishekmuthian.com/pages/access-ring/

As the other comment says, I suspect it to do with GPU as the built-in displays are usually driven by intel and external displays are always driven by my Nvidia 4090.

But surprisingly, when I drive my e-paper monitor using original Jetson Nano (Ubuntu 20.04 ESM / Nvidia Maxwell); the fractional scaling is quite good perhaps even better than macOS.

Did the GPT pish hf employee or did it go to blackhat forum and buy the credential? If so what financial instrument did it use?