I don't know if the model is picking up on a "need to lock in and be more rigorous" signal, or if the model providers are routing to smarter models if they detect a frustrated user. But if a model keeps making the same mistakes, swearing at it often helped kick it out of a glut and onto the right track.

Or it could just be catharsis.

Can you elaborate on this? I'm seen estimates of ~1.5bit per English letter, and tokens encode a lot more than that - sometimes full words, with multimodal even more. If KV cache embedding are storing more than just simple tokens but entire concepts with context and nuance, that'll bump the entropy up quite quickly.

Ultimately, there's too many factors to predetermine which approach is faster. Write clean code, and let a profiler guide optimizations when needed.

A lot of this seems to be due to modern multiplayer design, with shared town instances and (usually) private dungeon/outside instances.

[0] https://www.gamedeveloper.com/business/here-s-a-look-at-the-... (scroll down)

Some key points:

1. The Camera+Card was encased in a separate enclosure made of titanium+sapphire, and did not seem to be exposed to extreme pressures.

2. The encryption was done via a variant of LUKS/dm-crypt, with the key stored on the NVRAM of a chip (Edited; not in TrustZone).

3. The recovery was done by transplanting the original chip onto a new working board. No manufacturer backdoors or other hidden mechanisms were used.

4. Interestingly, the camera vendor didn't seem to realize there was any encryption at all.

[0] https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...

Can you elaborate on this? I didn't see anything in either link that would indicate unreasonable challenges. The PSL naturally has a a series of validation requirements, but I haven't heard of any undue shenanigans.

Is it great that such vital infrastructure is held together by a ragtag band of unpaid volunteers? No; but that's hardly unique in this space.

This is a pet project I've been building on-and-off. It ports the core of Chatterbox - a 0.5B Llama-architecture model - to vLLM. A lot of ugly hacks and workarounds were needed to make it work, but the end result works.

Outputting at around the same quality level as the original implementation, this port is roughly 5-10x faster, generating a 40min benchmark output in around 2min30s wall time on a 3090 (or 4m30s on a 3060ti). That's almost 16x faster than real-time.

High throughput like this can be itself transformative, enabling scale and efficiency that unblocks new use-cases. I look forward to seeing what the community can do with this!

Disclaimer: This is a personal community project not affiliated with ResembleAI, my employer, or any other entity. The project is based solely on publicly-available information. All opinions are my own and do not necessarily represent the views of my employer.

Comments URL: https://news.ycombinator.com/item?id=44777452

Points: 3

# Comments: 0

More restrictive requirements (ie `noImplicitAny`) could be turned on one at a time before eventually flipping the `strict` switch to opt in to all the checks.

This is a rough port of llm.c into a kernel module. A lot of hacks were needed to make this happen, so a lot of performance was left on the table. Nevertheless, it is a minimally functional GPT2 inference loop running in the kernel.

Comments URL: https://news.ycombinator.com/item?id=43558042

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=43175469

Points: 2

# Comments: 0

Partially. It can be fine for pretty much any real-life use case. But many naive implementations of formulae involve some gnarly intermediates despite having fairly mundane inputs and outputs.