M1 Max: FP16 hardware support, FP8 and Bfloat16 emulated in software (via dequantization)

H100: FP16 and FP8 hardware support

> which I ran both on a MacBook Pro M1 Max and a rented H100 SXM GPU

I actually found Facebook’s translations pretty good (better than Google Translate for things longer than a sentence). From my understanding of Khmer, Khmer is a bit more verbose and context dependent, hence LLMs in Khmer would be a big help understand those nuances.

In the inverse case (LLMs generating khmer from English) I heard from locals that it sounds formal and “robotic” which I found quite interesting.

- You’re right, data is very hard to come by. I’m curious, how do you plan to get around this? Outsourcing human labeling? We found it to be a very difficult task.

- The subcontractors and local construction companies we talked to were overwhelming excited about the idea.

- It’s entire people’s jobs to get this done and done correctly. They sit on site holding the pdfs in their hands, manually counting and calculating. You bet a lot of mistakes occur. They would absolutely love to have a digital assistant for this.

- Some of them (especially managers and owners) are quite technical and are using software such as BlueBeam and other CAD software to make these calculations. It’s quite manual currently, but gives great insight into a better solution. This led us to having the user manually select the symbol they wanted counted (which ML struggled to get right). Just getting the part counts (and highlighting them in the pdf) was a huge help!

- Impressive you got square footage calculations correct! In our experience, there was way too much variation between architects (and multistep dimension labeling) which made it hard (even for humans) to get right. How has your model generalized OOD thus far?

- Are you planning to integrate voice? Many of the subcontractors we worked with are very low tech. They usually talk with their clients in person, on the phone, or maybe text. But they don’t use email or their smart phones for much.

I will be following your work! I have friends who would love to use this once it passes the human threshold.

It’s an eye opening alternative explanation to the electrons flowing like a chain theory of this article.

They might be trying to create toxic back links to their domains and if those domains 301 to your domain, I believe this can negatively impact the SEO of your domain (from what I read). If so you can try to disavow them https://support.google.com/webmasters/answer/2648487?hl=en

I’d take this with a grain of salt (pun intended). There’s a lot of bugs that you cannot reproduce without certain permissions or a particular environment. Let alone the race conditions or user setup. In my experience, most bugs would not have been uncovered using this brute force approach. A few tests using your understanding of the code and critical thinking goes a lot further in my opinion.

There’s a few other expenses and some cons of living here but some research and YouTube videos will help you figure out if it’s right for you. And of course you can ask me :)

I’d also say £40k was way too low. I’d guess the founders have significantly higher upside. It would be worth asking for transparency in order to determine fair compensation.

Laser ($40): https://a.co/d/0wjNGBz

Diffraction grating ($12): https://a.co/d/6bpO8xm

Laser focusing lens: Not found

Fluorescence collection lens: Not found

Focusing lens: Not found

Collimating lenses: Not found

If you know of any implementations that can look at a spectrogram and say “hey there’s peaks at 150hz, 220hz and 300hz with standard deviations of 5hz, 7hz, and 10hz, decreasing in frequency over time thus this is a deep voice saying ‘ay’” and get it right every time I’d be really interested in seeing it (besides neural networks)

Personally I hypothesize that the reason it’s so hard is that the sources are intermixed sharing frequencies so isolating to certain frequencies doesn’t isolate a speaker. We’d need something like beam forming to know how much amplitude of each frequency to extract. I’d also hypothesize that humans, while able to focus on a directional source, also cannot “extract” clean signal either (imagine someone talking while a pan crashes on the floor - it completely drowns out what the person said)

1. STFT (get frequencies from the audio signal)

2. Log scale/ decibel scale (since we hear on the log scale)

3. Optionally convert to the Mel scale (filters to how humans hear)

Happy to answer any questions

keys = key_weights * x

query = query_weights * x

values = value_weights * what_to_look_at

For self attention, what_to_look_at = x

For regular attention, where_to_look_at could be a database, memory or anything else.

So in this example if we’re trying to predict the second “apple” the first “apples” is very helpful. If we’re predicting “juice” then we’d use one head of self-attention to look at the first “apples” and a second head to also look at the second “apple”

That’s my understanding at least