Llama is decent at auto optimizing it if you let it use both gpus. It’ll split the workload so the contiguous layers are all on one gpu. Once the model is loaded, you only transfer weights between gpus once (per token?), at the layer boundary.

I run on an 8gb 3070 and 12 gb 3060, and the only weird thing is that the weaker card gets more layers (and therefore work) because it has more ram.

Oh, if you’re barely fitting the models into your vram, you may need to explicitly adjust the layer balance between cards — sometimes it fails to realize it should have put certain things (like draft models) on the other card so you can fit one more layer in.

This is the approach I'm using at my job, which is incident detection with customer metrics. We're tagging our time series data with common features -- such as country, customer type, etc -- with the idea that we can do a graph-like search to find exogenous variables. We can also use this to identify time series that have a similar "data generating process" and are simply different "realizations" of each other.

We don't need great time series forecasts, just something that detects large deviations quickly. We can then add in an existing dataset of _known_ incidents, indexed by the same common features, as a training/validation set.

> What makes this especially frustrating is the lack of transparency. Every other ADS company in the NHTSA database, Waymo, Zoox, Aurora, Nuro, provides detailed narratives explaining what happened in each crash. Tesla redacts everything. We cannot independently assess whether Tesla’s system was at fault, whether the safety monitor failed to intervene in time, or *whether these were unavoidable situations caused by other road users*. Tesla wants us to trust its safety record while making it impossible to verify.

Rust has deliberately taken a “batteries not included” approach due to learned experiences in python, where many std-lib packages are basically abandoned/not recommended anymore.

This gives the ecosystem time to develop better approaches to things, rather than ossifying around the initial implementation.

Yes, it does lead to a larger set of dependencies, but some of the most common packages (including anyhow) were written and are maintained by members of the rust lang organization/team itself.

Rust is statically compiled, so it’s possible to have different versions of the same dependency in your dependency graph without build issues. Yes, that can lead to binary bloat and slower build times and some related problems, but in practice the compiler is good at dealing with those issues.

I can’t remember the exact article I read that in, but it’s pretty obvious when looking at Anandtech’s latency charts.

M1 latency: https://www.anandtech.com/show/17024/apple-m1-max-performanc...

Ryzen latency: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di...

There’s a whole host of subsidies for building/maintaining roads paid for by everyone [1]. There’s the regulatory advantages driving has (ex: the common zoning requirement that new construction ensure enough parking space [2]). There’s the negative externalities of highways/roadways. (Ex: the health impacts of pollution[3], and the health impacts of sitting in a car rather than walking/biking (part of the way) to their destination [4])

[1] https://usa.streetsblog.org/2013/01/23/drivers-cover-just-51... [2] https://www.planning.org/planning/2018/oct/peopleoverparking... [3] https://theicct.org/new-study-quantifies-the-global-health-i... [4] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8744747/

The gist is you’re trying to find the worst sequence of (hard mode) guesses possible for any word in the wordle set.

Their site shows submissions of up to 16 words and claims 17 is impossible [2]. I’ve been trying to do the same, but have yet to cut the search space enough[3].

Ex: calculating the worst chain for jazzy (with a 1000 word subset of the full word list) took an m1 core 1 hour and resulted in the ten-word chain:

civic, buggy, woody, array, leaky, mammy, fanny, happy, tasty, jazzy

[1] https://www.janestreet.com/puzzles/eldrow-index/ [2] https://www.janestreet.com/puzzles/eldrow-solution/ [3] https://github.com/roshangiyanani/wordle/blob/main/wordle/el...

It uses a couple of techniques, like wide/tagged pointers, object ids, and a special hardware managed bit to track illegal modifications.

If I had to hazard a guess, those object ids are probably useful for general capability systems.

I think apple (maybe as just an arm feature?) can do encrypted pointers, with a per application key tracked by the kernel.

[1] https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/

https://twitter.com/ErrataRob/status/1331735383193903104

If you mix arm and x86 code inside a process, you can't tell which ordering to use.

So for that List[int] example, you probably want to take a(n) Iterable[int], Iterator[int], or Collection[int] instead, depending on exactly how you use it.

What is interesting is how intel has 4-8 e cores even on the high end.