You can keep an eye on (gh) polli-labs/linnaeus (a bit stale; I'll rebase on my private repo later tonight-). There's some cool ideas in here to exploit the structure of taxonomic hierarchies to help the model approach recognition how a professional taxonomist might.. so working from coarse to fine, taxonomy-guided label smoothing (distributing alpha mass by taxonomic distance)..and (forthcoming) RL on expert consensus to teach abstention (if an expert could only identify a specimen to genus for some set of inputs; then our model should abstain from a species classification for the same inputs). Unfortunately I am very, very compute-constrained- but shooting for late April/first week of May for insect + flowering plant models. (Other taxa will come later; probably as unified model). I'm working on camera-based (automated) ecological monitoring systems for ~6yrs at this point; it's a really fun problem space! dropped out of grad school to go all-in on automating my favorite job I ever had (pollination ecology field research..watching flowers for visitations!); since I knew I'd always be a mediocre ecologist- but an engineer that happens to care about ecology could be very very valuable to my field.

a taxa recognition model turns out to be only a small piece of the system you need to extract structured observational data from cameras in the field :-) Working with one of my partners right now to launch a really cool demo of what's possible these days- Texas folks especially; keep an eye out on wildflower.org around May 1!

I'll spill more ink soon but (anyone) please get in touch if you find these things interesting. Or if you'd like to help me out with compute/expenses!

I have a small wrapper around rip2, aliased to `recycle`; files go to a `graveyard` zfs dataset. I deny `rm` usage for agents, a simple (global) instruction pointing to recycle seems to do the trick for Claude.

Seems like a quick win to remove some downside risk and make me a bit more comfortable letting agents run wild in local workspaces.

but as you mention, labeled data is the bottleneck. eventually I'll be able to skirt around this by just capturing more video data myself and learning sound features from the video component, but I have a hard time imagining how I can get the global coverage that I have in visual datasets. I would give anything to trade half of my labeled image data for labeled audio data!

I originally used the Backblaze when selecting the drive I'd build my storage pool around. Every time the updated stats pop up in my inbox, I check out the table and double-check that my drives are in fact the 001Gs.. the drives that Backblaze reports has having 0.99% AFR.. I guess the lesson is that YMMV.

Typically this might mean that you simulate an 8bit forward pass to ensure that the model is robust to quantization ‘noise’. You still use FP16/32 for backward pass & weight updates for numerical stability.

It’s just a way to optimize the model in anticipation of future quantization. The experience of using an 8-bit Nemo quant should more closely mirror that of using the full-fat bf16 model compared to if they hadn’t used QAT.

I may have some more time to debug my trace tonight (i.e. remove conditionals from model + make sure everything is on CPU) and will update if I have any new insights.

``` from torch.export import export ... example_args = (dummy_input) exported_program = export(model, args=example_args) ```

Links: - torch.export docs: https://pytorch.org/docs/stable/export.html#serialization - Using 24.02 container: https://docs.nvidia.com/deeplearning/frameworks/pytorch-rele...

I'm working on something to let you measure animal activity without pasting QRs [1]. I've been running a casual study of my bird feeders since September, and my system will be field-deployed by a few labs this summer. My background is in pollination ecology, so bee/pollinator tracking is a top priority. If you want to study your own backyard, you can use polliOS with your own IP cameras. Targeting March 1 for beta release, but you can submit your email to get notified.

[0] https://github.com/jamescrall/BEEtag [1] https://polli.ai

Some ESP variants have a built-in U.FL connector which makes things easier (e.g. AITHINKER ESP32-Cam and its many clones), but you still have to remove the 0Ω resistor, which I find very difficult with even the smallest tips on my Hakko iron. Could be a skill issue on my end; should be much easier if you have a simple reflow/rework station.

I’m working on a tool to make insect (focused esp. on pollinators) monitoring 10,000x cheaper. We do not currently understand the scale of insect declines, much less are we able to design targeted interventions. In the past decades, mammalian and avian declines were largely reversed, mostly thanks to ambitious multiyear monitoring efforts that enabled limited conservation resources to be allocated efficiently.

We should replicate this model with insects. I plan to make this possible by radically reducing the cost of data.

I’ve been toying with this idea for a few years, but I finally made the leap to building my startup^2 full-time a few months back (coming from entomology/ecology). Hoping for an alpha release & demo study this summer.

I'm happy to chat with anyone interested. Especially interested to meet those working and funding in this space.

1. https://www.nature.com/articles/s41559-020-1269-4 2. https://polli.ai Note: This site is austere (I blame CUDA troubles for stealing my time this month). Expect something more representative by April.

Not sure how well-evidenced this is, but something I've heard from relevant researchers is that bison hooves are more scoop-link vs. cattle hooves. Bison more thoroughly till topsoil and activate microbial/fungal processes that promote floral diversity (among other things).