I have a music collection that's grown rather large over the years and I've always had to choose between the lesser of three evils. I could either preserve the original files as they are, actually correct all the messy tags and organize them nicely, or keep the former and make a copy for the latter. Keeping both has worked, but now that my library is over 1TB the storage cost is definitely noticable. So, inspired by a long since defunct project called beetsfs; I built this, musefs.

Right now it supports MP3, FLAC, M4A/M4B, and Ogg (Opus, Vorbis, and FLAC, did you know you can put FLAC in Ogg? I didn't until I started working on this project). Implementing MP3 and FLAC were both straightforward, I'd call them the mobs. Making M4A the mini-boss, because it's a lot more loosey-goosey with how it's structured. Which leaves Ogg as the Final Boss. Ogg is a lovely container format, however, it was absolutely not designed for this. The way art has to be handled is unique to it, but more importantly each file is composed of individually checksummed pages. Now a key focus for me throughout development has been performance, so reading each page in full to rechecksum it was a non-starter. I'm honestly delighted by the solution I came to instead, it's by far the most technically interesting part of this project.

Let me give you the short version. Every Ogg page has a header and a payload. The header includes a page number and the aforementioned checksum. Since we're dynamically generating the preceding metadata every subsequent page (including the actual audio pages) needs to be renumbered. Which breaks the checksum. But! Due to the particular nature of CRC-32 implementation that Ogg uses, with a bit of math the actual payload drops right out of the equation. Then by applying some matrix algebra it's possible to compute the new CRC for the whole page with just the bytes of the header. I think that's really neat. For all the details you can look in docs/OGG.md in the repo.

In terms of support, right now it's 64-bit only on AMD64 and AARCH64, with first-class support for Linux and FreeBSD. macOS is theoretically supported, but I don't have the hardware to actually test that on, so caveat emptor. Caveat downloador? It's MIT licensed, so. At any rate, I decided that if I put off posting about it any longer I'd just keep polishing it forever, so I cut v1.0.0 and here we are. I've given it some pretty deep performance improvement passes and it's tested thoroughly, unit tests, prop tests, mutation testing, interop testing, we've got it all. It also comes with in tree plugins for beets, Picard, and Lidarr, as well as the common python package backing them if you want to build a plugin for something else. There are glibc and musl packages and containers available for both supported architectures too along with a sample systemd user service file for running it directly on a host.

It's near the start of the README but I'll reiterate it here as well for the sake of transparency. I built this using AI, Opus wrote specs and plans, MiMo v2.5 implemented them (along with some other models along the way). I put a lot of effort into making this not slop though and I think it paid off.

I'd love to hear any feedback that anyone has, especially if you run it against a large library!

Comments URL: https://news.ycombinator.com/item?id=48508883

Points: 2

# Comments: 0

This is like writing a paper about kids in a literal sandbox fighting over ‘territory’.

The models employed don’t indicate the actual extents of machine reasoning even as we currently recognize them. They certainly don’t have the metacognition necessary to accurately understand their own reasoning. As we’ve seen with recent papers on how LLMs do math there’s a complete disconnect between actual and reported mechanism.

“Chilling” shouldn’t be the take away here.

> But the real reason I do that is just because I just like to sit in anger whenever this breaks the user experience because of programming errors or inconsistencies.

Genuinely delighted by the fact that I’m not alone in that.

Not actually measuring crispness when you self report having the perfect equipment to do so is a cruel, cruel tease though.

The problem with DNS per the haiku isn’t that it’s difficult to understand, or even that running your own DNS server is particularly difficult. It’s that coordinating information and exchange at scale is a tricky problem with a lot of non-obvious edge cases and foot guns.

So trying to reduce complexity by sidestepping DNS really doesn’t do that, it just leaves you holding the bag on all the problems that DNS was quietly solving for you in the background.

It tried itself several times going “oh, I didn’t actually cd, let me add that and try again”. I tried correcting it several times “you MUST begin the command with `cd dir &&`”. There were a lot of variations back and forth to try to coax out the correct tool call. Including backing up the conversation and trying from earlier in the context.

It refused. Every time. It simply would not include the cd. Genuinely unhinged behavior.

“My Lord, you can attack them until March of next year!”

“Then March of next year it shall be!”

The main one is aquifer depletion. The consumptive vs withdrawal argument mostly holds water, but consumptive is a sliding scale. Evaporative cooling for data centers is solidly at the far ‘truly consumptive’ end of that scale because the consumed water will not reenter the local watershed. That’s problematic because aquifers are very slow to refill. So this is genuinely a concern in water stressed areas.

The other is the weak growth model. I suspect we’re only going to see faster and faster growth of data centers in coming years, making the consumption there more exponential than linear. Meanwhile the majority of the other consumptive consumers are strongly tied to demographics and population growth is slowing everywhere. For example agricultural water use in the US has held steady or even declined in recent years.

In fairness, part of that agricultural decline in use is from advancement in technology and methodology and we’ll likely see the same with data centers, but those numbers are unpredictable.

On the whole I agree that the concern over data centers in terms of water (and electricity) usage is overblown to an extent, but I think we do need to pay closer attention to the points that actually matter when looking at the situation.

Other series like The Culture are amazing too, but the aforementioned feel possible in a way that others don’t. For me, I can see the causal chains leading from here to there vividly in a way that you don’t get with a lot of other sci-fi.

That combination of plausible weirdness is unique and I’d highly recommend The Quantum Thief to anyone who enjoyed Accelerando or Stross’ other writing.