Without a checksum, hardware RAID has no way to KNOW it needs to use the parity to correct the block.

We successfully hired from HN in the previous round and are looking for another OpenZFS Developer (3+ years of experience) to join our team!

Klara Inc. provides development & solutions focused on open source software and the community-driven development of OpenZFS and FreeBSD. We develop new features, investigate/fix bugs, and support the community of these important open source infrastructure projects. Some of our recent work includes major ZFS features such as Fast Deduplication (OpenZFS 2.3: https://github.com/openzfs/zfs/discussions/15896) and AnyRAID: https://github.com/openzfs/zfs/pull/17567.

We’re looking for an OpenZFS Developer with:

- Strong C programming skills and solid understanding of data structures

- Experience with file systems, VFS, and OS internals (threading, locking, IPC, memory management)

- Familiarity with ZFS internals (DMU, MOS, vdevs, ZPL, datasets, boot environments)

- Ability to work across Linux, FreeBSD, or illumos environments

Previous upstream contributions to OpenZFS or other open source projects are a big plus.

Submit an application through our site: https://klarasystems.com/careers/openzfs-developer/

Klara provides open source development services with a focus on ZFS, FreeBSD, and Arm. Our mission is to advance technology through community-driven development while maintaining the ethics and creativity of open source. We help customers standardize and accelerate platforms built on ZFS by combining internal expertise with active participation in the community.

We are excited to share that we are looking to expand our OpenZFS team with an additional full-time Developer.

Our ZFS developer team works directly on OpenZFS for customers and with upstream to add features, investigating performance issues, and resolve complex bugs. Recently our team has upstreamed Fast Dedup, critical fixes for ZFS native encryption, improvements to gang block allocation, and has even more out for review (the new AnyRAID feature).

The ideal candidate will have experience working with ZFS or other Open Source projects in the kernel.

If you are interested in joining our team please contact us at zfs-hire@klarasystems.com or apply through the form here: https://klarasystems.com/careers/openzfs-developer/

Also, `mv` between different filesystems in the same ZFS pool. Traditionally when crossing filesystems doesn't allow just using `rename()`, `mv` resorted to effectively `cp` then `rm`, so at least temporarily required 2x the space, and that space might not be freed for a long time if you have snapshots.

With BRT, the copy to the 2nd filesystem doesn't need to write anything more than a bit of metadata, and then when you remove the source copy, it actually removes the BRT entry, so there is no long-term overhead.

One of the original developer's use cases was restoring a file from a snapshot, without having to copy it and have it take up additional space.

So you make a file (foo) 2 days ago. You change it each day. Today, the change you made was bad, and you want to restore the version from yesterday.

before BRT: you copied the file from the snapshot back to the live filesystem, and it took up all new space.

after BRT: we reference the existing blocks in the snapshot, so the copy to the live filesystem takes no additional space on disk. A small BRT entry is maintained in memory (and on disk).

If you remove the snapshot, the BRT entry is removed, and the file remains intact. No long term overhead.

The reason the parity ratio stays the same, is that all of the references to the data are by DVA (Data Virtual Address, effectively the LBA within the RAID-Z vdev).

So the data will occupy the same amount of space and parity as it did before.

All stripes in RAID-Z are dynamic, so if your stripe is 5 wide and your array is 6 wide, the 2nd stripe will start on the last disk and wrap around.

So if your 5x10 TB disks are 90% full, after the expansion they will contain the same 5.4 TB of data and 3.6 TB of parity, and the pool will now be 10 TB bigger.

New writes, will be 4+2 instead, but the old data won't change (they is how this feature is able to work without needing block-pointer rewrite).

See this presentation: https://www.youtube.com/watch?v=yF2KgQGmUic

Comments URL: https://news.ycombinator.com/item?id=35559318

Points: 2

# Comments: 1

In FreeBSD 12.x and older, you could opt out of using UMA with: vfs.zfs.zio.use_uma=0, but that is not necessary

Too many years of thinking of the aux vdev types like SLOG and L2ARC as caches, has made people confuse this one frequently.

The many video-streaming type workloads, the connection will go idle, for seconds or even minutes at a time. If the loss is at the tail end of some activity, before a period of idle, the recovery takes a lot longer than it would if there are further activity on the connection.

FreeBSD has features like 'bootonce', to switch ZFS boot environments on the next boot only. Better integration with the boot loader, more boot environment tooling, and other things that can only come when the filesystem is part of the OS.