The more general point is that if you need any of the many features of a general-purpose OS kernel, a full userspace driver may not be a very good fit, since you will end up reinventing a lot of wheels. Cases where it could be a good fit would be things like database backends or dedicated block storage appliances, situations where the OS would just get in the way and where it's viable to dedicate a whole storage device (or several) and a whole CPU (or several) to one task.

However, I was waiting for the touted memory safety to be mentioned beyond the introduction, but it never really came up again. I was hoping for the paper to make a stronger argument for memory-safe languages like Rust, something like "our driver did not have bugs X, Y, and Z, which were found in other drivers, because the compiler caught them".

Additionally, in a userspace device driver that is given control of a piece of hardware that can do DMA, like an NVMe controller, the most critical memory safety feature is an IOMMU, which the driver covered by the paper does not enable; no amount of memory safety in the driver code itself matters when the hardware can be programmed to read or write anywhere in the physical address space, including memory belonging to other processes or even the kernel, from totally "safe" (in Rust semantics) code.

While the driver from the paper may certainly have a "simplified API and less code", I don't expect much of this to be related to the implementation language; it's comparing a clean-sheet minimal design to a project that has been around for a while and has had additional features incrementally added to it over time, making the older codebase inevitably larger and more complex. This doesn't seem like a particularly surprising result or an endorsement of a particular language, though it perhaps does indicate that it would be useful to start from scratch now and again just to see what the minimum viable system can look like. I certainly would have liked to rewrite it in Rust, but that wasn't really feasible. :)

In any case, it's great to see proof that a Rust driver can have comparable performance to one written in C, since it will hopefully encourage new code to be written in a nicer language than C. I definitely don't miss having to deal with manual memory management and chasing down use-after-frees now that I write Rust instead of C.

(As a side note, I'd encourage anyone thinking of using a userspace storage driver on Linux to check out io_uring first before going all in; if io_uring had existed before SPDK, I don't know that SPDK would have been written, given that io_uring gets you most of the way there performance-wise and integrates nicely with the rest of the kernel. A userspace driver has its uses, but I would consider it to be a last resort after exhausting all other options, since you have to reinvent all of the other functionality normally provided by the kernel like I/O scheduling, filesystems, encryption, etc., not just the NVMe driver itself. That is, assuming the io_uring security issues get resolved over time, and I expect they will.)

I don't know anything about how MailChimp operates, but a quick search turns up this blog post about how to set up SPF records [1].

From there, you can get a list of which IPs MailChimp authorizes as a sender; following the SPF include directive, you can see they specify two IPv4 ranges, both of which are in class C space, so it seems unlikely that MailChimp has their own class A for SMTP senders.

[1]: https://blog.mailchimp.com/senderid-authentication-for-your-...

Comments URL: https://news.ycombinator.com/item?id=12123039

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=11962357

Points: 15

# Comments: 1

[1]: http://research.microsoft.com/en-us/news/features/prefast.as... [2]: https://msdn.microsoft.com/en-us/library/windows/hardware/ff...

However, the official word from Microsoft is that MSVCRT.DLL is only intended for operating system components to use, not user applications. For example, see Raymond Chen's blog on the subject. [1]

[1] https://blogs.msdn.microsoft.com/oldnewthing/20140411-00/?p=...

[1] Or any related tool (ffprobe, etc.), or any tool that uses the libav* libraries, or really any non-trivial multimedia processing tool...

I am not familiar enough with the Xeon Phi or GPU programming model to say for sure, but they could possibly be used to offload tasks like hashing/dedup or other storage-related functions.

Some of the straightforward use cases would be inside network-attached storage appliances (ideally in conjunction with a user-mode network stack) or in a database (database systems already typically want to avoid any OS interference with storage access). In general, the NVMe driver can be dropped in fairly easily when existing code is using something like Linux AIO with O_DIRECT on a raw block device; the AIO programming model maps quite directly to the NVMe driver programming model (create a queue, submit I/Os, and poll for completions).

Currently SPDK consists of a usermode NVMe (PCIe-attached SSD) driver. We will soon be releasing a usermode driver for the Intel I/OAT DMA engine (copy offload hardware) that is available on some server platforms.

It's definitely not a drop-in replacement for a kernel storage stack in the general case, but rather an optimization for specific applications (e.g. storage appliances) that can be structured to take advantage of the polled/no-interrupts model.