It's a big ass computer (technically many computers in a rack) that gives you a cloud provider style API to manage on-premises infrastructure. Plug it in, connect to the web console or API, and provision yourself some sweet sweet virtual machines, disks, and VPCs.

I get what you're saying though, and you're right that certain workloads like Kubernetes on the metal that can take advantage of hardware wouldn't be a good fit for Oxide today. We could decide to change that in the future, but I don't see that as a priority for Oxide given all else we have to build first.

Switching to Linux and KVM would gain the ecosystem benefits but Oxide would lose control of the host OS which can impact our security stance. Better to tightly integrate at this layer and expose the primitives that are needed. It's not impossible, but we're already maintaining the existing Helios host OS. Adding another wouldn't be beneficial.

The host OS is an implementation detail since you, the customer, aren't running workloads directly on the host OS. The VMs running on Oxide run on our host OS, much like other cloud providers.

That rack-scale hardware and software co-design allows Oxide to provide higher CPU and memory density in the same footprint at lower power than competitors. That's super helpful for companies operating at scale where data center power matters. The co-design allows us tackle security problems by eliminating the BIOS, providing attestation from the firmware up to the guest VM, and actually updating the firmware that's on Oxide rather than letting it rot like most customers do today.

This is why when people ask which verticals Oxide targets we kinda say "all of them" because different customers benefit from different Oxide features, but all customers need the core VM, disk, VPC abstraction. Some customers come to us because they don't have an API to manage their on-premises compute and Oxide solves that. Others come to us because they need absolute confidence that there's no malicious firmware running in their compute stack. Others come to us because they are tired of paying exorbitant amounts of recurring money just to run on-premises compute.

Our job is to make Oxide an appealing on-premises computing platform for customers to run their public cloud provider workloads and on-premises workloads without it feeling like it's an entirely different platform than you're used to.

The core primitive on Oxide is the instance (virtual machine). We could support some container primitive, but that's a larger product direction discussion. Our host OS is Helios (Illumos) so there are details to iron out there regarding what abstractions we would build and expose to the users. Not impossible but not something we're currently pursuing either given that we have other immediate product asks.

If you're at a scale where compute density, power efficiency, security, and rack-level API management matters then that's where Oxide makes sense for you.

[0] https://github.com/oxidecomputer/propolis

That's what we prototyped before local disk was released and before we started disk hot-plug work.

> Could you attach a single large volume, and do path-based provisioning on that?

Possibly. We'd still want disk hot-plug first. Otherwise, customers would have to create their cluster in a certain shape before using PVCs.

Autoscaling is a requested feature, and Karpenter fits that shape naturally. The nuance is to decide where something like the Cluster API provider ends and Karpenter begins since there's a bit of overlap in concerns. Specifically, both want to manage Kubernetes nodes but for different reasons. We're discussing it though. We have a Kubernetes watercooler meeting today where we'll likely discuss the comments in this post!

I don't necessarily want to match the public cloud experience if there's an opportunity for Oxide to exceed the public cloud experience. Eliminating the overlay is a good example of this. We have customers using external subnets to eliminate the overlay but we haven't integrated that into our controllers yet.

There's an opportunity to more tightly integrate at the network layer but we'll want to get our load balancer released first.

My colleague demo'd Karpeneter internally. We haven't committed releasing it yet but we're discussing it.

[0] https://github.com/kubernetes/cloud-provider/blob/master/clo...

[1] https://github.com/kubernetes/cloud-provider-aws