This trilemma shows up whenever we try to meter public goods in this fashion. We could, in theory, price public transit or water or what-have-you by evaluating willingness to pay and charging each person that value. However, in practice the invasive monitoring necessary to do so is an enormous burden. We can sort of spread this out with municipal services by (as you noted) replacing trunk lines/pipes with a general fund and asking individuals to pay for the last mile, but that mostly works because the real marginal cost is measurable and not near 0. Further, for utilities, the willingness to pay has good proxies: the end of the last mile is at a real house that isn’t going anywhere. For PACER I doubt that’s the case.

It is similar to the problem with the neutron bomb. On the surface the idea of the neutron bomb (a bomb which kills humans via hard radiation but leaves infra intact) is not “worse” than a regular nuclear weapon. The dead die the same way and the living envy them. What CHANGES is the use calculus. I might not want to bomb an industrial valley if doing so destroys the thing I am trying to capture. However, if I have a bomb which kills the people living there and spares the factory, I might pull the trigger.

Similarly, it is cheap (relatively) to indiscriminately launch weapons at a distant place. It is extraordinarily expensive to send human troops in. They need food, water, and generally have families that expect some of them to come home. If putting a rifle on an autonomous vehicle works, then a ground invasion becomes cheaper.

I know this is an analogy to AI, but I wish we would dispense with this idea that there’s some appropriate level of machinery which was reached just a hair before right now. There is no appropriate level of machinery, no point at which the nature of the system itself will unambiguously say “that’s enough.”