Vendors steadily provide faster and faster storage, evidently meeting a demand for higher speed and lower latency. Yet Linux easily throws most of this speed away, in all use cases where multiple entities compete for the same storage. Examples are:
- a host with several active virtual machines or containers;
- a server–of any kind–serving multiple clients, and possibly executing other administrative tasks at the same time.
The cause of this speed loss is as follows. I/O control is the only solution for providing at least some minimum fairness or bandwidth guarantees, needed in the above use cases. But, as we show in this presentation, I/O control easily slows storage down, to even just 10% of the available speed. This major loss is particularly problematic on lower-end storage, as well as on heavily loaded systems.
The BFQ I/O scheduler now recovers this throughput loss, but only in part of the problematic scenarios. Fortunately, we show that an improvement, still under development, seems to enable BFQ to reach a high throughput with all workloads we tested so far. Paradoxically, the gain is so high to make fully utilized lower-end storage outperform underutilized higher-end media, depending on the hardware at hand.