After he started studying the differences in performance between traditional hypervisor-based server virtualization and Linux containers – the alternative lightweight approach to virtualization that rose to prominence with the advent of Docker – Roberto Morabito quickly realized that one essential piece was missing from existing research on containers.
Nobody, it turned out, had studied the implications of replacing VMs with containers for server power consumption. This is why Morabito, a PhD student at Aalto University in Espoo, Finland, who also works as a researcher for Ericsson, decided to make power consumption the subject of his second study of container performance.
His work is part of a European project called Metrics, which is a collaborative effort between corporations and academia to study things that affect performance of the global internet. Results of Morabito’s study of power implications of containers were published in a research paper this month.
Power-Use Difference Substantial in Some Scenarios
Using a test bed consisting of two directly linked Intel Xeon-powered Dell servers running Ubuntu Linux and a Raritan Power Distribution Unit (to measure power consumption), Morabito ran a series of stress tests on open source KVM and Xen hypervisor-based environments and open source Docker and LXC container platforms, tracking how much power each of the technologies consumed under those stress tests.
In some of the scenarios he tested, the difference in power consumption between VMs and containers was negligible, but only before he added network traffic to the mix. Both Docker and LXC containers, he found, consumed substantially less power than VMs when performing the same tasks that involved sending or receiving network traffic.
It’s crucial to keep in mind that the variety of configurations and tests Morabito ran is extremely limited compared to the configurations that are possible in real-world scenarios. “Let’s say, it’s just a first step in this analysis,” he said in an interview. “This was just considering basic settings for containers and hypervisors.”
There is a maddening variety of potential configurations and customizations, and his first attempt to benchmark power-consumption characteristics of application containers did not aim to cover them all. For example, the most obvious and very common configuration he did not test was running containers on top of VMs. The work is useful as a starting point on the way to really understanding the power-usage implications of a technology whose popularity is skyrocketing, he explained.
Hypervisors Need More Power to Address Traffic
In his paper, Morabito explains the difference in power consumption by the fact that network packets go through more processing layers in hypervisors than they do in container platforms. Another notable conclusion was that KVM overall performed slightly better in terms of power consumption than Xen.
Morabito ran tests sending network traffic at 10 Gbps to VMs and containers using 1,500-byte frames and, in separate tests, 9,000-byte “jumbo frames.” He did not test Xen with 1,500-byte frames because it was impossible to saturate a 10 Gbps link to Xen with a standard network configuration. With jumbo frames, however, there wasn’t a noticeable difference between power consumption of Xen and KVM.
When receiving 10 Gbps traffic at 1,500 bytes per frame, a single Docker container in his setup consumed 176 watts on average, while an LXC container consumed 177 watts. KVM consumed about 8 watts more to do the same thing. At 9,000 bytes per frame, the difference in power consumption between the two hypervisors and two container solutions used in the test was about the same.
When containers and VMs themselves sent TCP traffic using 1,500-byte frames at an average rate of 9.40 Gbps, KVM, LXC, and Docker consumed about 10 watts less on average than Xen. The difference in power consumption between Xen and the other three technologies is even greater when UDP traffic is transmitted, ranging from 22 watts to 25 watts.
The difference in power consumption between containers and hypervisors was even greater when multiple VMs or multiple containers were sending or receiving network traffic.
Implications at Scale May be Huge
Developers love containers and the “composable” approach to building applications the technology enables, but as its adoption rises, it will be increasingly important to have a full picture of the effects the technology has on all aspects of the infrastructure. Eight watts per container may sound negligible by itself but when containers are deployed at scale, there is now some indication that they may potentially shave off a big chunk of the data center electricity bill as a result.
Of course, there is a multitude of other factors that impact data center power consumption, and, again, many other configuration options may yield different results, but it’s important to have some initial benchmarks to work from.
