On Tuesday evening, we met with some of the BreizhCamp team and friends for our first "BreizhCamp Hacking Party". Our plan was to give the RPiDocker challenge a try.
At DockerCon, hypriot folks demonstrated running 500 (basic) web servers on a Raspberry Pi. This was the starter for hackers to do more, and Damien Duportal reached actual record of ... 2334 webservers running on a Raspberry Pi 2.
So, we met together with various ideas to make a better score...
And the result was .... 70 containers running.
During the 4 hours of our hacking party we were not able to get more than 70 containers, whatever configuration change is made.
We later discovered Damien's hack recipe which is comparable in many ways, but do configure docker daemon LimitNPROC (max number of child processes) to infinity. It's odd this isn't the default. My naive vision of a Unix system is root can do anything, and don't have any restrictions. With capabilities this isn't the case - lesson learned.
Remaining question is to know how we should have understood deamon was blocked by some systemd settings. daemon.log didn't told us much, not sure what we should have been looking for.
Anyway, we now have a set of recipes to setup a RPiDocker system and experiment. We share our tweaks and still experiment on them to mix with Damien's ones. I don't have (yet) a RPi2 so can't give actual challenge metrics, but so far on my RPi 1BRev1 I can run up to 1400 web servers.
And we have some more ideas to get more :P
Wait and see ...
At DockerCon, hypriot folks demonstrated running 500 (basic) web servers on a Raspberry Pi. This was the starter for hackers to do more, and Damien Duportal reached actual record of ... 2334 webservers running on a Raspberry Pi 2.
So, we met together with various ideas to make a better score...
- don't use a IP stack per container, but --net=host
- configure RPi hardware for max performances
- use swap on disk and/or zram - tweak vm.swapiness
- disable everything (kernel modules and services) that we don't need. I even considered disabling sshd and connect to the RPi using serial console.
- enable memory overcommit
- use a tmpfs for /var/lib/docker so creating containers is faster and we don't spend hours waiting for thousands containers to start
- rebuild docker binary with Go 1.5, to benefit improved garbage collector
- run without docker. Really ? Yes! We wanted to know the system limits and then get some metrics on docker overhead.
And the result was .... 70 containers running.
During the 4 hours of our hacking party we were not able to get more than 70 containers, whatever configuration change is made.
We later discovered Damien's hack recipe which is comparable in many ways, but do configure docker daemon LimitNPROC (max number of child processes) to infinity. It's odd this isn't the default. My naive vision of a Unix system is root can do anything, and don't have any restrictions. With capabilities this isn't the case - lesson learned.
Remaining question is to know how we should have understood deamon was blocked by some systemd settings. daemon.log didn't told us much, not sure what we should have been looking for.
Anyway, we now have a set of recipes to setup a RPiDocker system and experiment. We share our tweaks and still experiment on them to mix with Damien's ones. I don't have (yet) a RPi2 so can't give actual challenge metrics, but so far on my RPi 1BRev1 I can run up to 1400 web servers.
And we have some more ideas to get more :P
Wait and see ...
