04 décembre 2015

Docker Garbage Collector

When you play a bit with Docker, you end up after some time with a very classic filesystem full issue.

The first time you hit this issue, you learn the docker rm command and start using --rm option to launch containers. The issue here is container aren't destroyed when stopped/killed. Most of us do even have some cleanup alias/script to run something like

docker rm $(docker ps --filter status=exited -q)


You also can hit such an abusive disk usage issue as you get lot's of obsolete image stored on your disk after you tested various things pulled from DockerHub. Same cause usually has the same effect, so you add to your script some :

docker rmi $(docker images | grep “^” | awk ‘{print $3}’)


(based on https://gist.github.com/ngpestelos/4fc2e31e19f86b9cf10b)

So, what's next ?
First issue with such a script is you have to run it by yourself when something goes wrong.
Second issue is such a script do remove untagged images, but not tagged ones you won't use anymore. Some other script could remove all unused images, but will then in many cases force you to re-pull few images you use on a daily basis, but weren't running at the time you ran the cleanup.

To avoid such an issue, I've created a small tool : docker-gc

This tool do listen docker daemon for destroy events, so it knows when a container is removed, and can capture the image it used. This information is used, when the gc process list the unused images, to determine which one where used recently and should be kept - as I assume you will reuse the same image on a regular basis - and which one weren't used for a long time and should be removed.

I'm sure there's many ways to improve the actual GC algorithm efficiency.

For convenience, tool is distributed as a docker image (what else ?) as ndeloof/docker-gc, just need to bind mount docker unix socket so it can interact with DockerHost daemon.

docker run -d -v /var/run/docker.sock:/var/run/docker.sock ndeloof/docker-gc


As a cool project needs a cool logo, I created one using a Jellyfish as a mascot to cleanup docker's ocean. 


Please note docker legal terms totally prohibit such a logo usage, so don't do such crazy logo hijack if you don't want Solomon's advocates to knock at your door. I'll welcome Laurel's pull-request to suggest another one :P