I'm quite fascinated with the growth in which we have witnessed with Docker. We have seen in 3 years Docker achieve over 6 Billion Pulls (Yes Billion with a B) in September. Amazing! A pull is essentially a download of a Docker image (Ubuntu, Redis, nginx, etc) from
My curiosity started when I started exploring various images on Docker Hub for a project. I clicked on Explore within Docker Hub and this is where my curiosity jumped up and grabbed hold of me. Within the Docker Hub Explore tab is where all the Official images live. Most of these images state they have 10+ Million Pulls. This ladies and gentleman is when I started asking myself "How many pulls does 10M+ really mean?" The journey begins..
Since I'm kinda a monitoring nerd I thought this might be a perfect opportunity to repurpose my Prometheus Monitoring project to track Docker Pulls. The hunt was on to find a container that uses the Docker Hub API to gather metrics from certain repos. I stumbled upon the Prometheus Docker Hub exporter by Infinity Works. This container exposes metrics of image pulls and stars on selected images within Docker Hub. Brilliant!
Configuring Prometheus to Track Docker Pulls
Now that we found an exporter we need to integrate this into our Prometheus stack. I've created a new docker-compose file which adds the Docker-Hub-Exporter service and connects it to Prometheus. The Docker-Hub-Exporter uses environment variables to determine which images to track on Docker Hub. Change this to the images you are interested to track.
metrics: image: infinityworks/docker-hub-exporter ports: - 9170:9170 environment: - "IMAGES=google/cadvisor, library/alpine, library/ubuntu" networks: - back-tier
Now that our Docker-Hub-Exporter is integrated into our Prometheus stack we now need to inform Prometheus what to do with the metrics from the new Docker-Hub-Exporter. This is performed by updating the prometheus.yml file's static_configs.
This configuration tells Prometheus to retrieve the metrics produced by Docker-Hub-Exporter. Hostname is metrics and the port is 9170 for our new target.
static_configs: - targets: ['node-exporter:9100','localhost:9090', 'metrics:9170']
Start your stack up and you should now be retrieving Docker Hub metrics from the images you selected into your Prometheus stack.
Graphing Docker Pulls
This is where it starts getting interesting. We need to start visualizing the data we are retrieving from our monitoring stack. I created a baseline Grafana Dashboard that tracks 3 different images from Docker Hub; Alpine, cAdvisor and Ubuntu which I declared in the prometheus.yml. This is a good starting point for tracking your images. I didn't include Repository Stars as it was not relevant for my testing.
I started this project out of interest to see how quickly images were being pulled and to understand the integration of 3rd party Prometheus Exporters. I learned a lot and hopefully you can use it to watch your Docker image pulls sky rocket.
The information that I gathered during this experiment shocked me so much I will write a follow-up blog to explain further.