How to avoid Docker Hub Rate Limits

‍

If your CI runs on Azure and you are pulling base images from Docker Hub, you might be hit by the Docker rate limits beginning June 30, 2024.

Most commonly, CI pipelines refer base image from Docker Hub. Also, the image pull is done without authentication, which triggers the pull as an anonymous user. As per Docker rate limit, you will be allowed only 100 pulls in every 6 hours window from a specific origin IP.

🔆What this means to you :

If you have many CI pipelines that use base image from Docker Hub and builds the image on ephemeral runners, you might hit the rate limit and build will fail. If you have self-hosted/dedicated runners, you might be good, since the bases images will be cached locally on subsequent builds.

🔆What options you have :

A few possible ways you can reduce/avoid the chance of build failure

1. Use authentication to pull image : You can use a free Docker Hub user login to double the rate limit (200 pulls every 6 hours). While the cost impact is zero, this will only reduce the chance of failure and not a complete fix.
2. Mirror or cache base images in Private Registry : if you use container based deployment, you probably use a private registry already. To avoid rate limits, either mirror or cache the public base images into your private registry, and change Dockerfile to point to private registry.
3. Purchase Docker Hub Subscription : If #2 doesn't work for some reason, you can always purchase a subscription of Docker and use that credential to authenticate, which will increase the limit to 5000 pulls per day (large enough for most organizations)

While #2 and #3 will impact your cloud cost budget, they will be a complete solution to the rate limit.