Understanding the Docker Registry
A Docker registry is a storage and content delivery system that holds named Docker images, available in different tagged versions. The most familiar example is Docker Hub, but for production environments, running your own private registry is often essential. At its core, the registry is a stateless HTTP server that stores image layers (blobs) and image manifests, conforming to the Docker Registry HTTP API V2 specification.
What Exactly Is a Docker Registry?
When you execute docker pull alpine:3.19, your Docker client reaches out to a registry server, authenticates (if required), fetches a manifest file describing the image, then downloads the individual compressed layer blobs. The registry itself is essentially a glorified file server with a specific API contract. A registry stores:
- Manifests — JSON documents that describe an image, listing its layers and configuration
- Blobs (layers) — the actual compressed tar archives containing filesystem changes
- Tags — pointers that map human-readable names like
latestorv2.1.0to specific manifests
Why Running Your Own Registry Matters
For many teams, the public Docker Hub isn't the right fit. Here's why a self-hosted registry becomes critical:
- Build speed and network egress costs — pulling images over the internet on every CI/CD pipeline run is slow and expensive. A local registry on the same network or cluster can serve layers at near line-speed
- Security and compliance — you can enforce strict image signing policies, vulnerability scanning gatekeepers, and keep proprietary code inside your own network boundary
- Rate limiting avoidance — Docker Hub imposes pull rate limits on anonymous and even authenticated free-tier accounts, which can break production deployments at scale
- Air-gapped environments — in disconnected or restricted networks, a local registry is the only way to distribute container images
Setting Up Your First Registry
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →The Simplest Local Registry with SSL
For development or a quick proof-of-concept, you can run the official registry image directly. However, Docker refuses to talk to registries over plain HTTP unless explicitly configured, so you need TLS. Let's create a proper setup using self-signed certificates:
# Create a directory structure
mkdir -p registry-data certs
# Generate a self-signed certificate (valid 365 days)
openssl req -newkey rsa:4096 -nodes -sha256 \
-keyout certs/registry.key \
-x509 -days 365 \
-out certs/registry.crt \
-subj "/CN=registry.local" \
-addext "subjectAltName=DNS:registry.local,DNS:localhost,IP:127.0.0.1"
# Run the registry with TLS on port 5000
docker run -d \
--name registry \
--restart=always \
-v $(pwd)/registry-data:/var/lib/registry \
-v $(pwd)/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/registry.key \
-p 5000:5000 \
registry:2
Now your registry is running on localhost:5000 with TLS. To push and pull images, you'll tag them with the registry hostname:
# Pull an official image and re-tag it for your registry
docker pull alpine:3.19
docker tag alpine:3.19 localhost:5000/alpine:3.19
# Push it to your private registry
docker push localhost:5000/alpine:3.19
# On another machine, pull from your registry
docker pull localhost:5000/alpine:3.19
Production-Ready Setup with Docker Compose
In production, you'll want the registry fronted by a proper reverse proxy, with persistent storage, and possibly authentication. Here's a complete Docker Compose stack using Nginx as the TLS terminator and basic auth:
# docker-compose.yml
version: '3.8'
services:
registry:
image: registry:2
container_name: registry
restart: always
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/lib/registry
REGISTRY_HTTP_RELATIVEURLS: "true"
# Headers for proxy behind Nginx
REGISTRY_HTTP_HEADERS_X-Content-Type-Options: "[nosniff]"
volumes:
- registry-data:/var/lib/registry
networks:
- registry-net
expose:
- "5000"
nginx:
image: nginx:alpine
container_name: registry-nginx
restart: always
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
- ./certs:/etc/nginx/certs:ro
- ./htpasswd:/etc/nginx/.htpasswd:ro
ports:
- "443:443"
networks:
- registry-net
depends_on:
- registry
volumes:
registry-data:
networks:
registry-net:
Here's the corresponding Nginx configuration that proxies to the registry and enforces basic authentication:
# nginx.conf
server {
listen 443 ssl;
server_name registry.yourcompany.com;
ssl_certificate /etc/nginx/certs/registry.crt;
ssl_certificate_key /etc/nginx/certs/registry.key;
ssl_protocols TLSv1.2 TLSv1.3;
# Maximum upload size for image layers (set generously)
client_max_body_size 0;
location / {
# Require authentication for all requests except /v2/ for health checks
satisfy any;
allow 127.0.0.1;
auth_basic "Docker Registry";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://registry:5000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Disable buffering for large layer uploads
proxy_request_buffering off;
proxy_buffering off;
# Extend timeouts for large layer pushes
proxy_read_timeout 900;
proxy_send_timeout 900;
}
# Health check endpoint without authentication
location /v2/ {
satisfy any;
allow all;
proxy_pass http://registry:5000;
proxy_set_header Host $host;
}
}
Create the htpasswd file for basic auth:
# Install apache2-utils if you don't have htpasswd
# apt-get install apache2-utils # Debian/Ubuntu
# yum install httpd-tools # RHEL/CentOS
# Create credentials (first run uses -c to create the file)
htpasswd -c -B htpasswd admin
# Add additional users without the -c flag
htpasswd -B htpasswd ci-bot
Best Practices for Registry Operations
1. Choose the Right Storage Backend
The registry supports multiple storage backends. For production, avoid the default local filesystem unless you're running on a single node with reliable, backed-up storage. Consider these options:
- Amazon S3 — ideal for cloud deployments, configure lifecycle policies to automatically clean up old untagged manifests
- Google Cloud Storage / Azure Blob — similar cloud object storage, great durability
- S3-compatible (MinIO, Ceph) — excellent for on-premises high-availability setups
- Shared NFS mount — workable but be cautious about concurrent write performance and locking
Configure S3 storage via environment variables:
# Example S3 storage configuration for docker-compose environment
REGISTRY_STORAGE: s3
REGISTRY_STORAGE_S3_REGION: us-east-1
REGISTRY_STORAGE_S3_BUCKET: my-registry-bucket
REGISTRY_STORAGE_S3_ROOTDIRECTORY: /registry-data
# Optional: use IAM roles instead of hardcoded credentials
REGISTRY_STORAGE_S3_CREDENTIALS_SOURCE: instance
2. Implement Garbage Collection Strategically
The Docker registry stores blobs referenced by manifests. When you delete an image tag or push a new version overwriting an old tag, the old layers aren't immediately removed — they become orphaned blobs. Over time, your storage fills with unreferenced layers. Garbage collection identifies and removes blobs not referenced by any manifest.
The registry ships with a garbage collector, but it requires the registry to be in read-only mode or completely stopped. Running GC while the registry accepts writes risks deleting layers that are in the process of being referenced by a new push:
# Run garbage collection safely
# Step 1: Put registry in read-only mode (or stop it entirely)
# If using docker-compose, you can set environment variable and restart:
# REGISTRY_MAINTENANCE_READONLY=true
# Step 2: Exec into the registry container and run GC
docker exec -it registry /bin/sh
registry garbage-collect /etc/docker/registry/config.yml
# Step 3: Remove read-only mode and restart if needed
For automated cleanup, schedule GC during maintenance windows. Some teams run it nightly via a cron job on a separate container that mounts the same storage volume (with the main registry stopped).
3. Enable Content Digest Validation
Ensure the registry validates content digests to prevent corruption during uploads. This is enabled by default in the v2 registry, but verify it hasn't been disabled:
# Ensure these are set in your registry config
REGISTRY_STORAGE_MAINTENANCE_UPLOADPURGING_ENABLED: "true"
REGISTRY_STORAGE_MAINTENANCE_UPLOADPURGING_AGEFREQUENCY: "168h" # 7 days
REGISTRY_STORAGE_MAINTENANCE_UPLOADPURGING_INTERVAL: "24h"
This automatically cleans up incomplete upload blobs older than the configured age, preventing storage leaks from interrupted pushes.
4. Set Up Vulnerability Scanning and Signing
A registry without scanning is a liability. Integrate a scanning tool like Trivy, Clair, or Grype. The modern approach runs scanning in your CI pipeline before images ever reach the registry:
# Example CI step using Trivy (run before pushing to registry)
trivy image --severity HIGH,CRITICAL --exit-code 1 my-image:latest
# If the exit code is non-zero, the pipeline fails and the image isn't pushed
# Push only after passing the scan
docker push registry.yourcompany.com/my-image:latest
For image signing, use Cosign with Sigstore to sign images and verify signatures before deployment:
# Generate a key pair
cosign generate-key-pair
# Sign an image after pushing
cosign sign --key cosign.key registry.yourcompany.com/my-image:latest
# Verify at deployment time
cosign verify --key cosign.pub registry.yourcompany.com/my-image:latest
5. Implement Tagging Policies and Immutability
Mutable tags like latest are convenient but dangerous in production — you can't know exactly which image digest a latest tag points to at any given moment. Enforce these conventions:
- Always deploy using image digests (SHA256 hashes) rather than mutable tags in production manifests
- Use semantic version tags (
1.2.3) that are never overwritten — push a new version instead - Consider enabling tag immutability on critical repositories if your registry supports it (Harbor, GitLab, and Quay all offer this)
For the upstream Docker registry, you can implement a proxy layer or admission controller that rejects mutable tag pushes. Many teams use Harbor, which provides this natively:
# In Harbor, configure project-level policies:
# - Enable "Tag Immutability" for production namespaces
# - Set retention policies to auto-delete old untagged artifacts
# - Enable vulnerability scanning at push time with blocking thresholds
6. Monitor Registry Health and Metrics
The registry exposes Prometheus-compatible metrics at /metrics. Scrape these to monitor request rates, error counts, storage usage, and push/pull latency:
# Enable metrics in the registry configuration
REGISTRY_METRICS_ENABLED: "true"
# Common Prometheus queries for alerting:
# High error rate:
# rate(registry_http_request_duration_seconds_count{status=~"5.."}[5m]) > 0.1
# Storage approaching limit:
# registry_storage_size_bytes > 0.9 * max_storage_bytes
Set up alerts for high 5xx error rates, which often indicate storage backend failures or disk exhaustion. Monitor the disk space on the filesystem storage or the object count in your S3 bucket.
Common Pitfalls and How to Avoid Them
Pitfall 1: Plain HTTP in Production
Docker refuses to communicate with registries over plain HTTP unless you add an explicit insecure-registries entry in the daemon configuration. In development, this might seem convenient, but in production, it exposes image layers to network interception. Always use TLS, ideally with certificates from an internal CA or Let's Encrypt, not self-signed certs that require client-side trust configuration.
# NEVER do this in production — it disables TLS for all communications
# /etc/docker/daemon.json
{
"insecure-registries": ["my-registry:5000"]
}
# Instead, properly distribute your internal CA root certificate
# Place it in /etc/docker/certs.d/my-registry:5000/ca.crt on each node
# Docker will then trust the registry without the insecure flag
Pitfall 2: Running Out of Disk Space During Large Pushes
When pushing a large image (several gigabytes), the registry temporarily stores the uploaded blobs before finalizing them. If the disk fills during this process, the push fails and leaves orphaned uploads. Monitor disk utilization and set storage quotas. For filesystem backends, mount a dedicated large partition for the registry data directory:
# Always mount /var/lib/registry on a dedicated volume
# In your docker-compose or systemd mount unit, ensure adequate space
# For cloud object storage, this is less of an issue but still monitor bucket capacity
Pitfall 3: Unauthenticated Registry Exposed to the Internet
An open registry allows anyone to push and pull images. This isn't just a security risk — it can be abused as a malware distribution vector or for cryptomining image storage. Always put authentication in front of the registry. At minimum, use HTTP basic auth with strong passwords. Better yet, integrate with your identity provider using OAuth2 or LDAP through a reverse proxy or use a registry with built-in auth like Harbor:
# Minimal protection: never expose registry port directly
# Always place it behind a reverse proxy with auth
# Use network policies in Kubernetes to restrict access to the registry service
# Only allow CI/CD pipeline service accounts and approved nodes
Pitfall 4: Ignoring the Garbage Collection Schedule
Many teams set up a registry, push thousands of images, and wonder why storage keeps growing even after deleting old tags. Remember: deleting a tag only removes the manifest reference, not the actual layer blobs. Without regular garbage collection, your storage fills with unreferenced blobs. Schedule GC as a recurring maintenance task — monthly for low-activity registries, weekly for active ones.
Pitfall 5: Pulling from Docker Hub Through a Caching Proxy Without Proper Configuration
The registry can act as a pull-through cache for Docker Hub, which saves bandwidth and speeds up pulls. However, the default configuration caches indefinitely, potentially serving stale images. Configure the cache expiry:
# registry config.yml for pull-through cache
proxy:
remoteurl: https://registry-1.docker.io
username: your_dockerhub_user # Optional but helps with rate limits
password: your_dockerhub_token
# Critical: set cache expiry so you don't serve stale images forever
ttl: 168h # Re-validate cached content every 7 days
# For private upstream registries, also configure:
# secret: "your-registry-secret"
Pitfall 6: Hardcoding Credentials in CI/CD Scripts
Never hardcode registry credentials in pipeline definitions. Use environment variables from a secrets manager, Kubernetes secrets, or your CI system's built-in secret store:
# CI example (GitLab CI)
docker_login:
script:
- echo "$REGISTRY_PASSWORD" | docker login registry.company.com -u "$REGISTRY_USER" --password-stdin
- docker push registry.company.com/my-image:${CI_COMMIT_SHA}
# Kubernetes: use imagePullSecrets from a secret
# Create the secret once:
kubectl create secret docker-registry regcred \
--docker-server=registry.company.com \
--docker-username=ci-bot \
--docker-password=secure-password
# Reference in pod spec:
# imagePullSecrets:
# - name: regcred
Pitfall 7: Not Planning for High Availability
A single registry instance is a single point of failure. If it goes down during a deployment, all image pulls fail, potentially bringing down your entire pipeline. For production, run multiple registry instances behind a load balancer, all pointing to the same shared storage backend (object storage like S3 makes this trivial since the registry is stateless):
# Run multiple registry replicas, all configured with the same S3 bucket
# The registry is stateless — all state is in the object store
# Use a load balancer (HAProxy, Nginx TCP stream, AWS ALB) in front
# Example: Kubernetes deployment with 3 replicas
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry
spec:
replicas: 3
selector:
matchLabels:
app: registry
template:
# ... container spec with S3 environment variables
# All replicas share the same S3 bucket — no coordination needed
Advanced Configurations Worth Considering
Cross-Region Replication
For globally distributed teams, pulling images across continents adds significant latency. The registry supports push-based replication to mirror images to multiple registries in different regions. Configure it to replicate specific namespaces:
# registry config.yml snippet for replication
# Note: this pushes images to a remote registry when they're pushed locally
sync:
onschedule:
- cron: "0 2 * * *" # Daily sync at 2am
source: "production-images"
destination: "registry-eu.company.com/production-images"
ondemand:
- source: "release-images"
destination: "registry-apac.company.com/release-images"
Notifications and Webhooks
The registry can send notifications on push and delete events, which integrates with your deployment pipeline to trigger automatic rollouts:
# registry config.yml
notifications:
endpoints:
- name: deployment-webhook
url: https://deploy.company.com/hooks/registry
timeout: 5000ms
threshold: 5
backoff: 1000ms
eventactions:
- push
- delete
headers:
Authorization: ["Bearer internal-api-token"]
Conclusion
Setting up a Docker registry is deceptively simple — the basic container starts in seconds. However, running one reliably in production requires thoughtful planning around TLS termination, authentication, storage backends, garbage collection, and high availability. The registry itself is intentionally minimal and stateless, which means most of the operational complexity lives in the surrounding infrastructure: your reverse proxy, your object storage, your monitoring, and your CI/CD integration.
Start with a reverse-proxied, TLS-protected, authenticated setup on reliable object storage. Schedule regular garbage collection. Scan images before they enter the registry and sign them so you can verify provenance at deploy time. Never expose the registry directly to the internet without authentication. If you treat your registry as a critical piece of infrastructure — monitoring it, securing it, and planning for its failure modes — it will serve as the reliable foundation your container delivery pipeline depends on every day.