Understanding Docker Logging Drivers
Every container running on Docker generates output that lands in stdout and stderr. Docker captures those streams and, by default, writes them to timestamped JSON files on the host disk. A logging driver is the component that decides where those logs actually go and in what format. It is the bridge between your application’s console output and your production observability stack.
What Are Docker Logging Drivers?
A logging driver is a plugin-like mechanism built directly into the Docker Engine. When a container starts, Docker attaches the driver to the container’s output pipes. The driver receives the raw stream of data and processes it according to its configuration — writing to local files, forwarding to a remote syslog server, pushing structured messages to a Fluentd aggregator, shipping directly to Amazon CloudWatch, and much more.
You can think of the logging driver as the output sink for container logs. The default driver is json-file, which stores logs in /var/lib/docker/containers/<id>/<id>-json.log. But for any non-trivial production workload, you will almost certainly replace it or heavily tune its behaviour.
Why Logging Drivers Matter in Production
In development, local log files are fine. In production, they become a liability without proper configuration. Here’s why choosing and tuning the right driver is critical:
- Disk exhaustion: The default json-file driver never rotates logs by default. A container writing at high volume can fill your host disk, causing outages.
- Performance: Writing synchronously to disk can block container stdout/stderr under heavy load. Some drivers like
localare designed specifically to avoid this bottleneck. - Centralization: You need logs from hundreds of containers across dozens of hosts to end up in a single searchable platform (ELK, Splunk, Graylog, CloudWatch). The logging driver is the first hop in that pipeline.
- Compliance and auditing: Some industries require tamper-proof, timestamped log streams forwarded directly to secure storage without touching local disk.
- Structured data: Modern observability expects structured logs (JSON or key-value pairs). The driver can add metadata like container name, image, and tags automatically.
Built-in Logging Drivers Overview
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →Docker ships with a rich set of drivers. Below is a summary of the most important ones you will encounter when designing a production logging strategy.
json-file (default)
Stores logs as JSON objects on the host filesystem. Each line contains a log field with the output, a time field, and a stream field (stdout/stderr). Simple, widely compatible, but requires log rotation configuration via the Docker daemon or a separate logrotate tool. Not ideal for high-throughput containers without tuning.
local
Introduced for better performance. It uses a custom binary format with per-container log rotation built-in and avoids the overhead of JSON serialisation on every write. It stores logs under /var/lib/docker/containers/<id>/local-logs/ and manages rotation internally. Great when you keep logs on disk but need higher throughput without blocking.
syslog
Sends logs to the local (or remote) syslog daemon via the traditional syslog protocol. Supports facilities like kern, user, daemon. Useful when you already have a syslog aggregation infrastructure like rsyslog or syslog-ng.
journald
Writes directly to the systemd journal. Works only on hosts using systemd. Provides tight integration with journalctl and forward-sealing, but limits portability if you run mixed-init systems.
GELF (Graylog Extended Log Format)
Sends structured JSON messages over UDP or TCP to a Graylog input. Supports optional compression and chunking. Perfect if your organisation uses Graylog as the central log platform.
Fluentd / Fluent Bit
Connects to the Fluentd forward protocol. Every log line becomes a Fluentd event with tags you define. This is one of the most flexible choices, as Fluentd can then route, transform, and buffer logs to dozens of backends (Elasticsearch, Kafka, S3, etc.).
awslogs (AWS CloudWatch Logs)
Pushes logs directly to CloudWatch Logs groups and streams. Essential when running on AWS ECS, but works anywhere with IAM credentials. Handles batching, retries, and stream creation automatically.
splunk
Streams logs over HTTP/HTTPS to a Splunk HTTP Event Collector (HEC). Supports token-based authentication, TLS, and batching. Ideal for Splunk-centric enterprises.
gcplogs (Google Cloud Logging)
Sends logs to Google Cloud Logging (Stackdriver). Requires appropriate GCP credentials. Integrates deeply with GKE and other GCP services.
etwlogs, logentries, and more
Docker also supports Event Tracing for Windows (ETW), Logentries, and third-party plugins via the plugin architecture. The list evolves, so always check docker info for the available drivers on your engine.
Configuring Logging Drivers
You can set the logging driver at two levels: globally for the Docker daemon, or per container. The per-container setting overrides the global default. In production, you will typically set a safe, rotated default for the daemon and then override it for specific workloads that need direct centralisation.
Setting the Default Driver for the Docker Daemon
Edit the Docker daemon configuration file, usually located at /etc/docker/daemon.json. The log-driver key sets the default driver, and log-opts configures its behaviour. After changing the file, reload or restart the Docker service.
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3",
"labels": "production,web",
"env": "os,image"
}
}
This example keeps the default json-file driver but enforces rotation (max 10 MB per file, 3 rotated files kept). It also includes container labels and environment variables in the log metadata, which is incredibly useful later for filtering.
Per-Container Logging Configuration
Use the --log-driver and --log-opt flags with docker run. In Docker Compose, use the logging key within a service definition.
Example: Configuring json-file with Rotation
This is the most common first step. Without it, production containers can fill a disk in minutes.
# Running a container with explicit log rotation
docker run -d \
--name my-app \
--log-driver json-file \
--log-opt max-size=50m \
--log-opt max-file=5 \
--log-opt labels=production \
nginx:latest
The same settings in a Docker Compose file:
version: '3.8'
services:
web:
image: nginx:latest
logging:
driver: json-file
options:
max-size: "50m"
max-file: "5"
labels: "production"
Example: Switching to the local Driver
The local driver is a drop-in replacement for json-file when you want to keep logs on disk but need better performance and built-in rotation. It uses a different storage format and is not compatible with tools that read the raw JSON log files directly, but for most use cases (forwarding via a separate shipper or using docker logs) it works fine.
# Run with local driver and size-based rotation
docker run -d \
--name high-throughput-app \
--log-driver local \
--log-opt max-size=100m \
--log-opt max-file=3 \
my-high-throughput-app:latest
Daemon-level configuration to make local the default for all containers:
{
"log-driver": "local",
"log-opts": {
"max-size": "20m",
"max-file": "5"
}
}
Example: Sending Logs to Fluentd
This pattern decouples log storage from the container runtime. Containers send logs to a Fluentd aggregator (which can run as a container itself), and Fluentd handles buffering, parsing, enrichment, and output to Elasticsearch, Kafka, or S3.
First, ensure you have a Fluentd instance listening on the forward protocol (default port 24224). Then configure the driver:
docker run -d \
--name web-logged \
--log-driver fluentd \
--log-opt fluentd-address=tcp://fluentd-host:24224 \
--log-opt tag=docker.{{.Name}} \
--log-opt labels=production \
nginx:latest
In Docker Compose, this becomes:
version: '3.8'
services:
web:
image: nginx:latest
logging:
driver: fluentd
options:
fluentd-address: "tcp://fluentd:24224"
tag: "docker.web.{{.ImageName}}"
labels: "production"
env: "ENV,REGION"
fluentd:
image: fluent/fluentd:v1.16
ports:
- "24224:24224"
volumes:
- ./fluentd.conf:/fluentd/etc/fluent.conf
The tag option is crucial — it allows Fluentd to route and filter logs based on container name or image. The Go template syntax {{.Name}}, {{.ImageName}}, etc. pulls metadata directly from the container spec.
Example: AWS CloudWatch Logs (awslogs)
When running on AWS, pushing logs directly to CloudWatch removes the need for a local aggregator. The driver will create log streams automatically if they don’t exist.
docker run -d \
--name ecs-app \
--log-driver awslogs \
--log-opt awslogs-region=eu-west-1 \
--log-opt awslogs-group=my-production-app \
--log-opt awslogs-stream=ecs-container-1 \
--log-opt awslogs-create-group=true \
my-app:latest
Docker Compose for ECS or AWS instances:
version: '3.8'
services:
app:
image: my-app:latest
logging:
driver: awslogs
options:
awslogs-region: "eu-west-1"
awslogs-group: "my-production-app"
awslogs-stream: "instance-{{.ID}}"
awslogs-create-group: "true"
Ensure the host has an IAM role or credentials with logs:CreateLogStream, logs:PutLogEvents, and logs:DescribeLogGroups permissions.
Example: GELF to Graylog
docker run -d \
--name graylogged-app \
--log-driver gelf \
--log-opt gelf-address=udp://graylog-server:12201 \
--log-opt gelf-compression-type=gzip \
--log-opt tag=production-web \
nginx:latest
In Compose:
version: '3.8'
services:
web:
image: nginx:latest
logging:
driver: gelf
options:
gelf-address: "udp://graylog:12201"
gelf-compression-type: "gzip"
tag: "production-web"
Docker Compose Logging Configuration: Full Example
Below is a complete docker-compose.yml that demonstrates a realistic production setup: a web service using the local driver with rotation, an application service using Fluentd to ship logs off-host, and the Fluentd aggregator itself using the json-file driver with rotation so its own logs are safe.
version: '3.8'
services:
reverse-proxy:
image: nginx:alpine
ports:
- "80:80"
logging:
driver: local
options:
max-size: "20m"
max-file: "3"
labels: "production,ingress"
app:
image: my-app:2.1.3
depends_on:
- fluentd
logging:
driver: fluentd
options:
fluentd-address: "tcp://fluentd:24224"
tag: "docker.app.{{.ImageName}}"
env: "ENVIRONMENT,APP_VERSION"
labels: "production,critical"
fluentd:
image: fluent/fluentd:v1.16
ports:
- "24224:24224"
volumes:
- ./fluentd/conf:/fluentd/etc
logging:
driver: json-file
options:
max-size: "10m"
max-file: "2"
Logging Best Practices for Production
- Always configure log rotation, even for centralized drivers. If the remote endpoint is unreachable, the driver may buffer to disk. A fallback disk buffer without rotation can fill your host. Set
max-sizeandmax-fileoptions everywhere possible. - Prefer structured output from your application. If your app emits JSON logs, the logging driver can forward them natively. Avoid parsing unstructured text later — ship JSON from the start.
- Tag logs with rich metadata. Use the
tag,labels, andenvoptions to include container name, image, environment name, region, and version. This metadata is invaluable when correlating logs across services. - Use the
localdriver for high-throughput workloads that must log to disk. It performs better than json-file under pressure and manages rotation internally without external cron jobs. - Centralize logs with a dedicated aggregator. Fluentd and Fluent Bit are battle-tested, highly configurable, and supported as a native driver. They allow you to decouple the logging pipeline from the container lifecycle.
- Monitor disk usage and driver errors. Set up alerts for
/var/lib/docker/containers/.../*-json.loggrowth. Watch for messages likeCannot connect to remote logging daemonindocker infoor daemon logs. - Keep configuration in version control. Your
daemon.json, Compose files, and Fluentd configs should all be managed via infrastructure-as-code. Never hand-edit production daemon settings. - Test driver failover behaviour. Simulate a network cut to your Fluentd or syslog target. Understand how long the driver buffers and whether it drops logs or blocks container output. Tune the
fluentd-buffer-limitand related options accordingly. - Consider log delivery guarantees. The json-file and local drivers are essentially “best-effort” on disk. The remote drivers like fluentd, awslogs, and splunk have retry logic but can still lose logs if the destination is unreachable for extended periods. Plan your pipeline to tolerate those gaps or add redundancy (e.g., dual logging via a sidecar).
Debugging and Troubleshooting Logging Drivers
When logs are not appearing where expected, start with these checks:
- Inspect container configuration:
docker inspect <container> | grep -A 10 "LogConfig"shows the active driver and options. - Verify driver availability:
docker info | grep "Logging Driver"lists supported drivers. If your desired driver isn’t listed, you may need to install a plugin or upgrade the engine. - Check daemon logs: The Docker daemon logs itself (via journald or syslog) and will often print driver-related errors there, such as connection refusals or malformed addresses.
- Test with a simple container: Run
docker run --log-driver <driver> --log-opt ... alpine echo "test"and observe if the message reaches the destination. - Watch for blocking: If containerized applications seem to stall when writing to stdout, the logging driver may be blocking due to back-pressure from a slow remote endpoint. Consider switching to a non-blocking driver like
localor adding buffering in Fluentd. - Validate options: Typos in option keys are silently ignored. Double-check exact names like
max-sizevsmaxSize. Refer to the official Docker documentation for the correct spelling and available options for each driver.
Conclusion
Docker logging drivers are not just an afterthought — they are a foundational piece of your production infrastructure. The default json-file driver works well for development but must be tuned with rotation and metadata for any serious deployment. For larger clusters, switching to a centralising driver like Fluentd, GELF, or awslogs moves logs off ephemeral container hosts and into durable, searchable storage. By applying the practices in this guide — setting rotation everywhere, enriching logs with tags, testing failure modes, and managing configuration as code — you can build a logging pipeline that is reliable, performant, and ready for production incidents.