Reducing Docker Logs Size: A Practical Guide to Log Management

Docker is an excellent tool for quickly setting up and running our needed applications. However, we often overlook a crucial aspect of its operation: the logs it produces.

Why do I say that? Recently, I discovered that one of my servers was running out of free disk space, almost hitting a critical low. Upon investigating, I found that a Docker container had accumulated a 14 GB log file over just a year.

So that you don’t fall into a similar situation, this guide will help you learn how to check the sizes of all Docker log files quickly, shrink them if needed, and set up Docker to prevent them from getting too big in the future.

Docker containers are stateless, meaning their logs are deleted when the container is recreated. But many overlook a crucial detail here. So, before we proceed, let’s clarify an important point.

Restart vs. Recreate Docker Container

Let’s return to the case above. Over the past year, the server has been restarted multiple times, meaning that Docker containers have also been stopped and started. However, their log files have persisted and continued to grow.

The answer to why this happened is that we are talking about restarting here and not recreating the containers. When the server is rebooted, the Docker daemon gracefully shuts down the running containers, and after boot, it starts them automatically if they’re set to do so.

Similarly, the behavior is the same when manually restarting the Docker service using the “systemctl restart docker” command. It’s important to understand that Docker logs are only reset when the Docker container is deleted and recreated. Simply stopping and starting the container does not reset the logs.

Having cleared that up, let’s get to the meat of this guide and learn how to quickly check how much space Docker container logs use on our system.

Checking Docker Log Sizes

Docker logs are typically saved (if you have not explicitly changed Docker’s default data directory) in the following location on the host machine:

/var/lib/docker/containers/<container-id>/<container-id>-json.logCode language: HTML, XML (xml)

Each Docker container has its directory under “/var/lib/docker/containers/.” Within each container’s directory, you’ll find a file ending in “-json.log,” which contains the logs for that specific container.

These files are structured in JSON format by default and capture the container’s standard output (stdout) and standard error (stderr) streams.

To find out the size of all Docker container logs sorted from largest to smallest, you can use the following find command:

find /var/lib/docker/containers/ -name "*json.log" | xargs du -h | sort -hrCode language: Bash (bash)

Great, now we have all the log file sizes from our Docker containers. But how do we match these IDs to the actual names of the containers to identify which one produced each log file? Here’s how.

Find Docker’s Container Name by Its ID

Let’s say we want to identify the Docker container responsible for creating the biggest log file listed earlier, which is 5.9M in size. We’ll use its ID, which is “d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977.”

The command we need is:

docker inspect --format='{{.Name}}' <container_id>Code language: Bash (bash)

So, we take the string containing the container ID and put it into the given command.

docker inspect --format='{{.Name}}' d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977Code language: Bash (bash)

And just like that, the magic happens.

We’ve confirmed that the Docker container responsible for the log file is named “php-fpm-valente” in our case.

To double-check that this is the correct container that produced the log file, you can verify this by using the docker inspect <container_name> command to get more details about it and confirm that it’s the source of the log file. In our case, it would look like this:

docker inspect php-fpm-valente

Or use the command below to get only the path information to the log file.

docker inspect --format='{{.LogPath}}' <container_name>Code language: Bash (bash)

Now that we have all the data we need, let me show you how to clear the contents of the corresponding log file to free up disk space.

Cleaning Docker Container Logs

You can use the truncate command to clear all the contents of an individual log file. Just provide the full path to the log file you want to clear as a parameter. For example:

truncate -s 0 /var/lib/docker/containers/d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977/d2e9228f92b66ac09fa35dcab36abba2eb4a7f46baa1d03b65d71ed8d42de977-json.logCode language: JavaScript (javascript)

If you choose a more aggressive approach, you can clear all Docker log files with just one command. But be sure you understand what you’re doing before you run the command below.

truncate -s 0 /var/lib/docker/containers/*/*-json.logCode language: Bash (bash)

Setting Up Docker Log Size Limits

The approach described above offers a temporary fix for managing large Docker log files and freeing up disk space. However, it requires regular use of certain commands, which might be inconvenient.

A way better and more permanent solution would be to limit the maximum size of Docker log files. Once this size is reached, the Docker daemon will automatically rotate those log files and archive them following naming conventions such as “<container_id>-json.log.1,” “<container_id>-json.log.2,” etc., depending on the options shown below.

This guarantees that the log files do not grow uncontrollably. Here’s how to do it.

To apply log rotation settings globally to all containers run by the Docker daemon, you can edit the Docker daemon configuration file, typically located at “/etc/docker/daemon.json.” If the file does not exist, you can create it by executing a command similar to:

sudo nano /etc/docker/daemon.json

Then, paste the following content into the file, save it and exit:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}Code language: JSON / JSON with Comments (json)

Okay, let me explain what this does. In short, it sets the log driver to “json-file,” setting the maximum allowed size of a single Docker log file to 10 megabytes and allowing up to 3 versions of the file to be archived.

Next, to apply the changes, simply restart the Docker service:

sudo systemctl restart dockerCode language: Bash (bash)

It’s important to remember that these changes will only affect newly created Docker containers, not the ones already running. To apply the changes to existing ones, you must first delete them and then create them again. Here’s how you can do it.

If you started the containers using a standard Docker command, use this to stop and then remove a container in one command:

docker rm -f <container_id_or_name>Code language: Bash (bash)

This is a handy command to stop and remove a container immediately. However, using “-f” might be risky as it does not allow a clean shutdown of applications running in the container. Then just run it again like you did before.

If you’re using Docker Compose, just navigate to the folder with the “docker-compose.yml” file and run the following command:

docker compose downCode language: Bash (bash)

It stops and removes all the resources defined in the file. Then, rerun the deployment:

docker compose up -dCode language: Bash (bash)

If you look at the container details again, you should see that the data you previously entered in the Docker “daemon.json” file has been applied to the container.

docker inspect <container_name>Code language: HTML, XML (xml)

Conclusion

Managing Docker log sizes is essential to maintaining an efficient and sustainable server environment.

This guide provides a strategy for managing the size of Docker log files and ensuring that logs do not accumulate uncontrollably using Docker’s log rotation and cleanup capabilities.

Ultimately, applying these practices allows for better resource management and operational efficiency in a Dockerized environment.

Refer to the official documentation for more detailed information on configuring Docker using the “daemon.json” file. We also recommend checking out our guide, especially dedicated to Docker log files.

Thanks for using our guide! Feel free to share your thoughts in the comments section below.