Docker From Start To Finish ( Part - 02 )

Docker is central to most deployment pipelines. This is because Docker gives us a consistent way to build, ship, and run applications across different environments.

Some of the biggest benefits are:

1. Environment reproducibility ensures that every developer, server, and deployment environment can use the same setup.
2. Dependency management becomes much simpler because we can package our application with the dependencies it needs to run.
3. Portability lets us run our containers across different machines, whether that is our dev laptop, a company server, or a cloud environment.
4. Dockerfiles can be included in version control. Since Dockerfiles are text files, we get a reliable way to track when changes were made, what changed, and why. If something breaks, we can inspect the history and roll back if needed. This makes deployment safer.

You are probably wondering, “How does Docker do all of this? What are Dockerfiles?”

Before we get into all that, let’s first understand what processes are.

Processes

Programs are just code files. I could write a Python file that prints "Hello World!" to the terminal, but until I execute it, it is still just a file with instructions.

When we execute a program, the operating system creates a process. This process is what actually does the work and allows us to interact with or see the result of our code.

A process is a running instance of a program that has its own memory space, CPU execution state, process ID, and access to system resources managed by the operating system.

Let’s walk through how a program is executed on a Linux machine.

On Linux, resource management for processes is handled by the Linux kernel. When you run something like a bash script, a Python file, or an Nginx server, the operating system does a few important things:

1. Creates a process.
2. Gives it a unique process ID, called a PID.
3. Loads the program’s code and required data into memory.
4. Sets up the process environment.
5. Schedules the process so it can run on the CPU.
6. Tracks the resources the process uses.

Containers

You may be asking, “Raghav, why do we care about what processes are? We are here for a Docker tutorial!”

Trust me, it will make sense once we understand what a container actually is.

A container is a lightweight, isolated environment for running a process on a shared operating system kernel. It packages the application code, libraries, environment variables, runtime, and configuration files needed to run the application.

The standout features that make containers so useful to us developers are:

1. Containers do not require a full guest operating system.
2. Containers share the host machine’s operating system kernel.
3. Each containerized process runs in an isolated environment.
4. Containers are fast to spin up and tear down compared to virtual machines.
5. Containers make applications easier to reproduce, deploy, and scale.

Containers are isolated processes with their own filesystem, dependencies, environment variables, networking, and runtime configuration. The container still shares the host machine’s kernel. This is why containers are usually faster to start and lighter than virtual machines.

But I would like to reiterate, this does not mean containers and virtual machines are enemies. Virtual machines virtualize hardware. Containers use operating system-level isolation.

Docker Architecture

I would now like to take some time exploring what Docker actually is and take a peek under the hood.

Docker is an open platform for developing, shipping, and running applications.

Docker separates our application code from the infrastructure it runs on. This means that what our app does and how our app runs become two separate concerns. Instead of configuring every server manually, we can describe our application environment using Dockerfiles, images, containers, networks, and volumes.

Docker achieves this through operating system-level isolation. On Linux, containers share the host machine’s kernel, but Docker uses kernel features like namespaces and cgroups to isolate processes and control resources.

So we are not booting a full operating system for every application like we would with virtual machines. We are rather running isolated processes that have their own filesystem, dependencies, networking, and runtime configuration.

The machine or server where Docker is installed is called the Docker host. This can be your laptop, a company server, a cloud VM, or any machine running Docker.

But how does Docker execute the commands I ask from it? Docker user a client-server architecture. Diagramatically, we can visualize it using the following flow chart:

Docker Client
    ↓
Docker API
    ↓
Docker Daemon / dockerd
    ↓
containerd
    ↓
runc / OCI runtime
    ↓
Container process

Docker also manages Docker objects such as:

1. Images
2. Containers
3. Volumes
4. Networks
5. Build cache
6. Registries

Let’s go through each part one by one.

Docker Client

The Docker client is the interface that lets us interact with Docker. Most of the time, this is the Docker CLI. When we run commands like:

docker run nginx
docker ps
docker build .
docker compose up

we are using the Docker client.

The Docker client does not run containers itself. It sends requests to the Docker daemon, called dockerd.

Docker API

The Docker API is the communication layer between the Docker client and the Docker daemon. When we run a Docker comand such as:

docker run nginx

the Docker CLI sends a request through the Docker API to dockerd.

This API layer also faciliates communication between Docker and external tooling such as CI/CD pipelines, SDKs, and orchestration tools.

Docker Daemon

The Docker daemon, also called dockerd, is the long-running background process that manages Docker. This is the heart of Docker from the user’s point of view. It listens for Docker API requests and manages Docker objects and high-level Docker operations such as:

1. Building images.
2. Pulling images from registries.
3. Creating containers.
4. Starting and stopping containers.
5. Managing networks.
6. Managing volumes.
7. Removing containers and images.

However, dockerd does not usually create the final isolated process by itself. For lower-level container lifecycle work, Docker uses containerd.

containerd

containerd is a lower-level container runtime used by Docker. It handles container lifecycle tasks like:

1. Image transfer
2. Image storage
3. Container creation
4. Container execution
5. Container supervision 6 .Low-level storage management

So when we run a command like:

docker run nginx

runc / OCI Runtime

runc is the low-level OCI runtime (OCI stands for Open Container Initiative) that creates and runs the actual container process. It defines standards for container images and runtimes.

When a container is started, the flow eventually reaches runc. It is responsible for creating the isolated process using operating system features like namespaces and cgroups.

Please do note that not every Docker command goes through every layer in the same way. For example, docker ps does not need to create a new container process. But commands like docker run usually go all the way down to runc.

Docker Objects

Docker objects are the fundamental components used to package, distribute, and run applications within the Docker platform. I would like to take some time to explore some of the previosuly stated Docker objects.

Docker Images

Docker images are read-only templates used to create containers. They contain the application code, libraries, dependencies, runtime, configuration, and default command needed to run the application. If we are building a Django API, our image might contain:

1. Python
2. System packages
3. Python dependencies
4. Django application code
5. Environment defaults
6. Startup command

An image is not the running application. It is the blueprint used to create the running container.

Docker Containers

A Docker container is a running or stopped instance of a Docker image. If the image is the template, the container is the actual environment created from that template.

For example:

docker run nginx

This tells Docker to create and start a container from the nginx image. A container has its own isolated environment, including:

1. Filesystem
2. Process tree
3. Network configuration
4. Environment variables
5. Runtime configuration

Docker Registry

A Docker registry stores Docker images. Docker Hub is the public registry most people are familiar with, but companies can also use private registries. Examples include:

1. Docker Hub
2. GitHub Container Registry
3. Amazon Elastic Container Registry
4. Google Artifact Registry
5. Azure Container Registry
6. Private self-hosted registries

When we run:

docker pull nginx

Docker pulls the nginx image from a registry.

When we run:

docker push my-app

Docker uploads our image to a registry. This is what allows us to build an image in one place and run it somewhere else.

That is everyhting I have for you this time. Next time I will show you how to write Dockerfiles to containerize you applications!

ILY.