It’s time to embark on the dockerless journey

https://zoues.com/posts/58bd45d9/

Docker, as one of the earliest and most widely used container runtimes, has become deeply ingrained in user workflows due to its intuitive interface and extensive features. In Kubernetes’ early days, it used the built-in dockershim component to integrate with Docker. However, as container technologies evolved, Kubernetes introduced the Container Runtime Interface (CRI) to standardize and expand container runtime support. This innovation enhanced Kubernetes’ compatibility with multiple container runtimes, offering users greater flexibility and choice.

The CRI enables Kubernetes system components to interact seamlessly with various container runtimes. This shift expanded Kubernetes’ scope beyond Docker and reduced its dependency on Docker and dockershim. Today, Kubernetes users can confidently adopt other robust runtimes like containerd and CRI-O, simplifying the Kubernetes architecture and improving operational efficiency.

Starting with Kubernetes version 1.24, official support for Docker (via dockershim) was removed. Similarly, Amazon EKS declared containerd as its sole supported runtime. Following this direction, our cluster’s container runtime was transitioned to containerd, replacing Docker CLI with nerdctl.

To better understand this shift, it’s important to introduce key container ecosystem components such as libcontainer, runc, containerd, CRI, and OCI. These components play critical roles in creating an efficient and flexible container runtime environment. By delving into these concepts, we can enhance operational expertise and lay a solid foundation for future application deployment and maintenance. For detailed information on nerdctl, please refer to the corresponding section.

Technical Background

OCI Standards (Open Container Initiative)

The Open Container Initiative (OCI), spearheaded by organizations like Docker and CoreOS (later acquired by Red Hat), establishes unified industry standards for container formats and runtimes. The OCI defines two primary specifications:

• Image Specification (image-spec): Specifies the format and structure of container images.
• Runtime Specification (runtime-spec): Governs the execution of container images. The most widely used low-level runtime today is runC, which includes libcontainer and supports operations like namespace and cgroup calls.

Main High-Level Container Runtimes

High-level container runtimes offer advanced functionality such as image management, unpacking, and API integration. The three most popular ones are:

1. containerd: A CNCF project originally contributed by Docker.
2. Podman: A Red Hat project, fully supported in RHEL 8.
3. CRI-O: Another CNCF project designed to integrate seamlessly with Kubernetes.

Container Relationships

The diagram below illustrates the relationships within the container ecosystem, highlighting the interaction between various components.

Using Docker based on containerd as a container runtime can be categorized as a “high-high-level” runtime.

Containerd

Launched as an independent project in early 2016 and donated to CNCF in March 2017, containerd is a lightweight, robust, and portable container runtime that supports Linux and Windows. It manages complete container lifecycles, including image management, container execution, and networking (via CNI plugins).

Key Components of Containerd

1. Runtime: Manages container lifecycle tasks, including creation, execution, and monitoring.
2. Storage: Provides image storage and management capabilities.
3. gRPC: Facilitates communication with upper-layer applications, offering container management APIs.
4. Metrics: Offers performance and resource utilization metrics, primarily based on cgroups.
5. Metadata: Stores essential information about images and containers.
6. Tasks: Organizes containers as tasks, encompassing runtime states and process data.
7. Events: Emits state change notifications to help applications monitor and react to container events.

Comparison Between Docker and Containerd

CRI (Container Runtime Interface)

The CRI, introduced by Kubernetes, standardizes interactions between Kubernetes and container runtimes. Initially, Kubernetes relied on Docker’s API but later adopted the CRI to reduce dependencies and support emerging runtimes.

CRI defines two gRPC services:

1. ImageService: Manages operations like pulling, listing, and deleting container images.
2. RuntimeService: Handles Pod and container lifecycle management.

Evolution of CRI and Dockershim

In early Kubernetes versions, dockershim acted as an adapter, enabling Docker to conform to CRI. Over time, this approach proved suboptimal due to the complexity of Docker’s architecture. By Kubernetes 1.20, dockershim was deprecated, and by 1.24, it was fully removed.

Switching to containerd offers significant efficiency improvements by simplifying the runtime stack. Kubernetes now interacts directly with CRI-compatible runtimes like containerd, without relying on dockershim. This evolution aligns with Kubernetes’ focus on modularity and scalability.

Command Line Overview

Docker, ctr, nerdctl, and crictl are container management tools, each designed for specific purposes and use cases. Here’s a comparison:

Docker

• Overview: Docker is a popular container management tool providing a comprehensive ecosystem for building, running, and managing containers.
• Features: Supports image building, distribution (integrated with Docker Hub), and container lifecycle management (start, stop, delete, etc.).
• Interface: Offers a powerful and user-friendly CLI and API, making it easy for developers and operators to adopt.
• Key Characteristics:
• Rich ecosystem (e.g., Docker Compose).
• Supports advanced orchestration (e.g., Swarm).
• Includes an additional daemon (dockerd) to collaborate with container runtimes.

ctr

• Overview: ctr is the native CLI provided by containerd for direct interaction with it.
• Features: Enables basic operations like pulling images, running containers, and managing image storage but is simplified for lower-level operations and debugging.
• Interface: Unlike Docker, ctr targets containerd users without offering advanced APIs or ecosystem support.
• Key Characteristics:
• Suitable for developers and system integrators needing direct access to containerd.
• Lacks advanced features like image building (focused solely on runtime management).

nerdctl

• Overview: nerdctl is a CLI tool based on containerd that supports Docker-style commands.
• Features: Provides a Docker-like user experience for running containers, building images, and supporting CNI plugins and volumes.
• Interface: Closely mirrors Docker CLI, making migration from Docker to nerdctl straightforward.
• Key Characteristics:
• Operates without the Docker daemon, relying solely on containerd.
• Integrates well with Kubernetes (when using containerd as the runtime) and supports Docker Compose-like functionality.

crictl

• Overview: crictl is a CLI tool for the Container Runtime Interface (CRI), designed specifically for Kubernetes container management.
• Features: Primarily interacts with CRI-compatible runtimes (e.g., containerd and CRI-O) for managing Pods, images, and containers.
• Interface: Focuses on Kubernetes cluster debugging and management, with commands tailored to Kubernetes scenarios.
• Key Characteristics:
• Highly integrated with Kubernetes, often used to debug or manage Pod and container states.
• Cannot build images, focusing exclusively on container and Pod lifecycle management.

Horizontal Comparison for Specific Use Cases

1. Image Management

2. Container Management

3. Pod Management

Summary

• Docker: User-friendly and feature-rich, ideal for standalone container management and development environments.
• ctr: Lightweight, low-level tool for direct interaction with containerd, mainly for debugging and backend management.
• nerdctl: Docker CLI-compatible, great for users wanting Docker-like commands while directly utilizing containerd.
• crictl: A Kubernetes-focused tool for managing Pods and containers via CRI; not suitable for general development.

nerdctl is a lightweight alternative to Docker based on containerd, while crictl specializes in Kubernetes CRI-based Pod management.

Guide to Using Containerd

nerdctl is a CLI tool compatible with the Docker CLI style, and it directly supports Docker Compose syntax. This significantly enhances efficiency when using containerd for local development, testing, or single-node container deployment.

1. Installation

You can download the appropriate package from the GitHub Release page and extract it to a directory in your PATH:

# If containerd is not installed, use the nerdctl-full-<VERSION>-linux-amd64.tar.gz package.
➜  ~ wget https://github.com/containerd/nerdctl/releases/download/v1.7.5/nerdctl-1.7.5-linux-amd64.tar.gz

➜  ~ mkdir -p /usr/local/containerd/bin/ && tar -zxvf nerdctl-1.7.5-linux-amd64.tar.gz nerdctl && mv nerdctl /usr/local/containerd/bin/
➜  ~ ln -s /usr/local/containerd/bin/nerdctl /usr/local/bin/nerdctl
➜  ~ nerdctl version

# If containerd is not installed, use the nerdctl-full-<VERSION>-linux-amd64.tar.gz package.
➜ ~ wget https://github.com/containerd/nerdctl/releases/download/v1.7.5/nerdctl-1.7.5-linux-amd64.tar.gz

➜ ~ mkdir -p /usr/local/containerd/bin/ && tar -zxvf nerdctl-1.7.5-linux-amd64.tar.gz nerdctl && mv nerdctl /usr/local/containerd/bin/
➜ ~ ln -s /usr/local/containerd/bin/nerdctl /usr/local/bin/nerdctl
➜ ~ nerdctl version

Sample output:

WARN[0000] unable to determine buildctl version: exec: "buildctl": executable file not found in $PATH 
Client:
 Version:    v1.7.5
 OS/Arch:    linux/amd64
 Git commit: cffed372371dcbea3dc9a646ce5a913fc1c09513

Server:
 containerd:
  Version: v2.0.0-beta.2-33-g96bf529cb
  GitCommit: 96bf529cbf55940ddb96bb8adc8be51b11922ebb
 runc:
  Version: 1.1.4
  GitCommit: v1.1.4-0-g5fd4c4d

2. Command Overview

Container Management

Run
The nerdctl run command works similarly to docker run. For example:

nerdctl run -d -p 80:80 --name=nginx --restart=always nginx:alpine

➜ ~ nerdctl run -d -p 80:80 — name=nginx — restart=always nginx:alpine

Available options are consistent with Docker, such as -i, -t, --cpus, and --memory. Use nerdctl run --help for more options.

Exec
Execute commands within a container:

nerdctl exec -it <container_id> date

➜ ~ nerdctl exec -it <container_id> date

List Containers (ps)
List all containers (active containers by default):

nerdctl ps

For Kubernetes containers, use the -n k8s.io namespace flag:

nerdctl -n k8s.io ps

Inspect
Display detailed container information:

➜ ~ nerdctl -n k8s.io inspect <container_id>

Logs
Fetch container logs:

➜ ~ nerdctl -n k8s.io logs <container_id>

Supports options like -f, -t, -n, --since, and --until.

Stop and Remove
Stop and delete containers:

➜ ~ nerdctl -n k8s.io stop <container_id>
➜ ~ nerdctl -n k8s.io rm <container_id>

Use -f or --force for forced removal.

Image Management

List Images
View available images:

➜ ~ nerdctl -n k8s.io images

Pull Images
Pull an image from a registry:

➜ ~ nerdctl -n k8s.io pull <image_name>:<tag>

Other Commands
nerdctl supports other image-related commands like build, tag, push, and inspect. Use nerdctl image --help for a complete list.

This guide highlights key nerdctl commands, demonstrating its compatibility with Docker-like workflows while leveraging containerd’s efficiency.

CRICTL Usage Guide

crictl traces its roots to the Container Runtime Interface (CRI) introduced in Kubernetes version 1.5. CRI standardizes the interface between Kubernetes and container runtimes, enabling Kubernetes to interact seamlessly with various runtimes like Docker, containerd, and CRI-O.

As Kubernetes evolved, the demand for more features and better performance from container runtimes grew. CRI addressed these needs by providing a standardized interface for runtime implementation, simplifying the development of custom runtimes.

In this context, crictl was developed as a command-line tool to interact with CRI-compliant container runtimes. It facilitates managing containers and images, querying container states and metadata, and viewing logs. By streamlining runtime management and debugging, crictl promotes CRI adoption.

Below are the primary uses and features of crictl:

1. Container Management: Start, stop, delete, and query container states.
2. Image Management: Pull, push, delete, and inspect container images.
3. Log Viewing: View stdout and stderr logs of containers.
4. Container Information: Query detailed container information such as ID, name, status, and IP.
5. Metadata Management: Annotate and tag containers.

crictl is a robust tool for managing containers and images within a runtime, debugging containerized applications, and integrating with orchestration systems like Kubernetes.

1. Installation

First, install crictl by downloading the appropriate binary from the cri-tools release page. Extract the binary to a PATH location:

➜ ~ VERSION="v1.22.0"
➜ ~ wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz
# Use a mirror for faster downloads if needed:
# wget https://download.fastgit.org/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz
➜ ~ tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin
➜ ~ rm -f crictl-$VERSION-linux-amd64.tar.gz
➜ ~ crictl -v
crictl version v1.22.0

If the version command runs successfully, the crictl tool is installed.

2. Command Overview

After installation, configure the default settings in /etc/crictl.yaml by specifying the runtime and image endpoint addresses:

runtime-endpoint: unix:///var/run/containerd/containerd.sock
image-endpoint: unix:///var/run/containerd/containerd.sock
debug: false
pull-image-on-create: false
disable-pull-on-run: false

Once configured, you can start using crictl.

Pod Management

Use the crictl pods command to list the running pods on a node:

➜ ~ crictl pods
POD ID CREATED STATE NAME NAMESPACE ATTEMPT RUNTIME
1f617ebf0524c 8 weeks ago Ready node-exporter-ggjwf node-exporter 0 (default)

Filter pods by name or label:

➜ ~ crictl pods - label app=node-exporter
POD ID CREATED STATE NAME NAMESPACE ATTEMPT RUNTIME
1f617ebf0524c 8 weeks ago Ready node-exporter-ggjwf node-exporter 0 (default)

Image Management

List all images with crictl images:

➜ ~ crictl images
IMAGE TAG IMAGE ID SIZE
hub.cloud.ctripcorp.com/prom/node-exporter v1.4.0 d3e443c987ef4 11.5MB

View detailed image information with the -v flag:

➜ ~ crictl images -v
ID: sha256:d3e443c987ef405e1be101647873d86b5729c9c47bb1dd1ab59ccb24bc9e322c
RepoTags: docker.io/prom/node-exporter:v1.4.0
…

Container Management

List running containers using crictl ps:

➜ ~ crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
e7f156d31942f d3e443c987ef4 2 days ago Running main 2 bf1704937991a

Filter containers by status with the -s option:

➜ ~ crictl ps -s Exited
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
22a0adacf702f d3e443c987ef4 2 days ago Exited main 1 bf1704937991a

Execute commands within a container:

➜ ~ crictl exec -it e7f156d31942f whoami
root

View container logs:

➜ ~ crictl logs e7f156d31942f
ts=2024–11–15T08:37:07.980Z caller=node_exporter.go:182 level=info msg="Starting node_exporter"
…

Follow logs in real time using -f or limit output with --tail N.

Monitor container resource usage with crictl stats:

➜ ~ crictl stats e7f156d31942f
CONTAINER NAME CPU % MEM DISK INODES
e7f156d31942f main 0.00 37.13MB 0B 18

For more features, refer to kubernetes-sigs/cri-tools.

Links

1. Kubernetes Container Runtimes
2. Zhihu Article
3. Qikqiak Guide
4. CRI API Reference
5. OpenNaru Article
6. Nerdctl Presentation
7. Alibaba Cloud Blog