Kubernetes Internals: How Container Orchestration Works

Kubernetes has become the de facto standard for deploying, managing, and scaling containerized applications. While its benefits—improved resource utilization, high availability, and simplified operations—are widely understood, the underlying mechanisms that enable this powerful orchestration often remain a black box. For technical audiences like software engineers and system architects, a deeper understanding of how Kubernetes works internally is crucial for effective design, troubleshooting, and optimization.

This article will pull back the curtain on Kubernetes, dissecting its core architecture and components. We’ll explore the fundamental principles that govern its operation, examine the intricate dance between its control plane and worker nodes, and illustrate how these elements collaborate to maintain your desired application state.

The Kubernetes Philosophy: Desired State Management

At its heart, Kubernetes operates on a declarative model of desired state management. Instead of issuing imperative commands (e.g., “start 3 instances of my app”), users declare what they want the system to look like (e.g., “I want a deployment named ‘my-app’ with 3 replicas of this container image, accessible via this service”). Kubernetes then continuously works to reconcile the current state of the cluster with this desired state.

This reconciliation is performed by controllers, which are specialized loops that watch for changes to specific resource types, compare the actual state to the desired state, and take corrective actions. This fundamental concept makes Kubernetes incredibly resilient and self-healing; if a component fails, the system automatically detects the discrepancy and works to restore the desired state without human intervention^[1].

Kubernetes cluster architecture with control plane and worker nodes — Photo by Fay Lee on Unsplash

Kubernetes Architecture Overview

A Kubernetes cluster is fundamentally composed of two main types of nodes:

The Control Plane (Master Node): This is the brain of the cluster. It exposes the API, schedules workloads, and manages the cluster’s state.
Worker Nodes: These are the machines where your actual applications (containers) run.

Each of these components runs several processes that collaborate to provide the orchestration capabilities. Understanding these individual pieces and their interactions is key to grasping Kubernetes’ operational model.

The Control Plane: The Brains of the Cluster

The control plane components are responsible for making global decisions about the cluster (e.g., scheduling) and detecting and responding to cluster events (e.g., starting up new pods when a replicas field is unsatisfied).

API Server (kube-apiserver)

The API Server is the front-end of the Kubernetes control plane. It exposes the Kubernetes API, which is the central communication hub for all cluster operations. All internal and external communication goes through the API Server.

Role: Processes REST requests, validates objects, and persists the cluster state.
Interaction: All other control plane components, worker nodes (via Kubelet), and external users (via kubectl) communicate exclusively with the API Server. It acts as the single source of truth for the cluster.

etcd

etcd is a consistent and highly available key-value store that Kubernetes uses as its backing store for all cluster data. All cluster configuration data, state, and metadata are stored here.

Role: Provides a reliable source of truth for the desired state of the cluster.
Interaction: The API Server is the only component that directly communicates with etcd. All other components indirectly interact with etcd by communicating with the API Server, which then reads from or writes to etcd. This ensures data consistency and prevents direct manipulation.

Scheduler (kube-scheduler)

The Scheduler is responsible for assigning newly created Pods to available Worker Nodes. It continuously monitors for new Pods with no assigned node and selects the best node for them based on various factors.

Role: Determines where Pods should run, considering resource requirements, policy constraints, affinity/anti-affinity rules, and quality of service (QoS) requirements.
Interaction: Watches the API Server for new Pods. Once a suitable node is found, it updates the Pod’s status via the API Server, binding the Pod to the chosen node.

Controller Manager (kube-controller-manager)

The Controller Manager runs various controller processes that regulate the state of the cluster. Each controller is a reconciliation loop attempting to move the current state closer to the desired state.

Role: Manages different types of controllers, such as:
- Node Controller: Notifies and responds when nodes go down.
- Replication Controller: Ensures the desired number of Pods for a ReplicaSet or ReplicationController is maintained.
- Endpoints Controller: Populates the Endpoints object (which maps Services to Pods).
- Service Account & Token Controllers: Create default service accounts and API access tokens for new namespaces.
Interaction: Monitors the API Server for changes in resources and updates them accordingly.

Cloud Controller Manager (kube-cloud-controller-manager)

This component runs controllers specific to the underlying cloud provider. It allows cloud-provider-specific control logic to be linked into the cluster without tightly coupling Kubernetes to any specific cloud.

Role: Manages cloud resources like load balancers, persistent volumes, and network routing.
Interaction: Interacts with the API Server and the cloud provider’s APIs.

Worker Nodes: Where the Work Happens

Worker nodes are the machines (physical or virtual) that run your containerized applications. Each worker node has the necessary components to run containers, communicate with the control plane, and handle network traffic.

Kubernetes worker node components diagram — Photo by Shubham Dhage on Unsplash

Kubelet

The Kubelet is an agent that runs on each worker node. It’s the primary “node agent” that ensures containers are running in a Pod.

Role: Registers the node with the cluster, receives Pod specifications from the API Server, and ensures that the containers described in those Pods are running and healthy. It does not manage containers that were not created by Kubernetes.
Interaction: Communicates with the API Server to register the node and report Pod status. It interacts with the container runtime to start, stop, and manage containers.

Kube-proxy

The Kube-proxy is a network proxy that runs on each node in the cluster. It maintains network rules on nodes, allowing network communication to your Pods from inside or outside of the cluster.

Role: Implements the Kubernetes Service abstraction, providing network connectivity and load balancing for Pods. It typically uses the operating system’s packet filtering layer (e.g., iptables or IPVS) to forward traffic to the correct Pods.
Interaction: Watches the API Server for changes to Service and Endpoint objects and updates the node’s network rules accordingly.

Container Runtime

The Container Runtime is the software responsible for running containers. Kubernetes supports several container runtimes through the Container Runtime Interface (CRI).

Role: Pulls container images, starts and stops containers, and manages container resources.
Examples: Containerd, CRI-O, or Docker Engine.
Interaction: Kubelet uses the CRI to interface with the chosen container runtime.

Key Abstractions: Building Blocks of Applications

Kubernetes provides higher-level abstractions that simplify the management of containerized applications, all of which ultimately interact with the control plane to achieve their desired state.

Pods: The smallest deployable unit in Kubernetes. A Pod represents a single instance of a running process in your cluster and can contain one or more tightly coupled containers that share network and storage resources.
Deployments: A controller that manages a set of identical Pods. Deployments provide declarative updates to Pods and ReplicaSets, enabling rolling updates, rollbacks, and scaling.
Services: An abstract way to expose an application running on a set of Pods as a network service. Services provide stable network identities and load balancing for dynamic Pods.
ConfigMaps & Secrets: Used to inject configuration data (ConfigMaps) and sensitive information (Secrets) into Pods, separating configuration from application code.
Volumes & Persistent Volumes: Provide durable storage for Pods, independent of their lifecycle, allowing data to persist even if Pods are restarted or moved.

The Reconciliation Loop in Action: A Deployment Example

Let’s trace how a simple application deployment unfolds within Kubernetes:

A developer uses kubectl to apply a Deployment YAML manifest.
The kubectl command sends this YAML to the API Server.
The API Server validates the request, authenticates the user, and persists the Deployment object into etcd.
The Controller Manager’s Deployment Controller continuously watches the API Server for new or updated Deployment objects. It detects the new Deployment.
Based on the Deployment’s specification (e.g., replicas: 3), the Deployment Controller creates a ReplicaSet object, also persisted via the API Server to etcd.
The Controller Manager’s ReplicaSet Controller detects the new ReplicaSet and realizes that no Pods exist for it yet. It then creates three Pod objects, which are also stored in etcd via the API Server.
The Scheduler watches the API Server for new Pods that don’t have a node assigned. It finds the three new Pods.
The Scheduler evaluates available worker nodes, considering resources, constraints, and other factors, and selects the optimal node for each Pod. It then updates each Pod object in etcd (via the API Server) with the chosen node’s name.
On each chosen Worker Node, the Kubelet is continuously watching the API Server for Pods scheduled to its node.
The Kubelet retrieves the Pod definition, instructs the Container Runtime (e.g., Containerd) to pull the specified container image(s) and start the containers within the Pod.
The Kubelet then continuously monitors the running containers, reporting their status and events back to the API Server. If a container fails, the Kubelet attempts to restart it. If the entire Pod fails, the ReplicaSet Controller will detect the discrepancy and create a new Pod.
Meanwhile, the Kube-proxy on each worker node watches for new Service objects. If a Service is created to expose our application, Kube-proxy configures the node’s network rules (e.g., iptables) to direct traffic to the Pods managed by the Deployment.

This continuous feedback loop, where controllers observe the actual state and drive it towards the desired state, is the essence of Kubernetes’ operational power.

Conclusion

Kubernetes is a sophisticated distributed system designed to manage the complexity of containerized applications. Its architecture, centered around a powerful control plane and distributed worker nodes, works tirelessly to maintain the desired state of your applications. By understanding the roles of the API Server, etcd, Scheduler, Controller Manager, Kubelet, and Kube-proxy, engineers gain critical insights into how their applications are deployed, scaled, and healed automatically. This foundational knowledge empowers teams to build more robust, resilient, and efficient cloud-native systems, truly harnessing the power of modern container orchestration.

References

[1] Kubernetes. (n.d.). Kubernetes Components. Available at: https://kubernetes.io/docs/concepts/overview/components/ (Accessed: November 2025) [2] Google Cloud. (n.d.). Kubernetes architecture. Available at: https://cloud.google.com/kubernetes-engine/docs/concepts/architecture (Accessed: November 2025) [3] Burns, B., Beda, J., & Faro, K. (2019). Kubernetes Up & Running: Dive into the Future of Infrastructure. O’Reilly Media. (Accessed: November 2025) [4] The Linux Foundation. (2020). Kubernetes Fundamentals. Available at: https://training.linuxfoundation.org/training/kubernetes-fundamentals/ (Accessed: November 2025)