Day-16 | AKS vs Self Managed Kubernetes Clusters | Pros and Cons | Azure Interview Questions

This chapter dives deep into the architecture, operational overhead, and trade-offs of running Kubernetes in the cloud. We explore the core differences between managed Kubernetes services like Azure Kubernetes Service (AKS) and Self-Managed Kubernetes Clusters (such as running clusters on raw virtual machines via kubeadm), evaluate their operational pros and cons, and prepare for top-tier cloud architecture interview questions.

1. Why Do We Need Kubernetes and Where Do Clusters Live?

Before comparing managed versus self-managed clusters, it is essential to review what a Kubernetes cluster actually consists of and how it functions in a production environment.

At its core, every Kubernetes cluster is divided into two primary planes:

• Control Plane (Master Nodes): Manages the state of the cluster, schedules pods, responds to cluster events, and runs the API server (kube-apiserver), etcd (database), controller manager, and scheduler.
• Worker Planes (Worker Nodes): The underlying virtual or physical machines where user containers and applications actually run, managed by kubelet and kube-proxy.

When deploying Kubernetes, infrastructure engineers face a fundamental architectural choice:

1. Self-Managed Kubernetes (DIY): Provision raw virtual machines (e.g., Azure VMs, AWS EC2, or bare-metal servers), install an operating system, configure networking, and manually bootstrap Kubernetes using tools like kubeadm, kops, or Ansible.
2. Managed Kubernetes (Platform-as-a-Service): Use a cloud-native managed service like AKS (Azure Kubernetes Service), EKS (Amazon Elastic Kubernetes Service), or GKE (Google Kubernetes Engine) where the cloud provider manages the control plane for you.

2. Core Mental Model: Renting a House vs. Building from Scratch

Real-Life Analogy:

• Self-Managed Kubernetes (Building a House): You buy an empty plot of land, pour the foundation, lay the bricks, install the plumbing, wire the electricity, and build the roof yourself. While you have 100% control over every single pipe and electrical outlet, if a pipe bursts at 3:00 AM, you are the plumber responsible for fixing it.
• Managed Kubernetes / AKS (Living in a Managed Apartment Complex / Gated Community): The property management company takes care of the structural integrity, security gates, elevators, plumbing infrastructure, and roof maintenance. You simply furnish your apartment and live in it. If the central water pump breaks down, the maintenance team fixes it. You focus entirely on your living experience (running applications).

3. Architecture Breakdown: Self-Managed vs. AKS

Understanding who owns which layer of the infrastructure stack is crucial for cloud architects and DevOps engineers.

4. Deep Dive into Self-Managed Kubernetes Clusters

Running your own Kubernetes cluster on raw virtual machines gives you absolute freedom, but it introduces massive operational complexity.

How a Self-Managed Cluster is Built

1. Provision VMs: Create 3 Master VMs and $N$ Worker VMs using Azure Virtual Machines or Terraform.
2. OS Configuration: Disable swap space, configure iptables, load kernel modules (br_netfilter), and configure container runtimes (such as containerd or Docker).
3. Cluster Initialization: Run kubeadm init on the primary master node to bootstrap the control plane components (etcd, API server, scheduler).
4. Join Worker Nodes: Run kubeadm join on worker nodes to connect them to the control plane.
5. Networking (CNI): Install a Container Network Interface plugin like Calico, Flannel, or Cilium to enable pod-to-pod communication across nodes.

Pros of Self-Managed Clusters

• Absolute Control: You have root access to master nodes, enabling custom API server flags, experimental features, and deep kernel-level tuning.
• Cloud Agnostic / Vendor Lock-in Avoidance: The exact same kubeadm scripts can run on AWS, Azure, On-Premises bare-metal servers, or GCP without relying on proprietary cloud APIs.
• Strict Compliance & Air-Gapped Environments: Ideal for disconnected or air-gapped data centers where external cloud APIs cannot be reached.

Cons of Self-Managed Clusters

• High Operational Burden: Your team must handle master node patching, OS upgrades, and storage maintenance 24/7.
• etcd Disaster Recovery Risk: If etcd corrupts or loses quorum due to a bad master node failure, recovering the cluster state can lead to catastrophic data loss without rigorous backup routines.
• Scaling Complexity: Implementing cluster autoscaling and node self-healing requires stitching together custom cloud provider integrations (e.g., Azure Cloud Provider plugins).

5. Deep Dive into Azure Kubernetes Service (AKS)

AKS is Microsoft Azure’s managed Kubernetes service that abstracts away the heavy lifting of control plane management.

Key Characteristics of AKS

• Free Control Plane: Azure provides the Kubernetes control plane completely free of charge; you only pay for the underlying worker node virtual machines.
• Managed Upgrades: AKS provides automated or one-click Kubernetes version upgrades for both control plane and worker nodes.
• Integrated Azure Ecosystem: Native integration with Azure Active Directory (Microsoft Entra ID) for Role-Based Access Control (RBAC), Azure Key Vault for secrets management, Azure Monitor, and Azure Container Registry (ACR).

Pros of AKS

• Drastically Reduced Operational Toil: No need to SSH into master nodes, manage etcd backups, or patch control plane operating systems.
• Built-in High Availability: AKS automatically replicates control plane components across Azure Fault Domains and Availability Zones behind a managed load balancer.
• Seamless Scalability: Effortless node pool scaling, cluster autoscaling, and integration with Azure Virtual Nodes (based on Virtual Kubelet for serverless container scaling via Azure Container Instances).
• Advanced Security: Integrated vulnerability scanning, Azure Defender for Kubernetes, and private cluster endpoints (--enable-private-cluster).

Cons of AKS

• Opinionated Defaults: While highly configurable, you are bound by Azure’s networking stack (Kubenet vs. Azure CNI), underlying VM sizes, and managed upgrade cadences.
• Cloud Vendor Lock-In: Deep integration with Azure services (Managed Identities, ACR, Azure CNI) makes migrating the entire cluster infrastructure to another cloud provider (e.g., AWS EKS) challenging.
• Hidden / Edge Costs: While the control plane is free, ancillary services like Azure Application Gateway Ingress Controller (AGIC), NAT Gateways, and Premium Managed Disks can increase overall cloud spend.

6. Comprehensive Feature Comparison Table

7. Production Pitfalls and Debugging Realities

When managing Kubernetes clusters in enterprise environments, engineers frequently encounter specific failure modes:

1. etcd Quorum Loss (Self-Managed):

   • Failure: In a 3-master self-managed cluster, if 2 masters crash simultaneously, etcd loses quorum (majority quorum = (n/2) + 1). The cluster becomes read-only, and scheduling halts.
   • Fix: Maintain strict automated snapshot backups of etcd using etcdctl snapshot save and practice disaster recovery restoration drills.

2. Node Pressure and Evictions:

   • Failure: Worker nodes run out of disk space (DiskPressure) or memory (MemoryPressure) due to runaway log files or uncleaned container images.
   • Fix: Configure proper resource requests and limits (resources.requests / resources.limits), set up Kubernetes eviction thresholds, and configure automatic container image pruning on worker nodes.

3. CNI IP Exhaustion (AKS with Azure CNI):

   • Failure: In default Azure CNI mode, every pod receives a real IP address from the virtual network (VNet) subnet. If pods scale rapidly, the VNet subnet quickly exhausts its IP address pool.
   • Fix: Use Azure CNI Overlay, where pod IPs are assigned from a private logical address space independent of the host VNet, conserving Azure VNet IP addresses.

8. Top Azure Interview Questions & Expert Answers

Interviewers frequently test candidates on their real-world understanding of managed versus self-managed Kubernetes architectures.

Q1: Why would any enterprise choose Self-Managed Kubernetes over AKS?

Answer: While AKS is the preferred choice for 90% of workloads, companies choose self-managed Kubernetes for specific edge cases:

• Multi-Cloud / Hybrid Portability: Running identical clusters across on-premises data centers, AWS, and Azure without relying on proprietary cloud control planes.
• Strict Regulatory / Air-Gapped Environments: Environments disconnected from the public internet where cloud provider API management is impossible.
• Deep Customization: Needing custom kernel modules, experimental API server flags, or specialized admission controllers that managed services restrict.

Q2: How does AKS handle control plane high availability?

Answer: Azure automatically deploys the AKS control plane across multiple fault domains and availability zones behind a load-balanced endpoint. If a master node hosting the API server or etcd fails in one zone, Azure instantly routes traffic to healthy replicas in other availability zones with zero downtime for cluster management.

Q3: Explain the difference between Kubenet and Azure CNI in AKS.

Answer:

• Kubenet: A basic, lightweight networking plugin where nodes get IPs from the Azure VNet, but pods receive internal IP addresses from a separate logical bridge network. It requires routing tables but conserves VNet IP addresses.
• Azure CNI: Every single pod gets a direct IP address from the Azure VNet subnet. This enables flat networking where pods can communicate directly with external VMs and resources on the VNet, but it consumes VNet IPs rapidly.

Summary and Key Takeaways

• Managed vs. DIY: AKS removes the heavy operational burden of managing control planes, etcd databases, and master node patching, allowing engineering teams to focus entirely on application delivery.
• Cost Efficiency: AKS offers a free control plane, meaning organizations only pay for worker node compute, whereas self-managed clusters require paying for 24/7 master VM compute instances.
• Architectural Control: Self-managed clusters provide absolute root access and custom kernel tuning at the cost of significantly higher operational toil and disaster recovery risks.
• High Availability: AKS handles control plane HA and automated backups out-of-the-box, whereas self-managed clusters require manual configuration of multi-master etcd quorums.
• Ecosystem Integration: AKS integrates natively with Azure Active Directory (Microsoft Entra ID), Azure Key Vault, Azure Monitor, and Azure Container Registry (ACR).
• Networking Strategy: Architects must carefully evaluate Azure CNI vs. Kubenet (or Azure CNI Overlay) to prevent subnet IP exhaustion as container workloads scale.
• Interview Readiness: Top cloud interviews focus heavily on trade-offs involving operational overhead, compliance, multi-cloud strategy, and disaster recovery.