ADR 002: Kubernetes Platform — Managed EKS Over Self-Managed

DevOps Studio › Docs › Architecture Decisions › ADR 002

Status: Accepted

Context

Lab 02 needs a Kubernetes platform that a small platform team can operate without taking on control-plane engineering as a full-time job. The realistic options for a customer running on AWS are:

1. Self-managed Kubernetes (kubeadm or similar on EC2) — full control over every component, but the customer owns etcd backups, control-plane HA, version upgrades, and CVE patching for the control plane itself.
2. Amazon EKS — AWS operates and patches the control plane; the customer still owns worker nodes, add-ons, and workload configuration.
3. A fully managed platform on top of Kubernetes (e.g. a vendor PaaS) — least operational burden, but the tradeoff is control and portability, which is out of scope for a lab about understanding the platform layer itself.

Most customers asking for "a Kubernetes platform" are asking for enough control to run their own workloads and enforce their own policies — not asking to operate etcd.

Decision

Use Amazon EKS for the managed control plane, with a managed node group for compute, and IAM Roles for Service Accounts (IRSA) instead of static credentials for pod-level AWS access.

Specific choices inside that decision, each visible in modules/eks/main.tf:

• Private worker nodes, public API endpoint by default. Nodes sit in private subnets behind NAT; the cluster API endpoint has both private and public access enabled so the lab can be reached from a laptop, but a customer with a VPN or Direct Connect would flip cluster_endpoint_public_access off.
• All five control-plane log types enabled (api, audit, authenticator, controllerManager, scheduler) shipped to CloudWatch — the audit trail a security team asks for first, at essentially no engineering cost since EKS already produces the logs.
• IRSA over node-level IAM roles for AWS access. The node IAM role only carries the three baseline worker-node policies (AmazonEKSWorkerNodePolicy, CNI, ECR read-only). Anything a pod needs — the EBS CSI driver's volume permissions, for example — gets its own IAM role scoped to a specific Kubernetes service account via the OIDC provider, not inherited by every pod on the node.
• The VPC is optional, reusing the same bring-your-own-VPC pattern as ADR 001: if a customer already has a VPC from Lab 01 (or their own), vpc_id is passed in and the module skips creating a second one.

Consequences

Positive:

• No etcd, no control-plane upgrade runbook, no control-plane HA to design — AWS's SLA covers it.
• IRSA means a compromised pod only has the AWS permissions explicitly bound to its service account, not the node's full permission set — a real blast-radius reduction over the node-role-for-everything pattern.
• Reusing Lab 01's VPC (when present) means one network to secure and monitor instead of two.

Negative / tradeoffs:

• EKS control-plane cost (~$0.10/hour flat) applies regardless of workload size — for a lab meant to be torn down after a few hours this is a rounding error, but it's a fixed cost a self-managed cluster on existing EC2 capacity wouldn't have.
• The customer still owns node patching, node-group scaling limits, and add-on version compatibility — "managed" only covers the control plane, not the whole platform.
• Public API endpoint access is a reasonable default for a lab but not for most production, regulated-industry deployments; it needs to be an explicit decision per environment, not an inherited default from this module.