Kubernetes on AWS - Kops

Introduction

Kubernetes is the leading container orchestration solution. It promises to standardize the way you run applications, without worrying if you are running on bare-metal, on a public cloud provider or on a private cloud.

AWS being the leading public cloud solution, it is important to be able to run Kubernetes easily on this provider. In this post, I will show you how to create a production-ready Kubernetes cluster on AWS from scratch. In a future post, I will explain how to run a fairly complex application -Gitlab- with complete high availability on this cluster.

This post assumes some knowledge of both AWS and Kubernetes. You should be familiar with the following elements.

In AWS:

For Kubernetes:

  • What Kubernetes is (obviously),
  • kubectl,
  • The basic architecture of Kubernetes.

Kops

So far, running Kubernetes (K8s) on AWS has always been challenging and quite long if you wanted a production cluster. The kube-up.sh script, while quickly creating a cluster, is nowhere near sufficient for production: no auto-scaling group, no VPC management, etc.

This is changing, mainly thanks to Kops. Kops is a tool under active development by the K8s AWS special interest group. Written in Go, its goal is to manage K8s clusters. You can now provision and customize far better installations:

  • Auto-scaling for the workers (for self-healing),
  • Possible HA for the master(s),
  • Customization of the VPC and subnets used,
  • Inter-pod networking is done via native AWS routing by default,
  • Possibility to export a corresponding Terraform stack.

Creating a simple cluster with Kops

For the purpose of this article, I will create a simple cluster: one master, 3 nodes in 3 availability zones. Here is how to do it with Kops. You need:

  • To have a profile declared in ~/.aws/credentials,
  • To create a S3 bucket to store Kops state,
  • To have a Route53 Hosted Zone in the same AWS account (Kops will use it to create records needed by the cluster).

You can then declare your cluster:

export AWS_PROFILE=my-profile
export KOPS_STATE_STORE=s3://my-kops-bucket
kops create cluster --cloud=aws \
  --dns-zone=k8s.myzone.net --master-size=t2.medium \
  --master-zones=eu-west-1a \
  --network-cidr=10.0.0.0/22 --node-count=3 \
  --node-size=m4.large \
  --zones=eu-west-1a,eu-west-1b,eu-west-1c \
  --name=k8s.myzone.net

At this stage, only the cluster’s description is created, not the cluster itself. You can still change any option by editing your cluster:

kops edit cluster k8s.myzone.net

And when you are satisfied, you can actually create the cluster:

kops update cluster k8s.myzone.net --yes

After a few minutes, your kubectl should be able to connect to the newly created cluster:

kubectl get nodes
NAME                                           STATUS    AGE
ip-10-0-0-159.eu-west-1.compute.internal       Ready     1d
ip-10-0-0-213.eu-west-1.compute.internal       Ready     1d
ip-10-0-1-105.eu-west-1.compute.internal       Ready     1d
ip-10-0-1-208.eu-west-1.compute.internal       Ready     1d

Upgrading a cluster

Kops also allows to easily upgrade a running Kubernetes cluster with no downtime, provided you have a multi-master setup. Just edit the cluster and change the kubernetesVersion:

kops edit cluster k8s.myzone.net

When you apply the update, Kops will automatically do a rolling-upgrade of the cluster:

kops update cluster k8s.myzone.net --yes
kops rolling-update cluster k8s.myzone.net --yes

Conclusion

Kops allows to create and easily manage Kubernetes clusters on AWS, with real production in mind. This solution is far easier and better than anything I have seen to provision K8s on AWS. It also allows a wide range of customizations.

In the next post, I will show how to leverage AWS managed services in conjunction with Kubernetes to run applications (with their databases and data) in a truly highly available fashion.