Kubernetes Federation With Google Global Load Balancer

LET’S FEDERATE AN APP

Let’s use GCK to create a 4 cluster federation, and Global Load Balancer to distribute the load. We will use a simple Python app that returns to the region in which it is located.
What you need here is an account in GCP and gcloud CLI installed. Let’s start!

First, we need the DNS zone and clusters’ nodes, the latter of which needs permission to add entries to it. The Federation uses it to provide DNS records for local cross cluster service discovery.

gcloud automatically add clusters to our ~/.kube/config so we can check them using kubectl config get-contexts

Let’s choose the cluster on which we will install the Federated Control Plane (it should be the closest one to you):

and check if we can communicate with it:

Now we can install the Control Plane:

It then appears as a new context:

Currently there are no clusters attached to it so let’s add all our clusters:

and wait until they become ready:

With our 4 cluster federation ready, it’s now time to deploy the get-region app and wait until we have 16 pods ready

Now we create the Federated Service of NodePort type with the 30040 port exposed on each clusters’ node, you can check that the Deployment and Service has been propagated by checking one of the clusters:

You can see that we have used the NodePort type, the reason behind this is that we can now easily create a Global Load Balancer that will connect with all our clusters.

The easiest way to do it is to create an Ingress on the Federation Control Plane and GKE will automatically create and connect the Global Load Balancer.

You can check the status of the Load Balancer here. After you see 3/3 Healthy for all backends, it means it is ready.

The next thing to do is is to test if it really works. Do you remember that our Python app returns to the region in which it is located? In order to check that I have created 3 DigitalOcean nodes in San Francisco, New York and London, I run a curl on each of them.

By going to the Monitoring tab in the Load Balancer list page you can see how the traffic is distributed:

CONCLUSIONS

The results look impressive and this simple setup can be the first step to some great enterprise grade solutions like a custom CDN or a multi-region web application.

Is it ready to use in practice? We believe so, we already use it with confidence but it does require some additional effort.

The Control Plane uses a single separate etcd – this has to be adjusted to use a dedicated etcd cluster to provide minimal fault tolerance.
On GKE, the Global Load Balancer is an ideal way to distribute traffic. It is very much the way to go as using AWS or Azure could be harder due to the reliance on DNS, which is not so flexible.

Last, but not least, it is also worth mentioning that there is a pending proposition to make Helm work with Federation here and here. This is great news because it would make it easier to maintain the federated application whether it be internal or open source.