Why I built a Kubernetes cluster with Raspberry Pi’s

 

 

What is Kubernetes?

If you haven’t heard of Kubernetes by now then you are either not working in tech, and the rest of this post is going to make zero sense; or you have been living under a rock for a few years.

Kubernetes is a “container orchestration platform” - to oversimplify it somewhat: it runs your docker images for you, manages scaling and redundancy for you. The reality is that it does much more than that.

Kubernetes is clearly winning the battle between container orchestration platforms; and while many people understand the fundamental concepts, many people don’t know the intricacies of what it all actually means.

 

Why build your own cluster?

Cloud providers make it easy for you to spin up clusters and deploy applications and do all the heavy lifting for you. “It’s not worth the hassle to actually deploy your own cluster” you are told; and for the most part this is sound advice, but doing so gives you a better understanding of the technology and if you are like me, a problem solver, it’s quite fun.

Having worked with and set up a few kubernetes platforms, I have been left with an itch to try out some new concepts and play around a bit more with some more advanced kubernetes concepts.

I am also forever not satisfied just accepting the inner workings of things, so was keen to have a go at actually setting up a cluster from scratch.

As a side project I could have done this on AWS, and spun up a few EC2 instances, but I also have a bit of an obsession with raspberry pi’s and it seemed like a great opportunity to bring these things together.

 

How did I do It?

A quick google and it comes as no surprise that I am not the first person to think of this, so much so that there are numerous blogs out there that will talk you through how to do this, and I am going to keep this as an overview rather than a technical guide, although I have created a repo with my setup scripts and steps here: https://github.com/weareslate/basic-bramble

Hardware

Well I already said I was going to use Raspberry Pi - but which ones? As I opted for full blown kubernetes I have opted for the latest Pi 4’s - so really all you need is a couple of these. But I want a cluster, and I want to play with redundancy, so I opted for four of them. I also want a neat solution, not a pile of little boards and power supplies, so I found a neat stackable case, and the idea of using PoE to power them, so my shopping list was:

  • 4 x Raspberry Pi 4
  • 4 x SD Cards
  • Stackable case
  • 5 port PoE switch
  • 4 x Raspberry Pi PoE hats
  • 5 CAT5 cables
  • Extra spacers (because the PoE hats make the Pis taller)

 

Software

I’ve got it down to a few simple steps (but please read the readme for more detail):

  • Flash an SD card for each of your Pi’s
  • Copy over my scripts onto the SD card before inserting it into your Pis
  • Boot your Pis
  • SSH into each one and run one of two scripts:
    • init-master.sh on your master node
    • init-node.sh on your worker nodes

 

What did I learn?

  • A cluster of Raspberry Pis is called a “Bramble”
  • It’s actually a lot easier than you would think to setup a cluster from scratch.
  • Solving some of the issues that I did come up with gave me a greater appreciation for the network providers, which use different techniques to allow pods to talk to each other. There are dozens available, it’s quite overwhelming at first - Cloud providers have their own offerings which work well with their platform; but there are plenty of other options to choose from. I went with Flannel, which is popular, and seems to play nicely with Raspberry Pis. Now that I have my cluster up and running I would like to try out some of the different options and understand their nuances better; watch this space.
  • Ingress controllers outside of the cloud are a bit trickier - the load balancers that they provide are quite essential to a production system - if you want to serve up applications running on your cluster you are going to need a router to act as a load balancer, or route traffic through a single node and accept that if it dies your app may move to another node, but you will need to find a way to redirect traffic, and it’s not going to be zero downtime.
  • Pi 4s get quite warm - buy a heatsink and fan!

 

What next?

In just an evening I was able to go from a handful of components, to a working kubernetes cluster, but there's plenty more to do, watch out for more posts!

  • Find a neat solution to the lack of load balancer, that will give minimal downtime.
  • Try out different network providers and service meshes
  • Seeing as each of my nodes has GPIO pins - what can I do with these? I’m going to start with some RGB LEDs to visualise the load on each node.

If you want to find out more, or need someone to setup or manage your kubernetes cluster - get in touch.

fsqs registered logo