Kubernetes: Orchestration Rant

What to do when Kubernetes isn't playing nice with your application orchestration needs?

Written by Rory Savage in October 16, 2018   |   Link

By now those of you who have worked with large microservice based applications will have discovered that the HELM packaging tool is a lifesaver. So what does this have to do with orchestration in Kubernetes? Everything!

Say you are like me, and you have to deploy a large number of PODs into your K8s ecosystem. Futhermore, you have been working with HELM, creating parent and child charts, only to find out that while HELM allows you do have dependencies, HELM nor K8s has the ability to track POD deployment status. In other words, if pieces of your application are dependant on other PODs starting up and becoming healthy first, there's no current way to support that control with HELM or Kurbernetes. So what can you do?

An idea of running AppController was presented to me today, however that project is very bleeding edge and it's lack of support and documentation turned me off really quick. The second idea I had was to add additional initContainers to my PODs which runs a script to poll the ecosystem for specific POD healthyness. Once that POD's healtyness returned sucessful, those initContainers would die off unblocking the other PODs from starting. This works, and works today -- but just imagine if you have a nested chain of application dependencies of 4 or even 5 levels deep? Ya, gets messy quick, Right? While I understand true microservices should be designed in a way to eliminate that behavior, unfortunatley that is not always the case in the real world. For now, initContainers with polling is the way to go.