May 18, 2022

Introduction

I have stumbled upon many projects that don’t share Kustomize manifests, or even Kubernetes manifests, but do have Helm charts.

Not everyone uses Helm, so I wanted to share a method can be semi-easily automated to generate k8s manifests from Helm charts.

What is Kustomize

I need the manifests to use them with Kustomize, which is a templating tool that allows you to “kustomize” the Kubernetes manifests, usually based on environments. As of a few Kubernetes versions ago, kustomize is also built into Kubernetes.

How does Kustomize Work

Introducing kustomize; Template-free Configuration Customization for Kubernetes

Authors: Jeff Regan (Google), Phil Wittrock (Google) If you run a Kubernetes environment, chances are you've customized a Kubernetes configuration - you've copied some API object YAML files and edited them to suit your needs. But there are drawbacks to this approach - it can be hard to go back to the source material and incorporate any improvements that were made to it.

kubernetes.io

Kustomize vs Helm

Here is some reading material if you are interested in the differences between Kustomize and Helm.

Comparing Helm vs Kustomize: Kubernetes Templating Tools

The Kubernetes ecosystem is still relatively new. With Kubernetes hitting GitHub in the Summer of 2014 and the first 1.0 release of Kubernetes available in the Summer of 2015, the pace and advancements are still fresh in the minds of many.

harness.io

K8s Tips : Manifests, Helm, Kustomize

In this article we will use the VotingApp, a sample microservices application, and deploy it in Kubernetes using different approaches and tools. We will see the particularities of each so you can have a good idea which one is the most appropriate for your needs.

itnext.io

Generate Manifests from Helm Charts

I will be using cluster-autoscaler for this example, you can see how the project has charts but not the manifests (external-dns has manifests, for example)

Let’s dive into it.

The first step is to add the autoscaler repository

helm repo add autoscaler https://kubernetes.github.io/autoscaler

Check the latest chart version

helm search repo autoscaler/cluster-autoscaler
NAME                               	CHART VERSION	APP VERSION	DESCRIPTION
autoscaler/cluster-autoscaler      	9.9.2        	1.20.0     	Scales Kubernetes worker nodes within autoscali...
autoscaler/cluster-autoscaler-chart	2.0.0        	1.18.1     	Scales Kubernetes worker nodes within autoscali...

Next, we need to download the latest version

helm fetch autoscaler/cluster-autoscaler --untar --untardir $PWD/tmp --version 9.9.2

I would suggest taking the values.yaml and modifying it to suite your needs. values.yaml should contain the unique values for the changes in our environment compared to the default values that the helm chart gives us.

helm template $PWD/tmp/cluster-autoscaler --values values.yaml --namespace cluster-autoscaler --output-dir $PWD/tmp --debug

Note that we are hardcoding the namespace above (not required, depends on your deployment strategy), make sure to change/remove it if required.

Once we generate the manifests, we still need to make a few tweaks:

1. (optional) Merge all RBAC related manifests into one. I like to do this as it simplifies the structure
Copy

TMP_LOC=$PWD/tmp/cluster-autoscaler/templates && cd $TMP_LOC && cat clusterrole.yaml clusterrolebinding.yaml role.yaml rolebinding.yaml serviceaccount.yaml > rbac.yaml && cd -

2. Move the following into the base directory from tmp/cluster-autoscaler/templates
• deployment.yaml
• pdb.yaml if there are any changes
• service.yaml if there are any changes
• newly generated rbac.yaml
3. There are some values that we can't replace via values.yaml as they are built into Helm, so we have to remove them
Copy

sed -i '' '/app.kubernetes.io\/instance/d' base/*.yaml
sed -i '' 's/RELEASE-NAME-cluster-autoscaler/cluster-autoscaler/g' base/*.yaml

Now all that is left is to create a kustomize manifest and test locally before adding it to version control.

I hope this has been useful, have a wonderful day.

Cover Photo by Syed Hussaini on Unsplash

Using external-dns to Migrate Services between Kubernetes Clusters

🔀

Using external-dns to Migrate Services between Kubernetes Clusters