bomonike

Package manager for Kubernetes

Overview

This article is a hands-on introduction about Helm “Charts” used to stand up a Kubernetes cluster.

NOTE: Content here are my personal opinions, and not intended to represent any employer (past or present). “PROTIP:” here highlight information I haven’t seen elsewhere on the internet because it is hard-won, little-know but significant facts based on my personal research and experience.

Helm simplifies discovering and deploying services to a Kubernetes cluster. Helm reduces “configuration sprawl” of different specifications for Dev, Staging, UAT, QA, Prod.

Thus, Helm competes with docker-compose.

Here is the guided tour (aimed to be succinct yet deep):

Visit helm.sh, Helm’s marketing home page (served from https://github.com/helm/helm-www using hugo). It calls helm a package manager for Kubernets (like Brew on macOS, Chocolately on Windows, apt on Debian/Ubuntu, yum on Red Hat, etc). Helm is a cloud industry consortium composed of Google, Microsoft, Bitnami, and others.

Why Helm?

Helm has become popular with cloud developers largely because it simplifies Kubernetes application management, the roll out of updates, and options to share applications. Package management features make it easier to:
- search available packages -
- provide information on packages
- download and install packages, along with dependencies, creation of folders, and insertion of those folders in the system’s PATH variable
- list installed packages
- lint installed packages
- update existing installed packages
- delete packages
PROTIP: Words in Chart names are separated by dashes, not underlines nor dots.
Visit https://github.com/helm/helm where Helm is open-sourced.

Helm was created by Matt Butcher (“The Mister Rogers of Cloud Native” living in Boulder, Colorado, USA) and others at Deislabs.io (acquired by Microsoft in May 2017).

In 2016, Helm was donated Helm to CNCF (the same organization who owns Kubernetes itself).
Visit the latest release at https://github.com/helm/helm/releases

CNCF graduated Helm2 in 2016. Helm3 released in 2019.

helm version on your laptop is a different version sequence.
Visit https://helm.sh/docs

https://helm.sh/docs/glossary

Architecture

The Helm CLI client running on your local machine sends requests to Kubernetes.

This CLI client is needed because operations such as rollback, running chart tests, etc. are done from the Helm CLI client.

No more Tiller in Helm 3

According to https://helm.sh/docs/community/history/

Until Helm3 was released November 2019 with Kubernetes 1.16, a Tiller server (and helm init that starts it) ran inside the Kubernetes cluster to manage (install, upgrade, query, and remove) Kubernetes resources via calls to the Kubernetes API server. [1] Helm3 removed Tiller and shifts to Helm itself the security, identity, and authorization features.

See https://github.com/helm/helm-2to3 for the strangler pattern (co-existing in the same cluster) or in situ (with migration).
```
helm3 plugin list
```
Helm 3 uses Secrets as the default storage driver instead of Helm 2’s ConfigMaps (default) to store release information.

The chart dependency management system has moved from requirements.yaml and requirements.lock in Helm 2 to Chart.yaml and Chart.lock in Helm3. An improved upgrade strategy, leveraging three-way strategic merge patches. Helm considers the old manifest, its live state, and the new manifest of when generating a patch.

Helm CLI client

On your Terminal on any folder:

This extends and summarizes https://helm.sh/docs/intro/install

Install Kubernetes first.

PROTIP: The Helm client learns about Kubernetes clusters by using files in the Kube config file. By default, Helm attempts to find this file in the place where kubectl creates it ($HOME/.kube/config).

See whether you already have it installed:

Helm CLI Version

helm version

If you see something like this, you already have it installed:

version.BuildInfo{Version:"v3.10.1", GitCommit:"9f88ccb6aee40b9a0535fcc7efea6055e1ef72c9", GitTreeState:"clean", GoVersion:"go1.19.2"}

What is the latest Kubernetes helm CLI client for macOS?

brew info helm

Response at time of writing:

helm: stable 3.2.1 (bottled), HEAD
The Kubernetes package manager
https://helm.sh/
/usr/local/Cellar/helm/3.2.1 (7 files, 43.3MB) *
  Poured from bottle on 2020-05-30 at 03:34:11
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/helm.rb
==> Dependencies
Build: go@1.13 ✘
==> Options
--HEAD
Install HEAD version
==> Caveats
Bash completion has been installed to:
  /usr/local/etc/bash_completion.d
 
zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Analytics
install: 36,371 (30 days), 110,830 (90 days), 251,843 (365 days)
install-on-request: 35,636 (30 days), 108,563 (90 days), 246,528 (365 days)
build-error: 0 (30 days)

Previously:

/usr/local/Cellar/helm/3.2.1 (7 files, 43.3MB) *
/usr/local/Cellar/helm/3.1.1... (7 files, 41.2MB)
/usr/local/Cellar/helm/3.1.0 (7 files, 41.2MB) *

To install helm CLI client for the first time:
```
brew install helm
```
To upgrade Kubernetes helm CLI client (if brew info returned a version):
```
brew upgrade helm
```
Sample response:
```
==> Downloading https://storage.googleapis.com/helm/releases/v3.1.0/helm-darwin-amd64
```
Obtain the version again after an upgrade.

PROTIP: Helm is written in the Go language, built using the Make utility.

helm env

Examine your local Helm enviorinment locations:

helm env

Notice that the MacOS Library is used as storage locations:

HELM_BIN="helm"
HELM_BURST_LIMIT="100"
HELM_CACHE_HOME="/Users/wilsonmar/Library/Caches/helm"
HELM_CONFIG_HOME="/Users/wilsonmar/Library/Preferences/helm"
HELM_DATA_HOME="/Users/wilsonmar/Library/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECAFILE=""
HELM_KUBECONTEXT=""
HELM_KUBEINSECURE_SKIP_TLS_VERIFY="false"
HELM_KUBETLS_SERVER_NAME=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/Users/wilsonmar/Library/helm/plugins"
HELM_REGISTRY_CONFIG="/Users/wilsonmar/Library/Preferences/helm/registry/config.json"
HELM_REPOSITORY_CACHE="/Users/wilsonmar/Library/Caches/helm/repository"
HELM_REPOSITORY_CONFIG="/Users/wilsonmar/Library/Preferences/helm/repositories.yaml"

NOTE: Helm stores its configuration files in XDG Base directory specifications created the first time helm is run.

Cache:  $XDG_CACHE_HOME - ${HOME}/.cache/helm/
Config: $XDG_CONFIG_HOME - ${HOME}/.config/helm/
Data:   $XDG_CONFIG_HOME - ${HOME}/.local/helm/

Helm 3 puts K8s CRD’s (Custom Resource Definitions) in the “crds” directory and can be skipped using --skip-crds on install. https://github.com/bitnami-labs/helm-crd is not under active development.

Helm 3 has a GoSDK CLI.

Open Container Initiative (OCI) with Docker Registry API.

Create new Helm Chart

A Helm “Chart” is a collective noun for a set of folders and files.

Create a new Helm Chart:
```
helm create mychart
```
Optionally, –starter option can be added to specify a “starter chart”.

Starter Charts are copied to $XDG_DATA_HOME/helm/starters. Chart developers author charts specifically designed to be used as starters. The Chart.yaml of starters are overwritten by the generator. Users will expect to modify such a chart’s contents, so documentation should indicate how users can do so. Starter charts can be used as templates, with all occurrences of CHARTNAME replaced with the specified chart name.
Examine the files created using a Tree command:
```
tree
```
```
├── Chart.yaml
├── charts
├── templates
└── values.yaml
   
```
REMEMBER: Each Helm Chart must contain a Chart.yaml file (with a capital C), a values.yaml file (with a lower case v) to override default values with your own information.

Those files provide files to “handlebars” within yaml files in the templates directory/folder containing Kubernetes deployment.yaml and other files,

Sometimes other charts are in a chart folder.
In an editor (such as VSCode), open the Chart.yaml file.

PROTIP: apiVersion is v2 starting with Helm3. Confusing, I know.

Yaml

REMEMBER: In yaml format files, indents use two spaces (and never tabs). (Kubernetes objects) into one unit. The charts file defines dependencies.

Templates folder

In the templates folder:

├── templates
│   ├── NOTES.txt
│   ├── _helpers.tpl
│   ├── deployment.yaml
│   ├── ingress.yaml
│   ├── service.yaml
│   ├── serviceaccount.yaml
│   └── tests
│       └── test-connection.yaml

Template yaml files contain placeholders:

\{\{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
  name: \{\{ include "my-chart.serviceAccountName" . }}
  labels:
    \{\{- include "my-chart.labels" . | nindent 4 }}
  \{\{- with .Values.serviceAccount.annotations }}
  annotations:
    \{\{- toYaml . | nindent 4 }}
  \{\{- end }}
\{\{- end -}}

From: VIDEO: What is Helm? (with Tiller) Dec 18, 2019 [9:05]:

Dependencies in requirements.yaml

The requirements.yaml file to Kubernetes specifies a MariaDB database:

dependencies:
- name: mariadb
  version: 0.6.0
  repository: https://kubernetes-charts.storage.googleapis.com

The chart folder is populated by the archive of “dependencies” of other charts with its own set of yaml files.

Lint a Chart

Validate that a Chart follows the conventions and requirements of the Helm chart standard JSON schema

Linting is automatic with helm install, upgrade, and template. But you can run it anytime:
```
helm lint
```
Sample output:
```
==> Linting .
[INFO] Chart.yaml: icon is recommended
 
1 chart(s) linted, 0 chart(s) failed
   
```
See https://www.waytoeasylearn.com/learn/helm-lint/

Specify app

Since Kubernetes works off Docker images, specify the Docker image, such as the simple “ToDo” app:

image:
   repository: prydonius/todo
   tag: latest
   pullPolicy: IfNotPresent

The client CLI knows to look for the repository within the https://hub.helm.sh public repository.

PROTIP: A Chart release number is an incremental counter that advances forward even on rollback. A Sematic version number (such as 1.2.3) is required on every chart.

Search for the ToDo chart this tutorial uses.
For a list of all apps in Hub:
```
help search hub
```
A repository of helm charts to “find, install, and publish Kubernetes packages”:

artifacthub.io

Search apps

Search for a specific Chart:
```
helm search hub vault
```
Note that the list contains “stable” and “incubator” editions.
To see logos among publicly available charts, view https://hub.helm.sh, search for “stable” Charts:
- Anchore, Clair
- web server Apache, Nginx, Tomcat, WordPress
- Argo-cd, GitLab
- Artifactory
- Databases: Cassandra, Mongodb, CockroadhDB, MySQL, Neo4j, Spark, Spinniker
- Secrets manager: Consul, Vault
- Testing tools: JMeter, Selenium
- Elastic Stack, Logstash, Prometheus, Kibana,
- Weave
Add repo
Define that folder in a system environment variable for use in shell file statements below:
```
MY_HELM_PATH="~/github-wilsonmar/helm-samples" 
```
Replace “github-wilsonmar” with your folder.

Create then Navigate to a folder holding your Helm chart.

cd 
echo "${MY_HELM_PATH}"
mkdir "${MY_HELM_PATH}"
cd "${MY_HELM_PATH}"

Add:

helm repo add dev https://hub.helm.sh

FIXME: ??? If you get back:

Error: looks like "https://hub.helm.sh" is not a valid chart repository or cannot be reached: error unmarshaling JSON: while decoding JSON: json: cannot unmarshal string into Go value of type repo.IndexFile

Install Chart in Kubernetes

This is a summary of https://helm.sh/docs/intro/using_helm/

Run it:

helm install --name todo "${MY_HELM_PATH}"
--dry-run --debug  --set service.type=NodePort

Highlight and copy the response to your Clipboard to paste in your local Terminal:

For example:

export NODE_PORT=$(kubectl get —-namespace default -o jsonpath="{.spec.ports[0].nodePort}" services todo-mychart)
export NODE_IP=$(kubectl get nodes —-namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT

Copy and paste the URL in the response (such as http://127.0.0.1:8080) in your browser’s address to see the app’s UI.

Komodor install

Alternately, Komodor.io provides this to install:

helm repo add komodorio https://helm-charts.komodor.io ; helm repo update; helm upgrade --install k8s-watcher komodorio/k8s-watcher --set watcher.actions.basic=true --set watcher.actions.advanced=true --set apiKey=12345678-abcd-3333-ccc-2edb0fe9e263 --set watcher.clusterName=default  --wait --timeout=90s && open https://app.komodor.com/main/services

In Kubernetes

See what is running in the Kubernetes cluster:
```
helm list --all
```
Uninstall
To uninstall
```
helm uninstall --keep-history
```
Package to Archive
Package a Chart folder:
```
helm package mychart
```
After a Chart is packaged by being tarred and gzipped (compressed/packed) to a .tgz file, and optionally signed, it is called an archive.
```
helm verify my-chart-0.1.0.tgz
```
A Chart may be accompanied by a .prov (provenance) file which details where the chart came from and what it contains. The cryptographic hash (signature OpenPGP “clearsign” block)) of the chart archive file is used to determine whether the chart file has been tampered with.

1) Package a Chart folder:

Ingress

VIDEO: Kubernetes Ingress Explained Completely For Beginners by KodeKloud

Interactive

“Helm Package Manager” on Qwiklabs covers installation and configure a Chart (based in MySQL) on GCP.

Videos

[3] CNCF Webinar Series – Getting Helm to be Enterprise-ready Apr 3, 2018

https://www.youtube.com/watch?v=TJ9hPLn0oAs Create a Helm chart Oct 3, 2019 https://docs.bitnami.com/tutorials/create-your-first-helm-chart

Articles

An Introduction to Helm, the Package Manager for Kubernetes August 6, 2018 by Brian Boucheron

https://www.katacoda.com/courses/docker-production/vault-secrets

Helmsman Desired State Configurator

Open-sourced at https://github.com/Praqma/helmsman, Helmsman from Praqma (by SAMI ALAJRAMI and others) provides an “autopilot” for Kubernetes clusters which automates the lifecycle management of Helm Charts using declarative (desired state) configuration files (DSF) to create, delete, upgrade, and move Kubernetes objects to different namespaces. This approach makes it easier to replicate a CI pipeline. This also takes care of secrets passing (from environment variables to Charts).

https://hub.docker.com/r/praqma/helmsman/

The desired state approach achieves idempotentcy - executing Helmsman several times gets the same result, and continues from failures.

Videos

Venkat’s playlist on Kubernetes includes:

[ Kube 63 ] Creating your first Helm chart Dec 2, 2019 based on v2.14 rather than v3, from scratch, drawing from with source in https://github.com/justmeandopensource

Helm (v1) and Kubernetes Tutorial - Introduction by Matthew Palmer

YouTube playlist

An Introduction to Helm [36:49] by Matt Farina, Samsung SDS & Josh Dolitsky, Blood Orange

Helm 3 Deep Dive a Nov 22, 2019 CNCF [Cloud Native Computing Foundation] video by Helm core maintainers Taylor Thomas (@_oftaylor, Microsoft Azure) and Martin Hickey (@mhickeybot IBM) say the security model is changed. Merge/upgrade does a 3 way compare that also includes cluster live state.

Managing Helm Deployments with Gitops at CERN a CNCF video by Ricardo Rocha, CERN [32:02]

Helm 3: Navigating to Distant Shores by Codefresh

If you have an O’Reilly subscription, the 10 minute “almost-live” hands-on scenario (from Katota) “Get Started with the Helm Package Manager” has you clicking each command and see it executed on an Ubuntu Bash terminal. This scenario teaches you how to use Helm, the package manager for Kubernetes, to deploy Redis.

Wait for Kubernetes to start. Then install it using a curl command.
The scenario is based on version 2 because it tells you to update the local cache to sync the latest available packages with the environment:
```
helm init
helm repo update
```
helm search redis
helm inspect stable/redis to see configuration policies.
To deploy the chart to your cluster:

helm install stable/redis
List package namespaces installed:

helm ls
Find out what pods, replication controllers, and services (master and slave) were deployed:

kubectl get all
List the persistent volumes available:

kubectl apply -f pv.yaml

The pod remains in a pending state while the Docker Image is downloaded.
Grant Redis data mount permissions to write:

chmod 777 -R /mnt/data*
Provide helm with a more friendly name “my-release”:

helm install –name my-release stable/redis
To get your password run:

export REDIS_PASSWORD=$(kubectl get secret –namespace default dinky-newt-redis-o jsonpath=”{.data.redis-password}” base64 –decode)
To connect to your Redis server, run a Redis pod that you can use as a client:

kubectl run –namespace default dinky-newt-redis-client –rm –tty -i –restart=’Never’
–env REDIS_PASSWORD=$REDIS_PASSWORD
–image docker.io/bitnami/redis:5.0.7-debian-10-r27 – bash
Connect using the Redis CLI:

redis-cli -h dinky-newt-redis-master -a $REDIS_PASSWORD redis-cli -h dinky-newt-redis-slave -a $REDIS_PASSWORD
To connect to your database from outside the cluster execute the following commands:

kubectl port-forward –namespace default svc/dinky-newt-redis-master 6379:6379 & redis-cli -h 127.0.0.1 -p 6379 -a $REDIS_PASSWORD

Cloud vendors

https://www.digitalocean.com/community/tutorials/an-introduction-to-helm-the-package-manager-for-kubernetes

https://docs.aws.amazon.com/eks/latest/userguide/helm.html

https://aws.amazon.com/blogs/startups/from-zero-to-eks-with-terraform-and-helm/

Helm installing Vault

https://www.hashicorp.com/blog/announcing-the-vault-helm-chart/

https://www.vaultproject.io/docs/platform/k8s/helm

https://github.com/hashicorp/vault-helm

https://github.com/hashicorp/consul-helm

Ansible is used in the aws_eks_cluster.py. Compre the Python 3.8 vs. 3.7 versions: diff /usr/local/Cellar/ansible/2.9.6_2/libexec/lib/python3.8/site-packages/ansible/modules/cloud/amazon/aws_eks_cluster.py $HOME/Library/Python/3.7/lib/python/site-packages/ansible/modules/cloud/amazon/aws_eks_cluster.py

https://learning.oreilly.com/live-events/helm-charts-with-kubernetes/0636920074683/0636920079146/ Helm Charts with Kubernetes

https://medium.com/@elliotgraebert/comparing-the-top-eight-managed-kubernetes-providers-2ae39662391b

More on DevOps

This is one of a series on DevOps:

bomonike

Why Helm?

Architecture

No more Tiller in Helm 3

Helm CLI client

Helm CLI Version

helm env

Create new Helm Chart

Yaml

Templates folder

Dependencies in requirements.yaml

Lint a Chart

Specify app

Search apps

Add repo

Install Chart in Kubernetes

Komodor install

In Kubernetes

Uninstall

Package to Archive

Ingress

Interactive

Videos

Articles

Helmsman Desired State Configurator

Videos

Cloud vendors

Helm installing Vault

More on DevOps