Golden Testing Helm Charts
We love tests at Twingate. When working on the Twingate’s helm charts repository I wanted to incorporate testing like we do with other code. For a long time our release process requires a long process of manual testing - every change to the chart’s templates required a manual process of testing the chart with various inputs to make sure it still works as expected. Enter golden tests as a way to automate this process…
What are Golden Tests? #
Golden tests, also known as snapshot testing, involves comparing the current output of your code with a “golden” reference or a previously stored correct version.
It’s a particularly useful method in situations where the output is complex and can change over time, like configuration files or templates.
Gold Testing a Helm Chart #
The concept is simple:
- We maintain a directory of different helm input files (different permutations of
values.yamlthat we want to test). - For each such file we also maintain the expected output from that file. The expected output for running our chart with
foo.yamlas input is stored infoo.golden.yaml - When we run our tests the simply render the chart’s result for every input file and compare the output with the content of the golden copy - if they’re different we fail.
- When we want to update the golden copy we can run the test with an
-updateflag in which case, instead of comparing, it will write the output to the golden file.
We can review the diff before committing the test changes.
Here’s the code to run our tests:
package golden
import (
"flag"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"testing"
"github.com/gruntwork-io/terratest/modules/helm"
"gopkg.in/yaml.v2"
)
var update = flag.Bool("update", false, "update golden test output files")
type ChartYaml struct {
Name string `yaml:"name"`
Version string `yaml:"version"`
}
func GetChartYaml(t *testing.T) ChartYaml {
chartYamlFile, err := os.ReadFile("./my-chart/Chart.yaml")
if err != nil {
t.Fatalf("Error reading Chart.yaml: %v", err)
}
var chartYaml ChartYaml
if err := yaml.Unmarshal(chartYamlFile, &chartYaml); err != nil {
t.Fatalf("Error unmarshaling YAML data: %v", err)
}
return chartYaml
}
func TestHelmRender(t *testing.T) {
files, err := os.ReadDir("./test/golden")
if err != nil {
t.Fatal(err)
}
chartYaml := GetChartYaml(t)
for _, f := range files {
if !f.IsDir() && strings.HasSuffix(f.Name(), ".yaml") && !strings.HasSuffix(f.Name(), ".golden.yaml") {
// Render this values.yaml file
output, error := helm.RenderTemplateE(t,
&helm.Options{
ValuesFiles: []string{"test/golden/" + f.Name()},
},
"./my-chart",
"test",
[]string{},
)
goldenFile := "test/golden/" + strings.TrimSuffix(f.Name(), filepath.Ext(".yaml")) + ".golden.yaml"
// If error, we write the error to the golden snapshot
if error != nil {
output = fmt.Sprintf("%v\n", error)
} else {
// Replace `cn.chart` helper value with a stable value for testing
// because we don't want to have to update all snapshots whenever Chart version changes
regex := regexp.MustCompile(fmt.Sprintf("%s-%s", chartYaml.Name, chartYaml.Version))
bytes := regex.ReplaceAll([]byte(output), []byte(fmt.Sprintf("%s-major.minor.patch-test", chartYaml.Name)))
output = fmt.Sprintf("%s\n", string(bytes))
}
// If we were called with `-update` param - write `output` to our golden snapshot file`
if *update {
err := os.WriteFile(goldenFile, []byte(output), 0644)
if err != nil {
t.Fatal(err)
}
}
// Read golden snapshot file and make sure its content is identical to output
expected, err := os.ReadFile(goldenFile)
if err != nil {
t.Fatal(err)
}
if string(expected) != output {
t.Fatalf("Expected %s, but got %s\n. Update golden files by running `go test -v ./... -update`", string(expected), output)
}
}
}
}
You’d run this simply by running go test -v ./... or go test -v ./... -update if you want to update the golden copies.
An example of how we use this in Twingate can be found on the helm-charts and Kubernetes Operator repositories on GitHub.