Créer des packages immutables avec vendir

Vendir est un outil de la suite Carvel, comme l’est kbld que nous avons vu auparavant, qui est bien pratique pour créer des packages immutables.

En effet, vendir permet à partir d’un fichier de déclaration au format yaml de synchroniser le contenu de sources de données distantes ou locales dans un répertoire local afin de pouvoir les exploiter. On peut par exemple lui indiquer de copier le contenu d’un repository GitHub avec un tag défini. Le résultat est que l’on retrouve dans le dossier local que les fichiers associés à ce tag.

On peut aussi récupérer des charts Helm, des images de containers, des images issus de imgpkg (un autre outil de la suite) et des données locales.

Installation de vendir

Comme pour kbld je vous propose d’installer toute la suite carvel :

wget -O- https://carvel.dev/install.sh > install.sh
sudo bash install.sh
vendir version
vendir version 0.23.0

Succeeded

Utilisation de vendir

Par défaut vendir va rechercher la présence d’un fichier vendir.yml dans le répertoire courant. Ce fichier est de la forme suivante (toutes les options avec les explications) :

apiVersion: vendir.k14s.io/v1alpha1
kind: Config

# declaration of minimum required vendir binary version (optional)
minimumRequiredVersion: 0.8.0

# one or more directories to manage with vendir
directories:
- # path is relative to vendir.yml location
  path: config/_ytt_lib

  contents:
  - # path lives relative to directory path # (required)
    path: github.com/cloudfoundry/cf-k8s-networking

    # uses git to clone Git repository (optional)
    git:
      # http or ssh urls are supported (required)
      url: https://github.com/cloudfoundry/cf-k8s-networking
      # branch, tag, commit; origin is the name of the remote (required)
      # optional if refSelection is specified (available in v0.11.0+)
      ref: origin/master
      # specifies a strategy to resolve to an explicit ref (optional; v0.11.0+)
      refSelection:
        semver:
          # list of semver constraints (see versions.md for details) (required)
          constraints: ">0.4.0"
          # by default prerelease versions are not included (optional; v0.12.0+)
          prereleases:
            # select prerelease versions that include given identifiers (optional; v0.12.0+)
            identifiers: [beta, rc]
      # skip downloading lfs files (optional)
      lfsSkipSmudge: false
      # verify gpg signatures on commits or tags (optional; v0.12.0+)
      verification:
        publicKeysSecretRef:
          name: my-git-gpg-auth
      # specifies name of a secret with auth details;
      # secret may include 'ssh-privatekey', 'ssh-knownhosts',
      # 'username', 'password' keys (optional)
      secretRef:
        # (required)
        name: my-git-auth

    # uses hg to clone Mercurial repository (optional; v0.22.0+)
    hg:
      # http or ssh urls are supported (required)
      url: https://hg.sr.ht/~sircmpwn/hg.sr.ht
      # branch, tag, commit (required)
      ref: 180c776fe29448afa8c756ab572bab7a1cf17a06
      # specifies name of a secret with auth details;
      # secret may include 'ssh-privatekey', 'ssh-knownhosts',
      # 'username', 'password' keys (optional)
      secretRef:
        # (required)
        name: my-hg-auth

    # fetches asset over HTTP (optional)
    http:
      # asset URL (required)
      url:
      # verification checksum (optional)
      sha256: ""
      # specifies name of a secret with basic auth details;
      # secret may include 'username', 'password' keys (optional)
      secretRef:
        # (required)
        name: my-http-auth

    # fetches asset from an image registry (optional; v0.11.0+)
    image:
      # image URL; could be plain, tagged or digest reference (required)
      url: gcr.io/repo/image:v1.0.0
      # specifies a strategy to choose a tag (optional; v0.22.0+)
      # if specified, do not include a tag in url key
      tagSelection:
        semver:
          # list of semver constraints (see versions.md for details) (required)
          constraints: ">0.4.0"
          # by default prerelease versions are not included (optional; v0.12.0+)
          prereleases:
            # select prerelease versions that include given identifiers (optional; v0.12.0+)
            identifiers: [beta, rc]
      # specifies name of a secret with registry auth details;
      # secret may include 'username', 'password' and/or 'token' keys;
      # as of v0.19.0+, dockerconfigjson secrets are also supported (optional)
      # of of v0.22.0+, multiple registry credentials are supported and
      #   are passed to imgpkg via env. registry hostname must match url
      #   for auth information to be chosen by imgpkg.
      #   (https://carvel.dev/imgpkg/docs/latest/auth/#via-environment-variables)
      secretRef:
        # (required)
        name: my-image-auth
      # specify wether to skip TLS verification; defaults to false (optional;v0.18.0+)
      dangerouSkipTLSVerify: false

    # fetches imgpkg bundle from an image registry (optional; v0.16.0+)
    imgpkgBundle:
      # could be plain, tagged or digest reference (required)
      image: gcr.io/repo/bundle:v1.0.0
      # specifies a strategy to choose a tag (optional; v0.22.0+)
      # if specified, do not include a tag in image key
      tagSelection:
        semver:
          # list of semver constraints (see versions.md for details) (required)
          constraints: ">0.4.0"
          # by default prerelease versions are not included (optional; v0.12.0+)
          prereleases:
            # select prerelease versions that include given identifiers (optional; v0.12.0+)
            identifiers: [beta, rc]
      # specifies name of a secret with registry auth details;
      # secret may include 'username', 'password' and/or 'token' keys;
      # as of v0.19.0+, dockerconfigjson secrets are also supported (optional)
      # of of v0.22.0+, multiple registry credentials are supported and
      #   are passed to imgpkg via env. registry hostname must match image
      #   for auth information to be chosen by imgpkg.
      #   (https://carvel.dev/imgpkg/docs/latest/auth/#via-environment-variables)
      secretRef:
        # (required)
        name: my-image-auth
      # specify wether to skip TLS verification; defaults to false (optional;v0.18.0+)
      dangerouSkipTLSVerify: false

    # fetches assets from a github release (optional)
    githubRelease:
      # slug for repository (org/repo) (required)
      slug: k14s/kapp-controller
      # use release tag (optional)
      # optional if tagSelection is specified (available in v0.22.0+)
      tag: v0.1.0
      # specifies a strategy to choose a tag (optional; v0.22.0+)
      tagSelection:
        semver:
          # list of semver constraints (see versions.md for details) (required)
          constraints: ">0.4.0"
          # by default prerelease versions are not included (optional; v0.12.0+)
          prereleases:
            # select prerelease versions that include given identifiers (optional; v0.12.0+)
            identifiers: [beta, rc]
      # use latest published version (optional)
      latest: true
      # use exact release URL (optional)
      url: https://api.github.com/repos/k14s/kapp-controller/releases/21912613
      # only download specific assets (optional; v0.12.0+)
      assetNames: ["release*.yml"]
      # checksums for downloaded files (optional)
      # (if release text body contains checksums, it's not necessary
      # to manually specify them here)
      checksums:
        release.yml: 26bf09c42d72ae448af3d1ee9f6a933c87c4ec81d04d37b30e1b6a339f5983a7
      # disables checking auto-found checksums for downloaded files (optional)
      # (checksums are extracted from release's text body
      # based on following format `<sha256>  <filename>`)
      disableAutoChecksumValidation: true
      # specifies which archive to unpack for contents (optional)
      unpackArchive:
        # (required)
        path: release.tgz
      # specifies name of a secret with github auth details;
      # secret may include 'token' key (optional)
      secretRef:
        # (required)
        name: my-gh-auth

    # fetch Helm chart contents (optional; v0.11.0+)
    helmChart:
      # chart name (required)
      name: stable/redis
      # use specific chart version (string; optional)
      version: "1.2.1"
      # specifies Helm repository to fetch from (optional)
      repository:
        # repository url; supports exprimental oci helm fetch via
        # oci:// scheme (required)
        url: https://...
        # specifies name of a secret with helm repo auth details;
        # secret may include 'username', 'password';
        # as of v0.19.0+, dockerconfigjson secrets are also supported (optional)
        # as of v0.22.0+, 0 or 1 auth credential is expected within dockerconfigjson secret
        #   if >1 auth creds found, error will be returned. (currently registry hostname
        #   is not used when found in provide auth credential.)
        secretRef:
          # (required)
          name: my-helm-auth
      # specify helm binary version to use;
      # '3' means binary 'helm3' needs to be on the path (optional)
      helmVersion: "3"

    # copy contents from local directory (optional)
    directory:
      # local file system path relative to vendir.yml
      path: some-path

    # states that directory specified by above path
    # is managed by hand; nothing to do for vendir (optional)
    manual: {}

    # specify contents inline within this file (optional; v0.11.0+)
    inline:
      # specifies mapping of paths to their content (optional)
      paths:
        dir/file.ext: file-content
      # specifies content via secrets and config maps (optional)
      pathsFrom:
      - secretRef:
          # (required)
          name: secret-name
          # specifies where to place files found in secret (optional)
          directoryPath: dir
      - configMapRef:
          # (required)
          name: cfgmap-name
          # specifies where to place files found in config map (optional)
          directoryPath: dir

    # includes paths specify what should be included. by default
    # all paths are included (optional)
    includePaths:
    - cfroutesync/crds/**/*
    - install/ytt/networking/**/*

    # exclude paths are "placed" on top of include paths (optional)
    excludePaths: []

    # specifies paths to files that need to be includes for
    # legal reasons such as LICENSE file. Defaults to few
    # LICENSE, NOTICE and COPYRIGHT variations (optional)
    legalPaths: []

    # make subdirectory to be new root path within this asset (optional; v0.11.0+)
    newRootPath: cfroutesync

Plutôt complet.

Création d’un package

Voyons comment créer un package. On commence par créer un répertoire et y créé le fichier vendir.yml :

mkdir test-vendir
cd test-vendir
vi vendir.yaml

On va y indiquer à vendir de récupérer un chart Helm et un fichier issu d’un tag d’un repo git.

apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
- path: plugins/
  contents:
    - path: .
      githubRelease:
        slug: neo4j-contrib/neo4j-apoc-procedures
        tag: 4.3.0.0
        disableAutoChecksumValidation: true
        assetNames: ["apoc-*-all.jar"]

- path: vendor
  contents:
  - path: custom-repo-custom-version
    helmChart:
      name: contour
      version: "1.2.1"
      repository:
        url: https://charts.bitnami.com/bitnami

On peut lancer la synchronisation :

vendir sync
Fetching: plugins/ + . (github release neo4j-contrib/neo4j-apoc-procedures@4.3.0.0)

Fetching: vendor + custom-repo-custom-version (helm chart from https://charts.bitnami.com/bitnami@contour:1.2.1)

Lock config

apiVersion: vendir.k14s.io/v1alpha1
directories:
- contents:
  - githubRelease:
      url: https://api.github.com/repos/neo4j-contrib/neo4j-apoc-procedures/releases/44495785
    path: .
  path: plugins/
- contents:
  - helmChart:
      appVersion: 1.8.0
      version: 1.2.1
    path: custom-repo-custom-version
  path: vendor
kind: LockConfig

Succeeded

Vous remarquez que l’on voit une phase de locking qui permet se charge de verrouiller les versions et ainsi rendre le package pour les rendre immutables :

• pour git et hg cela remplace le tag par une référence de commit
• pour une image son URL
• pour un githubRelease le lien direct
• pour un helmChart la version de l’application

Dans notre cas :

apiVersion: vendir.k14s.io/v1alpha1
directories:
- contents:
  - githubRelease:
      url: https://api.github.com/repos/neo4j-contrib/neo4j-apoc-procedures/releases/44495785
    path: .
  path: plugins/
- contents:
  - helmChart:
      appVersion: 1.8.0
      version: 1.2.1
    path: custom-repo-custom-version
  path: vendor
kind: LockConfig

Voyons le contenu :

tree
.
├── plugins
│   └── apoc-4.3.0.0-all.jar
├── vendir.lock.yml
├── vendir.yml
└── vendor
    └── custom-repo-custom-version
        ├── Chart.yaml
        ├── crds
        │   ├── httpproxies.yaml
        │   ├── ingressroutes.yaml
        │   └── tlscertificatedeligations.yaml
        ├── README.md
        ├── templates
        │   ├── configmap.yaml
        │   ├── crd-httpproxies.yaml
        │   ├── crd-ingressroutes.yaml
        │   ├── crd-tlscertificatedeligations.yaml
        │   ├── daemonset.yaml
        │   ├── deployment.yaml
        │   ├── _helpers.tpl
        │   ├── job.yaml
        │   ├── NOTES.txt
        │   ├── rbac.yaml
        │   ├── service-accounts.yaml
        │   ├── servicemonitor.yaml
        │   └── service.yaml
        ├── values-production.yaml
        └── values.yaml

5 directories, 23 files

Conclusion

Ça permet vraiment pas mal de choses et peut être vraiment pratique pour constituer des images de container, des configmap kubernetes et je pense plein d’autres cas d’usage

Plus d’infos sur la documentation de vendir ↗.