Skip to main content
Version: 1.36.0

SIGHUP Distribution Release v1.36.0

Welcome to SD release v1.36.0.

The distribution is maintained with ❤️ by the team SIGHUP by ReeVo.

This version adds support for Kubernetes 1.36, updates all core modules, and introduces the new utilities core module, with Headlamp as its first package, for the OnPremises and Immutable kinds. Also, includes several improvements to the recently introduced Immutable kind.

Note that this release includes Breaking Changes.

Installer Updates​

  • on-premises 📦 installer: v1.36.5
    • Installs Kubernetes version 1.36.5, adds support for Kubernetes v1.35.9 and v1.34.12.
    • Updates etcd to v3.6.15 and, for Kubernetes 1.36, containerd to v2.3.7 and runc to v1.5.2.
    • Renders the kubeadm configuration with kubeadm.k8s.io/v1beta4 and adds support for Debian 13.
  • eks 📦 installer: v3.6.0
    • Verified compatibility with Kubernetes 1.36.
  • immutable 📦 installer: v1.36.5
    • Installs Kubernetes version 1.36.5, adds support for Kubernetes v1.35.9.
    • Verified compatibility with Flatcar Container Linux 4757.2.1 (kernel 6.12.111), with containerd v2.3.7, etcd v3.6.15, keepalived v2.4.3 and HAProxy v3.2.23.

Module updates 📦​

  • aws 📦 provider module: v5.4.0
    • Bumps cluster-autoscaler to v1.36.1 and aws-load-balancer-controller to v3.5.0.
  • auth 📦 core module: v0.8.0
    • Bumps Gangplank to v1.3.0 and Pomerium to v0.33.1.
  • dr 📦 core module: v3.5.0
    • Bumps Velero to v1.18.2, Velero plugins to v1.14.2 (AWS, Microsoft Azure and GCP) and snapshot-controller to v8.6.0.
    • Updates the MinIO image.
  • ingress 📦 core module: v5.2.0
    • Bumps cert-manager to v1.21.2, HAProxy to v3.2.15 and NGINX (Chainguard fork) to v1.15.10.
    • Renames the HAProxy Ingress alerts with the HaproxyIngress prefix, and removes HaproxyBackendLatencyCritical, HaproxyBackendNoAliveServers and HaproxyHighHttp4xxErrorRateBackend.
  • logging 📦 core module: v5.5.0
    • Bumps OpenSearch to v3.8.0, OpenSearch Dashboards to v3.8.0, Logging operator to v6.8.0, Fluent Bit to v5.1.0 and Loki to v3.7.7.
    • Updates the MinIO image and adds MinIO alerts.
    • Routes the ingress-haproxy, forecastle and external-dns namespaces to the infra output.
  • monitoring 📦 core module: v4.3.0
    • Bumps AlertManager to v0.33.0, Grafana to v13.0.2, Kubernetes RBAC proxy to v0.22.0, Kubernetes state metrics to v2.19.0, Node exporter to v1.11.1, Prometheus operator to v0.92.0, Prometheus operated to v3.12.0, x509 exporter to v4.2.0 and Mimir to v3.2.0.
    • Updates the MinIO image.
    • Adds new control plane, MinIO and x509-exporter alerts. Some Mimir alerts change name with the upstream rules of Mimir v3.2.0.
    • The distribution also bumps the Prometheus image of the prometheusAgent monitoring type to v3.12.0.
  • networking 📦 core module: v4.1.0
    • Bumps Cilium to v1.19.8 and Tigera operator to v1.42.6 (Calico v3.32.2).
    • With Calico on OnPremises, Immutable and KFDDistribution, deploys the Calico API Server, which Whisker, Goldmane and Tiers need since Calico v3.32.
  • policy 📦 core module: v1.18.0
    • Bumps Gatekeeper to v3.23.1, Gatekeeper Policy Manager to v2.1.0 and Kyverno to v1.19.1.
  • tracing 📦 core module: v1.6.0
    • Bumps tempo-distributed to v2.10.8.
    • Updates the MinIO image and adds MinIO alerts.
  • utilities 📦 core module: v1.0.0
    • New core module, with Headlamp v0.40.1 as its first package.

New features 🌟​

  • [#575] OnPremises and Immutable: add support for the new furyctl renew kubeconfigs command. It renews the kubeconfig file of the admin and the kubeconfig files of the users in spec.kubernetes.advanced.users.names. Then it downloads them to the working directory. A user that you add to the configuration file gets a kubeconfig file. It is not necessary to apply the kubernetes phase.
  • [#600] Immutable: spec.infrastructure.proxy now applies to the whole node and not only to containerd. kubelet, etcd, the system extension downloads and command line tools like curl now use the proxy. Note that proxy configuration does not apply to the operating system installation step.
  • [#601] Immutable: the infrastructure phase now upgrades the load balancers. The new upgrade-load-balancers.yml playbook updates the operating system, the system extensions and the configuration. It upgrades one load balancer at a time, so the virtual IP address stays available. The playbook stops before it changes anything when a load balancer does not answer.
  • [#612] Adds module-utilities with Headlamp as its first package, for OnPremises and Immutable. Headlamp is exposed at dashboard.<baseDomain>. Set spec.distribution.modules.utilities.headlamp.type to token-auth or sso to install it. With token-auth, the users log in with their own Kubernetes token and get the permissions of that token. Set the optional clusterRole field to also bind a ClusterRole to the headlamp ServiceAccount, so that its token can be used to log in. With sso, Headlamp is behind Pomerium, and the group in oidcGroup gets read-only access to the cluster. With sso, the API server also accepts the pomerium audience in the OIDC tokens, unless it is already the client_id in spec.kubernetes.advanced.oidc.
  • [#618] Immutable: the upgrade now updates Flatcar on each node to the exact version that the release pins. The installer downloads the update payload, compares its SHA-256 checksum, and installs it with flatcar-update before it drains the node. Thus a missing or bad payload stops the upgrade while the node still serves. The nodes no longer ask an update server. Automatic OS updates stay off with SERVER=disabled in /etc/flatcar/update.conf, as the Flatcar documentation recommends, and update-engine is no longer masked. With spec.infrastructure.proxy, update-engine does not use the proxy.

Bug Fixes 🐛​

  • [#626] Immutable: a node reports booted to furyctl only after sshd listens. Before this release, the report could come some seconds before port 22 listened, and the first Ansible play of the apply could fail to connect to the node.
  • [#581] Immutable: node storage.installDisk now accepts persistent device paths like /dev/disk/by-id/wwn-..., /dev/disk/by-path/... and /dev/mapper/.... The field is validated the same way Butane/Ignition validates its own device fields — the path must be absolute and clean — instead of the previous alphanumeric-only pattern that rejected every /dev/disk/by-* symlink.
  • [#603] Immutable: the preflight check now stops when it cannot reach a control plane node. Before this release, a node that did not answer gave the same result as a cluster that does not exist. The apply then continued as if the cluster was new. The check also finds an existing cluster when a control plane node other than the first one holds the admin kubeconfig.
  • [#605] Immutable: the preflight check now reads every host, and not only the control plane hosts. It tells a cluster whose control plane does not answer from a cluster that does not exist, and furyctl stops the apply in the first case.
  • [#577] OnPremises: makes the admin.conf fetch in fetch-admin-conf-playbook.yaml pick the right master, and stops the playbook from failing on purpose to signal "cluster doesn't exist".
  • [#573] Immutable: you can now apply the configuration file that furyctl create config generates without changes to it. Each node must now declare arch, the schema declared a default of x86-64 for arch, but nothing applied that default. A node without the field made furyctl apply stop with a template error.
  • [#607] DR module: fix velero triggering immediate backups after every furyctl apply.
  • [#597] OnPremises and Immutable: the API server now uses the OIDC groups prefix from the configuration file. Before this release, the installers ignored spec.kubernetes.advanced.oidc.group_prefix and always used oidc:. The new field is groups_prefix, the same name as claimMappings.groups.prefix in the API server AuthenticationConfiguration. group_prefix is deprecated, but it still works when groups_prefix is not set. Refer to the Breaking Changes section.
  • [#589] All kinds: the schema now validates the storageSize fields of Prometheus, OpenSearch and MinIO as Kubernetes quantities, for example 10Gi. A value like 10GB now fails when furyctl validates the configuration, and not during the apply. The retentionSize field of Prometheus does not change, because it uses Prometheus units like 120GB.

Breaking changes 💔​

  • [#600] Immutable: spec.infrastructure.proxy now applies to every service on the node, so noProxy must list the addresses that the cluster uses to reach itself. Add the network of the nodes, the control plane address, the pod CIDR and the service CIDR. If these addresses are absent, kubelet and the other components try to reach the API server through the proxy, and the cluster does not start.
  • [#597] OnPremises and Immutable: if you set spec.kubernetes.advanced.oidc.group_prefix to a value that is not oidc:, the OIDC group names change when you apply. Before this release, the API server always used oidc:, so your RoleBinding and ClusterRoleBinding objects probably use oidc:. Update these bindings to the prefix that you set before you apply, or the users of these groups lose access. If you did not set the field, or you set it to oidc:, nothing changes.
  • Tracing: Tempo no longer supports the OpenCensus receiver.

Known issues ⚠️​

  • OnPremises and Immutable with Cilium: Cilium 1.18 and 1.19 have a bug (cilium/cilium#44891). When the upgrade replaces a Cilium pod that is not Ready, the hubble-relay pod on that node can stay not Ready, and furyctl apply waits on deployment/hubble-relay. The Immutable upgrade now waits until the pods of each node are Ready, unless you use --force pods-running-check.

    Before you upgrade, make sure that all the Cilium pods are Ready:

    kubectl -n kube-system rollout status ds/cilium

    If the fault occurs, drain the node of the hubble-relay pod, reboot it, then uncordon it. A restart of the Cilium pod does not clear the fault.

Upgrade procedure​

Check the upgrade docs for the steps to upgrade the SIGHUP Distribution from one version to the next using furyctl.