SIGHUP Distribution Ingress

Overview

SIGHUP Distribution Ingress supports multiple ingress controllers: HAProxy Kubernetes Ingress Controller (the new reference ingress controller, following the official retirement announcement of the Ingress NGINX Controller) and Ingress NGINX. It uses cert-manager to automate the issuing and renewal of TLS certificates from various issuing sources.

The module also includes additional tools like Forecastle, a web-based global directory of all the services offered by your cluster.

Module's repository: https://github.com/sighupio/module-ingress

Architecture

The reference architecture used to deploy the SIGHUP Distribution Ingress Module is shown in the following figure:

• The traffic from end users arrives first at a Load Balancer that distributes the traffic between the available Ingress Controllers (usually, one for each availability zone).
• Once the traffic reaches the Ingress Controller, the Ingress proxies the traffic to the Kubernetes service based on the URL path of the request.
• The service is a Kubernetes abstraction that makes the traffic arrive at the pods where the actual application is running, usually using iptables rules.

For more information, please refer to Kubernetes Ingress official documentation.

Packages

SIGHUP Distribution Ingress provides the following packages:

Package	Description
haproxy	The HAProxy Kubernetes Ingress Controller, the new reference ingress controller for SD.
nginx	The NGINX Ingress Controller for Kubernetes provides delivery services for Kubernetes applications.
cert-manager	cert-manager is a Kubernetes add-on to automate the management and issuance of TLS certificates from various issuing sources.
external-dns	external-dns allows you to manage DNS records natively from Kubernetes in EKS clusters.
forecastle	Forecastle gives you access to a control panel where you can see your ingresses and access them on Kubernetes.

info

NGINX Ingress Controller components are deployed in the ingress-nginx namespace. HAProxy Kubernetes Ingress Controller components are deployed in the ingress-haproxy namespace. Forecastle is deployed in the forecastle namespace. External-DNS is deployed in the external-dns namespace.

Compatibility

Kubernetes Version	Compatibility	Notes
1.31.x	✅	No known issues
1.32.x	✅	No known issues
1.33.x	✅	No known issues
1.34.x	✅	No known issues

Check the compatibility matrix for additional information on previous releases of the module.

Introduction: Ingress in Kubernetes

In Kubernetes, the Ingress resource provides a way to manage external access to services running within the cluster. Instead of exposing each service individually with LoadBalancers or NodePorts, an Ingress acts as a central entry point, routing external HTTP and HTTPS traffic to the appropriate backend services based on defined rules. Ingress resources enable functionalities like URL-based routing, SSL termination, and custom headers, streamlining how applications interact with external clients.

Using Ingress in Kubernetes simplifies managing traffic for large-scale applications by reducing the complexity of service exposure. It enables:

• Centralized Traffic Management: Control all ingress traffic at a single layer with customizable routing rules.
• SSL/TLS Termination: Simplify certificate management by handling encryption and decryption at the ingress layer.
• Scalability and Flexibility: Dynamically update routing configurations as workloads change without restarting services.

To use Ingress, a controller is required. The controller watches for Ingress resources in the cluster and implements the routing logic, translating it into configurations for an underlying proxy like NGINX.

SD: Ingress module

The Ingress module provides SD cluster with an ingress controller, an Ingress resource for each of the SD-managed services (i.e. Grafana, Alertmanager...), cert-manager to deploy self-signed certificates for all internal Ingresses and Forecastle, a nice portal to group all Ingress links in one place.

NGINX and HAProxy Kubernetes Ingress Controllers can be configured independently. You can enable one, both, or neither. When both are enabled, the infrastructureIngressController field controls which one is used for SD's own ingresses (e.g. Grafana, Alertmanager). By default, NGINX takes priority.

For each controller (nginx and haproxy) you can choose between:

• none: the ingress controller will not be installed. Note that cert-manager will still be included, as it's used also for Kubernetes certificates.
• single: a single ingress controller instance will be installed.
• dual: SD will deploy two separate ingress controller instances, one "internal" to be used for management purposes and one "external" to expose applications to end users.

SD also supports a BYOIC (Bring Your Own Ingress Controller) mode, which allows using a custom ingress controller deployed as a distribution plugin, without it being managed by the SD lifecycle.

If you are deploying SD using the EKSCluster provider, SD will also install external-dns in its own dedicated external-dns namespace to automatically manage Route53 DNS rules.

You can see all available configuration options in the Provider's documentation

tip

If you want to also add your own Ingress resources to Forecastle, you can do that by annotating them:

kubectl annotate ingress <YOUR_INGRESS> "forecastle.stakater.com/expose=true" --overwrite

HAProxy Kubernetes Ingress Controller

HAProxy Kubernetes Ingress Controller is the new reference ingress controller for SD, adopted following the official retirement announcement of the Ingress NGINX Controller. It leverages the high-performance HAProxy load balancer to manage and route HTTP(S) traffic.

In SD, you can choose between a single instance or a dual instance of HAProxy Kubernetes Ingress Controller. With the single instance, all Ingresses will be exposed through the same controller with IngressClass haproxy. With the dual configuration, SD will deploy two separate controllers:

• External controller with IngressClass haproxy-external, intended for public traffic towards end users.
• Internal controller with IngressClass haproxy-internal, intended for internal purpose.

The single instance will expose the TCP/30080 and TCP/30443 ports on nodes where it will be installed.

The dual instance will expose the following ports on nodes where they will be installed:

• TCP/30080 and TCP/30443 ports for the haproxy-external class
• TCP/32680 and TCP/32643 ports for the haproxy-internal class

tip

If you are migrating from NGINX, see the Migrating from NGINX to HAProxy guide.

NGINX Ingress Controller

The NGINX Ingress Controller is one of the most popular and widely used ingress controllers for Kubernetes. It leverages the robust and proven NGINX web server to manage and route HTTP(S) traffic. Key features include:

• Advanced Traffic Management: Supports URL-based routing, host-based routing, and complex rewrites.
• High Performance: Optimized for high traffic loads and low latency.
• TLS/SSL Support: Handles HTTPS traffic efficiently, supporting multiple certificates.

In SD, you can choose between a single instance or a double instance of NGINX Ingress Controller. With the single instance, all Ingresses will be exposed through the same controller. If you choose to have the dual configuration, SD will create two classes: internal, which is intended to be used to expose traffic for Ops teams, and external, which is intended to be used by end users.

The single instance will expose the TCP/31080 and TCP/31443 ports on nodes where it will be installed.

The double instance will expose the following ports on nodes where they will be installed:

• TCP/31080 and TCP/31443 ports for the external class
• TCP/32080 and TCP/32443 ports for the internal class

cert-manager

cert-manager, a CNCF open-source software, provides resources to manage X.509 certificates natively and automatically in Kubernetes. It can be configured to issue certificates from both well-known, public Issuers as well as private ones. It also manages the renewal of the issued certificates.

Both HAProxy and NGINX Ingress Controllers can automatically provision certificates for their Ingress resources using one of the Issuers provided by cert-manager in a Kubernetes cluster. Using these technologies together helps exposing applications to be accessed from outside a Kubernetes cluster in a more efficient way, managing the certificates for each route independently and automatically.

SD will install cert-manager regardless of the choice you make about the Ingress Controller, because it uses it to also create certificates used by Kubernetes.

You can also specify your own configuration for the ClusterIssuer resources, which will be responsible for the certificates used by all Ingresses.

external-dns

external-dns automatically manages DNS records from Kubernetes. Paired with NGINX Ingress Controller, it provides a powerful combination that can automate the lifecycle of DNS records for your Ingresses.

Currently, SD will use external-dns only when installed using the EKSCluster provider to automate the creation and deletion of Route53 records.

Forecastle

Forecastle is a control panel which dynamically discovers and provides a launchpad to access applications deployed on Kubernetes.

It will provide a simple frontend with a list of Ingresses that are available in a cluster, either auto-discovered or injected through annotations.

SD annotates every Ingress it creates to make Forecastle show them out of the box.

Monitoring

Both ingress controllers ship with Prometheus Rules out of the box:

NGINX Ingress Controller

Parameter	Description	Severity	Interval
NginxIngressDown	This alert fires if Prometheus target discovery was not able to reach ingress-nginx-metrics in the last 15 minutes.	critical	15m
NginxIngressFailureRate	This alert fires if the failure rate (the rate of 5xx responses) measured on a time window of 2 minutes was higher than 10% in the last 10 minutes.	critical	10m
NginxIngressFailedReload	This alert fires if the ingress' configuration reload failed in the last 10 minutes.	warning	10m
NginxIngressLatencyTooHigh	This alert fires if the ingress 99th percentile latency was more than 5 seconds in the last 10 minutes.	warning	10m
NginxIngressLatencyTooHigh	This alert fires if the ingress 99th percentile latency was more than 10 seconds in the last 10 minutes.	critical	10m
NginxIngressCertificateExpiration	This alert fires if the certificate for a given host is expiring in less than 7 days.	warning
NginxIngressCertificateExpiration	This alert fires if the certificate for a given host is expiring in less than 1 day.	critical

HAProxy Kubernetes Ingress Controller

Parameter	Description	Severity	Interval
HaproxyIngressTargetDown	This alert fires if Prometheus target discovery was not able to reach haproxy-ingress metrics in the last 15 minutes.	critical	15m
HaproxyHighHttp4xxErrorRateBackend	This alert fires if the 4xx error rate measured on a 2 minute window exceeds 10% for 5 minutes.	warning	5m
HaproxyHighHttp5xxErrorRateBackend	This alert fires if the 5xx error rate measured on a 2 minute window exceeds 10% for 10 minutes.	critical	10m
HaproxyServerResponseErrors	This alert fires if a specific backend server has a response error rate exceeding 5% for 5 minutes.	warning	5m
HaproxyBackendConnectionErrors	This alert fires if connection errors exceed 100 per second for 5 minutes on a backend.	warning	5m
HaproxyBackendNoAliveServers	This alert fires immediately when a backend has no healthy servers available.	critical	0m
HaproxyBackendPendingRequests	This alert fires if requests have been queued for a backend for more than 5 minutes.	warning	5m
HaproxyFrontendSecurityBlockedRequests	This alert fires if more than 10 connections per second are being denied on a frontend for 2 minutes.	warning	2m
HaproxyBackendLatencyHigh	This alert fires if average backend response time exceeds 5 seconds for 10 minutes.	warning	10m
HaproxyBackendLatencyCritical	This alert fires if average total time (including queue time) exceeds 10 seconds for 10 minutes.	critical	10m
HaproxyServerHealthcheckFailure	This alert fires if health check failures are detected for a server over a 5 minute period.	warning	5m
HaproxyIngressConfigSyncFailed	This alert fires if the ingress configuration sync failed in the last 10 minutes.	warning	10m

Read more

You can find more info about these topics at the following links:

• https://kubernetes.io/docs/concepts/services-networking/ingress/
• https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
• https://letsencrypt.org/
• https://cert-manager.io/