Terraform Module to Deploy a Highly Available, Production-Ready Talos Kubernetes Cluster on Hetzner Cloud
See the codeHcloud Kubernetes is a Terraform module for deploying a fully declarative, managed Kubernetes cluster on Hetzner Cloud. It utilizes Talos, a secure, immutable, and minimal operating system specifically designed for Kubernetes, featuring a streamlined architecture with only a handful of binaries and shared libraries. Just enough to run containerd and a small set of system services.
This project is committed to production-grade configuration and lifecycle management, ensuring all components are set up for high availability. It includes a curated selection of widely used and officially recognized Kubernetes components. If you encounter any issues, suboptimal settings, or missing elements, please file an issue to help us improve this project.
Provision a highly available and secure Kubernetes cluster on Hetzner Cloud, defined by these key features:
This project bundles essential Kubernetes components, preconfigured for seamless operation on Hetzner Cloud:
Talos Linux is a secure, minimal, and immutable OS for Kubernetes, removing SSH and shell access to reduce attack surfaces. Managed through a secure API with mTLS, Talos prevents configuration drift, enhancing both security and predictability. It follows NIST and CIS hardening standards, operates in memory, and is built to support modern, production-grade Kubernetes environments.
nodeID.[!IMPORTANT] Keep the CLI tools up to date. Ensure that
talosctlmatches your Talos version for compatibility, especially before a Talos upgrade.
Create kubernetes.tf file with the module configuration:
module "kubernetes" {
source = "hcloud-k8s/kubernetes/hcloud"
version = "<version>"
cluster_name = "k8s"
hcloud_token = "<hcloud-token>"
# Export configs for talosctl and kubectl (optional)
cluster_kubeconfig_path = "kubeconfig"
cluster_talosconfig_path = "talosconfig"
# Enable Cilium Gateway API and Cert Manager (optional)
cert_manager_enabled = true
cilium_gateway_api_enabled = true
control_plane_nodepools = [
{ name = "control", type = "cpx22", location = "nbg1", count = 3 }
]
worker_nodepools = [
{ name = "worker", type = "cpx22", location = "nbg1", count = 3 }
]
}
[!NOTE] Each Control Plane node requires at least 4GB of memory and each Worker node at least 2GB. For High-Availability (HA), at least 3 Control Plane nodes and 3 Worker nodes are required.
Initialize and deploy the cluster:
Terraform:
terraform init -upgrade
terraform apply
OpenTofu:
tofu init -upgrade
tofu apply
Set config file locations:
export TALOSCONFIG=talosconfig
export KUBECONFIG=kubeconfig
Display cluster nodes:
talosctl get member
kubectl get nodes -o wide
Display all pods:
kubectl get pods -A
For more detailed information and examples, please visit:
To destroy the cluster, first disable the delete protection by setting:
cluster_delete_protection = false
Apply this change before proceeding. Once the delete protection is disabled, you can teardown the cluster.
Terraform:
terraform destroy
OpenTofu:
tofu destroy
Bare metal worker nodes can be added through Hetzner Robot. The module enables rescue mode, installs Talos, attaches the servers to a vSwitch, and applies the Talos worker configuration.
Example kubernetes.tf snippet:
hcloud_robot_user = "<robot-user>"
hcloud_robot_password = "<robot-password>"
bare_metal_nodepools = [
{
name = "bare-metal"
servers = [
{ number = 1111111, private_ipv4 = "10.0.88.2" },
{ number = 2222222, private_ipv4 = "10.0.88.3" }
]
}
]
The module creates the vSwitch and connects it to the Hetzner Cloud Network. To use an existing vSwitch instead, set:
hcloud_vswitch_id = 12345
Each server needs a unique private_ipv4 from the vSwitch subnet. For the default 10.0.0.0/16 network, the bare metal subnet can also be calculated with:
echo "cidrsubnet(\"10.0.0.0/16\", 9, 176)" | tofu console
"10.0.88.0/25"
When bare metal servers are enabled for the first time, Cilium might need to be restarted to pick up the changed routing mode:
kubectl -n kube-system rollout restart ds/cilium ds/cilium-envoy
kubectl -n kube-system rollout restart deploy/cilium-operator
If install_disks and install_raid_level are omitted, the module automatically selects the first eligible non-removable disk. Explicit install_disks values must be disk IDs from /dev/disk/by-id:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_disks = ["ata-Samsung_SSD_870_ABC123"]
}
]
Setting install_raid_level to raid1 creates a bootable Linux MD RAID1 array with metadata format 1.0. RAID1 requires at least two disks, and specifying multiple installation disks without install_raid_level is rejected.
List the RAID members explicitly:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_disks = ["nvme-disk-1", "nvme-disk-2"]
install_raid_level = "raid1"
}
]
Alternatively, omit install_disks to include every eligible disk automatically:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_raid_level = "raid1"
}
]
preserve_other_disks defaults to false. Set it to true on an individual server to preserve eligible disks that are not used for the Talos installation or RAID.
⚠️ Warning: Installing Talos is destructive. By default, every eligible disk is wiped to prevent the server from booting a previous installation, and all data on those disks is lost. Preserved disks can still contain bootable installations, so the firmware might boot from them. RAID configuration is used only during initial provisioning; changing it does not reprovision existing servers.
By default, the cluster is accessible over the public internet. The firewall is automatically configured to use the IPv4 address and /64 IPv6 CIDR of the machine running this module. To disable this automatic configuration, set the following variables to false:
firewall_use_current_ipv4 = false
firewall_use_current_ipv6 = false
To manually specify source networks for the Talos API and Kube API, configure the firewall_api_source variable as follows:
firewall_api_source = [
"1.2.3.0/32",
"1:2:3::/64"
]
This allows explicit control over which networks can access your APIs, overriding the default behavior when set.
If your internal network is routed and accessible, you can directly access the cluster using internal IPs by setting:
cluster_access = "private"
For integrating Talos nodes with an internal network, configure a default route (0.0.0.0/0) in the Hetzner Network to point to your router or gateway. Additionally, add specific routes on the Talos nodes to encompass your entire network CIDR:
talos_extra_routes = ["10.0.0.0/8"]
# Optionally, disable NAT for your globally routed CIDR
network_native_routing_ipv4_cidr = "10.0.0.0/8"
# Optionally, use an existing Network
hcloud_network_id = 123456789
This setup ensures that the Talos nodes can route traffic appropriately across your internal network.
Optionally, a hostname can be configured to direct access to the Kubernetes API through a node IP, load balancer, or Virtual IP (VIP):
kube_api_hostname = "kube-api.example.com"
For accessing the Kubernetes API from the public internet, choose one of the following options based on your needs:
kube_api_load_balancer_enabled = true
control_plane_public_vip_ipv4_enabled = true
# Optionally, specify an existing Floating IP
control_plane_public_vip_ipv4_id = 123456789
When accessing the Kubernetes API via an internal network, an internal Virtual IP (Alias IP) is utilized by default to route API requests within the network. This feature can be disabled with the following configuration:
control_plane_private_vip_ipv4_enabled = false
To enhance internal availability, a load balancer can be used:
kube_api_load_balancer_enabled = true
This setup ensures secure and flexible access to the Kubernetes API, accommodating different networking environments.
Example kubernetes.tf snippet:
# Configuration for cluster autoscaler node pools
cluster_autoscaler_enabled = true
cluster_autoscaler_nodepools = [
{
name = "autoscaler"
type = "cpx22"
location = "nbg1"
min = 0
max = 6
labels = { "autoscaler-node" = "true" }
taints = [ "autoscaler-node=true:NoExecute" ]
}
]
Optionally, pass additional Helm values to the cluster autoscaler configuration:
cluster_autoscaler_helm_values = {
extraArgs = {
enforce-node-group-min-size = true
scale-down-delay-after-add = "45m"
scale-down-delay-after-delete = "4m"
scale-down-unneeded-time = "5m"
}
}
Cluster Autoscaler does not support upgrading nodes or changing their configuration, as its primary purpose is to manage short-lived nodes that handle load peaks. If you require long-lived autoscaled nodes, you can upgrade them manually using talosctl or use this Terraform module, which supports discovery of autoscaled nodes and manages their upgrades and configuration changes.
To enable this feature, add the following to your configuration:
cluster_autoscaler_discovery_enabled = true
Please note that errors may occur if a node pool has been scaled down recently, as Talos caches absent nodes for up to 30 minutes. You can pause automatic scaling by stopping the Cluster Autoscaler pods:
kubectl -n kube-system scale deployment cluster-autoscaler-hetzner-cluster-autoscaler --replicas=0
The module installs a curated set of Kubernetes components through Talos manifests and reconciles them during the normal module lifecycle.
| Component | Variable | Default |
|---|---|---|
| Cilium Container Network Interface (CNI) | cilium_enabled | true |
| Hcloud Cloud Controller Manager (CCM) | hcloud_ccm_enabled | true |
| Hcloud Container Storage Interface (CSI) | hcloud_csi_enabled | true |
| Metrics Server | metrics_server_enabled | true |
| Talos Backup | talos_backup_enabled | true |
| Talos Cloud Controller Manager (CCM) | talos_ccm_enabled | true |
| Component | Variable | Default |
|---|---|---|
| Cert Manager | cert_manager_enabled | false |
| Cert Manager Hetzner Webhook | cert_manager_webhook_hetzner_enabled | false |
| Cilium Gateway API | cilium_gateway_api_enabled | false |
| Cluster Autoscaler | cluster_autoscaler_enabled | false |
| Longhorn | longhorn_enabled | false |
| Component | Variable | Default |
|---|---|---|
| Gateway API CRDs | gateway_api_crds_enabled | true |
| Prometheus Operator CRDs | prometheus_operator_crds_enabled | true |
Extra manifests can be added alongside the built-in components:
talos_extra_remote_manifests = [
"https://example.com/extra-remote-manifest.yaml"
]
talos_extra_inline_manifests = [
{
name = "test-manifest"
contents = <<-EOF
---
apiVersion: v1
kind: Secret
metadata:
name: test-secret
data:
secret: dGVzdA==
EOF
}
]
By default, when using K8s NetworkPolicies CIDR-based selectors in Cilium do not match in-cluster entities (pods or nodes). This is however required when using NetworkPolicies that control traffic to the kube-apiserver pods as these use the Talos node IPs in our setup. Cilium's policy engine can be directed to select nodes by CIDR / ipBlock. This requires you to add the following to your configuration:
cilium_policy_cidr_match_mode = "nodes"
It is safe to toggle this option on a running cluster, and toggling the option affects neither upgrades nor downgrades More information can be found in the Cilium docs.
This module enables Cilium Transparent Encryption feature by default.
All pod network traffic is encrypted using WireGuard (Default) or protocols, includes automatic key rotation and efficient in-kernel encryption, covering all traffic types.
💡 Although WireGuard is the default option, Hetzner Cloud VMs supports AES-NI instruction set, making IPSec encryption more CPU-efficient compared to WireGuard. Consider enabling IPSec for CPU savings through hardware acceleration.
IPSec mode supports RFC4106 AES-GCM encryption with 128, 192 and 256 bits key sizes.
⚠️ IPSec encryption has the following limitations:
Source: Cilium Documentation
Example kubernetes.tf configuration:
cilium_encryption_enabled = true # Default true
cilium_encryption_type = "wireguard" # wireguard (Default) | ipsec
cilium_ipsec_algorithm = "rfc4106(gcm(aes))" # IPSec AES key algorithm (Default rfc4106(gcm(aes)))
cilium_ipsec_key_size = 256 # IPSec AES key size (Default 256)
cilium_ipsec_key_id = 1 # IPSec key ID (Default 1)
Keys automatically rotate when cilium_ipsec_key_id is incremented (1-15 range, resets to 1 after 15).
Cilium offers an Egress Gateway to ensure network compatibility with legacy systems and firewalls requiring fixed IPs. The use of Cilium Egress Gateway does not provide high availability and increases latency due to extra network hops and tunneling. Consider this configuration only as a last resort.
Example kubernetes.tf snippet:
# Enable Cilium Egress Gateway
cilium_egress_gateway_enabled = true
# Define worker nodepools including an egress-specific node pool
worker_nodepools = [
# ... (other node pool configurations)
{
name = "egress"
type = "cpx22"
location = "nbg1"
labels = { "egress-node" = "true" }
taints = [ "egress-node=true:NoSchedule" ]
}
]
Example Egress Gateway Policy:
apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
name: sample-egress-policy
spec:
selectors:
- podSelector:
matchLabels:
io.kubernetes.pod.namespace: sample-namespace
app: sample-app
destinationCIDRs:
- "0.0.0.0/0"
egressGateway:
nodeSelector:
matchLabels:
egress-node: "true"
Please visit the Cilium documentation for more details.
Each rule is defined with the following properties:
description: A brief description of the rule.direction: The direction of traffic (in for inbound, out for outbound).source_ips: A list of source IP addresses for inbound rules.destination_ips: A list of destination IP addresses for outbound rules.protocol: The protocol used (valid options: tcp, udp, icmp, gre, esp).port: The port number (required for tcp and udp protocols, must not be specified for icmp, gre, and esp).Example kubernetes.tf snippet:
firewall_extra_rules = [
{
description = "Custom UDP Rule"
direction = "in"
source_ips = ["0.0.0.0/0", "::/0"]
protocol = "udp"
port = "12345"
},
{
description = "Custom TCP Rule"
direction = "in"
source_ips = ["1.2.3.4", "1:2:3:4::"]
protocol = "tcp"
port = "8080-9000"
},
{
description = "Allow ICMP"
direction = "in"
source_ips = ["0.0.0.0/0", "::/0"]
protocol = "icmp"
}
]
For access to Talos and the Kubernetes API, please refer to the Cluster Access configuration section.
Kubernetes Gateway API is the modern replacement for Kubernetes Ingress. It fixes many Ingress limitations by offering a richer, more consistent model for traffic management, and it's designed to support multiple Gateway API implementations in parallel.
This module installs the Gateway API CRDs by default and deploys Cert Manager with Gateway API support enabled.
To use Cilium's Gateway API implementation, configure:
cilium_gateway_api_enabled = true
If Cert Manager and Cilium weren't initially deployed/configured with Gateway API support enabled, you may need to restart their controllers to pick up the new configuration:
kubectl -n cert-manager rollout restart deployment
kubectl -n kube-system rollout restart deployment/cilium-operator
Issuer (Let's Encrypt HTTP-01 via Gateway API)Issuer configures Cert Manager to request TLS certificates from Let's Encrypt using the ACME HTTP-01 challenge. The important part is the gatewayHTTPRoute solver: when Cert Manager needs to prove domain ownership, it will temporarily create/attach an HTTPRoute under the referenced Gateway and serve the /.well-known/acme-challenge/... response there.
Replace the placeholder email address, and note that privateKeySecretRef is the secret where Cert Manager stores your ACME account key (not the issued certificate).
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-http01
namespace: default
spec:
acme:
email: <[email protected]>
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-http01-key
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- name: cilium-gateway
namespace: default
kind: Gateway
ClusterIssuer (Let's Encrypt DNS-01 via Hetzner DNS)To request wildcard certificates or certificates that do not depend on public HTTP routing, enable the Hetzner DNS webhook:
cert_manager_enabled = true
cert_manager_webhook_hetzner_enabled = true
Your DNS zone must be managed by Hetzner DNS. Create a Secret containing a Hetzner API token with permission to create and remove DNS records. For ClusterIssuer resources, place the Secret in the cert-manager cluster resource namespace, which defaults to cert-manager.
apiVersion: v1
kind: Secret
metadata:
name: hetzner
namespace: cert-manager
stringData:
token: <hetzner-dns-token>
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01
spec:
acme:
email: <[email protected]>
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
webhook:
groupName: acme.hetzner.com
solverName: hetzner
config:
tokenSecretKeyRef:
name: hetzner
key: token
When using this ClusterIssuer with the Gateway shim, replace cert-manager.io/issuer: letsencrypt-http01 with cert-manager.io/cluster-issuer: letsencrypt-dns01.
Gateway (Cilium GatewayClass + Hetzner LB + Cert Manager integration)A Gateway defines the external entry point for traffic. With gatewayClassName: cilium, the resource is reconciled by the Cilium Gateway controller. The infrastructure.annotations are passed through as Hetzner-specific load balancer settings (interpreted by the Hetzner Cloud Controller Manager) to control how the Hetzner Cloud LB is created and configured. See: Load Balancer Annotations
The cert-manager.io/issuer: letsencrypt-http01 annotation is used by Cert Manager's Gateway shim so it knows which Issuer to use when populating the TLS secret referenced by the listener.
tls.certificateRefs points at the Kubernetes Secret that will contain the issued certificate and private key (example-com-tls). Cert Manager will keep that secret up to date, and the Gateway will use it to terminate TLS.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: cilium-gateway
namespace: default
annotations:
cert-manager.io/issuer: letsencrypt-http01
spec:
gatewayClassName: cilium
infrastructure:
annotations:
load-balancer.hetzner.cloud/name: "cilium-gateway-nbg1"
load-balancer.hetzner.cloud/location: "nbg1"
load-balancer.hetzner.cloud/uses-proxyprotocol: "true"
listeners:
- name: https
hostname: example.com
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: All
tls:
mode: Terminate
certificateRefs:
- name: example-com-tls
kind: Secret
group: ""
- name: http
hostname: example.com
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
HTTPRoute (bind hostname + route traffic to your service)Finally, the HTTPRoute attaches to the Gateway and defines routing rules for example.com. In this example it forwards all matching requests to a Kubernetes Service called example-service on port 8080.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: example-app
namespace: default
spec:
hostnames:
- example.com
parentRefs:
- name: cilium-gateway
namespace: default
rules:
- backendRefs:
- name: example-service
port: 8080
The Hetzner Cloud Container Storage Interface (CSI) driver can be flexibly configured through the hcloud_csi_storage_classes variable. You can define multiple storage classes for your cluster:
false).Delete or Retain, optional, defaults to Delete).Example:
hcloud_csi_storage_classes = [
{
name = "hcloud-volumes"
encrypted = false
defaultStorageClass = true
},
{
name = "hcloud-volumes-encrypted-xfs"
encrypted = true
reclaimPolicy = "Retain"
extraParameters = {
"csi.storage.k8s.io/fstype" = "xfs"
"fsFormatOption" = "-i nrext64=1"
}
}
]
Other settings:
hcloud_csi_encryption_passphrase: Optionally provide a custom encryption passphrase for LUKS-encrypted storage classes.
hcloud_csi_encryption_passphrase = "<secret-passphrase>"
Storage Class Immutability:
StorageClasses created by the Hcloud CSI driver are immutable. To change parameters after creation, you must either edit the StorageClass directly with kubectl, or delete it from both Terraform state and Kubernetes, then let this module recreate it.
For more details, see the HCloud CSI Driver documentation.
⚠️ Experimental: IPv6 Pod networking is opt-in. Hetzner Private Networks currently support only IPv4. When you enable IPv6 Pod networking with the CIDRs below and leave the routing settings unchanged, Cilium carries IPv6 Pod traffic between nodes through its IPv4 VXLAN overlay, and the module disables the Hetzner Cloud CCM Route Controller. For Internet egress, Cilium's IPv6 eBPF masquerading translates each Pod's IPv6 source address to its node's public IPv6 address. This NAT feature is beta, so dual-stack Pod networking is not supported for production use by this module.
To try it on a new cluster, provide separate IPv6 Pod and Service CIDRs:
network_pod_ipv6_cidr = "fd42:6:175::/56"
network_service_ipv6_cidr = "fd42:6:176::/112"
Choose unique IPv6 CIDRs for your cluster. Kubernetes allocates a /64 Pod CIDR per node from the Pod range. Cilium carries both IPv4 and IPv6 Pod traffic over its IPv4 VXLAN tunnel. Hetzner Cloud CCM still manages nodes and load balancers. IPv6 Internet egress uses each node's public IPv6 address, so keep talos_public_ipv6_enabled = true.
⚠️ Existing clusters: Enabling IPv6 Pod networking on an existing IPv4-only cluster requires additional migration steps and may disrupt Pod networking. Only do this if you know exactly what you are doing.
Longhorn is a lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes. It is fully independent from the Hetzner Cloud CSI driver.
You can enable Longhorn and configure it as the default StorageClass for your cluster via module variables:
Enable Longhorn:
Set longhorn_enabled to true to deploy Longhorn in your cluster.
Default StorageClass:
Set longhorn_default_storage_class to true if you want Longhorn to be the default StorageClass.
Example:
longhorn_enabled = true
longhorn_default_storage_class = true
Migrating existing Longhorn disks from /var/lib/longhorn to /var/mnt/longhorn:
Starting with module version 6, this module follows the Longhorn guidance for Talos Linux v1.10 and later, which recommends UserVolumeConfig and /var/mnt/longhorn instead of the legacy /var/lib/longhorn data path. Existing disks are not migrated automatically, so version 6 keeps the legacy path available for a rolling migration. Clusters created with an earlier module version must complete this migration before upgrading to module version 7 or higher.
/var/mnt/longhorn and uses it for new Longhorn disks while keeping existing /var/lib/longhorn disks available./var/lib/longhorn path. Note any custom storage reservation or tags that must be retained.true, and save.0 replicas and no backing images. Do not remove the disk before eviction finishes./var/lib/longhorn disk, and save./var/mnt/longhorn path, restore any custom storage reservation or tags, enable scheduling, and save.Do not enable scheduling on both paths of the same node at once. They share the node's EPHEMERAL filesystem, so Longhorn would count the same capacity twice. Complete the migration before upgrading to module version 7 or higher, which will remove the legacy /var/lib/longhorn mount.
For more information about Longhorn, see the Longhorn documentation.
By default, this module calculates optimal subnets based on the provided network CIDR (network_ipv4_cidr). The network is segmented automatically as follows:
network_node_ipv4_cidr)network_service_ipv4_cidr)network_pod_ipv4_cidr)Each Kubernetes node requires a /24 subnet within network_pod_ipv4_cidr. To support this configuration, the optimal node subnet size (network_node_ipv4_subnet_mask_size) is calculated using the formula:
32 - (24 - subnet_mask_size(network_pod_ipv4_cidr)).
With the default 10.0.0.0/16 network CIDR (network_ipv4_cidr), the following values are calculated:
/25 (Max. 128 Nodes per Subnet)10.0.64.0/19 (Max. 64 Subnets, each with /25)10.0.96.0/19 (Max. 8192 Services)/24 (Max. 256 Pods per Node)10.0.128.0/17 (Max. 128 Nodes, each with /24)The shared worker subnet and the shared Cluster Autoscaler subnet are created by default. Worker and Cluster Autoscaler node pools use their shared subnets unless they pin a legacy node subnet with subnet = "10.0.65.0/25". Treat pinned subnet values as immutable after the node pool has been created.
Please consider the following Hetzner Cloud limits:
A /16 Network CIDR is sufficient to fully utilize Hetzner Cloud's scaling capabilities. It supports:
/24 Pod subnet route.Here is a table with more example calculations:
| Network CIDR | Node Subnet Size | Node Subnets | Service IPs | Pod Subnets |
|---|---|---|---|---|
| 10.0.0.0/16 | /25 (128 IPs) | 10.0.64.0/19 (64) | 10.0.96.0/19 (8192) | 10.0.128.0/17 (128) |
| 10.0.0.0/17 | /26 (64 IPs) | 10.0.32.0/20 (64) | 10.0.48.0/20 (4096) | 10.0.64.0/18 (64) |
| 10.0.0.0/18 | /27 (32 IPs) | 10.0.16.0/21 (64) | 10.0.24.0/21 (2048) | 10.0.32.0/19 (32) |
| 10.0.0.0/19 | /28 (16 IPs) | 10.0.8.0/22 (64) | 10.0.12.0/22 (1024) | 10.0.16.0/20 (16) |
| 10.0.0.0/20 | /29 (8 IPs) | 10.0.4.0/23 (64) | 10.0.6.0/23 (512) | 10.0.8.0/21 (8) |
| 10.0.0.0/21 | /30 (4 IPs) | 10.0.2.0/24 (64) | 10.0.3.0/24 (256) | 10.0.4.0/22 (4) |
This module natively supports Hcloud Object Storage. Below is an example of how to configure backups with MinIO Client (mc) and Hcloud Object Storage. While it's possible to create the bucket through the Hcloud Console, this method does not allow for the configuration of automatic retention policies.
Create an alias for the endpoint using the following command:
mc alias set <alias> \
https://<location>.your-objectstorage.com \
<access-key> <secret-key> \
--api "s3v4" \
--path "off"
Create a bucket with automatic retention policies to protect your backups:
mc mb --with-lock --region <location> <alias>/<bucket>
mc retention set GOVERNANCE 14d --default <alias>/<bucket>
Configure your kubernetes.tf file:
talos_backup_s3_hcloud_url = "https://<bucket>.<location>.your-objectstorage.com"
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"
# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"
# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"
For users of other object storage providers, configure kubernetes.tf as follows:
talos_backup_s3_region = "<region>"
talos_backup_s3_endpoint = "<endpoint>"
talos_backup_s3_bucket = "<bucket>"
talos_backup_s3_prefix = "<prefix>"
# Use path-style URLs (set true if required by your provider)
talos_backup_s3_path_style = true
# Access credentials
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"
# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"
# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"
To recover from a snapshot, please refer to the Talos Disaster Recovery section in the Documentation.
By default, the module builds missing AMD64 and ARM64 Talos snapshots with Packer and selects images whose labels exactly match the cluster name, Talos version, and schematic ID. Image building and selection can be configured independently for each architecture:
talos_cloud_amd64_image_build_enabled = true
talos_cloud_amd64_image_selector = "exact"
talos_cloud_arm64_image_build_enabled = true
talos_cloud_arm64_image_selector = "exact"
The supported selector values are:
exact: Selects an image matching the cluster name, Talos version, and schematic ID.latest: Selects the newest image matching only the cluster name and architecture.Disabling an image builder is useful when matching snapshots are created outside this module. For example:
talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector = "exact"
If no exactly matching snapshot exists, the lookup fails instead of silently selecting an older image.
To disable the ARM64 builder and explicitly select the newest existing cluster snapshot:
talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector = "latest"
⚠️ Warning: The
latestselector is strongly discouraged and unsupported. It ignores the configured Talos version and schematic ID, so new nodes may boot with incompatible Talos versions, system extensions, or kernel arguments. This can cause bootstrap failures, unavailable nodes, or cluster disruption. It is not an officially supported recovery or emergency mechanism. Use it entirely at your own risk.
Talos uses the public discovery service operated by Sidero Labs to discover cluster members, including when Kubernetes is unavailable. Nodes require outbound access to TCP port 443 to use it.
The service is enabled by default and can be disabled with:
talos_discovery_service_enabled = false
For more details, refer to the official Talos discovery guide.
The module can install and configure the Talos Tailscale system extension on every control plane, worker, Cluster Autoscaler, and bare metal node:
talos_tailscale_enabled = true
talos_tailscale_auth_key = var.tailscale_auth_key
The module leaves containerboot defaults unchanged: Tailscale does not accept tailnet DNS settings and reruns tailscale up on each start. Additional settings can be supplied as KEY=VALUE entries:
talos_tailscale_environment = [
"TS_EXTRA_ARGS=--advertise-tags=tag:kubernetes",
]
TS_AUTH_ONCE=true avoids repeat login when state persists, but later changes to TS_EXTRA_ARGS are not reapplied. Use it only if that trade-off is acceptable.
The module leaves Cilium device selection unchanged. In testing, Cilium auto-detected tailscale0, used an MTU of 1280, and allowed pods to reach other nodes over Tailscale. Restricting Cilium to eth+ or excluding tailscale0 raised the MTU to 1450 but blocked that pod-to-tailnet traffic. If the lower MTU causes bandwidth problems, evaluate cilium_helm_values.devices for your network and test the traffic your workloads need.
If you also use the Tailscale Kubernetes Operator to expose Services or a Service CIDR while Cilium replaces kube-proxy, ensure socket load balancing is bypassed in pod namespaces. Set cilium_socket_lb_host_namespace_only_enabled = true unless Cilium Gateway API already enables this setting. The Talos node extension alone does not require it.
If talos_schematic_id is set, its custom schematic must already contain siderolabs/tailscale, because the module cannot modify a supplied schematic. The auth key is sensitive, but it is necessarily stored in Terraform state and the rendered Talos machine configuration.
This module allows you to create custom Kubernetes RBAC (Role-Based Access Control) roles and cluster roles that define specific permissions for users and groups. RBAC controls what actions users can perform on which Kubernetes resources. These custom roles can be used independently or combined with OIDC group mappings to automatically assign permissions based on user group membership from your identity provider.
rbac_cluster_roles)rbac_cluster_roles = [
{
name = my-cluster-role # ClusterRole name
rules = [
{
api_groups = [""] # Core API group (empty string for core resources)
resources = ["nodes"] # Cluster-wide resources this role can access
verbs = ["get", "list", "watch"] # Actions allowed on these resources
}
]
}
]
rbac_roles)rbac_roles = [
{
name = "my-role" # Role name
namespace = "target-namespace" # Namespace where the role will be created
rules = [
{
api_groups = [""] # Core API group (empty string for core resources)
resources = ["pods", "services"] # Resources this role can access
verbs = ["get", "list", "watch"] # Actions allowed on these resources
}
]
}
]
The Kubernetes API server supports OIDC (OpenID Connect) authentication, allowing integration with one or more external identity providers like Keycloak, Auth0, Authentik, Zitadel, etc. When enabled, users can authenticate using their existing organizational credentials instead of managing separate Kubernetes certificates or tokens.
OIDC authentication works by validating JWT tokens issued by your identity provider, extracting user information and group memberships, and mapping them to Kubernetes RBAC roles.
Each issuer must be unique. Configure multiple clients from the same issuer as audiences of a single provider. The module automatically configures MatchAny when a provider has multiple audiences.
kube_api_oidc_providers = [
{
issuer_url = "https://keycloak.example.com/realms/example"
audiences = ["kubectl", "headlamp"]
username_claim = "preferred_username"
groups_claim = "groups"
groups_prefix = "oidc:"
group_mappings = [
{
group = "cluster-admins-group"
cluster_roles = ["cluster-admin"]
},
{
group = "developers-group"
cluster_roles = ["view"]
roles = [
{
name = "developer-role"
namespace = "development"
}
]
}
]
}
]
Once OIDC is configured in your cluster, you'll need to configure your local kubectl to authenticate using OIDC tokens. This requires the kubelogin plugin.
# Homebrew (macOS and Linux)
brew install kubelogin
# Krew (macOS, Linux, Windows and ARM)
kubectl krew install oidc-login
# Chocolatey (Windows)
choco install kubelogin
First, verify that your OIDC provider is returning proper JWT tokens. Replace the placeholder values with your actual OIDC configuration:
kubectl oidc-login setup \
--oidc-issuer-url=https://your-oidc-provider.com \
--oidc-client-id=your-client-id \
--oidc-client-secret=your-client-secret \
--oidc-extra-scope=openid,email,profile # Add or change the scopes according to your IDP
This will open your browser for authentication. After successful login, you should see a JWT token in your terminal that looks like:
{
"aud": "your-client-id",
"email": "[email protected]",
"email_verified": true,
"exp": 1749867571,
"groups": [
"developers",
"kubernetes-users"
],
"iat": 1749863971,
"iss": "https://your-oidc-provider.com",
"nonce": "random-nonce-string",
"sub": "user-unique-identifier"
}
Verify that:
groups array contains your expected groupsemail field matches your user emailemail_verified is true (required by K8s)Add a new user to your ~/.kube/config file:
users:
- name: oidc-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
command: kubectl
args:
- oidc-login
- get-token
- --oidc-issuer-url=https://your-oidc-provider.com
- --oidc-client-id=your-client-id
- --oidc-client-secret=your-client-secret
- --oidc-extra-scope=groups
- --oidc-extra-scope=email
- --oidc-extra-scope=name # Add or change the scopes according to your IDP
Update your context to use the new OIDC user:
contexts:
- context:
cluster: your-cluster
namespace: default
user: oidc-user # Changed from certificate-based user
name: oidc@your-cluster # Updated context name
Now you can switch to the OIDC context and authenticate using your identity provider:
kubectl config use-context your-cluster-oidc
kubectl get pods # This will trigger OIDC authentication
Talos allows adding additional trusted root CA certificates to the system. This is useful for private image registries, OIDC providers with internal CAs, or other enterprise services.
The talos_certificates variable supports several input formats for maximum flexibility:
talos_certificates = {
"my-custom-ca" = "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"
}
talos_certificates = {
"enterprise-ca" = [
file("root.crt"),
file("intermediate.crt")
]
}
talos_certificates = {
"legacy-ca" = <<-EOT
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOT
}
This module manages the versions of Talos, Kubernetes, and all bundled components together. Each Hcloud K8s major version targets a defined Talos and Kubernetes minor version, while component versions are selected for compatibility with that Kubernetes release.
The table below lists the Talos and Kubernetes versions used by each Hcloud K8s version.
| Hcloud K8s | Talos Linux | Kubernetes |
|---|---|---|
| (7) | (1.15) | 1.36 |
| 6 | 1.14 | 1.35 |
| 5 | 1.13 | 1.34 |
Parenthesized versions are planned targets.
The table below lists the minimum required versions of each component to support the specified Kubernetes release.
| Kubernetes | Hcloud CCM | Hcloud CSI | Longhorn | Cilium | Cert Manager | Hetzner Webhook |
|---|---|---|---|---|---|---|
| 1.36 | ≥ 1.31 | ≥ 2.21 | ≥ 1.12 | ≥ 1.20 | ≥ 1.21 | ≥ 0.8 |
| 1.35 | ≥ 1.30 | ≥ 2.19 | ≥ 1.11 | ≥ 1.19.2 | ≥ 1.19 | ≥ 0.6.4 |
| 1.34 | ≥ 1.27 | ≥ 2.18 | ≥ 1.11 | ≥ 1.19 | ≥ 1.19 | ≥ 0.6.1 |
Any minor or major upgrade to Talos or Kubernetes results in a major version change for this module. Downgrades are generally neither supported nor tested.
Changing software versions manually is not recommended. Component versions are selected and tested for compatibility with the Kubernetes release shown above.
[!IMPORTANT]
- Do not combine major or minor module upgrades with infrastructure changes.
- Upgrade to the latest minor release before upgrading to the next major version.
- Verify that the cluster is healthy before and after upgrades.
If you'd like to support this project, please consider leaving a ⭐ on GitHub!
Your sponsorship supports the ongoing development, improvement, and maintenance of this project 🙏
Become a Sponsor:
We welcome everyone to join the Discussions, report Issues, and help improve this project.
Contributions are always welcome!
Hcloud Kubernetes is an independent, open source project built for the Hetzner Community. Its goal is to make Kubernetes deployments as predictable, repeatable, and flexible as possible.
This project is supported by Hetzner through testing infrastructure ❤️
Distributed under the MIT License. See LICENSE for more information.
342 followers · starred Sep 2025
188 followers · starred Aug 2025
608 followers · starred May 2026
860 followers · starred Sep 2025
Terraform Module to Deploy a Highly Available, Production-Ready Talos Kubernetes Cluster on Hetzner Cloud
See the codeHcloud Kubernetes is a Terraform module for deploying a fully declarative, managed Kubernetes cluster on Hetzner Cloud. It utilizes Talos, a secure, immutable, and minimal operating system specifically designed for Kubernetes, featuring a streamlined architecture with only a handful of binaries and shared libraries. Just enough to run containerd and a small set of system services.
This project is committed to production-grade configuration and lifecycle management, ensuring all components are set up for high availability. It includes a curated selection of widely used and officially recognized Kubernetes components. If you encounter any issues, suboptimal settings, or missing elements, please file an issue to help us improve this project.
Provision a highly available and secure Kubernetes cluster on Hetzner Cloud, defined by these key features:
This project bundles essential Kubernetes components, preconfigured for seamless operation on Hetzner Cloud:
Talos Linux is a secure, minimal, and immutable OS for Kubernetes, removing SSH and shell access to reduce attack surfaces. Managed through a secure API with mTLS, Talos prevents configuration drift, enhancing both security and predictability. It follows NIST and CIS hardening standards, operates in memory, and is built to support modern, production-grade Kubernetes environments.
nodeID.[!IMPORTANT] Keep the CLI tools up to date. Ensure that
talosctlmatches your Talos version for compatibility, especially before a Talos upgrade.
Create kubernetes.tf file with the module configuration:
module "kubernetes" {
source = "hcloud-k8s/kubernetes/hcloud"
version = "<version>"
cluster_name = "k8s"
hcloud_token = "<hcloud-token>"
# Export configs for talosctl and kubectl (optional)
cluster_kubeconfig_path = "kubeconfig"
cluster_talosconfig_path = "talosconfig"
# Enable Cilium Gateway API and Cert Manager (optional)
cert_manager_enabled = true
cilium_gateway_api_enabled = true
control_plane_nodepools = [
{ name = "control", type = "cpx22", location = "nbg1", count = 3 }
]
worker_nodepools = [
{ name = "worker", type = "cpx22", location = "nbg1", count = 3 }
]
}
[!NOTE] Each Control Plane node requires at least 4GB of memory and each Worker node at least 2GB. For High-Availability (HA), at least 3 Control Plane nodes and 3 Worker nodes are required.
Initialize and deploy the cluster:
Terraform:
terraform init -upgrade
terraform apply
OpenTofu:
tofu init -upgrade
tofu apply
Set config file locations:
export TALOSCONFIG=talosconfig
export KUBECONFIG=kubeconfig
Display cluster nodes:
talosctl get member
kubectl get nodes -o wide
Display all pods:
kubectl get pods -A
For more detailed information and examples, please visit:
To destroy the cluster, first disable the delete protection by setting:
cluster_delete_protection = false
Apply this change before proceeding. Once the delete protection is disabled, you can teardown the cluster.
Terraform:
terraform destroy
OpenTofu:
tofu destroy
Bare metal worker nodes can be added through Hetzner Robot. The module enables rescue mode, installs Talos, attaches the servers to a vSwitch, and applies the Talos worker configuration.
Example kubernetes.tf snippet:
hcloud_robot_user = "<robot-user>"
hcloud_robot_password = "<robot-password>"
bare_metal_nodepools = [
{
name = "bare-metal"
servers = [
{ number = 1111111, private_ipv4 = "10.0.88.2" },
{ number = 2222222, private_ipv4 = "10.0.88.3" }
]
}
]
The module creates the vSwitch and connects it to the Hetzner Cloud Network. To use an existing vSwitch instead, set:
hcloud_vswitch_id = 12345
Each server needs a unique private_ipv4 from the vSwitch subnet. For the default 10.0.0.0/16 network, the bare metal subnet can also be calculated with:
echo "cidrsubnet(\"10.0.0.0/16\", 9, 176)" | tofu console
"10.0.88.0/25"
When bare metal servers are enabled for the first time, Cilium might need to be restarted to pick up the changed routing mode:
kubectl -n kube-system rollout restart ds/cilium ds/cilium-envoy
kubectl -n kube-system rollout restart deploy/cilium-operator
If install_disks and install_raid_level are omitted, the module automatically selects the first eligible non-removable disk. Explicit install_disks values must be disk IDs from /dev/disk/by-id:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_disks = ["ata-Samsung_SSD_870_ABC123"]
}
]
Setting install_raid_level to raid1 creates a bootable Linux MD RAID1 array with metadata format 1.0. RAID1 requires at least two disks, and specifying multiple installation disks without install_raid_level is rejected.
List the RAID members explicitly:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_disks = ["nvme-disk-1", "nvme-disk-2"]
install_raid_level = "raid1"
}
]
Alternatively, omit install_disks to include every eligible disk automatically:
servers = [
{
number = 1234567
private_ipv4 = "10.0.88.2"
install_raid_level = "raid1"
}
]
preserve_other_disks defaults to false. Set it to true on an individual server to preserve eligible disks that are not used for the Talos installation or RAID.
⚠️ Warning: Installing Talos is destructive. By default, every eligible disk is wiped to prevent the server from booting a previous installation, and all data on those disks is lost. Preserved disks can still contain bootable installations, so the firmware might boot from them. RAID configuration is used only during initial provisioning; changing it does not reprovision existing servers.
By default, the cluster is accessible over the public internet. The firewall is automatically configured to use the IPv4 address and /64 IPv6 CIDR of the machine running this module. To disable this automatic configuration, set the following variables to false:
firewall_use_current_ipv4 = false
firewall_use_current_ipv6 = false
To manually specify source networks for the Talos API and Kube API, configure the firewall_api_source variable as follows:
firewall_api_source = [
"1.2.3.0/32",
"1:2:3::/64"
]
This allows explicit control over which networks can access your APIs, overriding the default behavior when set.
If your internal network is routed and accessible, you can directly access the cluster using internal IPs by setting:
cluster_access = "private"
For integrating Talos nodes with an internal network, configure a default route (0.0.0.0/0) in the Hetzner Network to point to your router or gateway. Additionally, add specific routes on the Talos nodes to encompass your entire network CIDR:
talos_extra_routes = ["10.0.0.0/8"]
# Optionally, disable NAT for your globally routed CIDR
network_native_routing_ipv4_cidr = "10.0.0.0/8"
# Optionally, use an existing Network
hcloud_network_id = 123456789
This setup ensures that the Talos nodes can route traffic appropriately across your internal network.
Optionally, a hostname can be configured to direct access to the Kubernetes API through a node IP, load balancer, or Virtual IP (VIP):
kube_api_hostname = "kube-api.example.com"
For accessing the Kubernetes API from the public internet, choose one of the following options based on your needs:
kube_api_load_balancer_enabled = true
control_plane_public_vip_ipv4_enabled = true
# Optionally, specify an existing Floating IP
control_plane_public_vip_ipv4_id = 123456789
When accessing the Kubernetes API via an internal network, an internal Virtual IP (Alias IP) is utilized by default to route API requests within the network. This feature can be disabled with the following configuration:
control_plane_private_vip_ipv4_enabled = false
To enhance internal availability, a load balancer can be used:
kube_api_load_balancer_enabled = true
This setup ensures secure and flexible access to the Kubernetes API, accommodating different networking environments.
Example kubernetes.tf snippet:
# Configuration for cluster autoscaler node pools
cluster_autoscaler_enabled = true
cluster_autoscaler_nodepools = [
{
name = "autoscaler"
type = "cpx22"
location = "nbg1"
min = 0
max = 6
labels = { "autoscaler-node" = "true" }
taints = [ "autoscaler-node=true:NoExecute" ]
}
]
Optionally, pass additional Helm values to the cluster autoscaler configuration:
cluster_autoscaler_helm_values = {
extraArgs = {
enforce-node-group-min-size = true
scale-down-delay-after-add = "45m"
scale-down-delay-after-delete = "4m"
scale-down-unneeded-time = "5m"
}
}
Cluster Autoscaler does not support upgrading nodes or changing their configuration, as its primary purpose is to manage short-lived nodes that handle load peaks. If you require long-lived autoscaled nodes, you can upgrade them manually using talosctl or use this Terraform module, which supports discovery of autoscaled nodes and manages their upgrades and configuration changes.
To enable this feature, add the following to your configuration:
cluster_autoscaler_discovery_enabled = true
Please note that errors may occur if a node pool has been scaled down recently, as Talos caches absent nodes for up to 30 minutes. You can pause automatic scaling by stopping the Cluster Autoscaler pods:
kubectl -n kube-system scale deployment cluster-autoscaler-hetzner-cluster-autoscaler --replicas=0
The module installs a curated set of Kubernetes components through Talos manifests and reconciles them during the normal module lifecycle.
| Component | Variable | Default |
|---|---|---|
| Cilium Container Network Interface (CNI) | cilium_enabled | true |
| Hcloud Cloud Controller Manager (CCM) | hcloud_ccm_enabled | true |
| Hcloud Container Storage Interface (CSI) | hcloud_csi_enabled | true |
| Metrics Server | metrics_server_enabled | true |
| Talos Backup | talos_backup_enabled | true |
| Talos Cloud Controller Manager (CCM) | talos_ccm_enabled | true |
| Component | Variable | Default |
|---|---|---|
| Cert Manager | cert_manager_enabled | false |
| Cert Manager Hetzner Webhook | cert_manager_webhook_hetzner_enabled | false |
| Cilium Gateway API | cilium_gateway_api_enabled | false |
| Cluster Autoscaler | cluster_autoscaler_enabled | false |
| Longhorn | longhorn_enabled | false |
| Component | Variable | Default |
|---|---|---|
| Gateway API CRDs | gateway_api_crds_enabled | true |
| Prometheus Operator CRDs | prometheus_operator_crds_enabled | true |
Extra manifests can be added alongside the built-in components:
talos_extra_remote_manifests = [
"https://example.com/extra-remote-manifest.yaml"
]
talos_extra_inline_manifests = [
{
name = "test-manifest"
contents = <<-EOF
---
apiVersion: v1
kind: Secret
metadata:
name: test-secret
data:
secret: dGVzdA==
EOF
}
]
By default, when using K8s NetworkPolicies CIDR-based selectors in Cilium do not match in-cluster entities (pods or nodes). This is however required when using NetworkPolicies that control traffic to the kube-apiserver pods as these use the Talos node IPs in our setup. Cilium's policy engine can be directed to select nodes by CIDR / ipBlock. This requires you to add the following to your configuration:
cilium_policy_cidr_match_mode = "nodes"
It is safe to toggle this option on a running cluster, and toggling the option affects neither upgrades nor downgrades More information can be found in the Cilium docs.
This module enables Cilium Transparent Encryption feature by default.
All pod network traffic is encrypted using WireGuard (Default) or protocols, includes automatic key rotation and efficient in-kernel encryption, covering all traffic types.
💡 Although WireGuard is the default option, Hetzner Cloud VMs supports AES-NI instruction set, making IPSec encryption more CPU-efficient compared to WireGuard. Consider enabling IPSec for CPU savings through hardware acceleration.
IPSec mode supports RFC4106 AES-GCM encryption with 128, 192 and 256 bits key sizes.
⚠️ IPSec encryption has the following limitations:
Source: Cilium Documentation
Example kubernetes.tf configuration:
cilium_encryption_enabled = true # Default true
cilium_encryption_type = "wireguard" # wireguard (Default) | ipsec
cilium_ipsec_algorithm = "rfc4106(gcm(aes))" # IPSec AES key algorithm (Default rfc4106(gcm(aes)))
cilium_ipsec_key_size = 256 # IPSec AES key size (Default 256)
cilium_ipsec_key_id = 1 # IPSec key ID (Default 1)
Keys automatically rotate when cilium_ipsec_key_id is incremented (1-15 range, resets to 1 after 15).
Cilium offers an Egress Gateway to ensure network compatibility with legacy systems and firewalls requiring fixed IPs. The use of Cilium Egress Gateway does not provide high availability and increases latency due to extra network hops and tunneling. Consider this configuration only as a last resort.
Example kubernetes.tf snippet:
# Enable Cilium Egress Gateway
cilium_egress_gateway_enabled = true
# Define worker nodepools including an egress-specific node pool
worker_nodepools = [
# ... (other node pool configurations)
{
name = "egress"
type = "cpx22"
location = "nbg1"
labels = { "egress-node" = "true" }
taints = [ "egress-node=true:NoSchedule" ]
}
]
Example Egress Gateway Policy:
apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
name: sample-egress-policy
spec:
selectors:
- podSelector:
matchLabels:
io.kubernetes.pod.namespace: sample-namespace
app: sample-app
destinationCIDRs:
- "0.0.0.0/0"
egressGateway:
nodeSelector:
matchLabels:
egress-node: "true"
Please visit the Cilium documentation for more details.
Each rule is defined with the following properties:
description: A brief description of the rule.direction: The direction of traffic (in for inbound, out for outbound).source_ips: A list of source IP addresses for inbound rules.destination_ips: A list of destination IP addresses for outbound rules.protocol: The protocol used (valid options: tcp, udp, icmp, gre, esp).port: The port number (required for tcp and udp protocols, must not be specified for icmp, gre, and esp).Example kubernetes.tf snippet:
firewall_extra_rules = [
{
description = "Custom UDP Rule"
direction = "in"
source_ips = ["0.0.0.0/0", "::/0"]
protocol = "udp"
port = "12345"
},
{
description = "Custom TCP Rule"
direction = "in"
source_ips = ["1.2.3.4", "1:2:3:4::"]
protocol = "tcp"
port = "8080-9000"
},
{
description = "Allow ICMP"
direction = "in"
source_ips = ["0.0.0.0/0", "::/0"]
protocol = "icmp"
}
]
For access to Talos and the Kubernetes API, please refer to the Cluster Access configuration section.
Kubernetes Gateway API is the modern replacement for Kubernetes Ingress. It fixes many Ingress limitations by offering a richer, more consistent model for traffic management, and it's designed to support multiple Gateway API implementations in parallel.
This module installs the Gateway API CRDs by default and deploys Cert Manager with Gateway API support enabled.
To use Cilium's Gateway API implementation, configure:
cilium_gateway_api_enabled = true
If Cert Manager and Cilium weren't initially deployed/configured with Gateway API support enabled, you may need to restart their controllers to pick up the new configuration:
kubectl -n cert-manager rollout restart deployment
kubectl -n kube-system rollout restart deployment/cilium-operator
Issuer (Let's Encrypt HTTP-01 via Gateway API)Issuer configures Cert Manager to request TLS certificates from Let's Encrypt using the ACME HTTP-01 challenge. The important part is the gatewayHTTPRoute solver: when Cert Manager needs to prove domain ownership, it will temporarily create/attach an HTTPRoute under the referenced Gateway and serve the /.well-known/acme-challenge/... response there.
Replace the placeholder email address, and note that privateKeySecretRef is the secret where Cert Manager stores your ACME account key (not the issued certificate).
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-http01
namespace: default
spec:
acme:
email: <[email protected]>
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-http01-key
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- name: cilium-gateway
namespace: default
kind: Gateway
ClusterIssuer (Let's Encrypt DNS-01 via Hetzner DNS)To request wildcard certificates or certificates that do not depend on public HTTP routing, enable the Hetzner DNS webhook:
cert_manager_enabled = true
cert_manager_webhook_hetzner_enabled = true
Your DNS zone must be managed by Hetzner DNS. Create a Secret containing a Hetzner API token with permission to create and remove DNS records. For ClusterIssuer resources, place the Secret in the cert-manager cluster resource namespace, which defaults to cert-manager.
apiVersion: v1
kind: Secret
metadata:
name: hetzner
namespace: cert-manager
stringData:
token: <hetzner-dns-token>
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01
spec:
acme:
email: <[email protected]>
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
webhook:
groupName: acme.hetzner.com
solverName: hetzner
config:
tokenSecretKeyRef:
name: hetzner
key: token
When using this ClusterIssuer with the Gateway shim, replace cert-manager.io/issuer: letsencrypt-http01 with cert-manager.io/cluster-issuer: letsencrypt-dns01.
Gateway (Cilium GatewayClass + Hetzner LB + Cert Manager integration)A Gateway defines the external entry point for traffic. With gatewayClassName: cilium, the resource is reconciled by the Cilium Gateway controller. The infrastructure.annotations are passed through as Hetzner-specific load balancer settings (interpreted by the Hetzner Cloud Controller Manager) to control how the Hetzner Cloud LB is created and configured. See: Load Balancer Annotations
The cert-manager.io/issuer: letsencrypt-http01 annotation is used by Cert Manager's Gateway shim so it knows which Issuer to use when populating the TLS secret referenced by the listener.
tls.certificateRefs points at the Kubernetes Secret that will contain the issued certificate and private key (example-com-tls). Cert Manager will keep that secret up to date, and the Gateway will use it to terminate TLS.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: cilium-gateway
namespace: default
annotations:
cert-manager.io/issuer: letsencrypt-http01
spec:
gatewayClassName: cilium
infrastructure:
annotations:
load-balancer.hetzner.cloud/name: "cilium-gateway-nbg1"
load-balancer.hetzner.cloud/location: "nbg1"
load-balancer.hetzner.cloud/uses-proxyprotocol: "true"
listeners:
- name: https
hostname: example.com
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: All
tls:
mode: Terminate
certificateRefs:
- name: example-com-tls
kind: Secret
group: ""
- name: http
hostname: example.com
port: 80
protocol: HTTP
allowedRoutes:
namespaces:
from: All
HTTPRoute (bind hostname + route traffic to your service)Finally, the HTTPRoute attaches to the Gateway and defines routing rules for example.com. In this example it forwards all matching requests to a Kubernetes Service called example-service on port 8080.
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: example-app
namespace: default
spec:
hostnames:
- example.com
parentRefs:
- name: cilium-gateway
namespace: default
rules:
- backendRefs:
- name: example-service
port: 8080
The Hetzner Cloud Container Storage Interface (CSI) driver can be flexibly configured through the hcloud_csi_storage_classes variable. You can define multiple storage classes for your cluster:
false).Delete or Retain, optional, defaults to Delete).Example:
hcloud_csi_storage_classes = [
{
name = "hcloud-volumes"
encrypted = false
defaultStorageClass = true
},
{
name = "hcloud-volumes-encrypted-xfs"
encrypted = true
reclaimPolicy = "Retain"
extraParameters = {
"csi.storage.k8s.io/fstype" = "xfs"
"fsFormatOption" = "-i nrext64=1"
}
}
]
Other settings:
hcloud_csi_encryption_passphrase: Optionally provide a custom encryption passphrase for LUKS-encrypted storage classes.
hcloud_csi_encryption_passphrase = "<secret-passphrase>"
Storage Class Immutability:
StorageClasses created by the Hcloud CSI driver are immutable. To change parameters after creation, you must either edit the StorageClass directly with kubectl, or delete it from both Terraform state and Kubernetes, then let this module recreate it.
For more details, see the HCloud CSI Driver documentation.
⚠️ Experimental: IPv6 Pod networking is opt-in. Hetzner Private Networks currently support only IPv4. When you enable IPv6 Pod networking with the CIDRs below and leave the routing settings unchanged, Cilium carries IPv6 Pod traffic between nodes through its IPv4 VXLAN overlay, and the module disables the Hetzner Cloud CCM Route Controller. For Internet egress, Cilium's IPv6 eBPF masquerading translates each Pod's IPv6 source address to its node's public IPv6 address. This NAT feature is beta, so dual-stack Pod networking is not supported for production use by this module.
To try it on a new cluster, provide separate IPv6 Pod and Service CIDRs:
network_pod_ipv6_cidr = "fd42:6:175::/56"
network_service_ipv6_cidr = "fd42:6:176::/112"
Choose unique IPv6 CIDRs for your cluster. Kubernetes allocates a /64 Pod CIDR per node from the Pod range. Cilium carries both IPv4 and IPv6 Pod traffic over its IPv4 VXLAN tunnel. Hetzner Cloud CCM still manages nodes and load balancers. IPv6 Internet egress uses each node's public IPv6 address, so keep talos_public_ipv6_enabled = true.
⚠️ Existing clusters: Enabling IPv6 Pod networking on an existing IPv4-only cluster requires additional migration steps and may disrupt Pod networking. Only do this if you know exactly what you are doing.
Longhorn is a lightweight, reliable, and easy-to-use distributed block storage system for Kubernetes. It is fully independent from the Hetzner Cloud CSI driver.
You can enable Longhorn and configure it as the default StorageClass for your cluster via module variables:
Enable Longhorn:
Set longhorn_enabled to true to deploy Longhorn in your cluster.
Default StorageClass:
Set longhorn_default_storage_class to true if you want Longhorn to be the default StorageClass.
Example:
longhorn_enabled = true
longhorn_default_storage_class = true
Migrating existing Longhorn disks from /var/lib/longhorn to /var/mnt/longhorn:
Starting with module version 6, this module follows the Longhorn guidance for Talos Linux v1.10 and later, which recommends UserVolumeConfig and /var/mnt/longhorn instead of the legacy /var/lib/longhorn data path. Existing disks are not migrated automatically, so version 6 keeps the legacy path available for a rolling migration. Clusters created with an earlier module version must complete this migration before upgrading to module version 7 or higher.
/var/mnt/longhorn and uses it for new Longhorn disks while keeping existing /var/lib/longhorn disks available./var/lib/longhorn path. Note any custom storage reservation or tags that must be retained.true, and save.0 replicas and no backing images. Do not remove the disk before eviction finishes./var/lib/longhorn disk, and save./var/mnt/longhorn path, restore any custom storage reservation or tags, enable scheduling, and save.Do not enable scheduling on both paths of the same node at once. They share the node's EPHEMERAL filesystem, so Longhorn would count the same capacity twice. Complete the migration before upgrading to module version 7 or higher, which will remove the legacy /var/lib/longhorn mount.
For more information about Longhorn, see the Longhorn documentation.
By default, this module calculates optimal subnets based on the provided network CIDR (network_ipv4_cidr). The network is segmented automatically as follows:
network_node_ipv4_cidr)network_service_ipv4_cidr)network_pod_ipv4_cidr)Each Kubernetes node requires a /24 subnet within network_pod_ipv4_cidr. To support this configuration, the optimal node subnet size (network_node_ipv4_subnet_mask_size) is calculated using the formula:
32 - (24 - subnet_mask_size(network_pod_ipv4_cidr)).
With the default 10.0.0.0/16 network CIDR (network_ipv4_cidr), the following values are calculated:
/25 (Max. 128 Nodes per Subnet)10.0.64.0/19 (Max. 64 Subnets, each with /25)10.0.96.0/19 (Max. 8192 Services)/24 (Max. 256 Pods per Node)10.0.128.0/17 (Max. 128 Nodes, each with /24)The shared worker subnet and the shared Cluster Autoscaler subnet are created by default. Worker and Cluster Autoscaler node pools use their shared subnets unless they pin a legacy node subnet with subnet = "10.0.65.0/25". Treat pinned subnet values as immutable after the node pool has been created.
Please consider the following Hetzner Cloud limits:
A /16 Network CIDR is sufficient to fully utilize Hetzner Cloud's scaling capabilities. It supports:
/24 Pod subnet route.Here is a table with more example calculations:
| Network CIDR | Node Subnet Size | Node Subnets | Service IPs | Pod Subnets |
|---|---|---|---|---|
| 10.0.0.0/16 | /25 (128 IPs) | 10.0.64.0/19 (64) | 10.0.96.0/19 (8192) | 10.0.128.0/17 (128) |
| 10.0.0.0/17 | /26 (64 IPs) | 10.0.32.0/20 (64) | 10.0.48.0/20 (4096) | 10.0.64.0/18 (64) |
| 10.0.0.0/18 | /27 (32 IPs) | 10.0.16.0/21 (64) | 10.0.24.0/21 (2048) | 10.0.32.0/19 (32) |
| 10.0.0.0/19 | /28 (16 IPs) | 10.0.8.0/22 (64) | 10.0.12.0/22 (1024) | 10.0.16.0/20 (16) |
| 10.0.0.0/20 | /29 (8 IPs) | 10.0.4.0/23 (64) | 10.0.6.0/23 (512) | 10.0.8.0/21 (8) |
| 10.0.0.0/21 | /30 (4 IPs) | 10.0.2.0/24 (64) | 10.0.3.0/24 (256) | 10.0.4.0/22 (4) |
This module natively supports Hcloud Object Storage. Below is an example of how to configure backups with MinIO Client (mc) and Hcloud Object Storage. While it's possible to create the bucket through the Hcloud Console, this method does not allow for the configuration of automatic retention policies.
Create an alias for the endpoint using the following command:
mc alias set <alias> \
https://<location>.your-objectstorage.com \
<access-key> <secret-key> \
--api "s3v4" \
--path "off"
Create a bucket with automatic retention policies to protect your backups:
mc mb --with-lock --region <location> <alias>/<bucket>
mc retention set GOVERNANCE 14d --default <alias>/<bucket>
Configure your kubernetes.tf file:
talos_backup_s3_hcloud_url = "https://<bucket>.<location>.your-objectstorage.com"
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"
# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"
# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"
For users of other object storage providers, configure kubernetes.tf as follows:
talos_backup_s3_region = "<region>"
talos_backup_s3_endpoint = "<endpoint>"
talos_backup_s3_bucket = "<bucket>"
talos_backup_s3_prefix = "<prefix>"
# Use path-style URLs (set true if required by your provider)
talos_backup_s3_path_style = true
# Access credentials
talos_backup_s3_access_key = "<access-key>"
talos_backup_s3_secret_key = "<secret-key>"
# Optional: AGE X25519 Public Key for encryption
talos_backup_age_x25519_public_key = "<age-public-key>"
# Optional: Change schedule (cron syntax)
talos_backup_schedule = "0 * * * *"
To recover from a snapshot, please refer to the Talos Disaster Recovery section in the Documentation.
By default, the module builds missing AMD64 and ARM64 Talos snapshots with Packer and selects images whose labels exactly match the cluster name, Talos version, and schematic ID. Image building and selection can be configured independently for each architecture:
talos_cloud_amd64_image_build_enabled = true
talos_cloud_amd64_image_selector = "exact"
talos_cloud_arm64_image_build_enabled = true
talos_cloud_arm64_image_selector = "exact"
The supported selector values are:
exact: Selects an image matching the cluster name, Talos version, and schematic ID.latest: Selects the newest image matching only the cluster name and architecture.Disabling an image builder is useful when matching snapshots are created outside this module. For example:
talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector = "exact"
If no exactly matching snapshot exists, the lookup fails instead of silently selecting an older image.
To disable the ARM64 builder and explicitly select the newest existing cluster snapshot:
talos_cloud_arm64_image_build_enabled = false
talos_cloud_arm64_image_selector = "latest"
⚠️ Warning: The
latestselector is strongly discouraged and unsupported. It ignores the configured Talos version and schematic ID, so new nodes may boot with incompatible Talos versions, system extensions, or kernel arguments. This can cause bootstrap failures, unavailable nodes, or cluster disruption. It is not an officially supported recovery or emergency mechanism. Use it entirely at your own risk.
Talos uses the public discovery service operated by Sidero Labs to discover cluster members, including when Kubernetes is unavailable. Nodes require outbound access to TCP port 443 to use it.
The service is enabled by default and can be disabled with:
talos_discovery_service_enabled = false
For more details, refer to the official Talos discovery guide.
The module can install and configure the Talos Tailscale system extension on every control plane, worker, Cluster Autoscaler, and bare metal node:
talos_tailscale_enabled = true
talos_tailscale_auth_key = var.tailscale_auth_key
The module leaves containerboot defaults unchanged: Tailscale does not accept tailnet DNS settings and reruns tailscale up on each start. Additional settings can be supplied as KEY=VALUE entries:
talos_tailscale_environment = [
"TS_EXTRA_ARGS=--advertise-tags=tag:kubernetes",
]
TS_AUTH_ONCE=true avoids repeat login when state persists, but later changes to TS_EXTRA_ARGS are not reapplied. Use it only if that trade-off is acceptable.
The module leaves Cilium device selection unchanged. In testing, Cilium auto-detected tailscale0, used an MTU of 1280, and allowed pods to reach other nodes over Tailscale. Restricting Cilium to eth+ or excluding tailscale0 raised the MTU to 1450 but blocked that pod-to-tailnet traffic. If the lower MTU causes bandwidth problems, evaluate cilium_helm_values.devices for your network and test the traffic your workloads need.
If you also use the Tailscale Kubernetes Operator to expose Services or a Service CIDR while Cilium replaces kube-proxy, ensure socket load balancing is bypassed in pod namespaces. Set cilium_socket_lb_host_namespace_only_enabled = true unless Cilium Gateway API already enables this setting. The Talos node extension alone does not require it.
If talos_schematic_id is set, its custom schematic must already contain siderolabs/tailscale, because the module cannot modify a supplied schematic. The auth key is sensitive, but it is necessarily stored in Terraform state and the rendered Talos machine configuration.
This module allows you to create custom Kubernetes RBAC (Role-Based Access Control) roles and cluster roles that define specific permissions for users and groups. RBAC controls what actions users can perform on which Kubernetes resources. These custom roles can be used independently or combined with OIDC group mappings to automatically assign permissions based on user group membership from your identity provider.
rbac_cluster_roles)rbac_cluster_roles = [
{
name = my-cluster-role # ClusterRole name
rules = [
{
api_groups = [""] # Core API group (empty string for core resources)
resources = ["nodes"] # Cluster-wide resources this role can access
verbs = ["get", "list", "watch"] # Actions allowed on these resources
}
]
}
]
rbac_roles)rbac_roles = [
{
name = "my-role" # Role name
namespace = "target-namespace" # Namespace where the role will be created
rules = [
{
api_groups = [""] # Core API group (empty string for core resources)
resources = ["pods", "services"] # Resources this role can access
verbs = ["get", "list", "watch"] # Actions allowed on these resources
}
]
}
]
The Kubernetes API server supports OIDC (OpenID Connect) authentication, allowing integration with one or more external identity providers like Keycloak, Auth0, Authentik, Zitadel, etc. When enabled, users can authenticate using their existing organizational credentials instead of managing separate Kubernetes certificates or tokens.
OIDC authentication works by validating JWT tokens issued by your identity provider, extracting user information and group memberships, and mapping them to Kubernetes RBAC roles.
Each issuer must be unique. Configure multiple clients from the same issuer as audiences of a single provider. The module automatically configures MatchAny when a provider has multiple audiences.
kube_api_oidc_providers = [
{
issuer_url = "https://keycloak.example.com/realms/example"
audiences = ["kubectl", "headlamp"]
username_claim = "preferred_username"
groups_claim = "groups"
groups_prefix = "oidc:"
group_mappings = [
{
group = "cluster-admins-group"
cluster_roles = ["cluster-admin"]
},
{
group = "developers-group"
cluster_roles = ["view"]
roles = [
{
name = "developer-role"
namespace = "development"
}
]
}
]
}
]
Once OIDC is configured in your cluster, you'll need to configure your local kubectl to authenticate using OIDC tokens. This requires the kubelogin plugin.
# Homebrew (macOS and Linux)
brew install kubelogin
# Krew (macOS, Linux, Windows and ARM)
kubectl krew install oidc-login
# Chocolatey (Windows)
choco install kubelogin
First, verify that your OIDC provider is returning proper JWT tokens. Replace the placeholder values with your actual OIDC configuration:
kubectl oidc-login setup \
--oidc-issuer-url=https://your-oidc-provider.com \
--oidc-client-id=your-client-id \
--oidc-client-secret=your-client-secret \
--oidc-extra-scope=openid,email,profile # Add or change the scopes according to your IDP
This will open your browser for authentication. After successful login, you should see a JWT token in your terminal that looks like:
{
"aud": "your-client-id",
"email": "[email protected]",
"email_verified": true,
"exp": 1749867571,
"groups": [
"developers",
"kubernetes-users"
],
"iat": 1749863971,
"iss": "https://your-oidc-provider.com",
"nonce": "random-nonce-string",
"sub": "user-unique-identifier"
}
Verify that:
groups array contains your expected groupsemail field matches your user emailemail_verified is true (required by K8s)Add a new user to your ~/.kube/config file:
users:
- name: oidc-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
command: kubectl
args:
- oidc-login
- get-token
- --oidc-issuer-url=https://your-oidc-provider.com
- --oidc-client-id=your-client-id
- --oidc-client-secret=your-client-secret
- --oidc-extra-scope=groups
- --oidc-extra-scope=email
- --oidc-extra-scope=name # Add or change the scopes according to your IDP
Update your context to use the new OIDC user:
contexts:
- context:
cluster: your-cluster
namespace: default
user: oidc-user # Changed from certificate-based user
name: oidc@your-cluster # Updated context name
Now you can switch to the OIDC context and authenticate using your identity provider:
kubectl config use-context your-cluster-oidc
kubectl get pods # This will trigger OIDC authentication
Talos allows adding additional trusted root CA certificates to the system. This is useful for private image registries, OIDC providers with internal CAs, or other enterprise services.
The talos_certificates variable supports several input formats for maximum flexibility:
talos_certificates = {
"my-custom-ca" = "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----"
}
talos_certificates = {
"enterprise-ca" = [
file("root.crt"),
file("intermediate.crt")
]
}
talos_certificates = {
"legacy-ca" = <<-EOT
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
EOT
}
This module manages the versions of Talos, Kubernetes, and all bundled components together. Each Hcloud K8s major version targets a defined Talos and Kubernetes minor version, while component versions are selected for compatibility with that Kubernetes release.
The table below lists the Talos and Kubernetes versions used by each Hcloud K8s version.
| Hcloud K8s | Talos Linux | Kubernetes |
|---|---|---|
| (7) | (1.15) | 1.36 |
| 6 | 1.14 | 1.35 |
| 5 | 1.13 | 1.34 |
Parenthesized versions are planned targets.
The table below lists the minimum required versions of each component to support the specified Kubernetes release.
| Kubernetes | Hcloud CCM | Hcloud CSI | Longhorn | Cilium | Cert Manager | Hetzner Webhook |
|---|---|---|---|---|---|---|
| 1.36 | ≥ 1.31 | ≥ 2.21 | ≥ 1.12 | ≥ 1.20 | ≥ 1.21 | ≥ 0.8 |
| 1.35 | ≥ 1.30 | ≥ 2.19 | ≥ 1.11 | ≥ 1.19.2 | ≥ 1.19 | ≥ 0.6.4 |
| 1.34 | ≥ 1.27 | ≥ 2.18 | ≥ 1.11 | ≥ 1.19 | ≥ 1.19 | ≥ 0.6.1 |
Any minor or major upgrade to Talos or Kubernetes results in a major version change for this module. Downgrades are generally neither supported nor tested.
Changing software versions manually is not recommended. Component versions are selected and tested for compatibility with the Kubernetes release shown above.
[!IMPORTANT]
- Do not combine major or minor module upgrades with infrastructure changes.
- Upgrade to the latest minor release before upgrading to the next major version.
- Verify that the cluster is healthy before and after upgrades.
If you'd like to support this project, please consider leaving a ⭐ on GitHub!
Your sponsorship supports the ongoing development, improvement, and maintenance of this project 🙏
Become a Sponsor:
We welcome everyone to join the Discussions, report Issues, and help improve this project.
Contributions are always welcome!
Hcloud Kubernetes is an independent, open source project built for the Hetzner Community. Its goal is to make Kubernetes deployments as predictable, repeatable, and flexible as possible.
This project is supported by Hetzner through testing infrastructure ❤️
Distributed under the MIT License. See LICENSE for more information.
342 followers · starred Sep 2025
188 followers · starred Aug 2025
608 followers · starred May 2026
860 followers · starred Sep 2025