Build a Kubernetes cluster using K3s via Ansible

Author: https://github.com/itwars
Current Maintainer: https://github.com/dereknola

Easily bring up a cluster on machines running:

• Debian
• Ubuntu
• Raspberry Pi OS
• RHEL Family (CentOS, Redhat, Rocky Linux, Oracle Linux...)
• SUSE Family (SLES, OpenSUSE Leap, Tumbleweed...)
• ArchLinux

on processor architectures:

• x64
• arm64
• armhf

System requirements

The control node must have Ansible 8.0+ (ansible-core 2.15+) and the Python netaddr library (pip install netaddr), required by the ansible.utils.ipwrap filter for IPv6 api_endpoint support.

All managed nodes in inventory must have:

• Passwordless SSH access
• Root access (or a user with equivalent permissions)

It is also recommended that all managed nodes disable firewalls and swap. See K3s Requirements for more information.

Installation

With ansible-galaxy

k3s-ansible is an Ansible collection and can be installed with the ansible-galaxy command:

$ ansible-galaxy collection install git+https://github.com/k3s-io/k3s-ansible.git

Alternatively, add it to the requirements.yaml file of your Ansible project as follows:

collections:
  - name: https://github.com/k3s-io/k3s-ansible.git
    type: git
    version: <comit-ish>

As a comit-ish, you can use a branch, a version tag, or a specific commit.

From source

Alternatively to an installation with ansible-galaxy, the k3s-ansible repository can simply be cloned from github:

$ git clone https://github.com/k3s-io/k3s-ansible.git
$ cd k3s-ansible

Usage

First copy the sample inventory to inventory.yml.

cp inventory-sample.yml inventory.yml

If you have installed k3s-ansible with ansible-galaxy, you can grab the inventory-sample.yml from github.

Second edit the inventory file to match your cluster setup. For example:

k3s_cluster:
  children:
    server:
      hosts:
        192.16.35.11:
    agent:
      hosts:
        192.16.35.12:
        192.16.35.13:

If needed, you can also edit vars section at the bottom to match your environment.

If multiple hosts are in the server group the playbook will automatically setup k3s in HA mode with embedded etcd. An odd number of server nodes is required (3,5,7). Read the official documentation for more information.

Setting up a loadbalancer or VIP beforehand to use as the API endpoint is possible but not covered here.

Start provisioning of the cluster using one of the following commands. The command to be used depends on whether you installed k3s-ansible with ansible-galaxy or if you run the playbook from within the cloned git repository:

Installed with ansible-galaxy

ansible-playbook k3s.orchestration.site -i inventory.yml

Running the playbook from inside the repository

ansible-playbook playbooks/site.yml -i inventory.yml

Alternatively, to run the playbook from your existing project setup, run the playbook from within your own playbook:

Installed with ansible-galaxy

- name: Import kube cluster playbook
  ansible.builtin.import_playbook: k3s.orchestration.site

Running the playbook from inside the repository

- name: Import kube cluster playbook
  ansible.builtin.import_playbook: k3s-ansible/playbooks/site.yml

Using an external database

If an external database is preferred, this can be achieved by passing the --datastore-endpoint as an extra server argument as well as setting the use_external_database flag to true.

k3s_cluster:
  children:
    server:
      hosts:
        192.16.35.11:
        192.16.35.12:
    agent:
      hosts:
        192.16.35.13:

  vars:
    use_external_database: true
    extra_server_args: "--datastore-endpoint=postgres://username:password@hostname:port/database-name"

The use_external_database flag is required when more than one server is defined, as otherwise an embedded etcd cluster will be created instead.

The format of the datastore-endpoint parameter is dependent upon the datastore backend, please visit the K3s datastore endpoint format for details on the format and supported datastores.

Server config file permissions

The server role writes /etc/rancher/k3s/config.yaml at mode 0644. That file can contain the cluster token, so any local account on a server node can read the credential that joins a node to the cluster. Set k3s_server_config_mode to narrow it:

k3s_server_config_mode: "0640"

K3s reads the file as root, so 0600 is the safe floor: no other account needs the file. The default stays 0644, which is the mode the file already has on a running cluster, so a re-run changes nothing until the variable is set. The upgrade role rewrites the same file and honors the same variable.

Upgrading

A playbook is provided to upgrade K3s on all nodes in the cluster. To use it, update k3s_version with the desired version in inventory.yml and run one of the following commands. Again, the syntax is slightly different depending on whether you installed k3s-ansible with ansible-galaxy or if you run the playbook from within the cloned git repository:

Installed with ansible-galaxy

ansible-playbook k3s.orchestration.upgrade -i inventory.yml

Running the playbook from inside the repository

ansible-playbook playbooks/upgrade.yml -i inventory.yml

Re-running the site.yml playbook after bumping k3s_version performs the same upgrade declaratively: it restarts the k3s services so the cluster picks up the new runtime. On a multi-server (HA) cluster, add -e k3s_server_serial=1 so the servers restart one at a time and etcd keeps quorum. With embedded etcd, the next server restarts only after the restarted ones report as etcd voters. By default all servers restart together. Set it through extra vars or import_playbook vars, not inventory: serial is a play keyword, so inventory variables don't apply to it.

ansible-playbook playbooks/site.yml -i inventory.yml -e k3s_server_serial=1

The dedicated upgrade.yml playbook remains available and unchanged.

Airgap Install

Airgap installation is supported via the airgap_dir variable. This variable should be set to the path of a directory containing the K3s binary and images. The release artifacts can be downloaded from the K3s Releases. You must download the appropriate images for you architecture (any of the compression formats will work). Additionally, you must run the airgap role to set up the airgapped environment.

An example folder for an x86_64 cluster:

$ ls ./playbooks/my-airgap/
total 248M
-rwxr-xr-x 1 $USER $USER  58M Nov 14 11:28 k3s
-rw-r--r-- 1 $USER $USER 190M Nov 14 11:30 k3s-airgap-images-amd64.tar.gz

$ cat inventory.yml
...
airgap_dir: ./my-airgap # Paths are relative to the playbooks directory

Additionally, if deploying on an OS with SELinux, you will also need to download the latest k3s-selinux RPM and its dependencies selinux-policy and container-selinux RPMs and place them in the airgap folder.

It is assumed that the control node has access to the internet. The playbook will automatically download the k3s install script on the control node, and then distribute all three artifacts to the managed nodes.

Kubeconfig

After successful bringup, the kubeconfig of the cluster is copied to the control node and merged with ~/.kube/config under the k3s-ansible context unless overwritten by the cluster_context variable. Assuming you have kubectl installed, you can confirm access to your Kubernetes cluster with the following:

kubectl config use-context k3s-ansible
kubectl get nodes

If you wish for your kubeconfig to be copied elsewhere and not merged, you can set the kubeconfig variable in inventory.yml to the desired path.

If you wish to get a new copy of the kubeconfig after installation you can run the site playbook again using the kubeconfig tag.

Installed with ansible-galaxy

ansible-playbook k3s.orchestration.site -i inventory.yml --tags kubeconfig

Running the playbook from inside the repository

ansible-playbook playbooks/site.yml -i inventory.yml --tags kubeconfig

Local Testing

A Vagrantfile is provided that provision a 5 nodes cluster using Vagrant (LibVirt or Virtualbox as provider). To use it:

vagrant up

By default, each node is given 2 cores and 2GB of RAM and runs Ubuntu 24.04. You can customize these settings by editing the Vagrantfile.

Need More Features?

This project is intended to provide a "vanilla" K3s install. If you need more features, such as:

• Advanced Storage (Longhorn, Ceph, etc)
• External Database Management
• External Load Balancer or VIP
• Alternative CNIs

See these other projects:

• https://github.com/PyratLabs/ansible-role-k3s
• https://github.com/techno-tim/k3s-ansible
• https://github.com/jon-stumpf/k3s-ansible
• https://github.com/alexellis/k3sup
• https://github.com/axivo/k3s-cluster