CertMate - Certificate Lifecycle Management

---

Ecosystem

CertMate is the open-source core of a small, focused toolset:

• certmate-tools — free, privacy-first, client-side TLS / certificate / ACME diagnostics (runs entirely in your browser).
• certmate-agent — conversational assistant over the docs.
• nis2-public — NIS2 continuous posture management & remediation.

Enterprise / high-scale — multi-tenant, mTLS, white-label and NIS2-aligned deployments are available through CertMate-ng (source-available, BSL 1.1, EU-built). For access or a deployment discussion, email [email protected].

---

Command-line interface

The whole certificate lifecycle from your terminal — pip install certmate-cli:

pip install certmate-cli

export CERTMATE_URL=https://certmate.example.com
export CERTMATE_TOKEN=...                 # omit on a fresh (setup-mode) instance

certmate health
certmate cert create app.example.com --dns cloudflare --wait   # issue, block until live
certmate cert ls
certmate cert info app.example.com
certmate audit verify

certmate-cli is a thin layer over certmate-sdk (pip install certmate-sdk) — a small, httpx-based Python client for the same REST API the web UI and MCP server drive. Both are first-party, live in clients/, and are published to PyPI. The clip above is a real issuance over DNS-01 (Let's Encrypt staging); see demo/.

---

AI agents (MCP server)

CertMate ships a first-party Model Context Protocol server, so an assistant like Claude can drive the same REST API the web UI and the CLI use — with the same auth and the same audit trail. It lives in mcp/, is Node.js (>= 20), and exposes 16 tools: inventory and status (certmate_list_certificates, certmate_get_certificate, certmate_diagnostics, certmate_get_activity, …), lifecycle operations (certmate_create_certificate, certmate_renew_certificate, certmate_get_job, certmate_set_auto_renew, …), and delivery (certmate_download_certificate, certmate_deploy_certificate).

cd mcp && npm install

// claude_desktop_config.json — or any MCP-capable client
{
  "mcpServers": {
    "certmate": {
      "command": "node",
      "args": ["/path/to/certmate/mcp/index.js"],
      "env": {
        "CERTMATE_URL": "https://certmate.example.com",
        "CERTMATE_TOKEN": "<a scoped API key with is_agent: true>"
      }
    }
  }
}

Give it a scoped key, not your admin token. A key created with is_agent: true (a checkbox under Settings → API Keys, or is_agent in POST /api/keys) makes every action the agent takes land in the audit chain as actor.kind="agent" rather than being indistinguishable from a human operator — which is the difference between an audit trail and a rumour. Scope the key to the domains the agent is allowed to touch.

Full tool reference, attribution model and safety notes: docs/mcp.md.

---

Why CertMate?

CertMate solves the complexity of SSL certificate management in modern distributed architectures. Whether you're running a single application or managing certificates across multiple datacenters, CertMate provides:

• Zero-Downtime Automation - Certificates renew automatically (30 days before expiry by default; earlier in the life of a certificate that lives under 60 days — see when a certificate renews), with deploy hooks to reload services
• Multi-Cloud Support - Cloudflare, AWS, Azure, GCP, Akamai Edge DNS, Hetzner, Porkbun, GoDaddy and the rest — docs/dns-providers.md has the full list
• Enterprise-Ready - RBAC, scoped API keys, Docker, Kubernetes, REST API, and monitoring built-in
• Simple Integration - One-URL certificate downloads for easy automation
• Security-First - Role-based access control, scoped API keys, audit logging, HMAC-signed webhooks
• Unified Backup System - Atomic backups of settings and certificates ensuring data consistency
• Real-Time Dashboard - SSE-powered live updates, command palette, keyboard shortcuts, dark mode

CertMate runs as a single instance. Its renewal scheduler runs inside the web process, so a second replica is a second scheduler issuing against the same certificate store — duplicate ACME orders, and the CA's duplicate-certificate rate limit. The Helm chart refuses to render more than one replica rather than let that happen quietly.

This is a deliberate design choice, not a missing feature: one writer is what makes certificate state safe without a distributed lock service. It scales up (a single instance manages thousands of certificates across many datacenters), not out. For availability, run active/standby with the data volume on shared storage and fail over — see Availability and failover.

Key Features

Certificate Management

• Multiple CA Providers - Support for Let's Encrypt, ZeroSSL, Google Trust Services, Actalis, DigiCert ACME, Sectigo, SSL.com, and Private CAs
• Let's Encrypt Integration - Free, automated SSL certificates, with the staging environment available as a dedicated CA entry for testing
• DigiCert ACME Support - Enterprise-grade certificates with External Account Binding (EAB)
• Actalis Support - Free 90-day DV certificates from a European CA via ACME with EAB
• Private CA Support - Internal/corporate CAs with custom trust bundles and ACME compatibility
• Wildcard Support - Single certificate for *.example.com and example.com
• Multi-Domain Certificates - SAN certificates for multiple domains
• DNS Alias via CNAME Delegation - Delegate ACME DNS validation to an alternative domain using standard CNAME records
• Automatic Renewal - One rule for every lifetime, and the CA's renewal window (ARI) can move it; each certificate says when it renews (renews_at)
• Certificate Validation - Real-time SSL certificate status checking
• Per-Certificate CA Selection - Choose different CAs for different certificates
• Zombie Certificate Scanner - multi-threaded probe that asks whether the names in your managed certificates still resolve and answer, so a domain you have let go is visible before its renewals start failing

Multi-DNS Provider Support

• Multi-Account Support - Manage multiple accounts per provider for enterprise environments
• Cloudflare - Global CDN with edge locations worldwide (Multi-Account)
• AWS Route53 - Amazon's scalable DNS service (Multi-Account)
• Azure DNS - Microsoft's cloud DNS solution (Multi-Account)
• Google Cloud DNS - Google's high-performance DNS (Multi-Account)
• DigitalOcean - Cloud infrastructure DNS (Multi-Account)
• PowerDNS - Open-source DNS server with REST API (Multi-Account)

Enterprise Features

• Role-Based Access Control - Three-tier RBAC with viewer, operator, and admin roles
• Scoped API Keys - Create, revoke, and manage API keys with per-key role and optional expiration
• Multi-Account Management - Support multiple accounts per DNS provider for enterprise workflows
• REST API - Complete programmatic control with Swagger/OpenAPI docs
• Web Dashboard - Modern, responsive UI built with Tailwind CSS and Alpine.js
• Setup Wizard - Guided first-run configuration for DNS, CA, and authentication
• Real-Time Updates - Server-Sent Events (SSE) push live status to the dashboard
• Docker Ready - Full containerization with Docker Compose
• Kubernetes Compatible - Deploy in any Kubernetes cluster
• Monitoring Integration - Health checks, Prometheus metrics, and structured JSON logging

Backup and Recovery

• Unified Backups - Atomic snapshots of both settings and certificates ensuring data consistency
• Automatic Backups - Settings and certificates backed up automatically on changes
• Manual Backup Creation - On-demand backup creation via web UI or API
• Comprehensive Coverage - Backs up DNS configurations, certificates, and application settings
• Retention Management - 50 most recent archives per type, and nothing older than 30 days — both constants; keep disaster-recovery archives off the host
• Easy Restore - Simple restore process from any backup point with atomic consistency
• Download Support - Export backups for external storage and disaster recovery

Certificate Storage Backends

• Local Filesystem - Default secure local storage with proper file permissions (600/700)
• Azure Key Vault - Enterprise-grade secret management with Azure integration and HSM protection
• AWS Secrets Manager - Scalable secret storage with AWS ecosystem integration and cross-region replication
• HashiCorp Vault - Industry-standard secret management with versioning, audit logging, and fine-grained policies
• Infisical - Modern open-source secret management with team collaboration and end-to-end encryption
• S3-Compatible Object Storage - One backend for any S3 endpoint via a configurable endpoint URL (Hetzner, Contabo, OVHcloud, Scaleway, Exoscale, Wasabi, MinIO, AWS) — ideal for EU-sovereign object storage; no extra dependency
• Pluggable Architecture - Easy to extend with additional storage backends
• Migration Support - Seamless migration between storage backends without downtime
• Backward Compatibility - Existing installations continue working without changes

Notifications & Automation

• Multi-Channel Notifications - Email (SMTP), Slack, Discord, Google Chat, Telegram, ntfy, Gotify, and generic webhooks
• Webhook HMAC Signatures - SHA-256 signed payloads for secure webhook verification
• Deploy Hooks - Post-issuance shell commands to reload Nginx/Apache or run custom scripts
• Expiry Warnings - certificate_expiring at 14/7/3/1 days (at the renewal threshold when auto-renew is off), and domain_expiring for the domain registration itself at 60/30/14/7/1 days. Each threshold is announced once per expiry date
• Weekly Digest - Scheduled email summary of certificate status and upcoming renewals
• SSE Real-Time Events - Live push updates for certificate operations and deploy hook results

Security & Compliance

• Role-Based Access Control - Viewer, operator, and admin roles with hierarchical permissions
• Scoped API Keys - Create keys with specific roles and optional expiration dates
• Bearer Token Authentication - Secure API access control
• File Permissions - Proper certificate file security (600/700)
• Audit Logging - Complete certificate lifecycle tracking with timeline view
• Environment Variables - Secure credential management
• Rate Limit Handling - Let's Encrypt rate limit awareness
• Log Sanitizer - Automatically redacts sensitive parameters, private keys, and API tokens from application logs

User Interface

• Command Palette - Cmd+K / Ctrl+K quick search and navigation
• Keyboard Shortcuts - Power-user shortcuts for navigation and common actions
• Dark Mode - System-aware dark/light theme toggle
• Mobile-Friendly - Responsive layout with bottom tab bar on small screens
• Activity Timeline - Chronological view of all certificate and system events

Developer Experience

• One-URL Downloads - Simple certificate retrieval for automation (/api/certificates/{domain}/download)
• Individual Component Downloads - Fetch cert, key, chain, or fullchain separately
• Multiple Output Formats - PEM, ZIP, individual files
• SDK Examples - Python, Bash, Ansible, Terraform examples
• Webhook Support - Certificate lifecycle notifications with HMAC verification
• Deploy Hook API - Configure and test post-issuance hooks via REST API
• Backup API - Programmatic backup creation and restoration
• Swagger & ReDoc - Interactive API documentation at /docs/ and /redoc/
• Model Context Protocol (MCP) Server - Built-in Node.js MCP server providing tools for agentic AI assistants to manage certificates and run diagnostics

Supported DNS Providers

CertMate supports a wide range of DNS providers through Let's Encrypt DNS-01 challenge via individual certbot plugins that provide reliable, well-tested DNS challenge support. The complete list is in the table below. Multi-account support is available for major providers, enabling enterprise-grade deployments with separate accounts for production, staging, and disaster recovery.

What the availability column means. Until now it said Stable on all twenty-nine rows, which is not a status — a column with one value cannot be wrong about any particular provider, and four of them were wrong:

Namecheap is the Unavailable one: the only release on PyPI is certbot-dns-namecheap 1.0.0, an alpha targeting Python 2.7-3.8, incompatible with certbot 2.x on Python 3.12 — as NamecheapStrategy's own docstring has said all along. Use ACME-DNS or the custom-script hook for Namecheap domains. Scaleway is alpha (0.0.7, declares Python 2.7 support) and PowerDNS pulls dns-lexicon<=3.5.6, which conflicts with the rest of the extended set — hence its own environment.

Provider Categories

• Enterprise Multi-Account: Cloudflare, AWS Route53, Azure DNS, Google Cloud DNS, DigitalOcean, PowerDNS, RFC2136
• Cloud Providers: AWS Route53, Azure DNS, Google Cloud DNS, DigitalOcean, Linode, Akamai Edge DNS, Vultr, Hetzner
• Enterprise DNS: Cloudflare, DNS Made Easy, NS1, PowerDNS, EfficientIP SOLIDserver
• Domain Registrars: Gandi, OVH, Namecheap, Porkbun, GoDaddy
• European Providers: OVH, Gandi, Hetzner
• Free Services: Hurricane Electric, Dynu
• Standard Protocols: RFC2136 (for BIND and compatible servers)

Multi-Account Benefits

For supported providers, you can configure multiple accounts to enable:

• Environment Separation: Different accounts for production, staging, and development
• Multi-Region Management: Separate accounts for different geographical regions
• Team Isolation: Department-specific accounts with tailored permissions
• Disaster Recovery: Backup accounts for high-availability scenarios
• Permission Scoping: Accounts with minimal required permissions for security

Detailed Setup Instructions: See DNS Providers Guide for provider-specific configuration. Step-by-Step Installation: See Installation Guide for complete setup guide. Multi-Account Examples: See DNS Providers Guide for enterprise configuration examples.

Quick Start with Docker

Get CertMate running in under 5 minutes with Docker Compose. One file, the published image, nothing to clone or build.

Prerequisites

• Docker 20.10+ with Docker Compose v2 (docker compose)
• A domain with DNS managed by a supported provider

1. Download and configure

mkdir certmate && cd certmate
curl -fsSLO https://raw.githubusercontent.com/fabriziosalmi/certmate/main/deploy/docker-compose.yml
printf 'API_BEARER_TOKEN=%s\nSECRET_KEY=%s\nCERTMATE_BACKUP_PASSPHRASE=%s\n' \
  "$(openssl rand -hex 32)" "$(openssl rand -hex 32)" "$(openssl rand -hex 32)" > .env
chmod 600 .env

.env now holds the three secrets CertMate needs: the API token (also asked once by the first-run screen to create the admin), the session signing key, and the backup passphrase. Optionally add CLOUDFLARE_TOKEN=... to bootstrap a Cloudflare DNS account; every other DNS provider is configured in the web UI. What each setting does, and how to upgrade: docs/docker.md.

Storage Backends: By default, certificates are stored locally. For enterprise deployments, you can configure Azure Key Vault, AWS Secrets Manager, HashiCorp Vault, Infisical, or any S3-compatible object storage via the web interface after startup. See Storage Backends for details.

Backup Best Practices: CertMate includes a unified backup system that creates atomic snapshots of both settings and certificates. Set CERTMATE_BACKUP_PASSPHRASE. With it, every automatic backup is complete and encrypted at rest, so it can actually restore this instance; without it, automatic backups keep their credentials masked and are configuration snapshots that cannot. The backup list marks which archives can restore, and Settings disables Restore on the ones that cannot. Then keep a copy off the host — archives on the host are pruned after 30 days, and a lost volume takes them with it. POST /api/backups/upload brings one back.

2. Start

docker compose up -d
docker compose ps
docker compose logs -f certmate

CertMate listens on 127.0.0.1:8000 only. For remote access, put a reverse proxy in front and set BEHIND_PROXY=true in .env.

To build from source instead, clone the repository and use its root docker-compose.yml (development).

3. Access CertMate

4. Create Your First Certificate

Using the Web Interface:

1. Navigate to http://localhost:8000
2. Go to Settings and configure your DNS provider
3. Add your domain (e.g. example.com)
4. Click "Create Certificate"

Using the API:

curl -X POST "http://localhost:8000/api/certificates/create" \
 -H "Authorization: Bearer your_api_token_here" \
 -H "Content-Type: application/json" \
 -d '{"domain": "example.com"}'

Installation Methods

Choose the installation method that best fits your environment:

Docker (Recommended)

Isolated, reproducible, and the way CertMate is tested and released. Run one container (see the single-instance note above); give it more CPU and memory rather than more replicas. Published images cover two architectures: AMD64 (Intel/AMD) and ARM64 (Apple Silicon, ARM servers). ARM v7 (32-bit Raspberry Pi) is not published; build it yourself with docker buildx build --platform linux/arm/v7 -t YOUR_DOCKERHUB_USERNAME/certmate:armv7 --push . (see Docker).

The production Compose bundle is one file that runs the published image; the Quick Start above shows the commands. To build the image yourself instead, clone the repository and run docker compose up -d with its root docker-compose.yml, after preparing its bind-mounted directories as its comments describe.

Multi-Platform Support:

# Build for multiple architectures (ARM64 + AMD64); add --push to publish the result
docker buildx build --platform linux/amd64,linux/arm64 -t YOUR_DOCKERHUB_USERNAME/certmate:latest .

# Build and push to Docker Hub for all platforms
docker buildx build --platform linux/amd64,linux/arm64 -t YOUR_DOCKERHUB_USERNAME/certmate:latest --push .

# Use pre-built multi-platform image (bound to localhost; put it behind a
# reverse proxy and enable authentication before exposing it externally)
docker run --platform linux/arm64 -d --name certmate --env-file .env -p 127.0.0.1:8000:8000 fabriziosalmi/certmate:latest

Multi-Platform Guide: See Docker Guide for comprehensive multi-architecture setup instructions.

Python Virtual Environment

Ideal for development and testing environments.

# Create and activate virtual environment
python3 -m venv certmate-env
source certmate-env/bin/activate # On Windows: certmate-env\Scripts\activate

# Install dependencies
git clone https://github.com/fabriziosalmi/certmate.git
cd certmate
pip install -r requirements.txt

# Set environment variables
export API_BEARER_TOKEN="your_token_here"
export CLOUDFLARE_TOKEN="your_cloudflare_token"

# Run the application
python app.py

Kubernetes

For container orchestration and managed rollouts. replicas: 1 is not an example value — it is the supported configuration, and the Helm chart fails at template time if you change it.

Install the Helm chart, published to GHCR on every release:

kubectl create namespace certmate
kubectl -n certmate create secret generic certmate-secrets \
  --from-literal=API_BEARER_TOKEN="$(openssl rand -hex 32)" \
  --from-literal=SECRET_KEY="$(openssl rand -hex 32)" \
  --from-literal=CERTMATE_BACKUP_PASSPHRASE="$(openssl rand -hex 32)"
helm install certmate oci://ghcr.io/fabriziosalmi/charts/certmate \
  --namespace certmate --set secrets.existingSecret=certmate-secrets

CertMate on Kubernetes covers the first login, Argo CD and Flux, production sizing and OOM troubleshooting. Every chart value is in the chart README.

Linux server with systemd

One command on Debian, Ubuntu, Fedora, RHEL, Rocky or Alma Linux:

curl -fsSL https://raw.githubusercontent.com/fabriziosalmi/certmate/main/deploy/install.sh | sudo sh

The installer puts CertMate in /opt/certmate with its own Python 3.12 (fetched by uv, so the system Python is not touched), generates the secrets into /etc/certmate/certmate.env, and starts the certmate systemd service on 127.0.0.1:8000. Run it again to upgrade: certificates, data and configuration stay. Details and options: docs/installation.md.

To set the service up by hand instead, follow the next section.

Service Setup

For production deployments, CertMate should run as a system service. This section provides comprehensive instructions for setting up CertMate with systemd on Linux distributions.

Prerequisites

• Linux system with systemd
• Python 3.12
• Root/sudo access

1. Create Dedicated System User

Create a dedicated user for running CertMate:

# Create system user and group
sudo useradd --system --shell /bin/false --home-dir /opt/certmate --create-home certmate

# Set proper ownership
sudo chown -R certmate:certmate /opt/certmate

2. Prepare Application Directory

Set up the application in /opt/certmate:

# Clone straight into the path. `useradd --create-home` above populated
# /opt/certmate from /etc/skel, so it is not empty and `mv certmate /opt/`
# fails with "Directory not empty" — leaving the next steps to run in a
# directory with no application in it.
sudo -u certmate git clone https://github.com/fabriziosalmi/certmate.git /opt/certmate
cd /opt/certmate

# Create Python virtual environment
sudo -u certmate python3 -m venv venv
sudo -u certmate ./venv/bin/pip install -r requirements.txt

# Create necessary directories. All four: the startup writeability probe
# in modules/factory.py checks certificates, data, backups AND logs,
# and raises at boot if any of them is not writable.
sudo -u certmate mkdir -p certificates data backups logs

3. Configure Environment Variables

Create environment file for the service:

# Create environment file
# The unit reads /etc/certmate/certmate.env (see certmate.service), not a
# .env in the application directory.
sudo install -d -m 750 /etc/certmate
sudo tee /etc/certmate/certmate.env > /dev/null <<EOF
# SECURITY: Change this token!
API_BEARER_TOKEN=your_super_secure_api_token_here_change_this
EOF

# PORT has no effect here: the port is the literal in the unit's ExecStart
# (--bind 0.0.0.0:8000), and nothing in the application reads PORT under
# gunicorn. To change it, edit ExecStart. PORT is honoured only by the
# container image.

# Set proper permissions
sudo chown root:certmate /etc/certmate/certmate.env
sudo chmod 640 /etc/certmate/certmate.env

4. Install systemd Service

Install and configure the systemd service:

# Copy service file
sudo cp /opt/certmate/certmate.service /etc/systemd/system/

# Reload systemd configuration
sudo systemctl daemon-reload

# Enable service to start on boot
sudo systemctl enable certmate

# Start the service
sudo systemctl start certmate

5. Verify Service Status

Check that the service is running correctly:

# Check service status
sudo systemctl status certmate

# View recent logs
sudo journalctl -u certmate --lines=50

# Follow logs in real-time
sudo journalctl -u certmate -f

6. Service Management Commands

Common commands for managing the CertMate service:

# Start service
sudo systemctl start certmate

# Stop service
sudo systemctl stop certmate

# Restart service
sudo systemctl restart certmate

# Reload service configuration
sudo systemctl reload certmate

# Check if service is enabled
sudo systemctl is-enabled certmate

# Check if service is active
sudo systemctl is-active certmate

# Disable service from starting on boot
sudo systemctl disable certmate

7. File Permissions

Ensure proper file permissions for security:

# Set ownership
sudo chown -R certmate:certmate /opt/certmate

# Set directory permissions
sudo chmod 755 /opt/certmate
sudo chmod 750 /opt/certmate/certificates /opt/certmate/data /opt/certmate/backups /opt/certmate/logs

# Set file permissions
sudo chmod 644 /opt/certmate/*.py /opt/certmate/*.md
sudo chmod 640 /etc/certmate/certmate.env
sudo chmod 755 /opt/certmate/venv/bin/*

Security Notes

• API Bearer Token: Always change the default API bearer token in /etc/certmate/certmate.env
• File Permissions: The service runs with restricted permissions and limited filesystem access
• Network Access: The service binds to 0.0.0.0:8000 by default - consider using a reverse proxy for production
• Environment File: The .env file contains sensitive data and should be readable only by the certmate user
• Certificates: Generated certificates are stored in /opt/certmate/certificates with restricted access

Troubleshooting Service Setup

If the service fails to start:

1. Check service status: sudo systemctl status certmate
2. View logs: sudo journalctl -u certmate --lines=100
3. Verify permissions: Ensure the certmate user can read all necessary files
4. Test manually: sudo -u certmate /opt/certmate/venv/bin/python /opt/certmate/app.py
5. Check dependencies: sudo -u certmate /opt/certmate/venv/bin/certbot --version — the ACME client is the dependency that breaks first when the pins drift.

For more detailed installation instructions, see the Installation Guide.

Single Sign-On (OIDC/SSO)

CertMate supports authenticating users against an external OpenID Connect provider (Keycloak, Authentik, Okta, Google Workspace, Microsoft Entra, ...) using the Authorization Code + PKCE flow. SSO is additive: local username/password login and API keys keep working alongside it.

Configuring an IdP

1. Open the CertMate UI as an admin → Settings → SSO.

2. Set the Issuer URL to the IdP's base URL (the path before /.well-known/openid-configuration). For example:

   • Keycloak: https://idp.example.com/realms/main
   • Authentik: https://idp.example.com/application/o/certmate/
   • Google: https://accounts.google.com

3. Fill in the Client ID and Client Secret issued by the IdP for the CertMate application.

4. In the IdP, register CertMate's callback URL as a valid redirect URI:

   https://your-certmate.example.com/api/auth/oidc/callback
   

5. Pick the claim names your IdP uses for username (preferred_username by default), email (email), and role (groups). Add Role mappings to translate IdP group/role claim values to CertMate roles — first match wins. Anything that doesn't match falls back to the configured Default role (viewer recommended).

6. Toggle Enable OIDC/SSO on and save. Visit /login in a new browser session to see the Sign in with button.

Role mapping example

For a Keycloak realm that exposes a groups claim, the configuration block in settings.json looks like:

"oidc": {
  "enabled": true,
  "provider_name": "Keycloak",
  "issuer_url": "https://idp.example.com/realms/main",
  "client_id": "certmate",
  "client_secret": "********",
  "scopes": ["openid", "email", "profile", "groups"],
  "role_claim": "groups",
  "role_mappings": [
    { "claim_value": "certmate-admins",    "role": "admin" },
    { "claim_value": "certmate-operators", "role": "operator" }
  ],
  "default_role": "viewer",
  "auto_create_users": true,
  "link_by_email": true,
  "sync_role_on_login": true
}

Provisioning and linking

• Just-in-time provisioning (auto_create_users) creates a CertMate user row on first login. The row has an empty password hash so JIT-provisioned SSO accounts cannot fall back to local login.
• Email linking (link_by_email) detects collisions with existing local users and merges identities — the user keeps their existing password hash, so a local-then-linked account can still log in either way during a rollout. Their role is preserved at the moment of linking, and from the next login onwards it is governed by sync_role_on_login like anyone else's (see below): with the default true, a linked local admin whose IdP groups map to viewer becomes a viewer on their second login. Disable link_by_email if you want JIT-only provisioning with no local-password fallback.
• Subject (sub + iss) lookup always wins over email matching, so an already-linked SSO user is never accidentally re-merged when their IdP email changes.
• Role sync (sync_role_on_login, default true) re-derives the role from the current claims on every login, so removing someone from an admin group in the IdP demotes them in CertMate too. Set it to false when the IdP only authenticates and roles are managed inside CertMate — an admin promoting someone by hand then survives their next login.
• A disabled CertMate user is refused at SSO login exactly as at local login: disabling an account locks it out regardless of how it authenticates.

Security

• PKCE (S256) is enforced for every flow regardless of client type.
• The id_token's signature, audience, issuer, expiry and nonce are validated server-side by Authlib using the IdP's published JWKS.
• client_secret is masked (********) in every GET response and round-tripped safely through the Settings UI.
• The oidc settings block is on the bulk-POST reject list — only the dedicated /api/auth/oidc/settings endpoint can mutate it, with full audit.
• Failed callbacks count against the same per-IP rate limit as local login.

API Usage

CertMate provides a comprehensive REST API for programmatic certificate management. All endpoints require Bearer token authentication.

Authentication

Include the Authorization header in all API requests:

Authorization: Bearer your_api_token_here

Endpoints

The full reference is docs/api.md: every endpoint, the role it needs, the shape it answers with, and the error format.

This section used to be a second catalogue, and the two had drifted: of the endpoints each named, six were in both. A reader had no way to tell which was current, and the reference was missing surfaces this file carried, which is where the storage backend, backup, metrics and zombie-scan sections in docs/api.md came from.

A worked example, so the shape is here even if the list is not:

# Every request carries the token.
curl -H "Authorization: Bearer $CERTMATE_TOKEN" \
  https://certmate.example.com/api/certificates

# Issue one. The response is the certificate record, not a job id.
curl -X POST -H "Authorization: Bearer $CERTMATE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domain": "api.example.com", "dns_provider": "cloudflare"}' \
  https://certmate.example.com/api/certificates/create

Automation-Friendly Download URL

Certificate downloads for infrastructure automation:

# Download every certificate file as one ZIP
GET /api/certificates/{domain}/download
Authorization: Bearer your_token_here

This endpoint returns a ZIP file containing all certificate files:

• cert.pem - Server certificate
• chain.pem - Intermediate certificate chain
• fullchain.pem - Full certificate chain (cert + chain)
• privkey.pem - Private key

For a single file, use the path form GET /api/certificates/{domain}/download/{cert|chain|fullchain|privkey|combined}. privkey and combined require operator; a viewer may pull the public material only.

Integration Examples

cURL Download

curl -H "Authorization: Bearer your_token_here" \
 -o example.com-tls.json \
 https://your-certmate-server.com/api/certificates/example.com/download?format=json

Python SDK Example

import requests
from pathlib import Path

class CertMateClient:
 def __init__(self, base_url, token):
 self.base_url = base_url.rstrip('/')
 self.headers = {"Authorization": f"Bearer {token}"}
 
 def download_certificate(self, domain):
 """Download certificate bundle as JSON for domain"""
 url = f"{self.base_url}/api/certificates/{domain}/download?format=json"
 
 response = requests.get(url, headers=self.headers)
 response.raise_for_status()
 return response.json()
 
 def list_certificates(self):
 """List all managed certificates"""
 response = requests.get(f"{self.base_url}/api/certificates", 
 headers=self.headers)
 response.raise_for_status()
 return response.json()
 
 def create_certificate(self, domain, dns_provider=None):
 """Create new certificate for domain"""
 data = {"domain": domain}
 if dns_provider:
 data["dns_provider"] = dns_provider
 
 response = requests.post(f"{self.base_url}/api/certificates/create",
 json=data, headers=self.headers)
 response.raise_for_status()
 return response.json()
 
 def renew_certificate(self, domain):
 """Renew existing certificate"""
 response = requests.post(f"{self.base_url}/api/certificates/{domain}/renew",
 headers=self.headers)
 response.raise_for_status()
 return response.json()

# Usage example
client = CertMateClient("https://certmate.company.com", "your_token_here")

# List and download certificates
certs = client.list_certificates()
bundle = client.download_certificate("api.company.com")
Path("/etc/ssl/certs/api").mkdir(parents=True, exist_ok=True)
for name, key in {
    "cert.pem": "cert_pem",
    "chain.pem": "chain_pem",
    "fullchain.pem": "fullchain_pem",
    "privkey.pem": "private_key_pem",
}.items():
    Path("/etc/ssl/certs/api", name).write_text(bundle[key])

The same JSON response shape can be consumed directly by Ansible's uri module or Salt's HTTP helpers without unpacking an archive.

Infrastructure as Code Examples

Terraform: there is no CertMate Terraform provider. This README used to show one, with source = "local/certmate" and certmate_certificate resources; neither the registry nor any repository has it, so the example could not be run. Drive the REST API from Terraform with the http provider or a local-exec, or use the Ansible and shell examples below, which are against the real API.

Bash Automation Script:

#!/bin/bash
set -euo pipefail

# Configuration
CERTMATE_URL="https://certmate.company.com"
API_TOKEN="${CERTMATE_TOKEN}"
DOMAIN="${1:-example.com}"
CERT_DIR="/etc/ssl/certs/${DOMAIN}"
BACKUP_DIR="/backup/certs/${DOMAIN}/$(date +%Y%m%d_%H%M%S)"

# Functions
log() {
 echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*" >&2
}

create_backup() {
 if [[ -d "$CERT_DIR" ]]; then
 log "Creating backup of existing certificates"
 mkdir -p "$BACKUP_DIR"
 cp -r "$CERT_DIR"/* "$BACKUP_DIR/" || true
 fi
}

download_certificate() {
 log "Downloading certificate for ${DOMAIN}"
 
 # Download with retry logic
 for i in {1..3}; do
 if curl -f -H "Authorization: Bearer $API_TOKEN" \
 -o "${DOMAIN}-tls.json" \
 "$CERTMATE_URL/api/certificates/$DOMAIN/download?format=json"; then
 log "Certificate downloaded successfully"
 return 0
 else
 log "Download attempt $i failed, retrying..."
 sleep 5
 fi
 done
 
 log "Failed to download certificate after 3 attempts"
 return 1
}

extract_certificate() {
 log "Extracting certificate to ${CERT_DIR}"
 mkdir -p "$CERT_DIR"
 jq -r '.cert_pem' "${DOMAIN}-tls.json" > "$CERT_DIR/cert.pem"
 jq -r '.chain_pem' "${DOMAIN}-tls.json" > "$CERT_DIR/chain.pem"
 jq -r '.fullchain_pem' "${DOMAIN}-tls.json" > "$CERT_DIR/fullchain.pem"
 jq -r '.private_key_pem' "${DOMAIN}-tls.json" > "$CERT_DIR/privkey.pem"
 
 # Set proper permissions
 chmod 600 "$CERT_DIR"/*.pem
 chown root:ssl-cert "$CERT_DIR"/*.pem
}

reload_services() {
 log "Reloading web services"
 systemctl reload nginx || log "Failed to reload nginx"
 systemctl reload apache2 || log "Failed to reload apache2"
 systemctl reload haproxy || log "Failed to reload haproxy"
}

cleanup() {
 rm -f "${DOMAIN}-tls.json"
}

# Main execution
main() {
 log "Starting certificate update for ${DOMAIN}"
 
 create_backup
 download_certificate
 extract_certificate
 reload_services
 cleanup
 
 log "Certificate update completed for ${DOMAIN}"
}

# Trap cleanup on exit
trap cleanup EXIT

# Run main function
main "$@"

Advanced Ansible Playbook:

---
- name: Manage SSL certificates with CertMate multi-account support
 hosts: web_servers
 vars:
 certmate_url: "https://certmate.company.com"
 certmate_token: "{{ vault_certmate_token }}"
 
 tasks:
 - name: Configure Cloudflare accounts
 uri:
 url: "{{ certmate_url }}/api/dns/cloudflare/accounts"
 method: POST
 headers:
 Authorization: "Bearer {{ certmate_token }}"
 Content-Type: "application/json"
 body_format: json
 body:
 account_id: "{{ item.account_id }}"
 config:
 name: "{{ item.name }}"
 description: "{{ item.description }}"
 api_token: "{{ item.api_token }}"
 loop:
 - account_id: "production"
 name: "Production Environment"
 description: "Main production Cloudflare account"
 api_token: "{{ vault_cloudflare_prod_token }}"
 - account_id: "staging"
 name: "Staging Environment"
 description: "Development and testing account"
 api_token: "{{ vault_cloudflare_staging_token }}"
 
 - name: Create certificates with specific accounts
 uri:
 url: "{{ certmate_url }}/api/certificates/create"
 method: POST
 headers:
 Authorization: "Bearer {{ certmate_token }}"
 Content-Type: "application/json"
 body_format: json
 body:
 domain: "{{ item.domain }}"
 dns_provider: "{{ item.provider }}"
 account_id: "{{ item.account_id }}"
 loop:
 - domain: "api.company.com"
 provider: "cloudflare"
 account_id: "production"
 - domain: "staging.company.com"
 provider: "cloudflare"
 account_id: "staging"
 - domain: "test.company.com"
 provider: "route53"
 account_id: "backup-aws"
 
 - name: Download and deploy certificates
 block:
 - name: Download certificate bundle as JSON
 uri:
 url: "{{ certmate_url }}/api/certificates/{{ item }}/download?format=json"
 headers:
 Authorization: "Bearer {{ certmate_token }}"
 return_content: yes
 register: cert_bundle
 
 - name: Write certificate files
 copy:
 dest: "/etc/ssl/certs/{{ item.0.item }}/{{ item.1.name }}"
 content: "{{ item.0.json[item.1.key] }}"
 owner: root
 group: ssl-cert
 mode: "{{ item.1.mode }}"
 loop: "{{ cert_bundle.results | product(cert_files) | list }}"
 vars:
 cert_files:
 - { name: "cert.pem", key: "cert_pem", mode: "0644" }
 - { name: "chain.pem", key: "chain_pem", mode: "0644" }
 - { name: "fullchain.pem", key: "fullchain_pem", mode: "0644" }
 - { name: "privkey.pem", key: "private_key_pem", mode: "0600" }
 loop:
 - "api.company.com"
 - "staging.company.com"
 - "test.company.com"

Production-Ready Ansible Playbook:

---
- name: Enterprise SSL certificate management with CertMate
 hosts: web_servers
 become: yes
 vars:
 certmate_url: "https://certmate.company.com"
 api_token: "{{ vault_certmate_token }}"
 certificate_domains:
 - name: "api.company.com"
 dns_provider: "cloudflare"
 nginx_sites: ["api"]
 services_to_reload: ["nginx"]
 - name: "web.company.com"
 dns_provider: "route53"
 nginx_sites: ["web", "admin"]
 services_to_reload: ["nginx", "haproxy"]
 
 tasks:
 - name: Create certificate directories
 file:
 path: "/etc/ssl/certs/{{ item.name }}"
 state: directory
 owner: root
 group: ssl-cert
 mode: '0750'
 loop: "{{ certificate_domains }}"
 
 - name: Check certificate expiry
 uri:
 url: "{{ certmate_url }}/api/certificates/{{ item.name }}/deployment-status"
 method: GET
 headers:
 Authorization: "Bearer {{ api_token }}"
 register: cert_status
 loop: "{{ certificate_domains }}"
 
 - name: Create new certificates if needed
 uri:
 url: "{{ certmate_url }}/api/certificates/create"
 method: POST
 headers:
 Authorization: "Bearer {{ api_token }}"
 Content-Type: "application/json"
 body_format: json
 body:
 domain: "{{ item.name }}"
 dns_provider: "{{ item.dns_provider }}"
 loop: "{{ certificate_domains }}"
 when: cert_status.results[ansible_loop.index0].json.needs_renewal | default(false)
 
 - name: Download certificates
 uri:
 url: "{{ certmate_url }}/api/certificates/{{ item.name }}/download?format=json"
 method: GET
 headers:
 Authorization: "Bearer {{ api_token }}"
 return_content: yes
 register: cert_bundle
 loop: "{{ certificate_domains }}"
 
 - name: Write certificates
 copy:
 dest: "/etc/ssl/certs/{{ item.0.item.name }}/{{ item.1.name }}"
 content: "{{ item.0.json[item.1.key] }}"
 owner: root
 group: ssl-cert
 mode: "{{ item.1.mode }}"
 loop: "{{ cert_bundle.results | product(cert_files) | list }}"
 vars:
 cert_files:
 - { name: "cert.pem", key: "cert_pem", mode: "0644" }
 - { name: "chain.pem", key: "chain_pem", mode: "0644" }
 - { name: "fullchain.pem", key: "fullchain_pem", mode: "0644" }
 - { name: "privkey.pem", key: "private_key_pem", mode: "0600" }
 notify: 
 - reload nginx
 - reload haproxy
 - restart services
 
 - name: Verify certificate installation
 openssl_certificate:
 path: "/etc/ssl/certs/{{ item.name }}/fullchain.pem"
 provider: assertonly
 has_expired: no
 valid_in: 86400 # Valid for at least 1 day
 loop: "{{ certificate_domains }}"
 
 - name: Update nginx SSL configuration
 template:
 src: "nginx-ssl.conf.j2"
 dest: "/etc/nginx/sites-available/{{ item.1 }}"
 backup: yes
 loop: "{{ certificate_domains | subelements('nginx_sites') }}"
 notify: reload nginx
 
 - name: Cleanup temporary files
 file:
 path: "/tmp/{{ item.name }}-tls.json"
 state: absent
 loop: "{{ certificate_domains }}"
 
 handlers:
 - name: reload nginx
 systemd:
 name: nginx
 state: reloaded
 
 - name: reload haproxy
 systemd:
 name: haproxy
 state: reloaded
 
 - name: restart services
 systemd:
 name: "{{ item }}"
 state: restarted
 loop: "{{ services_to_restart | default([]) }}"

API contract version

Two versions appear in API responses and they answer different questions.

A client deciding whether it still works should read the second. The first moves on every patch whether or not anything a caller depends on moved with it.

• minor bump — the surface grew in a way you can ignore: a new endpoint, a new field on a response, a new optional request field;
• major bump — something you may depend on went away or changed meaning: an endpoint removed, a response field removed or retyped, a request field that became required, a status code that changed for an existing condition.

Deprecating something bumps neither — that is the point of deprecating rather than removing.

Deprecation

An endpoint on its way out answers normally and says so in its headers, so a client can warn instead of failing:

Deprecation: @1757289600                      # RFC 9745 — when it became deprecated
Sunset: Mon, 08 Mar 2027 00:00:00 GMT         # RFC 8594 — earliest it may stop answering
Link: <https://…/docs/api.md#…>; rel="deprecation"

Nothing is deprecated today. The headers appear only on an endpoint that is.

Request validation

The API validates request bodies against the models published in /api/swagger.json. A missing required field or a wrong type is refused with 400 and a body naming the field, before the request reaches the application.

One consequence worth knowing: send a field or leave it out — do not send null. Omitting an optional field means "use the default"; an explicit null is a type error ("None is not of type 'integer'"), because null is not an integer.

Values outside a documented enum are refused by the application rather than by the schema, with a message naming the accepted set — for example key_size must be one of [2048, 3072, 4096], got 1024.

Configuration Guide

Environment Variables

Deployment and networking

Security behaviour

Timing and diagnostics

Bind address is not an environment variable. The container always binds 0.0.0.0 inside its own network namespace; to expose it only on loopback, publish it that way — -p 127.0.0.1:8000:8000. Running app.py directly (development only) takes --host / --port / --debug as CLI flags. HOST and FLASK_DEBUG are read by nothing, and setting them has never had any effect (#429).

DNS Provider Configuration

Cloudflare Setup

1. Go to Cloudflare API Tokens
2. Click "Create Token" → "Custom token"
3. Set permissions:

• Zone: DNS:Edit + Zone:Read
• Zone Resources: Include specific zones or all zones

4. Copy the generated token

# Environment variable
CLOUDFLARE_TOKEN=your_cloudflare_api_token_here

AWS Route53 Setup

1. Create IAM user with Route53 permissions
2. Attach policy: Route53FullAccess or custom policy:

{
 "Version": "2012-10-17",
 "Statement": [{
 "Effect": "Allow",
 "Action": ["route53:ListHostedZones",
 "route53:GetChange",
 "route53:ChangeResourceRecordSets"
 ],
 "Resource": "*"
 }
 ]
}

# Environment variables
AWS_ACCESS_KEY_ID=your_access_key_id
AWS_SECRET_ACCESS_KEY=your_secret_access_key
AWS_DEFAULT_REGION=us-east-1

Azure DNS Setup

1. Create Service Principal:

az ad sp create-for-rbac --name "CertMate" --role "DNS Zone Contributor" --scopes "/subscriptions/{subscription-id}/resourceGroups/{resource-group}"

# Environment variables
AZURE_SUBSCRIPTION_ID=your_subscription_id
AZURE_RESOURCE_GROUP=your_resource_group_name
AZURE_TENANT_ID=your_tenant_id
AZURE_CLIENT_ID=your_client_id
AZURE_CLIENT_SECRET=your_client_secret

Google Cloud DNS Setup

1. Create service account with DNS Administrator role
2. Download JSON key file

# Environment variables
GOOGLE_PROJECT_ID=your_project_id
GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

PowerDNS Setup

# Environment variables
POWERDNS_API_URL=https://your-powerdns-server:8081
POWERDNS_API_KEY=your_api_key

Certificate Storage Configuration

CertMate supports multiple storage backends for certificates, providing flexibility for different deployment scenarios and security requirements. By default, certificates are stored locally on the filesystem, but you can configure enterprise-grade storage backends for enhanced security and compliance.

Choosing the Right Storage Backend:

• Local Filesystem: Perfect for development, testing, and small deployments
• Azure Key Vault: Best for Azure-native environments and Microsoft ecosystem integration
• AWS Secrets Manager: Ideal for AWS infrastructure and cross-region deployments
• HashiCorp Vault: Excellent for multi-cloud environments and advanced secret management
• Infisical: Great for teams wanting open-source secret management with collaboration features
• S3-compatible object storage: One bucket on any S3 endpoint (Hetzner, Contabo, OVHcloud, Scaleway, Exoscale, Wasabi, MinIO, AWS)

Local Filesystem (Default)

The default storage backend stores certificates in the local filesystem with secure permissions:

# Default certificate directory
certificates/
 example.com/
 cert.pem # Server certificate
 chain.pem # Certificate chain
 fullchain.pem # Full chain
 privkey.pem # Private key (600 permissions)
 metadata.json # Certificate metadata (600 permissions)

Configuration:

• Directory: certificates (configurable)
• Permissions: 600 for private keys, 644 for certificates
• Backup: Included in automatic backups
• Use Cases: Development, testing, single-server deployments

Benefits:

• Zero configuration required
• No external dependencies
• Fast access and operations
• Perfect for getting started

Azure Key Vault

Store certificates securely in Azure Key Vault for enterprise-grade secret management:

Required Dependencies:

pip install -r requirements-azure-storage.txt

Configuration:

{
 "certificate_storage": {
 "backend": "azure_keyvault",
 "azure_keyvault": {
 "vault_url": "https://yourvault.vault.azure.net/",
 "tenant_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
 "client_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
 "client_secret": "your_client_secret"
 }
 }
}

Benefits:

• Azure-native secret management
• Compliance and audit capabilities (SOC 2, ISO 27001, FIPS 140-2)
• Hardware security module (HSM) protection
• Azure RBAC integration and managed identity support
• Automatic backup and disaster recovery

Use Cases:

• Azure-based infrastructure
• Enterprise compliance requirements
• Multi-region Azure deployments
• Integration with Azure DevOps and ARM templates

AWS Secrets Manager

Integrate with AWS Secrets Manager for scalable secret storage:

Required Dependencies:

pip install -r requirements-aws-storage.txt

Configuration:

{
 "certificate_storage": {
 "backend": "aws_secrets_manager",
 "aws_secrets_manager": {
 "region": "us-east-1",
 "access_key_id": "AKIAIOSFODNN7EXAMPLE",
 "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
 }
 }
}

For IAM authentication, choose AWS credentials / IAM role (no keys) in Settings → Storage, enter the region and leave access keys empty. CertMate uses the AWS credential chain (for example an EC2 instance profile, ECS task role or EKS pod identity). An optional Assume role ARN uses sts:AssumeRole and refreshes the temporary credentials automatically. Existing key-based settings still work. If auth_mode is absent in a manual configuration, access keys remain required. Equivalent settings without an additional assumed role:

{"certificate_storage":{"backend":"aws_secrets_manager","aws_secrets_manager":{"region":"eu-west-1","auth_mode":"iam_role"}}}

The IAM identity needs secretsmanager:ListSecrets (account-wide, resource *) and GetSecretValue, DescribeSecret, CreateSecret, UpdateSecret and DeleteSecret for the certmate/certificates/* secrets. Assuming a second role also requires permission to call sts:AssumeRole and a trust policy on the target role. The connection test reports AWS permission errors rather than treating them as an empty store.

Benefits:

• AWS-native secret management
• Automatic encryption at rest with AWS KMS
• Cross-region replication for high availability
• IAM-based access control and fine-grained permissions
• Integration with AWS CloudTrail for audit logging
• Automatic rotation capabilities

Use Cases:

• AWS-based infrastructure
• Multi-region deployments
• Integration with ECS, EKS, Lambda
• Compliance with AWS security best practices

HashiCorp Vault

Use industry-standard HashiCorp Vault for advanced secret management:

Required Dependencies:

pip install -r requirements-vault-storage.txt

Configuration:

{
 "certificate_storage": {
 "backend": "hashicorp_vault",
 "hashicorp_vault": {
 "vault_url": "https://vault.example.com:8200",
 "vault_token": "hvs.xxxxxxxxxxxxxxxxxxxx",
 "mount_point": "secret",
 "engine_version": "v2"
 }
 }
}

Benefits:

• Industry-standard secret management
• Secret versioning and rollback capabilities
• Comprehensive audit logging and monitoring
• Fine-grained access policies and dynamic secrets
• Multi-cloud and hybrid cloud support
• Advanced authentication methods (LDAP, Kubernetes, AWS IAM)

Use Cases:

• Multi-cloud environments
• Complex organizational security requirements
• Dynamic secret generation
• Integration with CI/CD pipelines and Kubernetes

Infisical

Modern open-source secret management with team collaboration:

Required Dependencies:

pip install -r requirements-infisical-storage.txt

Configuration:

{
 "certificate_storage": {
 "backend": "infisical",
 "infisical": {
 "site_url": "https://app.infisical.com",
 "client_id": "your_client_id",
 "client_secret": "your_client_secret",
 "project_id": "your_project_id",
 "environment": "prod"
 }
 }
}

site_url must be https:// (plain http:// is accepted only for a loopback address: localhost, 127.0.0.1 or ::1). The Infisical SDK follows a redirect and sends the request body on, and here the body is the certificate and its private key; its HTTP client cannot be configured from CertMate, so the one thing CertMate can refuse is the position where anyone on the path could answer with a redirect. Point it at a server you trust with the key, which is what storing it there means anyway.

This backend did not work in v2.44.0 or earlier. It could not run at all: it imported a module name the pinned infisical-python does not have, and called methods that SDK does not have.

Benefits:

• Open-source secret management with transparency
• End-to-end encryption for maximum security
• Team collaboration features and role-based access
• Multi-environment support (dev, staging, prod)
• Git-like versioning for secrets
• Self-hostable for complete control

Use Cases:

• Team-based development workflows
• Open-source preference
• Self-hosted secret management
• Multi-environment certificate management

S3-Compatible Object Storage

One backend for S3-compatible object storage: Hetzner, Contabo, OVHcloud, Scaleway, Exoscale, Wasabi, self-hosted MinIO, or AWS S3 itself. Each domain is stored as one JSON object, <prefix>/<domain>.json, holding the certificate files and their metadata.

Required Dependencies: boto3, already in requirements.txt and requirements-storage-all.txt (not in requirements-minimal.txt).

Configuration:

{
 "certificate_storage": {
 "backend": "s3_compatible",
 "s3_compatible": {
 "endpoint_url": "https://fsn1.your-objectstorage.com",
 "bucket": "certmate",
 "access_key_id": "your_access_key_id",
 "secret_access_key": "your_secret_access_key",
 "region": "us-east-1",
 "prefix": "certmate/certificates"
 }
 }
}

For S3-compatible services, use Access keys and provide both keys, the bucket and the service's endpoint. Existing key-based configurations continue to work unchanged. region defaults to us-east-1 and prefix to certmate/certificates.

For AWS S3 with an IAM role, select AWS credentials / IAM role (no keys) in Settings → Storage, provide the bucket and region, and leave the endpoint URL and access keys empty. Boto3 uses its standard credential chain (for example an EC2 instance profile, ECS task role or EKS pod identity). You can optionally enter an Assume role ARN (arn:aws:iam::123456789012:role/CertMateS3): the current identity must have sts:AssumeRole permission and the destination role must trust it. CertMate refreshes the temporary STS credentials automatically. The role needs s3:ListBucket on the bucket and s3:GetObject, s3:PutObject, and s3:DeleteObject on <prefix>/* (including object existence checks). The Test Storage Backend action lists the prefix and reports authentication/permission failures.

For manual settings.json, explicitly set "auth_mode":"iam_role" to use AWS credentials without keys or an endpoint: {"backend":"s3_compatible","s3_compatible":{"bucket":"certmate","region":"eu-west-1","auth_mode":"iam_role","assume_role_arn":"arn:aws:iam::123456789012:role/CertMateS3"}}. Omit assume_role_arn to use the attached role directly. Without auth_mode, legacy key-pair requirements still apply. The optional off-site S3 backup storage has separate credential settings; this option configures certificate storage only.

Quick Installation Guide

Install All Storage Backends:

# Install all storage backends at once
pip install -r requirements-storage-all.txt

Install Individual Storage Backends:

# Azure Key Vault only
pip install -r requirements-azure-storage.txt

# AWS Secrets Manager only 
pip install -r requirements-aws-storage.txt

# HashiCorp Vault only
pip install -r requirements-vault-storage.txt

# Infisical only
pip install -r requirements-infisical-storage.txt

# S3-compatible storage: no separate file — boto3 is in requirements.txt

Requirements File Overview:

• requirements-storage-all.txt - All storage backends (recommended for production)
• requirements-azure-storage.txt - Azure Key Vault dependencies
• requirements-aws-storage.txt - AWS Secrets Manager dependencies
• requirements-vault-storage.txt - HashiCorp Vault dependencies
• requirements-infisical-storage.txt - Infisical dependencies
• requirements-minimal.txt - Base CertMate without storage backends

Configuring Storage Backends

Via Web Interface:

1. Navigate to Settings → Certificate Storage Backend
2. Select your preferred backend from the dropdown
3. Configure the required credentials and settings
4. Test the connection to verify configuration
5. Save settings and optionally migrate existing certificates

Via API:

# Test storage backend connectivity before switching
curl -X POST "http://localhost:8000/api/storage/test" \
 -H "Authorization: Bearer your_token" \
 -H "Content-Type: application/json" \
 -d '{
 "backend": "azure_keyvault",
 "config": {
 "vault_url": "https://yourvault.vault.azure.net/",
 "tenant_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
 "client_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
 "client_secret": "your_client_secret"
 }
 }'

# Get current storage backend information
curl -X GET "http://localhost:8000/api/storage/info" \
 -H "Authorization: Bearer your_token"

# Update storage backend configuration. Unlike /api/storage/test, the settings
# go under a key named after the backend, not under "config" — a "config" key
# here is ignored and the backend is saved with no credentials.
curl -X POST "http://localhost:8000/api/storage/config" \
 -H "Authorization: Bearer your_token" \
 -H "Content-Type: application/json" \
 -d '{
 "backend": "hashicorp_vault",
 "hashicorp_vault": {
 "vault_url": "https://vault.example.com:8200",
 "vault_token": "hvs.xxxxxxxxxxxxxxxxxxxx",
 "mount_point": "secret",
 "engine_version": "v2"
 }
 }'

Migrating Between Backends:

Migration copies every certificate from a source backend to a target backend, one domain at a time. When no source_backend is given, the source is the backend currently saved in settings, so migrate before switching:

1. Test the new backend (POST /api/storage/test)
2. Migrate (POST /api/storage/migrate, or the migration tool in Settings)
3. Check the response: migrated_count, failed_count, and migration_results, one true/false per domain
4. Switch the active backend (POST /api/storage/config)
5. Optionally clean up the old storage — CertMate never deletes it

If you already saved the new backend, choose Local filesystem (or the previously configured backend) as Source backend in the Settings migration dialog. The dialog always requires an explicit source choice: when migrating from bucket A to bucket B within S3, choose the saved S3 backend as source, even though S3 is also selected as the target. The target remains the backend selected in Settings. Via the API, include "source_backend": "local_filesystem" in the migration request. A zero-domain result means no certificates were found in that source; check the source selection and certificate directory rather than assuming the certificates were copied.

Migration via API:

# Migrate all certificates from current backend to new backend
curl -X POST "http://localhost:8000/api/storage/migrate" \
 -H "Authorization: Bearer your_token" \
 -H "Content-Type: application/json" \
 -d '{
 "target_backend": "aws_secrets_manager",
 "target_config": {
 "region": "us-east-1",
 "access_key_id": "AKIAIOSFODNN7EXAMPLE", 
 "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
 }
 }'

Each certificate's metadata is copied along with its files. There is no separate verification pass and no rollback: a domain whose copy failed is reported as false in migration_results, and the source is left untouched, so re-running the migration or staying on the old backend are both safe.

Backward Compatibility:

• Existing installations continue working without changes
• New storage backends are opt-in
• Migration is non-destructive (copies certificates)
• Local filesystem remains the default backend

Directory Structure

certmate/
 app.py # Main Flask application
 requirements.txt # Python dependencies
 docker-compose.yml # Docker Compose configuration
 Dockerfile # Container build instructions
 nginx.conf.example # Nginx reverse proxy config template (copy to nginx.conf)
 .env.example # Environment template
 README.md # This documentation
 CONTRIBUTING.md # Contribution guidelines
 docs/ # Comprehensive documentation
 certificates/ # Certificate storage
 {domain}/
 cert.pem # Server certificate
 chain.pem # Certificate chain
 fullchain.pem # Full chain
 privkey.pem # Private key
 data/ # Application data
 settings.json # Persistent settings
 logs/ # Application logs
 letsencrypt/ # Let's Encrypt working directory
 config/ # Certbot configuration
 work/ # Certbot working files
 logs/ # Certbot logs
 modules/ # Modular backend
 core/ # Core modules
 deployer.py # Deploy hooks manager
 notifier.py # Notification channels
 events.py # SSE event bus
 digest.py # Weekly digest
 audit.py # Audit logger
 shell.py # Shell executor
 web/ # Web routes
 api/ # API resources
 templates/ # Web interface templates
 index.html # Main dashboard
 settings.html # Settings page
 activity.html # Activity timeline
 help.html # Help & documentation
 static/js/ # Frontend modules
 dashboard.js # Dashboard logic
 settings.js # Settings components
 cmd-palette.js # Cmd+K palette
 shortcuts.js # Keyboard shortcuts

Security & Best Practices

Security Considerations

Authentication & Authorization

• Role-Based Access Control: Assign viewer, operator, or admin roles to each user
• Scoped API Keys: Create API keys with specific role permissions and optional expiration
• Per-Domain Scoping (allowed_domains): Restrict a scoped API key to a list of domain patterns. Supports exact (example.com) and wildcard (*.example.com) forms; the wildcard matches subdomains only, not the apex. An operation on an existing certificate needs every name the certificate covers to be in scope, as creating it does. Client certificates, backups and the inventory configuration belong to the instance, not to a domain, so a key restricted this way cannot use those routes; /metrics answers it with the series about its own domains only. An admin key cannot be restricted to domains: creating one is refused, and one stored by an earlier release acts as an operator within its domains. Out-of-scope requests return 403 DOMAIN_OUT_OF_SCOPE and are recorded in the audit log. Leave the field empty for unrestricted access (legacy behavior).
• HMAC-SHA256 Token Hashing: API tokens are hashed with a server-side HMAC secret, preventing offline brute-force even if the settings file is leaked (backward compatible with pre-2.2.6 SHA-256 hashes)
• Strong Bearer Tokens: Use cryptographically secure tokens (32+ characters)
• Token Rotation: Regularly rotate API tokens and revoke unused keys
• Environment Variables: Never commit tokens to version control
• HTTPS Only: Always use HTTPS in production environments
• IP Restrictions: Implement firewall rules to restrict access

Settings API Hardening

• Strict Field Whitelist on POST /api/settings: only documented configuration keys are accepted. Sensitive fields (api_bearer_token, deploy_hooks, users, api_keys, local_auth_enabled) cannot be written through the generic settings endpoint — each has its own dedicated endpoint with its own audit:
  • Users → POST /api/users
  • API keys → POST /api/keys
  • Deploy hooks → POST /api/deploy/config
  • Local auth toggle → POST /api/auth/config (admin-only) Unknown or rejected keys are returned in a 400 response with a hint field pointing at the correct endpoint.
• Audit Trail for Configuration Changes: every mutation to settings, the auth-config toggle, users, scoped API keys, and deploy hooks is recorded with operator identity and source IP. Authorization denials (out-of-scope domain access, blocked field writes) are recorded too. Logs are written to data/audit/certificate_audit.log as one JSON object per line — pipeable into your SIEM of choice.

The Setup Window

Until the first operator credential exists — a local admin with local auth enabled, API_BEARER_TOKEN, or OIDC — every request is served as admin to anyone who can reach the instance. So during setup CertMate only bootstraps: it creates the first admin and turns login on. API keys and further users are refused with 409 SETUP_BOOTSTRAP_ONLY until setup is complete, because anything created in that window would be created by whoever was there and would outlive it. API keys that an older version let be created during setup stay valid but are flagged in Settings → API Keys and in the startup log, until an operator confirms or revokes each one.

Certificate Security

• File Permissions: Private keys stored with 600 permissions
• Directory Permissions: Certificate directories with 700 permissions
• Backup Encryption: Encrypt certificate backups
• Access Logging: Monitor certificate access patterns

Secret Storage Hardening

CertMate stores DNS provider credentials, ACME account keys, and (legacy) bearer tokens inside data/settings.json. The bearer token itself is migrated to an HMAC-SHA256 hash on first save, but DNS provider credentials remain in the file in their original form so they can be passed to certbot plugins. The file is created with 0600 permissions, which is the first line of defense. Recommended hardening, in order of effort:

1. Use an external secret backend (already supported): point certificate_storage.backend at HashiCorp Vault, Infisical, AWS Secrets Manager, or Azure Key Vault. Issued certificates are stored there transparently, keeping the bearer token + DNS credentials in settings.json as the only on-disk secret surface.
2. Encrypt the underlying volume: run CertMate's data/ directory on a LUKS-encrypted partition (Linux), an encrypted APFS volume (macOS), or a Kubernetes Secret mounted in a tmpfs. This protects credentials at rest even if the host disk is removed or imaged.
3. Run as a dedicated non-root user with data/ owned by that user and 0700 mode. Verify with ls -la data/ — only the CertMate process user should be able to read the file.
4. Avoid bind-mounting data/ from untrusted sources in Docker. Use a named volume managed by Docker, or a CSI-provisioned volume in Kubernetes, rather than a bind from a multi-tenant host.
5. Rotate credentials regularly: DNS provider credentials can be rotated independently of CertMate — issue a new token in the provider console, update Settings, then revoke the old token. CertMate will pick up the new credentials on the next renewal.

CertMate does not currently encrypt secrets at the application layer. If your threat model requires that the same operator who can read settings.json should still not be able to read DNS credentials, use option (1) above and configure the per-provider credentials in the external secret backend rather than in CertMate's settings.

Network Security

# Example firewall rules (iptables)
# Allow only specific IPs to access CertMate
iptables -A INPUT -p tcp --dport 8000 -s 10.0.0.0/8 -j ACCEPT
iptables -A INPUT -p tcp --dport 8000 -s 192.168.0.0/16 -j ACCEPT
iptables -A INPUT -p tcp --dport 8000 -j DROP

The rules above confine inbound access. You can also confine outbound traffic: route CertMate's HTTP(S) egress through a forward proxy and deny it any other route to the internet, so it can reach only your CA, DNS provider, object storage, and notification endpoints. With Secure Proxy Manager v3.9.0+ this is a built-in default-deny egress allowlist. See Confining outbound traffic for a worked example.

Performance Optimization

Production Deployment

# docker-compose.prod.yml
version: '3.8'
services:
  certmate:
    image: certmate:latest
    deploy:
      # CertMate runs an in-process renewal scheduler and per-domain locks, so
      # it must run as a SINGLE writer. Multiple replicas (or >1 gunicorn
      # worker) each fire the renewal check and would issue duplicate ACME
      # orders and hit the CA's duplicate-certificate rate limit. A host-local
      # flock guards multiple workers/containers on a *shared* data volume, but
      # the safe default is one writer. For HA, front a single active instance
      # rather than scaling this service.
      replicas: 1
      resources:
        limits:
          cpus: '1.0'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
    environment:
      - FLASK_ENV=production
      # The worker and thread counts are fixed in the image
      # (`--workers 1 --threads 8`): one worker because the scheduler runs
      # in-process and a second would duplicate every renewal. GUNICORN_TIMEOUT
      # is the one that is read.
      - GUNICORN_TIMEOUT=300   # the image default; raise it for slow DNS providers
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s

Load Balancing with Nginx

upstream certmate_backend {
 server certmate1:8000;
 server certmate2:8000;
 server certmate3:8000;
}

server {
 listen 443 ssl http2;
 server_name certmate.company.com;
 
 ssl_certificate /etc/ssl/certs/certmate.company.com/fullchain.pem;
 ssl_certificate_key /etc/ssl/certs/certmate.company.com/privkey.pem;
 
 location / {
 proxy_pass http://certmate_backend;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 
 # API rate limiting
 limit_req zone=api burst=10 nodelay;
 }
}

Backup and Recovery

CertMate provides comprehensive backup and recovery capabilities built directly into the application, ensuring your certificates and configuration data are always protected.

Unified Backup System

What is Unified Backup?

• Atomic Operation: Creates a single ZIP file containing both settings and certificates
• Data Consistency: Ensures settings and certificates are always in sync
• Prevents Corruption: Eliminates configuration/certificate mismatches
• Simplified Management: One file per snapshot. A disaster-recovery archive (include_secrets=true) contains everything needed for complete restoration; the default share-safe archive contains certificates, chains, metadata, the audit chain and the inventory — no credentials and no private keys

Two kinds of archive:

• Share-safe: settings with every credential masked, and no private keys — no ACME privkey.pem, no ACME account key, no private-CA key, no .pfx. Certificates, chains, metadata, the audit chain and the inventory are all there. The manifest says so (secrets_masked, key_material_excluded). Such an archive cannot restore an instance on its own: GET /api/backups reports it with can_restore: false and Settings disables its Restore button. The restore API does not read can_restore. It refuses a share-safe archive when this instance already holds certificates; otherwise it restores it, keeping the credentials already in this instance's settings.json, or — on an instance with no settings.json — leaving every credential masked, to be re-entered before the next renewal.
• Disaster recovery: plaintext settings and every key — encrypted at rest when CERTMATE_BACKUP_PASSPHRASE is set. This is the archive to keep off-site.

Which one you get automatically depends on one thing: whether a backup passphrase is configured. With CERTMATE_BACKUP_PASSPHRASE set, automatic backups are disaster-recovery archives, encrypted at rest. Without it they stay share-safe, because a complete archive that cannot be encrypted would be a plaintext credential dump written to disk on every settings change. Without a passphrase, the instance logs a notice once per process, at the first automatic backup it takes.

CertMate never generates or stores that passphrase: keeping it beside the archive it protects would make the encryption meaningless. It is yours to set and to keep.

POST /api/backups/create with {"include_secrets": true} (admin, audited) still takes a disaster-recovery archive on demand, passphrase or not — a deliberate opt-in.

Getting a backup back onto a new instance: POST /api/backups/upload (admin, multipart/form-data, field file), or the Restore From Elsewhere control in Settings. Uploading only stores the archive under a name CertMate generates — the name you send is discarded. Restoring it is a separate, explicit step, so an upload is never destructive on its own.

Automatic Backups:

• Unified Snapshots - Automatically created when DNS providers, domains, certificates, or application settings are modified. Disaster-recovery archives when CERTMATE_BACKUP_PASSPHRASE is set, share-safe otherwise (see above)
• Retention - Two rules, whichever hits first: the 50 most recent archives per type are kept (MAX_BACKUPS_PER_TYPE), and any archive older than 30 days is deleted regardless of how few there are (BACKUP_RETENTION_DAYS). Both are constants, not settings. This is why a disaster-recovery archive must be kept off the host: one left in backups/unified/ is deleted after 30 days like any other. Logging in no longer writes a backup, so routine sign-ins do not consume restore points
• Automatic Cleanup - Pruning runs after every backup, applying both rules above

Manual Backups:

• On-Demand Creation - Create backups anytime via the web interface or API
• Download Support - Export backups for external storage and disaster recovery
• Comprehensive Coverage - Includes all DNS configurations, certificates, and application settings

Web Interface Backup Management

Access backup features from the Settings page:

<!-- Create backup -->
<button onclick="createBackup('unified', this)">Create Backup</button>

<!-- View and manage existing backups -->
- Download backups for external storage
- Restore from any backup point with atomic consistency
- View backup contents and metadata
- Delete specific backups manually

API Backup Operations

Create Backup:

# Create backup (settings + certificates); "type" is required
curl -X POST "http://localhost:8000/api/backups/create" \
 -H "Authorization: Bearer your_token" \
 -H "Content-Type: application/json" \
 -d '{"type": "unified", "reason": "manual_backup"}'

# Response (201)
{
 "message": "Backup created successfully",
 "backups": [
 {"type": "unified", "filename": "backup_20241225_120000_123456_manual_backup.zip"}
 ],
 "secrets_masked": true,
 "recommendation": null
}

Archives are named backup_<YYYYmmdd_HHMMSS_ffffff>_<reason>.zip (UTC timestamp), or .zip.enc when CERTMATE_BACKUP_PASSPHRASE is set.

List and Download Backups:

# List all backups
curl -H "Authorization: Bearer your_token" \
 "http://localhost:8000/api/backups"

# Download backup
curl -H "Authorization: Bearer your_token" \
 "http://localhost:8000/api/backups/download/unified/backup_20241225_120000_123456_manual_backup.zip" \
 -o backup.zip

Backup File Structure

Backup File (ZIP):

backup_20260821_120000_000000_manual.zip
 backup_metadata.json  # manifest: secrets_masked, key_material_excluded, key_files_excluded, encrypted
 settings.json         # application settings — credentials masked in a share-safe archive, plaintext in a DR one
 certificates/
   domain1.com/
     cert.pem
     chain.pem
     fullchain.pem
     privkey.pem       # DR archive only (include_secrets=true); absent from a share-safe archive
     metadata.json
     live/ archive/ renewal/ accounts/   # certbot lineage (keys under accounts/ and live/ DR only)
 data/
   certs/ca/ca.crt      # ca.key: DR only
   certs/client/...     # client certificates (.key/.pfx: DR only)
   audit/ inventory/

Recovery Procedures

Backup Restoration:

Web Interface:

1. Navigate to Settings → Backup Management
2. Select the backup to restore from
3. Confirm restoration (restores both settings and certificates atomically)
4. The page reloads the restored settings; nothing is restarted
5. Verify all certificates and configurations are working

API Restoration:

# Restore from backup
curl -X POST "http://localhost:8000/api/backups/restore/unified" \
 -H "Authorization: Bearer your_token" \
 -H "Content-Type: application/json" \
 -d '{"filename": "backup_20241225_120000_123456_manual_backup.zip", "create_backup_before_restore": true}'

External Backup Integration

For additional protection, integrate with external backup systems:

Automated External Backup Script:

#!/bin/bash
# /opt/scripts/backup-certmate-external.sh

BACKUP_DIR="/backup/certmate/$(date +%Y%m%d_%H%M%S)"
CERT_DIR="/opt/certmate/certificates"
DATA_DIR="/opt/certmate/data"
RETENTION_DAYS=30

# Create backup directory
mkdir -p "$BACKUP_DIR"

# Download the newest backup that can restore (there is no "latest" alias;
# the list is ordered oldest to newest by name)
LATEST=$(curl -s -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/api/backups \
 | jq -r '[.unified[] | select(.can_restore)] | last | .filename // empty')
[ -n "$LATEST" ] || { echo "No restorable backup on this instance" >&2; exit 1; }
curl -H "Authorization: Bearer $API_TOKEN" \
 "http://localhost:8000/api/backups/download/unified/$LATEST" \
 -o "$BACKUP_DIR/$LATEST"

# Backup certificates directory
tar -czf "$BACKUP_DIR/certificates.tar.gz" "$CERT_DIR"

# Backup application data
tar -czf "$BACKUP_DIR/data.tar.gz" "$DATA_DIR"

# Encrypt backups (optional)
gpg --cipher-algo AES256 --compress-algo 1 --symmetric \
 --output "$BACKUP_DIR/$LATEST.gpg" \
 "$BACKUP_DIR/$LATEST"

# Cleanup old backups
find /backup/certmate -type d -mtime +$RETENTION_DAYS -exec rm -rf {} \;

echo "External backup completed: $BACKUP_DIR"

Recovery Procedure

#!/bin/bash
# Recovery from backup

BACKUP_DATE="20241225_120000"
BACKUP_DIR="/backup/certmate/$BACKUP_DATE"

# Stop services
docker-compose down

# Restore certificates
tar -xzf "$BACKUP_DIR/certificates.tar.gz" -C /opt/certmate/

# Restore data
tar -xzf "$BACKUP_DIR/data.tar.gz" -C /opt/certmate/

# Set permissions
chown -R 1000:1000 /opt/certmate/certificates
chmod -R 700 /opt/certmate/certificates

# Start services
docker-compose up -d

echo "Recovery completed from backup: $BACKUP_DATE"

Monitoring & Observability

Health Monitoring

Built-in Health Checks

# Basic health check
curl -f http://localhost:8000/health

# Detailed health with auth
curl -H "Authorization: Bearer your_token" \
 http://localhost:8000/api/certificates

Prometheus Metrics Integration

Nothing to add: /metrics is already a registered route (modules/web/misc_routes.py), and the metrics themselves are declared in modules/core/metrics.py. This section used to show a snippet headed "Add to app.py", which would have registered a second, unauthenticated /metrics on top of the real one.

A ready-to-import Grafana dashboard, Prometheus alert rules, and an authenticated scrape config ship in monitoring/ — see monitoring/README.md. The /metrics endpoint requires the viewer role, so scrape it with a viewer-scoped API token (Bearer) — a scraper reads, so it does not need admin.

Log Aggregation

# docker-compose.logging.yml
version: '3.8'
services:
 certmate:
 logging:
 driver: "fluentd"
 options:
 fluentd-address: localhost:24224
 tag: certmate
 fluentd-async-connect: "true"
 
 fluentd:
 image: fluent/fluentd:v1.14
 volumes:
 - ./fluentd/conf:/fluentd/etc
 - ./logs:/var/log/fluentd
 ports:
 - "24224:24224"
 - "24224:24224/udp"

Grafana Dashboard Example

{
 "dashboard": {
 "title": "CertMate SSL Certificate Monitoring",
 "panels": [{
 "title": "Certificate Expiry Status",
 "targets": [{
 "expr": "certmate_certificate_expiry_days < 30",
 "legendFormat": "Expiring Soon ({{domain}})"
 }
 ],
 "alert": {
 "conditions": [{
 "query": {"queryType": "", "refId": "A"},
 "reducer": {"type": "last", "params": []},
 "evaluator": {"params": [30], "type": "lt"}
 }
 ],
 "executionErrorState": "alerting",
 "frequency": "1h",
 "handler": 1,
 "name": "Certificate Expiring",
 "noDataState": "no_data",
 "notifications": []
 }
 }
 ]
 }
}

Built-in Notifications

CertMate includes a built-in notification system configurable from Settings > Notifications:

• Email (SMTP) - Certificate expiry warnings and renewal confirmations
• Slack - Incoming webhook integration for team channels
• Discord - Webhook notifications for Discord servers
• Telegram - Bot API messages (bot token + chat ID)
• ntfy - Push to an ntfy topic (self-hostable); optional access token, per-message priority
• Gotify - Push to a self-hosted Gotify server (server URL + app token, numeric priority)
• Generic Webhooks - HTTP POST/PUT/PATCH with HMAC-SHA256 signed payloads, bearer/basic/header authentication, and a JSON payload template with {{placeholders}} so the body fits whatever receives it — see docs/webhooks.md
• Weekly Digest - Scheduled summary of certificate status and upcoming renewals

All notification channels support per-event filtering (created, renewed, expiring, failed) and can be tested from the settings UI.

Microsoft Teams: no dedicated adapter is needed. Use a Teams channel's built-in email address (channel → ... → Get email address) as a recipient on the Email (SMTP) channel — Teams posts the message into the channel for you.

Downgrades & Recovery

Downgrading to a version older than the one that wrote settings.json is not supported and may result in a broken configuration or loss of accounts. If you see a DOWNGRADE DETECTED message in the logs after rolling back, restore the most recent backup that can actually restore.

Not every backup is a restore point. Unless CERTMATE_BACKUP_PASSPHRASE was set when the backup was taken, automatic backups are written with secrets masked and cannot restore this instance — they are configuration snapshots. On an instance that has never had a passphrase set, the newest archive is almost certainly one of those, so picking "the latest" by timestamp picks one that will be refused. Ask the API which ones qualify: it reports can_restore per archive, and restore_blocked_reason when it is false.

# Which backups can actually restore, newest first
curl -s -H "Authorization: Bearer $API_TOKEN" http://localhost:8000/api/backups \
  | jq -r '.unified[] | "\(.filename)  can_restore=\(.can_restore)  \(.restore_blocked_reason // "")"'

# Restore the newest one whose can_restore is true
curl -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"filename": "backup_YYYYmmdd_HHMMSS_ffffff_<reason>.zip", "create_backup_before_restore": true}' \
  http://localhost:8000/api/backups/restore/unified

If nothing reports can_restore: true, this instance has no restore point. Set CERTMATE_BACKUP_PASSPHRASE, take a backup immediately, and keep a copy off this node — see Backup and Recovery.

If you have lost the admin password and cannot log in, use the emergency reset script (requires container shell access):

docker exec -it certmate python scripts/reset_admin_password.py

Troubleshooting Guide

Common Issues & Solutions

Certificate Creation Failures

Issue: Certificate creation failed: ... with a DNS-01 challenge error (the text after the prefix is certbot's own)

# Check DNS propagation
dig TXT _acme-challenge.example.com @8.8.8.8

# Verify DNS provider credentials
curl -H "Authorization: Bearer cf_token" \
 "https://api.cloudflare.com/client/v4/user/tokens/verify"

Issue: Rate limit exceeded

# Let's Encrypt rate limits:
# - 50 certificates per registered domain per week
# - 5 duplicate certificates per week
# - 300 new orders per account per 3 hours

# Check rate limit status
curl "https://crt.sh/?q=example.com&output=json" | jq length

Issue: Permission denied accessing certificate files

# Fix file permissions
sudo chown -R certmate:certmate /opt/certmate/certificates
sudo chmod -R 700 /opt/certmate/certificates
sudo chmod 600 /opt/certmate/certificates/*/privkey.pem

API Authentication Issues

Issue: 401 Unauthorized

# Verify token format
curl -H "Authorization: Bearer your_token_here" \
 http://localhost:8000/api/certificates

# The token cannot be read back: settings.json stores only an HMAC of it
# (api_bearer_token_hash). Compare against what you set in API_BEARER_TOKEN
# or API_BEARER_TOKEN_FILE.

Issue: lost or forgotten API token

# Set a new token and restart: at startup, the token from API_BEARER_TOKEN
# (or API_BEARER_TOKEN_FILE) replaces the stored one.
# (32+ characters; a malformed token is ignored and the old one stays.)
openssl rand -hex 32   # put the output in .env as API_BEARER_TOKEN=...
docker-compose up -d --force-recreate certmate

Docker & Container Issues

Issue: Container won't start

# Check logs
docker-compose logs certmate

# Verify environment variables
docker-compose config

# Check port conflicts
netstat -tulpn | grep :8000

Issue: Volume mount issues

# Fix volume permissions
sudo chown -R 1000:1000 ./certificates ./data ./logs

# Check volume mounts
docker inspect certmate | jq '.[0].Mounts'

Deployment Status Shows "Backend: Unreachable"

Q: All my certificates show "Backend: Unreachable" even though they are issued, served, and downloadable. Is something broken?

No. The deployment-status badge is an optional health indicator — it does not affect issuance, renewal, or download. CertMate's own process opens a TLS connection to <domain>:<port> and compares the served certificate's fingerprint against the stored one. The badge only answers whether CertMate itself can reach the service and see the expected certificate.

• Deployed — handshake succeeded and the fingerprint matches.
• Wrong Cert — handshake succeeded but a different certificate is served.
• Unreachable — CertMate could not open a TLS connection to the domain at all.

Configuring the probe per certificate

By default the probe connects on port 443 with a direct TLS handshake (HTTPS). You can change the port and protocol per certificate via the API or the dashboard:

From the dashboard, open a certificate's detail panel and click Configure Probe, then enter the port and protocol.

From the API:

# Set probe to SMTP STARTTLS on port 587
curl -X PATCH https://certmate.example.com/api/certificates/example.com \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"deployment_port": 587, "deployment_protocol": "smtp-starttls"}'

# Reset to defaults (HTTPS on port 443)
curl -X PATCH https://certmate.example.com/api/certificates/example.com \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"deployment_port": null, "deployment_protocol": "https-tls"}'

The probe protocol defaults to the global timeout (3s) and can be tuned via the CERTMATE_TLS_PROBE_TIMEOUT_SECONDS environment variable.

Global timeout tuning:

# Accepts 1–30 seconds; default is 3
CERTMATE_TLS_PROBE_TIMEOUT_SECONDS=10

"Unreachable" for every certificate is common in Kubernetes/ingress setups where the CertMate pod cannot dial the public/ingress IP directly (split-horizon DNS, an egress NetworkPolicy, or TLS terminated by an ingress controller / load balancer). If the target is merely slow, raise the probe budget or configure the port/protocol to match your topology.

See docs/kubernetes.md for the Kubernetes-specific note. (Reference: #263.)

DNS Provider Specific Issues

Cloudflare:

# Verify API token permissions
curl -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify" \
 -H "Authorization: Bearer your_token_here"

# Check zone access
curl -X GET "https://api.cloudflare.com/client/v4/zones" \
 -H "Authorization: Bearer your_token_here"

AWS Route53:

# Test AWS credentials
aws sts get-caller-identity

# Check Route53 permissions
aws route53 list-hosted-zones

Azure DNS:

# Verify service principal
az login --service-principal \
 -u $AZURE_CLIENT_ID \
 -p $AZURE_CLIENT_SECRET \
 --tenant $AZURE_TENANT_ID

# Check DNS zone access
az network dns zone list

Debug Mode

Enable debug logging for troubleshooting:

# Docker / gunicorn: CERTMATE_LOG_LEVEL. The stock docker-compose.yml does not
# pass it through, so add it under the certmate service's environment: and
# recreate the container
      - CERTMATE_LOG_LEVEL=DEBUG

# Development server: --log-level overrides CERTMATE_LOG_LEVEL (default INFO)
python app.py --debug --log-level DEBUG

# FLASK_ENV=production makes --debug refuse to start, by design
FLASK_ENV=development

Reporting Bugs

CertMate ships an in-app bug reporter that produces an actionable issue with one click. When an action fails and you're signed in as admin, the error toast surfaces a Report this issue button. Clicking it:

1. Calls GET /api/diagnostics/snapshot (admin-only) to collect sanitised operational state — CertMate version, Python version, OS, scheduler status, certificate count, DNS provider name, CA name, challenge type, storage backend, disk free, and the last 5 audit-log entries with all identifiers (resource_id, user, IP address, details, error) stripped.
2. Merges the snapshot with browser-side context (user agent, current page, viewport, the specific error envelope endpoint / status / code / message / hint).
3. Formats the result as Markdown and copies it to your clipboard.
4. Opens github.com/fabriziosalmi/certmate/issues/new?template=bug_report.md with the title pre-filled as [Bug] <status> <code> on <method> <endpoint>.

Paste the clipboard contents into the issue body, edit if you want, submit. The flow is fully manual — nothing leaves the install without your explicit click, and you can read everything before submitting (it's right there in the textarea). If your browser blocks the clipboard write or pop-ups, the reporter falls back to a modal with the markdown in an editable textarea and a clickable GitHub link.

Support Checklist (manual reporting)

If the in-app reporter is unavailable (you're not signed in as admin, or you hit the bug before reaching the UI), please provide:

• CertMate version/commit hash
• DNS provider being used
• Error messages from logs
• Steps to reproduce the issue
• Environment details (Docker, Python version, OS)

# Collect system information manually (equivalent of the in-app snapshot)
echo "=== CertMate Debug Info ==="
echo "Version: $(docker exec certmate python -c 'import modules; print(modules.__version__)')"
echo "Python: $(docker exec certmate python --version)"
echo "OS: $(docker exec certmate cat /etc/os-release | head -2)"
echo "Certbot: $(docker exec certmate certbot --version)"
echo "DNS Plugins: $(docker exec certmate pip list | grep certbot-dns)"
# DO NOT paste settings.json verbatim into a public issue — it contains
# credentials. The /api/diagnostics/snapshot endpoint returns a redacted
# subset; use it instead:
curl -sS -H "Authorization: Bearer $TOKEN" http://localhost:8000/api/diagnostics/snapshot | jq .

Documentation

Prefer watching? CertMate in 60 seconds is a video guide of 24 one-minute shorts, made from the documentation on certmate.org.

Complete Documentation Set

Online Resources

• API Documentation: http://your-server:8000/docs/ (Swagger UI)
• Alternative API Docs: http://your-server:8000/redoc/ (ReDoc)
• GitHub Repository: https://github.com/fabriziosalmi/certmate
• Docker Hub: https://hub.docker.com/r/fabriziosalmi/certmate
• Issue Tracker: https://github.com/fabriziosalmi/certmate/issues

Examples Repository

Check out our examples repository for:

• Production deployment configurations
• Integration scripts for popular tools
• Terraform modules
• Kubernetes manifests
• CI/CD pipeline examples

Community Contributions

We welcome contributions! Areas where we need help:

• Documentation - Tutorials, use cases, translations
• Testing - DNS provider testing, edge cases
• Integrations - New DNS providers, monitoring tools
• Features - UI improvements, API enhancements

Commercial Support & Consulting

Running CertMate in production? I offer paid support, custom development, and security consulting — certificate/TLS automation, hardening, WAF, and cloud detection & alerting. Reach out: [email protected].

Contributing

We love contributions! CertMate is an open-source project and we welcome:

Types of Contributions

• Bug Reports - Help us identify and fix issues
• Feature Requests - Suggest new functionality
• Documentation - Improve guides and examples
• Testing - Test new features and edge cases
• Code - Submit pull requests with improvements

Quick Start for Contributors

# Fork and clone the repository
git clone https://github.com/fabriziosalmi/certmate.git
cd certmate

# Create the development environment (.venv, from requirements.lock + the test requirements)
make setup
source .venv/bin/activate

# Lint, security-scan and test: the checks CI gates on (`make help` lists the rest)
make check

# Start development server
python app.py

A bare pytest is not the way to run the suite: it includes the Playwright ui tests, which need Docker. make test runs the selection CONTRIBUTING.md documents.

Contribution Guidelines

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

---