Falcosidekick

Description

A simple daemon for connecting Falco to your ecosystem. It takes a Falco events and forward them to different outputs in a fan-out way.

It works as a single endpoint for as many as you want Falco instances :

Table of contents

• Falcosidekick
  • Description
  • Table of contents
  • Outputs
    • Chat
    • Metrics / Observability
    • Alerting
    • Logs
    • Object Storage
    • FaaS / Serverless
    • Message queue / Streaming
    • Email
    • Database
    • Web
    • SIEM
    • Workflow
    • Traces
    • Other
    • Response engine
  • Installation
    • Localhost
      • With docker
      • With systemd
    • In Kubernetes
      • With Helm
    • Connect Falco
      • with falco.yaml
      • with Helm
    • Configuration
      • YAML File
  • Usage
  • Endpoints
  • Logs
  • Mutual TLS
  • Metrics
    • Golang ExpVar
    • Prometheus
    • StatsD / DogStatsD
  • Try
  • Development
    • Build
    • Quicktest
    • Test & Coverage
  • Author

Outputs

Falcosidekick manages a large variety of outputs with different purposes.

[!NOTE] Follow the links to get the configuration of each output.

Chat

• Slack
• Rocketchat
• Mattermost
• Teams
• Webex
• Discord
• Google Chat
• Zoho Cliq
• Telegram

Metrics / Observability

• Datadog
• Influxdb
• StatsD (for monitoring of falcosidekick)
• DogStatsD (for monitoring of falcosidekick)
• Prometheus (for both events and monitoring of falcosidekick)
• Wavefront
• Spyderbat
• TimescaleDB
• Dynatrace
• OTEL Metrics (for both events and monitoring of falcosidekick)

Alerting

• AlertManager
• Opsgenie
• PagerDuty
• Grafana OnCall

Logs

• Elasticsearch
• Loki
• AWS CloudWatchLogs
• Grafana
• Syslog
• Zincsearch
• OpenObserve
• SumoLogic
• Quickwit
• Datadog Logs
• Logstash

Object Storage

• AWS S3
• GCP Storage
• Yandex S3 Storage

FaaS / Serverless

• AWS Lambda
• GCP Cloud Run
• GCP Cloud Functions
• Fission
• KNative (CloudEvents)
• Kubeless
• OpenFaaS
• Tekton

Message queue / Streaming

• NATS
• STAN (NATS Streaming)
• AWS SQS
• AWS SNS
• AWS Kinesis
• GCP PubSub
• Apache Kafka
• Kafka Rest Proxy
• RabbitMQ
• Azure Event Hubs
• Yandex Data Streams
• MQTT
• Gotify

Email

• SMTP

Database

• Redis

Web

• Webhook
• Node-RED
• WebUI

SIEM

• AWS Security Lake

Workflow

• n8n

Traces

• OTEL Traces

Other

• Policy Report

Response engine

• Falco Talon

Installation

Run the daemon as any other daemon in your architecture (systemd, k8s deployment, swarm service, ...).

Localhost

With docker

Use the environment variables to set up the outputs:

docker run -d -p 2801:2801 -e SLACK_WEBHOOKURL=XXXX -e DATADOG_APIKEY=XXXX falcosecurity/falcosidekick

With systemd

• Download the latest release:

  VER=$(curl --silent -qI https://github.com/falcosecurity/falcosidekick/releases/latest | awk -F '/' '/^location/ {print  substr($NF, 1, length($NF)-1)}')
  wget -c https://github.com/falcosecurity/falcosidekick/releases/download/${VER}/falcosidekick_${VER}_linux_arm64.tar.gz -O - | tar -xz
  or
  wget -c https://github.com/falcosecurity/falcosidekick/releases/download/${VER}/falcosidekick_${VER}_linux_amd64.tar.gz -O - | tar -xz
  chmod +x falcosidekick
  sudo mv falcosidekick /usr/local/bin/
  

• Create the /etc/falcosidekick/config.yaml file, see Configuration.

• Create the systemd unit files /etc/systemd/system/falcosidekick.service:

  sudo touch /etc/systemd/system/falcosidekick.service
  sudo chmod 664 /etc/systemd/system/falcosidekick.service
  

  [Unit]
  Description=Falcosidekick
  After=network.target
  StartLimitIntervalSec=0
  
  [Service]
  Type=simple
  Restart=always
  RestartSec=1
  ExecStart=/usr/local/bin/falcosidekick -c /etc/falcosidekick/config.yaml
  
  [Install]
  WantedBy=default.target
  

• Reload systemd and start Falcosidekick:

  sudo systemctl daemon-reload
  sudo systemctl enable falcosidekick
  sudo systemctl start falcosidekick
  

• Check if Falcosidekick runs:

  curl localhost:2801/healthz
  

In Kubernetes

With Helm

See https://github.com/falcosecurity/charts/blob/master/charts/falcosidekick/README.md

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

helm install falcosidekick --set config.debug=true falcosecurity/falcosidekick

[!NOTE] You can also deploy falcosidekick as a dependency of the falco chart, the settings for the communication between falco and falcosidekick are automatically set. Just prefix all falcosidekick settings with falcosidekick.:

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

helm install falco --set falcosidekick.enabled=true falcosecurity/falco

Connect Falco

To connect Falco with Falcosidekick, you need to change it configuration as following:

with falco.yaml

If managing falco.yaml manually, set this:

json_output: true
json_include_output_property: true
http_output:
  enabled: true
  url: "http://localhost:2801/"

with Helm

If installing falco with Helm, set this (adapted to your environment) in your values.yaml :

falcosidekick:
  enabled: true

or

jsonOutput: true
jsonIncludeOutputProperty: true
httpOutput:
  enabled: true
  url: "http://falcosidekick:2801/"

or

jsonOutput: true
jsonIncludeOutputProperty: true
programOutput:
  enabled: true
  keepAlive: false
  program: "curl -d @- falcosidekick:2801/"

Configuration

Configuration is made by file (yaml) and env vars, both can be used but env vars override values from file.

YAML File

See config_example.yaml :

#listenaddress: "" # ip address to bind falcosidekick to (default: "" meaning all addresses)
#listenport: 2801 # port to listen for daemon (default: 2801)
debug: false # if true all outputs will print in stdout the payload they send (default: false)
customfields: # custom fields are added to falco events, if the value starts with % the relative env var is used
  # Akey: "AValue"
  # Bkey: "BValue"
  # Ckey: "CValue"
templatedfields: # templated fields are added to falco events and metrics, it uses Go template + output_fields values
  # Dkey: '{{ or (index . "k8s.ns.labels.foo") "bar" }}'
customtags: # custom tags are added to the falco events, if the value starts with % the relative env var is used
  # - tagA
  # - tagB
# bracketreplacer: "_" # if not empty, replace the brackets in keys of Output Fields
outputFieldFormat: "<timestamp>: <priority> <output> <custom_fields> <templated_fields>" # if not empty, allow to change the format of the output field. (default: "<timestamp>: <priority> <output>")
mutualtlsfilespath: "/etc/certs" # folder which will used to store client.crt, client.key and ca.crt files for mutual tls for outputs, will be deprecated in the future (default: "/etc/certs")
mutualtlsclient: # takes priority over mutualtlsfilespath if not empty
  certfile: "/etc/certs/client/client.crt" # client certification file
  keyfile: "/etc/certs/client/client.key" # client key
  cacertfile: "/etc/certs/client/ca.crt" # for server certification
tlsclient:
  cacertfile: "/etc/certs/client/ca.crt" # CA certificate file for server certification on TLS connections, appended to the system CA pool if not empty
tlsserver:
  deploy: false # if true, TLS server will be deployed instead of HTTP
  certfile: "/etc/certs/server/server.crt" # server certification file
  keyfile: "/etc/certs/server/server.key" # server key
  mutualtls: false # if true, mTLS server will be deployed instead of TLS, deploy also has to be true
  cacertfile: "/etc/certs/server/ca.crt" # for client certification if mutualtls is true
  notlsport: 2810 # port to serve http server serving selected endpoints (default: 2810)
  notlspaths: # if not empty, and tlsserver.deploy is true, a separate http server will be deployed for the specified endpoints
    - "/ping"
    # - "/metrics"
    # - "/healthz"

[!NOTE] For the confiuration of the outputs, see the docs.

Usage

Usage :

usage: falcosidekick [<flags>]

Flags:
      --help                     Show context-sensitive help (also try --help-long and --help-man).
  -c, --config-file=CONFIG-FILE  config file

Endpoints

Different endpoints (handlers) are available :

• / : main and default handler, your falco config must be configured to use it
• /ping : you will get a pong as answer, useful to test if falcosidekick is running and its port is opened (for healthcheck purpose for example). This endpoint is deprecated and it will be removed in 3.0.0.
• /healthz: you will get a HTTP status code 200 response as answer, useful to test if falcosidekick is running and its port is opened (for healthcheck or purpose for example)
• /test : (for debug only) send a test event to all enabled outputs.
• /debug/vars : get statistics from daemon (in JSON format), it uses classic expvar package and some custom values are added
• /metrics : prometheus endpoint, for scraping metrics about events and falcosidekick

Logs

All logs are sent to stdout.

2019/05/10 14:32:06 [INFO] : Enabled Outputs : Slack Datadog

Mutual TLS

Outputs with mutualtls enabled in their configuration require the client.crt, client.key and ca.crt filepaths to be configured in the mutualtlsclient_certfile, mutualtlsclient_keyfile and mutualtlsclient_cacertfile global parameter.

docker run -d -p 2801:2801 -e MUTUALTLSCLIENT_CERTFILE=/etc/certs/client/client.crt -e MUTUALTLSCLIENT_KEYFILE=/etc/certs/client/client.key -e MUTUALTLSCLIENT_CACERTFILE=/etc/certs/client/ca.crt -e ALERTMANAGER_HOSTPORT=https://XXXX -e ALERTMANAGER_MUTUALTLS=true -e INFLUXDB_HOSTPORT=https://XXXX -e INFLUXDB_MUTUALTLS=true -e WEBHOOK_ADDRESS=XXXX -v /localpath/myclientcert.crt:/etc/certs/client/client.crt -v /localpath/myclientkey.key:/etc/certs/client/client.key -v /localpath/ca.crt:/etc/certs/client/ca.crt falcosecurity/falcosidekick

Alternately the path where the client.crt, client.key and ca.crt files are stored can be configured in mutualtlsfilespath global parameter. (Important: file names must be preserved)

docker run -d -p 2801:2801 -e MUTUALTLSFILESPATH=/etc/certs -e ALERTMANAGER_HOSTPORT=https://XXXX -e ALERTMANAGER_MUTUALTLS=true -e INFLUXDB_HOSTPORT=https://XXXX -e INFLUXDB_MUTUALTLS=true -e WEBHOOK_ADDRESS=XXXX -v /localpath/myclientcert.crt:/etc/certs/client.crt -v /localpath/myclientkey.key:/etc/certs/client.key -v /localpath/ca.crt:/etc/certs/ca.crt falcosecurity/falcosidekick

In above example, the same client certificate will be used for both Alertmanager & InfluxDB outputs which have mutualtls flag set to true.

Metrics

Golang ExpVar

The daemon exposes the common Golang metrics and some custom values in JSON format. It's useful for monitoring purpose.

Prometheus

The daemon exposes a prometheus endpoint on URI /metrics.

See the docs for more info.

StatsD / DogStatsD

The daemon is able to push its metrics to a StatsD/DogstatsD server. See Configuration section for how-to.

See the statsd docs and dogstastd docs for more info.

Try

Run you daemon and try (from Falco's documentation):

curl -XPOST "http://localhost:2801/" -d'{"output":"16:31:56.746609046: Error File below a known binary directory opened for writing (user=root command=touch /bin/hack file=/bin/hack)","hostname": "localhost", "priority":"Error","rule":"Write below binary dir","time":"2019-05-17T15:31:56.746609046Z", "output_fields": {"evt.time":1507591916746609046,"fd.name":"/bin/hack","proc.cmdline":"touch /bin/hack","user.name":"root"}}'

Development

Build

make falcosidekick

Quicktest

Create a debug event

curl -X POST -H "Content-Type: application/json" -H "Accept: application/json" localhost:2801/test

Test & Coverage

make test

With Coverage

make test-coverage

Author

Thomas Labarussias (https://github.com/Issif)