LoggerFileBackend

A simple Elixir Logger backend which writes logs to a file. It does not handle log rotation, but it does tolerate log file renames, so it can be used in conjunction with external log rotation.

Note The renaming of log files does not work on Windows, because File.Stat.inode is used to determine whether the log file has been (re)moved and, on non-Unix, File.Stat.inode is always 0.

Note If you are running this with the Phoenix framework, please review the Phoenix specific instructions later on in this file.

Configuration

LoggerFileBackend is a custom backend for the elixir :logger application. As such, it relies on the :logger application to start the relevant processes. However, unlike the default :console backend, we may want to configure multiple log files, each with different log levels, formats, etc. Also, we want :logger to be responsible for starting and stopping each of our logging processes for us. Because of these considerations, there must be one :logger backend configured for each log file we need. Each backend has a name like {LoggerFileBackend, id}, where id is any elixir term (usually an atom).

For example, let's say we want to log error messages to "/var/log/my_app/error.log". To do that, we will need to configure a backend. Let's call it {LoggerFileBackend, :error_log}.

Our config.exs would have an entry similar to this:

# tell logger to load a LoggerFileBackend processes
config :logger,
  backends: [{LoggerFileBackend, :error_log}]

With this configuration, the :logger application will start one LoggerFileBackend named {LoggerFileBackend, :error_log}. We still need to set the correct file path and log levels for the backend, though. To do that, we add another config stanza. Together with the stanza above, we'll have something like this:

# tell logger to load a LoggerFileBackend processes
config :logger,
  backends: [{LoggerFileBackend, :error_log}]

# configuration for the {LoggerFileBackend, :error_log} backend
config :logger, :error_log,
  path: "/var/log/my_app/error.log",
  level: :error

Check out the examples below for runtime configuration and configuration for multiple log files.

LoggerFileBackend supports the following configuration values:

• path - the path to the log file
• level - the logging level for the backend
• level_mode - how to interpret the level (:minimum (default) or :exact)
• format - the logging format for the backend
• truncate - truncate output to specified number of characters (default: 4096, use :infinity to disable)
• formatter - custom formatter module and options (defaults to Logger.Formatter)
• metadata - the metadata to include
• metadata_filter - metadata terms which must be present in order to log
• metadata_reject - metadata terms which must be present in order to do not log
• rotate - log rotation configuration (see Rotation section below)

Examples

Runtime configuration

Logger.add_backend {LoggerFileBackend, :debug}
Logger.configure_backend {LoggerFileBackend, :debug},
  path: "/path/to/debug.log",
  format: ...,
  metadata: ...,
  metadata_filter: ...

Application config for multiple log files

config :logger,
  backends: [{LoggerFileBackend, :info},
             {LoggerFileBackend, :error}]

config :logger, :info,
  path: "/path/to/info.log",
  level: :info

config :logger, :error,
  path: "/path/to/error.log",
  level: :error

Configuring log format

The format is configured separately from the backend definition. Each backend needs its own configuration block:

# Backend configuration
config :logger,
  backends: [{LoggerFileBackend, :app}]

# Format configuration for that backend
config :logger, :app,
  path: "/path/to/app.log",
  format: "$date $time [$level] $message\n"

Available format patterns: $date, $time, $level, $levelpad, $message, $metadata

Output Truncation

By default, log output is truncated to 4096 characters to prevent extremely long lines from filling up logs. You can customize this:

config :logger, :debug,
  path: "/path/to/debug.log",
  truncate: :infinity  # disable truncation

or

config :logger, :compact,
  path: "/path/to/compact.log",
  truncate: 500  # truncate to 500 characters

When truncation occurs, the output is appended with (truncate).

Log Level Filtering

By default (level_mode: :minimum), the level option acts as a minimum level filter. For example, setting level: :error logs error, critical, and alert messages.

To log only messages of a specific level, set level_mode: :exact:

config :logger, :errors_only,
  path: "/path/to/errors_only.log",
  level: :error,
  level_mode: :exact

Log Rotation

You can configure automatic log rotation by setting the rotate option:

config :logger,
  backends: [{LoggerFileBackend, :debug}]

config :logger, :debug,
  path: "/path/to/debug.log",
  level: :debug,
  rotate: %{max_bytes: 10400, keep: 5}

This creates log files with a maximum size of 10,400 bytes and keeps up to 5 rotated files:

debug.log
debug.log.1
debug.log.2
debug.log.3
debug.log.4

Configuration options:

• max_bytes - maximum size in bytes before rotation (required)
• keep - number of rotated files to keep (required)

Custom Formatters

You can use a custom formatter instead of the default Logger.Formatter:

config :logger, :json_log,
  path: "/path/to/json.log",
  formatter: {LoggerJSON.Formatters.Basic, [metadata: [:domain, :module]]}

The formatter is specified as a tuple {module, options} where the module's format/2 function will be called with the log event and options.

Filtering specific metadata terms

This example only logs :info statements originating from the :ui OTP app; the :application metadata key is auto-populated by Logger.

config :logger,
  backends: [{LoggerFileBackend, :ui}]

config :logger, :ui,
  path: "/path/to/ui.log",
  level: :info,
  metadata_filter: [application: :ui]

This example only writes log statements with a custom metadata key to the file.

# in a config file:
config :logger,
  backends: [{LoggerFileBackend, :device_1}]

config :logger, :device_1,
  path: "/path/to/device_1.log",
  level: :debug,
  metadata_filter: [device: 1]

# Usage:
# anywhere in the code:
Logger.info("statement", device: 1)

# or, for a single process, e.g., a GenServer:
# in init/1:
Logger.metadata(device: 1)
# ^ sets device: 1 for all subsequent log statements from this process.

# Later, in other code (handle_cast/2, etc.)
Logger.info("statement") # <= already tagged with the device_1 metadata

Additional Phoenix Configurations

Phoenix makes use of its own mix.exs file to track dependencies and additional applications. Add the following to your mix.exs:

def application do
    [applications: [
      ...,
      :logger_file_backend,
      ...
      ]
    ]
end

defp deps do
  [ ...
    {:logger_file_backend, "~> 0.0.10"},
  ]
end

Copyright and License

Copyright (c) 2014 Kurt Williams

This library licensed under the MIT license.

Image Attribution

"log" by Matthew Weatherall from the Noun Project.