We are bringing the best of Tectonic to Red Hat OpenShift to build the most secure, hybrid Kubernetes application platform.
The fluentd-configmap.yaml provided has been designed to be easily customizable. Generally you'll want to avoid modifying anything other than the fluentd.conf
and output.conf
sections of the configmap.
The customizing log destination document explains how to configure where logs are sent.
To add additional filters or parsers, add them to the extra.conf
section in the fluentd-configmap.yaml. The extra.conf
already has a very brief example of how to add an extra field to log entries, and a more detailed example is shown below.
For details on Fluentd post-processing, check out the Fluentd filters and parsers documents.
Fluentd routes event based on tags. Events flowing through Fluentd can be routed based on the value of the tag
using <match>
and <filter>
directives. The configuration tags events using the following conventions:
systemd.<unit-name-here>
kube.<namespace-name>.<container-name>
kube-apiserver-audit
The existing configuration already does additional post processing based on some of these tags.
For example, the host's kubelet.service
log's are parsed by matching on the tag systemd.kubelet
, and we do the same for parsing the Docker engine's logs using the tag systemd.docker
. These filters set their key_name
parameter to MESSAGE
which is the actual field name for the log message when it originates from journald.
Similarly, we parse the logs of the kube-apiserver, kube-scheduler, and other controller components by performing a wildcard match on the tag: kube.kube-system.**
. This filter set its key_name
parameter to log
, which is the field for log messages originating from Docker.
The following configuration will parse the frontend
component's logs from the guestbook example app deployed in the "Deploy your second app" tutorial. To use it, copy and paste the snippet below fluentd-configmap.yaml's extra.conf
section (make sure you indent to the correct level).
<filter kube.default.php-redis>
@type parser
# Fluentd provides a few built-in formats for popular and common formats such as "apache" and "json".
format apache2
key_name log
# Retain the original "log" field after parsing out the data.
reserve_data true
# The access logs and error logs are interleaved with each other and have
# different formats, so ignore parse errors, as they're expected
suppress_parse_error_log true
</filter>
<filter kube.default.php-redis>
@type parser
format apache_error
key_name log
reserve_data true
# The access logs and error logs are interleaved with each other and have
# different formats, so ignore parse errors, as they're expected
suppress_parse_error_log true
</filter>
Once you've updated your config, you will need to delete and recreate your fluentd
pods in order for the configuration to take effect:
$ kubectl delete pods --namespace logging -l app=fluentd
Once your pods have restarted, any new logs being parsed should be using the new configuration.