The fluentd-configmap.yaml provided has been designed to be easily customizable. Generally you'll want to avoid modifying anything other than the
output.conf sections of the configmap.
The customizing log destination document explains how to configure where logs are sent.
To add additional filters or parsers, add them to the
extra.conf section in the fluentd-configmap.yaml. The
extra.conf already has a very brief example of how to add an extra field to log entries, and a more detailed example is shown below.
Fluentd routes event based on tags. Events flowing through Fluentd can be routed based on the value of the
<filter> directives. The configuration tags events using the following conventions:
The existing configuration already does additional post processing based on some of these tags.
For example, the host's
kubelet.service log's are parsed by matching on the tag
systemd.kubelet, and we do the same for parsing the Docker engine's logs using the tag
systemd.docker. These filters set their
key_name parameter to
MESSAGE which is the actual field name for the log message when it originates from journald.
Similarly, we parse the logs of the kube-apiserver, kube-scheduler, and other controller components by performing a wildcard match on the tag:
kube.kube-system.**. This filter set its
key_name parameter to
log, which is the field for log messages originating from Docker.
The following configuration will parse the
frontend component's logs from the guestbook example app deployed in the "Deploy your second app" tutorial. To use it, copy and paste the snippet below fluentd-configmap.yaml's
extra.conf section (make sure you indent to the correct level).
<filter kube.default.php-redis> @type parser # Fluentd provides a few built-in formats for popular and common formats such as "apache" and "json". format apache2 key_name log # Retain the original "log" field after parsing out the data. reserve_data true # The access logs and error logs are interleaved with each other and have # different formats, so ignore parse errors, as they're expected suppress_parse_error_log true </filter> <filter kube.default.php-redis> @type parser format apache_error key_name log reserve_data true # The access logs and error logs are interleaved with each other and have # different formats, so ignore parse errors, as they're expected suppress_parse_error_log true </filter>
Once you've updated your config, you will need to delete and recreate your
fluentd pods in order for the configuration to take effect:
$ kubectl delete pods --namespace logging -l app=fluentd
Once your pods have restarted, any new logs being parsed should be using the new configuration.