Send syslogs to remote endpoint

Example 1: Configure Rsyslog on Linux Ubuntu to forward syslogs to a remote server.

Add the following line to the syslog daemon configuration file /etc/rsyslog.conf (UDP protocol):

*.* @<fluentd host IP>:5140

*.* - instructs the daemon to forward all messages to the specified Fluentd instance listening on port 5140 and <fluentd host IP> needs to point to the IP address of Fluentd.

@@ - if you are using TCP, type two @ symbols

Example 2: Configuring the F5 BIG-IP system to log to a remote syslog server (11.x - 17.x)

Refer to F5 BIG-IP documentation for procedures regarding remote Syslog configuration.

Decorate syslogs with meaningful attributes

The Dynatrace software intelligence platform and its Davis AI engine depend on context-rich, high-quality data. You can provide the context for your data ingested via Generic log ingest API that supports a set of keys and semantic attributes. You can also provide custom attributes that don't require indexing in Dynatrace Grail database.

Example: Add log.source attribute based on the source of syslogs in Fluentd.

The syslog message often needs additional context to differentiate sources while analysing. In this example, there are two separate syslog endpoints exposed in Fluentd. One for linux syslogs and the second for f5 syslogs. This helps decorate log streams with meaningful log.source attribute. Fluentd configuration file would look like this:

<source>
    @type syslog
    port 5140
    bind 0.0.0.0
    tag system-linux
</source>

<source>
    @type syslog
    port 5141
    bind 0.0.0.0
    tag system-f5
</source>

Add log.source attribute based on fluentd tag.

<filter system-linux.**>
    @type record_transformer
    <record>
    log.source "linux syslogs"
    </record>
</filter>

<filter system-f5.**>
    @type record_transformer
    <record>
    log.source "f5 syslogs"
    </record>
 </filter>

Refer to Fluentd record_transformer filter plugin documentation for more details.

Set up the flow from Syslog producer over Fluentd to Dynatrace with the following steps:

1. Get a Dynatrace API token with the logs.ingest (Ingest Logs) scope

2. Deploy Fluentd

• Fluentd can also run as a DaemonSet in a Kubernetes cluster. Built-in resiliency ensures data completeness and consistency even if Fluentd or an endpoint service goes down temporarily.

3. Enable Fluentd to accept incoming Syslog messages.

• The in_syslog input plugin enables Fluentd to retrieve records via the Syslog protocol on UDP or TCP. It is included in Fluentd's core so no additional installation is needed in this step.

4. Use the Dynatrace Fluentd plugin to stream logs to Dynatrace cluster.

• Open source Dynatrace Fluentd plugin uses generic log ingestion API to send logs to dynatrace.

5. Point your devices to send syslogs to Fluentd