Fluent Bit is an open source and multiplatform log processor and forwarder that allows you to collect data/logs from different sources, unify and send them to multiple destinations and is fully compatible with Docker and Kubernetes environments.
While Fluent Bit has configurable HTTP output, Dyntrace can be configured as the target log management and analytics environment for your data.
This is intended for users who want to:
- Complement observability data in Dynatrace, extend the Davis AI engine, and enable App Owners, DevOps and SRE teams to centrally analyze all log data.
This enables you to:
- Leverage any of Fluent Bit input plugins, which connect to numerous data sources to get logs and events from your application into Dynatrace where you can immediately benefit from Dynatrace AI-driven root cause analysis.
Quickly search, explore, and analyze log and event data from a multitude of sources.
Fluent Bit is the preferred solution when resource consumption is critical because it is a lightweight component.
In this example, we look at the AWS Fargate context, where FireLens makes it easy to set up Fluent Bit more quickly.
Ingest AWS Fargate logs with Fluent Bit
When creating a new task definition using the AWS Management Console, the FireLens integration section makes it easy to add a log router container. Just pick the built-in Fluent Bit image.
Next, edit the container in which your app-generating logs are running. In the “Storage and Logging” section, select “awsfirelens” as the log driver.
The settings for the log driver should point to the log ingest API of your SaaS tenant. Note that you normally need to provide two headers for Fluent Bit: content type and authorization token. As FireLens supports only one header, you can pass the token as part of the URL. Your configuration for AWS FireLens should have the following:
- Name: http
- TLS: on
- TLS.verify: Off
- Format: json
- Header: Content-Type application/json; charset=utf-8
- Host: {your-environment-id}.live.dynatrace.com
- Port: 443
- URI: /api/v2/logs/ingest?api-token={your-API-token-here}
- Allow_Duplicated_Headers: false
- Json_Date_Format: iso8601
- Json_Date_Key: timestamp
To avoid publishing the token in plaintext, use AWS Secrets Manager to manage the token.
Referr to the AWS samples repository for task definition json with Dynatrace configuration.
As your application starts publishing logs, you can view them in Dynatrace.