Automatic log enrichment
powered by Grail
Dynatrace enables you to transform logs ingested via OneAgent.
Transform the OneAgent-ingested logs
During log ingest via OneAgent, the severity of logs is determined.
Log severity
By default, the log event severity is detected through a keyword search performed on the first 100 characters of the log content, within the first two lines of text.
To adjust these limits
- In the Dynatrace menu, go to Settings.
- Select Log Monitoring > OneAgent settings.
- Adjust the following settings as needed.
- Severity search chars limit is the number of characters in each log line, starting from the first character, to search for severity.
- Severity search lines limit is the number lines in each log entry, starting from the first line, to search for severity.
There are 19 keywords that correspond with 9 severity levels as per the table below:
Keyword | Severity level |
---|---|
trace | DEBUG |
debug | DEBUG |
fine | DEBUG |
finer | DEBUG |
finest | DEBUG |
notice | NOTICE |
info | INFO |
information | INFO |
warn | WARN |
warning | WARN |
severe_warning | WARN |
severe | SEVERE |
err | ERROR |
error | ERROR |
crit | CRITICAL |
critical | CRITICAL |
alert | ALERT |
fatal | EMERGENCY |
emerg | EMERGENCY |
A match occurs and severity is determined when
- The keyword found is a single word/phrase from the above list, and it is preceded and followed by a space.
- The keyword found is a single word/phrase from the above list, and it is preceded and followed by one of the four predefined non-alphanumeric symbols, as in the example below:
[error]
{error}
{{error}}
<error>
Transform all types of logs
Additionally, for each log event, a status
attribute is created with a value that is a sum of loglevel
values based on the following grouping:
Included loglevel values | Combined status attribute value |
---|---|
|
|
|
|
|
|
|
|
For example:
The level
severity key in the generic log ingestion API request parameter contains the value serious
.
- The
level
severity key is transformed into theloglevel
attribute with theserious
value mapped toSEVERE
based on the above table. - The
loglevel
attribute containing theSEVERE
value is grouped intostatus
attribute. Based on the grouping table above, thestatus
attribute will contain theERROR
value. - For the log event details, the log viewer will report the following:
- status -
ERROR
- loglevel -
SEVERE
Attributes added during a log ingest via OneAgent
During the log ingestion via OneAgent, the following attributes are added automatically:
General attributes (via OneAgent)
container.name
container.image.name
container.id
dt.host_group.id
dt.kubernetes.cluster.id
dt.kubernetes.cluster.name
dt.kubernetes.node.system_uuid
dt.process.name
event.type
host.name
k8s.cluster.name
k8s.namespace.name
k8s.pod.name
k8s.pod.uid
k8s.container.name
k8s.deployment.name
log.iostream
loglevel
log.source
process.technology
span_id
status
trace_id
web_server.iis.site_id
web_server.iis.site_name
web_server.iis.application_pool
dt entity model attributes (via OneAgent)
dt.entity.cloud_application
dt.entity.cloud_application_instance
dt.entity.cloud_application_namespace
dt.entity.container_group
dt.entity.container_group_instance
dt.entity.host
dt.entity.kubernetes_cluster
dt.entity.kubernetes_node
dt.entity.process_group
dt.entity.process_group_instance
dt.source_entity
Attributes automatically extracted from log content via OneAgent
OneAgent automatically extract attributes found in form \[!dt key1=value1, key2=value2]
and the section itself is removed from content.
For instance:
127.0.0.1 - [21/Oct/2021:10:33:28 +0200] GET /index.htm HTTP/1.1 404 597 [!dt dt.trace_id=aa764ee37ebaa764ee37eaa764ee37e, dt.span_id=b93ede8b93ede8]
will result in additional dt.trace_id
and dt.span_id
attributes for log record and actuall content sent will be:
127.0.0.1 - [21/Oct/2021:10:33:28 +0200] GET /index.htm HTTP/1.1 404 597