Log content transformation

powered by Grail

Generic log ingestion automatically transforms status, severity, level, and syslog.severity severity keys to the loglevel attribute.

The input values for the status, severity, level, and syslog.severity severity keys are transformed (transformation is not case sensitive) into output values for the loglevel attribute based on the following mapping:

Additionally, for each log event, a status attribute is created with a value that is a sum of loglevel values based on the following grouping:

For example: The level severity key in the generic log ingestion API request parameter contains the value serious.

1. The level severity key is transformed into the loglevel attribute with the serious value mapped to SEVERE based on the above table.
2. The loglevel attribute containing the SEVERE value is grouped into status attribute. Based on the grouping table above, the status attribute will contain the ERROR value.
3. For the log event details, the log viewer will report the following:

• status - ERROR
• loglevel - SEVERE

Log data queue

You can customize the log data queue properties by editing the custom.properties file (see Configuration properties and parameters of ActiveGate on your ActiveGate to set the following values:

plaintextcopydownload
[generic_ingest]
#disk_queue_path=<custom_path> # defaults to temp folder
#disk_queue_max_size_mb=<limit> # defaults to 300 MB

Example

In this example, the API request ingests log data that will create a log event with defined log attributes content, status, service.name, and service.namespace.

The API token is passed in the Authorization header.

The response contains response code 204.

Curl

bashcopydownload
curl -X POST \
   https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest \
  -H 'Content-Type: application/json; charset=utf-8' \
  -H 'Authorization: Api-Token dt0c01.abc123.abcdefjhij1234567890' \
  -d '[
    {
    "content": "Exception: Custom error log sent via Generic Log Ingest",
    "status": "error",
    "service.name": "log-monitoring-tenant",
    "service.namespace": "dev-stage-cluster"
    }
]'

Request URL

bashcopydownload
https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest

Response content

plaintextcopydownload
Success

Response code

204

Automatically added attributes

During the log ingestion via OneAgent, the following attributes are added automatically:

General attributes (via OneAgent)

• container.name
• container.image.name
• container.id
• dt.host_group.id
• dt.kubernetes.cluster.id
• dt.kubernetes.cluster.name
• dt.kubernetes.node.system_uuid
• dt.process.name
• event.type
• host.name
• k8s.cluster.name
• k8s.namespace.name
• k8s.pod.name
• k8s.pod.uid
• k8s.container.name
• k8s.deployment.name
• log.iostream
• loglevel
• log.source
• process.technology
• span_id
• status
• trace_id
• web_server.iis.site_id
• web_server.iis.site_name
• web_server.iis.application_pool

dt entity model attributes (via OneAgent)

• dt.entity.cloud_application
• dt.entity.cloud_application_instance
• dt.entity.cloud_application_namespace
• dt.entity.container_group
• dt.entity.container_group_instance
• dt.entity.host
• dt.entity.kubernetes_cluster
• dt.entity.kubernetes_node
• dt.entity.process_group
• dt.entity.process_group_instance
• dt.source_entity