Set log alert

Build and run a DQL query to retrieve logs that trigger an alert.

Create a log processing rule to extract the additional information from the log content.

Build an alert by creating a log event.

Check if there are logs that match your alert.

Go to Logs or Logs & Events (latest Dynatrace).

On the Logs and events page, turn on Advanced mode.

Select Copy for the code sample below.

fetch logs
| filter matchesValue(process.technology, "nginx")
| filter matchesValue(loglevel, "ERROR")
| filter matchesPhrase(content, "Connection refused")
| fields timestamp,content, process.technology
| parse content, "LD '[error] ' INT:error_number '#' INT LD 'Connection refused)' LD 'client:' SPACE? IPADDR:client_ip LD 'request:' SPACE? DQS:http_request"
| sort timestamp desc

Paste the query into the query edit box and select Run query.

Select a row in the Search results from the query above to display a side panel with details about the selected row.

Select Create processing rule to add a rule.

Define a Rule name—In this example, Copy and paste the following name.

NGINX connection refused: extract error, client IP, request

Define a Matcher query.

Your matcher query is a one-line query that contains conditions set using matcher-specific functions. Your matcher can be extracted directly from the filter fields in your DQL query.

For this example, Copy and paste the following matcher:

matchesValue(loglevel, "ERROR") and matchesPhrase(content, "Connection refused") and matchesValue(process.technology, "nginx")

Define a Processor definition. In this example, Copy and paste the following processor definition:

PARSE(content, "LD '[error] ' INT:error.number '#' INT LD 'Connection refused' LD 'client:' SPACE? IPADDR:client_ip LD 'request:' SPACE? DQS:http_request")

Select Copy below and paste the sample log into Paste log/JSON sample.

{
"content":"2023/04/13 08:53:56 [error] 32#32: *100507 connect() failed (111: Connection refused) while connecting to upstream, client: 123.45.67.890, server: , request: \"GET /cart HTTP/1.1\", upstream: \"http://01.222.3.44:55/cart\", host: \"HOST-1\", referrer: \"https://HOST-1/cart\", dt.trace_id: 1abc2, dt.span_id: d123e, dt.trace_sampled: true",
"process.technology": "nginx",
"loglevel": "ERROR"
}

Select Test the rule and examine Test result.

Select Copy below and paste the sample log into Paste log/JSON sample.

{
"content": "2023/04/13 08:53:56 [error] 32#32: *100507 connect() failed (111: Connection refused) while connecting to upstream, client: 123.45.67.890, server: , request: \"GET /cart HTTP/1.1\", upstream: \"http://01.222.3.44:55/cart\", host: \"HOST-1\", referrer: \"https://HOST-1/cart\", dt.trace_id: 1abc2, dt.span_id: d123e, dt.trace_sampled: true",
"timestamp": "2023-04-17T14:20:20.222000000 +0000",
"process.technology": "nginx",
"loglevel": "ERROR",
"error.number": "32",
"client.ip": "123.45.67.89",
"http.request": "GET /cart HTTP/1.1"
}

Select Test the rule and examine Test result.

Go to Settings and select Log Monitoring > Events extraction.

Select Add log event.

Enter a Summary. For this example, you can Copy below and paste into Summary.

Alert on Nginx connection refused

Enter a Log query. For this example, you can Copy below and paste into Log query.

matchesValue(loglevel, "ERROR") and matchesPhrase(content, "Connection refused") and matchesValue(process.technology, "nginx")

Optionally, you can add more conditions. For example, to include only logs with error numbers, Copy and paste the following matcher:

matchesValue(loglevel, "ERROR") and matchesPhrase(content, "Connection refused") and matchesValue(process.technology, "nginx") and isNotNull (error_number)

In the Event template section, enter a Title and Description, and set Event type.

Select Save changes.

Go to Problems.

Find the problem you defined during custom log event creation. For this example, you can Copy below and paste into Filter by.

Nginx Connection Refused

Select Analyze logs.

On the Logs and events Simple Mode page, select Run query.

In Results, you can see the errors that were captured according to the criteria that you set.

Sample results: