Business event processing

powered by Grail

Business event processing uses the capabilities of the business event ingest pipeline and Dynatrace Query Language (DQL) to refine your data.

Within the business event ingest pipeline, you can define how your data will be processed further.

• You can create a processing rule that transforms incoming events.
• You can also specify the retention time for your events by assigning them to buckets.

If you create multiple rules, the rules are performed in the order in which you have defined them.

Configure business events processing

To configure Processing

1. In the Dynatrace menu, go to Settings > Business Analytics > Ingest Pipeline > Processing
2. Select Add rule and name your rule.
3. Add a Matcher to your rule by pasting your matcher-specific DQL query.
4. Select Add item to choose the fields you will transform via the processor definition. Determine the data types and names for your fields. You can choose if the rule is Optional, Is Array, or Read-only.
5. Add a Processor definition. Processor definitions are instructions that tell Dynatrace how to transform the data you’ve selected in the matcher DQL query. It is created using processing commands, functions and pattern matching (Dynatrace Pattern Language) that allows you to add, modify or remove incoming records.
6. Select Save changes.

Business events processing examples

In the below examples, you can see how ingest pipeline processing can be used to meet your technical and administrative requirements.

• Example 1. Add a new calculated field to the pipeline.
  As a user, you need to add a calculated dollar trading volume value to the pipeline.

  jsoncopydownload
  {
   "action":"buy",
   "accountId":6,
   "amount":10,
   "instrumentId":1,
   "price":157.025
  }

  • Rule name: Add field
  • Matcher: matchesValue(action, "buy")
  • Transformation fields:

  Name	Type	Optional	is Array	Read-only
  amount	double	false	false	true
  price	double	false	false	true

  • Processor definition: FIELDS_ADD(trading_volume: price*amount)

• Example 2. Mask your data.
  You need to hide the CVV field on your credit card in an incoming JSON payload.

  jsoncopydownload
  {
   "action": "payment",
   "creditCardNumber":"5570001112223344",
   "valid":"12/27",
   "cvv":"001"
  }

  • Rule name: Mask field
  • Matcher: matchesValue(action, "payment")
  • Transformation fields:

  Name	Type	Optional	is Array	Read-only
  cvv	string	false	false	false

  • Processor definition: FIELDS_ADD(cvv: SHA1(cvv))

• Example 3. Drop event attribute.
  You need to drop the birthdate field in an incoming JSON payload.

  jsoncopydownload
  {
   "action": "newUser",
   "firstName":"Frank",
   "lastName": "Underwud",
   "birthDate": "10.01.1967"
  }

  • Rule name: Drop field
  • Matcher: matchesValue(action, "newUser")
  • Transformation fields:

  Name	Type	Optional	is Array	Read-only
  birthDate	string	false	false	false

  • Processor definition: FIELDS_REMOVE(birthDate)

• Example 4. Parse nested JSON.
  You need to parse attributes from a nested JSON in order to have them as top-level attributes in Grail.

  jsoncopydownload
  {
    "action":"sell",
    "details":{
       "accountId":6,
       "amount":10,
       "instrumentId":1,
       "price":157.025
     }
   }

  • Rule name: Parse field
  • Matcher: matchesValue(action, "sell")
  • Transformation fields:

  Name	Type	Optional	is Array	Read-only
  details	string	false	false	false

  • Processor definition: PARSE(details,"JSON{INTEGER:accountId,INTEGER:amount,INTEGER:instrumentId,DOUBLE:price}(flat=true)") | FIELDS_REMOVE(details)