How do I mask sensitive information in Log Analytics?

Requires OneAgent 1.147 or later

Your log data contains information that may be considered sensitive. Specific log messages may include user names, email addresses, URL parameters, and other information that you may not want to disclose. Log Analytics features the ability to mask any information by modifying the configuration file on each of the OneAgents that handles information you consider to be sensitive.

Mask personal data

Each OneAgent provides a ruxitagentloganalytics.conf file where you can set configuration options. To mask personal data in Log Analytics, a masking rule and a masking rule scope need to be added to the configuration file for each OneAgent. The Masking rule defines what data is masked, while the masking rule scope defines to what log files the rule is applied.

The masking is done in the OneAgent. Thus no personal data will be sent to and stored in the Dynatrace server.

For each OneAgent:

Log in to OneAgent and open the Windows command prompt or Linux terminal.

Edit the ruxitagentloganalytics.conf file, where you can set the configuration options:

  • On Linux, the file is located in:
    /var/lib/dynatrace/oneagent/agent/config/

  • On Windows, the file is located in:
    C:\ProgramData\dynatrace\oneagent\agent\config\

Create MaskingRule.
Append all your masking rules to the ruxitagentloganalytics.conf file.

This MaskingRule is a prerequisite for defining a scope. Whether you plan to use MaskingRulePGIDScope, MaskingRuleFileScope or both, first you must define the MaskingRule. This rule defines what to search for and how to replace it.

See MaskingRule

Define the scope where the MaskingRule should be applied.
Append all your defined scopes to the ruxitagentloganalytics.conf file.

You can create:

  • MaskingRulePGIDScope
    This scope specifies that the MaskingRule should be applied to the specified process group ID.

    See MaskingRulePGIDScope.

  • MaskingRuleFileScope
    This scope specifies that the MaskingRule should be applied to the specified file or file groups.

    See MaskingRuleFileScope.

  • Both MaskingRulePGIDScope and MaskingRuleFileScope In this case the MaskingRule rule is applied to each defined (associated with it) scope in the order in which they were defined.

Save the file.
The masking configuration will be applied (ready for use) in less than a minute.
Changes in the configuration don't require OneAgent restart.

Your log files will now have sensitive information masked based on the rules you've defined.

Priority for multiple rules

Each consecutive masking rule is applied to the result of the previous masking rule.

For example, assume that you have created three masking rules:

  1. The first masking rule is applied to the original log files.
  2. The second masking rule is applied to the result of the first masking rule.
  3. The third masking rule is applied to the result of the first and second masking rules.

The result is a log file sequentially processed by masking rules based on the rule order in the configuration file.

Entering log paths

MS Windows system logs

Masking sensitive information works on all Windows-based applications logs, but not on MS Windows operating system logs.

The log file paths should be presented in a specific way:

  • The log file path is case sensitive.
    Certain Windows log file paths are presented with the drive letter capitalized. Make sure that your Windows log file path reflects the capitalization of your Windows version.

  • No spaces are accepted between definition components.
    Valid: MaskingRuleFileScope=Username_replace,"C:\logs\errors.log"
    Invalid: MaskingRuleFileScope=Username_replace, "C:\logs\errors.log"

  • Do not use regular expressions or asterisk wild card (*) to indicate multiple log files. Instead, use the Dynatrace mapped names. For example, access.#.log.

MaskingRule

Syntax: MaskingRule=<name>,"<search_pattern>","<replace_string_or_hash>"

Masking rule name

A masking rule must have a unique name which may only consist of Letters [a-zA-Z], Numbers [0-9], and an _ (underscore) character.

Masking rule search pattern

A search pattern consists of any literal string including a {*} wildcard which specifies the part of the string that should be masked. The search pattern must be placed in double quotes (" ") and cannot begin with the {*} wildcard. The {*} wildcard may be used at the end or in the middle of a search pattern.

Input line Expression Result
text123_text234_text123_text234 {*}text not allowed (invalid)
text123text234text123text234 text1{*}text2 text1###text234text1###text234
text123text234text123text234 text{*} text123text234text123text###
text123text234text123text234 text{*}text1 text###text123text234

Masking rule replace string or hash

The replace string may either be a fixed phrase or an SHA-1 hash.

The provided string will literally replace the {*} wildcard in the search pattern result. Be sure to place the replace string in double quotes. The length of the replace string is fixed and does not correspond to the length of the pattern matched.

MaskingRulePGIDScope

Syntax: MaskingRulePGIDScope=<name>,<pgid_as_hexadecimal_value>

MaskingRulePGIDScope defines where the linked MaskingRule should be applied based on the specified Process Group ID.

  • The MaskingRulePGIDScope is linked with the MaskingRule through a common name.

    For example, MaskingRulePGIDScope=ApplicationErrorRule,0x6714946EC3018AB3 indicates that the MaskingRule named ApplicationErrorRule should be applied to all log files in the 0x6714946EC3018AB3 process group (the process group is represented as a hexadecimal value).

  • You can have multiple definitions of MaskingRulePGIDScope for the same MaskingRule.

  • The process group is represented as a hexadecimal value that must begin with either 0x or 0X to indicate that it is a hex value.

MaskingRuleFileScope

Syntax: MaskingRuleFileScope=<name>,"<path_to_physical_or_mapped_filename>"

MaskingRuleFileScope defines where the linked MaskingRule should be applied based on the specified file or file groups.

  • The MaskingRuleFileScope is linked with the MaskingRule through a common name.

    For example, MaskingRuleFileScope=ApplicationErrorRule,"/var/log/apache2/access.#.log" indicates that the MaskingRule named ApplicationErrorRule should be applied to all access.#.log files.

  • You can have multiple definitions of MaskingRuleFileScope using the same MaskingRule.

  • The file scope definition can point to a specific log file on the host (for example, /var/log/apache2/error.log) or a mapped name that is used for rotated files (for example, /var/log/apache2/access.#.log). In either case, be sure to place the file path in double quotes.

  • The mapped names that represent rotated files use the same wildcard syntax as the Log viewer.

    For example, the access.#.log mapped name represents files access.1.log, access.2.log, and any consecutive log files matching that pattern.