What log format does Log Analytics support?

Log Analytics can read and analyze:

  • Windows format event logs.
    System, Security, and Application logs are automatically discovered on hosts. Other custom event-log format logs can be added manually on the process group level.

  • Any plain-text log file as long as it contains a timestamp that meets these basic requirements:

    • A timestamp must appear at the beginning of each log entry.
      If you're using a JSON file, the timestamp is automatically detected through the time or timestamp tag.
      The only supported timestamp format in a JSON file is: ::*_year_*:-:*_month_*:-:*_day_*:T:*_time_*:0::
      For example: 2018-02-28T16:17:50.000

    • The timestamp date can be separated using either of the following:
      (space)
      / (slash)
      - (dash)
      . (period)
      T (Combined date and time in UTC, ISO 8601 format)

    • For the timestamp date, either month abbreviations or full names can be used in the date format.

    • Timestamp must include BOTH, the date and time.

    • Timestamp time is in the following format:

  [0-9]{1,2}:[0-9]{2}:[0-9]{2}((\.|,)[0-9]{1,9})?( *AM|PM)? *((GMT|UTC)?[+-][0-9]{2,4})?(GMT|CEST|CET|OTHER_TZ_ABBREVIATION)?

Examples of valid log file date formats

2018 04 06
2018/04/06
2018-04-06
2018.04.06
2018-04-06T18:46:19Z

Examples of valid log file time formats

12:23:34.123
12:23:34.123GMT+0100
12:23:34.123 GMT+0100
0:00:00 GMT
0:00:00   GMT+0100
00:00:00
12:13:01+0100
12:13:02.123
12:13:03.123123+0100
12:13:02,123
12:13:03,123123+0100
12:13:04GMT
12:13:05GMT+0100
12:13:06GMT+01
12:13:09+0100
12:13:10+01
12:13:12+0200
12:13:13.123pm
12:13:14.123 AM
12:13:15.123PM+01
12:13:16.123 AM+02
12:13:17CEST
12:13:18 CET
00:13:19

Examples of valid log file timestamps

2018-04-06 09:54:04.839 UTC
2018-04-06 11:01:19,625
2018/04/06 11:06:23 UTC
Apr  6 12:23:52
Apr-6 13:35:57.621

Incorrect date formats

Any log file containing an invalid timestamp will generate an Incorrect date format error. You can use the LogTimeSource setting in the OneAgent ruxitagentloganalytics.conf file and set it to AUTO-OS.

LogTimeSource=path:/var/ossec/logs/alerts/alerts.log, pattern: AUTO-OS

This should be used for logs with unsupported formats or with no timestamp. OneAgent will then stamp each logline with the operating system clock time.

For more information on the ruxitagentloganalytics.conf file, see Log Analytics configuration file.