We’re continuously working to extend the reach of Dynatrace log analytics beyond OneAgent-instrumented data sources (for details, see our recent syslog and API import log-streaming announcement). Another source of valuable log files that’s particularly important for cloud applications deployed in AWS is CloudTrail. You can now use Environment ActiveGate (version 1.157 and above) to retrieve CloudTrail logs that are stored in your AWS S3 buckets.
AWS CloudTrail is Amazon’s cloud service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail provides event history for your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services. This event history simplifies security analysis, resource change tracking, and troubleshooting.
Troubleshooting insights are particularly valuable for Dynatrace users as actions logged in CloudTrail (for example, reconfiguration events) often have a significant impact on the performance of your applications. Imagine the effect that a simple reconfiguration (for example, changing the parameters of an auto-scaling group) can have on the performance of your application. Such events are logged in CloudTrail and can now be made available in Dynatrace CloudTrail analysis in the context of analyzed problems.
ActiveGate version 1.157 and above takes care of the secure and reliable transportation of this log data to your Dynatrace cluster (either SaaS or Managed). Note that due to the fact that OneAgent isn’t used to receive these logs, this functionality requires central log storage, which isn’t provided with the free tier of Dynatrace Log Analytics.
To configure CloudTrail log import to central log storage
- From the navigation menu, select Settings > Cloud and virtualization > AWS.
- Click the Edit button (pencil icon) of the AWS instance you want to configure.
- Set the Import CloudTrail logs from SQS-based S3 buckets switch to the On position.
- Type the ARN of the SQS queue that will contain the events within the log data from your S3 bucket. For details on this, see how to configure SQS on the AWS side.
All functionality that’s available for other log types, such as querying, grouping, searching for tags, and names is also possible for CloudTrail logs. You can also analyze mixtures of logs that are received both with using this approach and the standard OneAgent approach.
Note the special device names that are used for this type of log file in the example below.
The most powerful way of leveraging this new source of information is by analyzing CloudTrail logs in the context of problem time frames. You can drill down to the Log Viewer directly from any problem details page (or any host or process overview page when viewed in the context of an analyzed problem). In this way, you can easily focus on the time frame that’s important for analysis. Whether your entire application infrastructure relies on AWS resources or your infrastructure only partially relies on AWS resources, you can now add AWS CloudTrail logs to your Dynatrace analysis to see what events were recorded around the time that problems first occurred.
Note: Definition of log events, and log content masking and filtering within within AWS CloudTrail logs will be provided in a future Dynatrace release.