The ChatOps Approach to Software Delivery

Dynatrace synthetic monitoring runs tests against a web application and provides users with information on application performance and availability. This includes alerts whenever performance or availability go below defined thresholds.
However, the process by which alerts are received, evaluated, and diagnosed is changing rapidly. Some of this change is driven by our new Retry on Error feature, which automatically retries a test that for some reason produces an error.

But advances in communication and collaboration are the true impetus behind this change. In particular, the ChatOps concept is getting relevant information into the hands of those best qualified to analyze, diagnose, and fix the issue fast.

How does it work? ChatOps tools like Slack, HipChat and VictorOps let organizations quickly form groups for the purpose of instantaneous communication. You might already be a part of a tiger team responsible for addressing immediate issues. Your organization may have associate teams, in DevOps, or performance, for example, as the issue becomes better defined.

Further, these teams can easily exchange screen captures, other images, support documents, and other files that may be relevant to the issue. Testers may share test cases and results, while operations professionals can supply real time updates from other alerts or monitoring tools.

ChatOps tools also let team members kick off bots or other programs that can perform more detailed analyses or collect more data, or even take corrective actions. This can accelerate fixing an issue that has a known solution.

Consider an alert generated by a high-volume ecommerce site from a remote server farm that indicates that response time SLAs aren’t being met. This goes to the on-duty administrator in the operations room, a lead front-end and back-end developer for the site, the DBA, and a security specialist. This team confirms the validity of the alert within minutes, and also confirms that it isn’t a network or server error.

They pass it off to the performance teams, which also includes a developer and tester. The tester examines pre-production performance tests, and makes a suggestion on changing some configuration settings. An operations supervisor checks with other team members, then kicks off a bot that changes the appropriate settings.

Issue resolved.  Total elapsed time:  20 minutes.

For effective use, ChatOps tools should be integrated with a testing and monitoring solution such as Dynatrace Synthetic. Dynatrace Synthetic has an API that enables users to write custom integrations, but in this case the product will soon have prepackaged integrations with leading ChatOps solutions, including Slack, HipChat and VictorOps.

This means that alerts coming from Dynatrace synthetic monitoring can be fed into a ChatOps tool, making them immediately available for review and analysis by a cross-functional ChatOps team. Within seconds, the alert is on the screens of a dozen functional specialists prepared to work together to resolve the issue.

Of course, thanks to the new Retry on Error feature, Dynatrace synthetic monitoring can automatically re-run a test, confirming whether an issue exists or was a spurious result. The combination of Retry on Error with ChatOps integration means that teams can collaborate in addressing alerts faster than ever.

If you are using ChatOps tools, connect them to your Dynatrace synthetic solution and let us know how much time it saves you.

Peter Varhol is community evangelist and a product manager for Dynatrace Synthetic Monitoring. He has been an evangelist, product manager, technology journalist, and university professor in his career, and speaks regularly at industry technical conferences. He has graduate degrees in computer science, applied mathematics, and psychology. You can contact him at