End-to-end request monitoring for Go applications: No code changes required

…to boldly Go where no one has monitored before.

Dynatrace is proud to announce request-level visibility of your Go-based applications—fully automated and with no code changes required!

That’s right. The Dynatrace Go crew returned from another adventure into the widely uncharted expanse of Go standard libraries. It explored binary-only systems, made first contact with new Go library packages, and placed their probes advisedly, to finally return with another Dynatrace first. You can now monitor your Go applications and any third-party Go application requests, end to end, with no changes to a single line of Go source code.

Our inspiration

Recently, watching a video of a GothamGo 2017 monitoring session, the presenter demonstrated the painful challenges of manual code instrumentation and asked, “Does it really have to be so complicated?” To this, my answer is absolutely not. It’s even less complicated than you might imagine.

InfluxDB example revisited

In a recent blog post, we introduced Go monitoring using an InfluxDB example. We’ll now pick up from that example and utilize an in-house Java application to load InfluxDB with high-frequency query requests.

After installing InfluxDB directly from the influxdata website, Dynatrace OneAgent automatically monitors the InfluxDB daemon process called influxd.

Our previous Go monitoring blog post explained the details of the Go metrics that are available in Processes view (see below). However, we’ve introduced something new since that last post: Dynatrace now identifies the services that are hosted by the influxd process.

Let’s drill down into the metrics and insights that Dynatrace provides for the InfluxDB service.

As always, Dynatrace provides service details related to response times, instances, and executed web requests.

Service flow reveals more about the transactions and gives you easy access to the details of individual requests, including the related PurePaths.

As you can see, OneAgent has faithfully captured the details of this web request, including request and response headers.

Impressive, right? But what about your own applications and client requests?

Introducing foxy

To extend our example, we’ve implemented a forwarding proxy called foxy to add to the setup.

Our foxy is a basic, application-specific forwarding proxy. It accepts requests on port 8888 and forwards to the default InfluxDB port, 8086. The implementation is fewer than 25 lines of Go source code.

Besides an alarming lack of error handling, the foxy source code has another remarkable omission: no source code instrumentation whatsoever.

But with a quick go build foxy.go and reconfiguration of the load test application’s destination port number, the new setup is ready to run.

Dynatrace identifies foxy as a Go-specific process and starts monitoring it automatically. You can see the web request service hosted by the foxy process almost immediately.

The Service flow and PurePath views reflect the new setup. The PurePath provides basic code-level insights into the foxy request-processing mechanics.

Go monitoring features

• Auto-injection and instrumentation of any Go executable
• Go-specific metrics
  • Suspension time
  • Committed, Used, Idle, and Live heap memory sizes
  • Go routine count (application, system)
  • Go managed memory heap: Off-heap, Stack, and overall Committed / Used memory
  • Allocated Go object count
  • Garbage collector invocation count
  • Go runtime system call count, CGO call count
  • Global Go routine run queue size
  • Parked, Out of work, and Overall worker thread count
  • Idle scheduling context count
• Incoming / Outgoing Web-request monitoring (available since OneAgent v1.139)

What’s next

Rumor has it that the restless Dynatrace OneAgent Go crew has already set course on a new mission to full source code level visibility and an exploration of Cloud Foundry metrics. So stay tuned for the next episode.

Attention

• Request-level visibility requires OneAgent v1.139 or higher.
• Go support is still in beta. To enable it, go to Settings > Monitoring > Monitored technologies and set the Go switch to the On position.