Five reasons why you should monitor your Cloud Foundry Auctioneers

Dynatrace currently monitors close to 13,000 Cloud Foundry processes, including Gorouters and Auctioneers on more than 4,500 hosts. Gorouter processes route incoming traffic to the appropriate components. Auctioneer processes take care of distributing applications and tasks onto Diego cells.

As Auctioneers are crucial to achieving successful application- and task-placement onto Diego cells, their overall health is crucial, not only to platform operators but also to application owners—unhealthy Cloud Foundry components can have a negative impact on your applications.

With a “single pane of glass” monitoring experience in mind, Dynatrace has once again extended its Cloud Foundry monitoring capabilities by enabling you to keep a close eye on the following key performance indicators related to Auctioneer processes (which are also recommended by Pivotal):

• Failed application instance placements
• Failed task placements
• Time to fetch Diego cell state
• Successful application instance starts

Five reasons why you should start monitoring your Cloud Foundry Auctioneers today

As Dynatrace now detects Cloud Foundry Auctioneer metrics, you can understand how well (or poorly) Auctioneers handle the distribution of applications and tasks onto Diego cells:

1. You can now inspect failed application instance placements and failed task placements and understand whether your Diego cells have enough resources or not. These metrics are also an indicator of whether Diego cells are going back and forth between healthy and unhealthy states.
2. Increases in the average time to fetch states from Diego cells indicate that application staging requests to Diego might be failing. This metric also points out whether Diego cells themselves are healthy or not.
3. When observing high or low activity trends in successful application instance starts over an extended period, it’s recommended to scale Cloud Foundry foundation resources up or down as needed.
4. Successful application instance starts not only provide a sense of overall system activity levels, they can also indicate a significant amount of container churn. And even though Cloud Foundry can cope with a huge amount of churn in replacing applications, if all Diego cells run at maximum capacity, rescheduling applications will only work if additional Diego cells have been added.
5. With Dynatrace, you’re able to improve Cloud Foundry foundation management with deeper insights into platform metrics. You can benefit from Cloud Foundry foundation monitoring today without being required to intercept the Firehose stream of application logs and Cloud Foundry component metrics. You’re also not required to deploy an additional nozzle—all you need is a single OneAgent to collect all Gorouter and Auctioneer metrics at the process level.

Secret sauce

The secret sauce of Dynatrace Cloud Foundry foundation monitoring is clearly our Dynatrace Go monitoring support, which provides code-level insight not only into your own Go applications but also into third-party Go-based applications like Cloud Foundry’s Gorouter and Auctioneer components. This means that the OneAgent Go code module collects Gorouter and Auctioneer metrics, without deploying an additional nozzle.

Gorouter metrics are HTTP metrics (i.e., web request response codes) that are captured by our web request sensor. For detecting Auctioneer metrics and further Cloud Foundry metrics, we added support for the component diego-logging-client (i.e., metron/loggregator v2).

Two examples of Auctioneer metric usage in the lab

In our first example of Auctioneer metric usage, we noticed an increase in failed application instance placements during platform upgrades of one of our internal Pivotal Cloud Foundry PAS foundations. It was interesting to see that during the upgrade, two Diego cells were barely enough to properly run all the applications we had deployed in this environment. But because we had insights into the Auctioneer, we were quickly able to address the issue by scaling the foundation and adding another Diego cell.

In the second example, we were able to use Auctioneer metrics during acceptance tests where we re-push all of our Cloud Foundry testing applications. The Auctioneer metrics pointed us towards continuously failing application instance placements in our testing foundation. An investigation of the issue showed that because the re-push of all testing applications was done in parallel, there was high Diego cell CPU usage. As we wanted to avoid scaling the foundation, we decided to adapt the way we re-push testing applications to solve the issue. The images below illustrate diagnosing the issue of failing application instance placements. Davis®, the Dynatrace AI causation engine detects Diego cell CPU usage. Gorouters and Auctioneers are also integrated with Davis at the process level.

Auctioneer and Gorouter metrics are not only available on the Cloud Foundry foundation overview page, but also for custom charting.

How to start monitoring your Cloud Foundry Auctioneers

With the release of Dynatrace version 1.164 and OneAgent version 1.163, you’ll find Cloud Foundry Auctioneer metrics as part of the Cloud Foundry overview pages. The Cloud Foundry Auctioneer metrics are also available for custom charting. We’ve improved the presentation of Gorouter metrics to address the scalability requirements of large Cloud Foundry environments. With the new and improved Gorouters card, you’ll be able to understand relevant Gorouter metrics as well as problematic Gorouter processes. If you haven’t yet connected your Cloud Foundry foundations to Dynatrace, please review the guidelines in Dynatrace Help.

What’s next

We’re committed to improving Dynatrace Cloud Foundry monitoring capabilities with each product release. For example, we’ll start working on introducing Cloud Foundry Organization quota metrics soon. However, we’d like to include you in the process of prioritizing upcoming features. We’ve listed all planned enhancements on Dynatrace Community. Please share your feedback and ideas there as comments to the post.

Save the date

We’ll demo the latest Dynatrace product enhancements focusing on Gorouter and Auctioneer platform component metrics at the upcoming Performance Clinic: Mastering Cloud Foundry with Dynatrace – Episode 2 on Wednesday, March 27 at 3:00 PM CET / 10:00 AM EST. Meanwhile, you can check out the recording of Mastering Cloud Foundry with Dynatrace – Episode 1.