The Rise of the Liquid Data Center – How the Cloud, Docker and Micro Services change the game

Our industry’s perception of what a data center is and how it is provisioned has significantly changed with the adoption of cloud and virtualization technologies. What started as pure infrastructure on-demand has evolved to a powerful technology stack which we refer to as the Liquid Datacenter.

The Liquid Data Center

So, what is the Liquid Data Center all about? In short the Liquid Data Center is the fully virtualized software-defined datacenter focussed on optimal application delivery. Next to dynamically provisioned infrastructure continuous deployment technologies like Ansible, Chef or Puppet are a key ingredient to liquid datacenters. Dynamic infrastructure management does not even have the value without dynamically provisioned application services.  Containers start to play a central role as the new smallest self-contained unit for shipping software seamlessly integrating infrastructure and application elasticity.

These highly dynamic and agile environments even go further. Software Defined Networks are the next logical step in making the network as programmable so they can adjust to infrastructure dynamics in real-time.

Taking all of this ingredients together results in a totally dynamic – or liquid – application environment.

Monitoring the Liquid Data Center

At Dynatrace we focus our efforts on building monitoring and management for exactly these highly dynamic environments. This makes us constantly rethink traditional monitoring approaches, adjust and enrich them to the need of emerging Liquid data centers. We identified a number of key aspects for “liquid” monitoring:

  • Automatic adjustment of monitoring configuration without manual intervention.
  • Real-time visualisation of multi-datacenter environments providing a full-time view.
  • Tracking changes in datacenter and application configurations.
  • Integrating data sources of all platforms seamlessly with “classical” monitoring data.
  • Automated analysis of problems caused by misbehaviour of the underlying infrastructure.

Dynatrace’s Liquid Datacenter initiative is still in it’s early stages and we expect it to evolve as we see more and more of the above technologies become commonly used in data centers and cloud operations. However, already today Dynatrace comes with wide support for a number of liquid environments.

Support for dynamic VMware environments

VMware can be seen as the pioneer of dynamically provisioned and managed infrastructure, spanning from virtual machines to storage and network. Dynatrace comes with an integrated view to easily understand and manage complex VMware environments visualizing the dynamics and analysing key changes like vMotion events.

Visualizing dynamic VMware environments
Visualizing dynamic VMware environments

Support for dynamic AWS Environments

Amazon Web Services is undoubtedly the pioneer in the cloud computing space. While they started with EC2 and an on-demand Infrastructure-as-a-Service offering the portfolio has been growing over the years adding Platform-as-a-Service offerings and services like S3 for storage, RDS for database service and many more.

Dynatrace AWS Monitoring  integrates information for each of these key cloud services seamlessly with application and real user information.

Overview of AWS services and infrastructure
Overview of AWS services and infrastructure

Support for dynamic Docker environments

Docker – and containers in general –  represent the latest innovation in this space offering a lightweight approach to deploy packaged application components across the entire continuous delivery pipeline without any infrastructure dependencies. The increasing popularity of Docker and also cloud services using Docker whether it is Azure or AWS Beanstalk show this hits a nerve in the industry.

Dynatrace provides automatic monitoring of Docker environments without any manual configuration. Docker specific metrics are extended with support for Apache Mesos, Kubernetes and Docker Swarm

Application and Infrastructure Monitoring for a Dockerized Service
Application and Infrastructure Monitoring for a Dockerized Service

Support for Software Defined Networks

Once we change and modify servers on the fly, we need ways for the infrastructure to keep up with dynamics, too. Software-defined networks (SDNs) provide the solution to this challenge. While SDNs are in the early stages of adoption, there is no doubt that they will place a central role in the datacenter of the future.

Dynatrace used a novel network monitoring approach that monitors network activity on the actual machine without requiring access to the physical network while providing in-depth network monitoring.

Deep Network Monitoring for Software-Defined Networks
Deep Network Monitoring for Software-Defined Networks

Micro Service Environments

Micro Services might not seem an obvious building block of a liquid datacenter. However, as the focus moves more and more to the infrastructure adjusting to application functionality, services become the central building block of the infrastructure.

Dynatrace visualizes how services are deployed and how they interact with each other and links this data to the underlying infrastructure. The beauty of this approach is, that it scales much better and also is more stable in web-scale architectures than typical infrastructure focussed visualizations. Virtual machine or runtime-based visualisation usually do not scale well with increasingly large environments.

Full-Stack Visualization of a Micro Service Environment
Full-Stack Visualization of a Micro Service Environment

What’s next?

If you are already running your liquid datacenter we would love to get your feedback on our ideas. If you think this is a topic that is still in notp-so-near future for you, I hope you find the ideas interesting. We still recommend that if you define your monitoring strategy for the next two years, to have these emerging requirements in mind. Eventually they will become your daily business.

Alois is Chief Technology Strategist of Dynatrace. He is fanatic about monitoring, DevOps and application performance. He spent most of his professional career in building monitoring tools and speeding up applications. He is a regular conference speaker, blogger, book author and Sushi maniac.