Top DevOps Tools We Love

The word DevOps is a portmanteau of “development” and “operations”. However, anyone who knows “The Phoenix Project” by Gene Kim et al. will agree that its radius is much wider than what the term suggests: it’s a melting pot that combines principles from Agile Software Development and Lean Manufacturing with the aim to reduce friction and improve collaboration by honing a culture of communication, respect and trust across Product Management, Development, Testing, IT Operations, Information Security, Marketing, Sales and Technical Support departments, as well as to continuously improve organizational performance. Aside from the social, DevOps also has a technological side to it: tooling. Picking the right tools that support or even fully automate software development, testing and delivery processes empower everyone involved in getting things done more effectively and efficiently, lessen the risk of the human factor, reduce waste and allow for a vastly improved time to market.

“The DevOps movement is built around a group of people who believe that the application of a combination of appropriate technology and attitude can revolutionize the world of software development and delivery.”
Stephen Nelson-Smith, founding member of the DevOps movement in What Is This DevOps Thing, Anyway?

In this post, I will present to you our DevOps toolset, a collection of tools that we enjoy using at Dynatrace for our internal processes because they foster collaboration amongst our Product Management, Development, IT Operations, and Technical Support teams, allow us to build more quality into our products, and support us in establishing better feedback loops.

Development

GitHub

GitHub is arguably the best tool for sharing and collaborating on code. On our Dynatrace GitHub Organization, we provide the Dynatrace OneAgent SDK to the public.

Virtual Machines

Vagrant, Packer and VeeWee

Vagrant is a software for configuring and running virtualized environments on top of desktop virtualization software such as VirtualBox, KVM or VMware, but can also be extended to run environments on various cloud providers. Vagrant offers integration with configuration management tools such as Ansible, Chef or Puppet, and spinning up a virtual machine is as simple as saying `vagrant up` on the command line.

We use Packer to automatically create base images for Vagrant for a wide variety of Linux and Microsoft Windows operating systems. Packer is inspired by, and meanwhile replaces, VeeWee, a tool created by DevOps movement founder Patrick Debois. However, VeeWee still comes in handy since it provides an impressive library of base image templates that can easily be converted to Packer configurations using the VeeWee-to-Packer conversion tool by Mitchell Hashimoto, the creator of Packer and Vagrant.

In combination with a configuration management tool, VeeWee, Packer, and Vagrant allow our Technical Support teams to literally select an operating system and a Dynatrace version from a drop-down list and recreate customer-like environments quickly and automatically for targeted resolution of customer issues. It also allows us to create tailor-made demo environments that we use for product enablement or presentations.

Configuration Management

Ansible, Chef and Puppet

Ansible, Chef and Puppet are amongst the most prominent configuration management and deployment automation tools. While we believe that each one of these tools has its particular strengths and weaknesses, we also think that Ansible’s in-order execution of tasks, its extensive and easily extensible modules library, plugin system, as well as its agentless architecture that allows for the orchestration of zero-downtime releases make it the best pick for orchestrating the provisioning of instances inside our Dynatrace offering.

Test Kitchen

Whether on-prem or in the cloud: we don’t rely on just a single tool when it comes to provisioning Dynatrace. Test Kitchen is an awesome tool that allows you to automatically test your configuration management and deployment automation scripts against a list of platforms (Vagrant images). A configuration file specifies a set of scripts (could be Ansible, Chef or Puppet), a list of platforms and a list of test suites. Test suites can be expressed in Bats, Cucumber, RSpec, Serverspec and even in good old Bash scripts. All our configuration management and deployment automation scripts are automatically validated using Test Kitchen. You can check out and learn from our integration tests in any of our offerings (see above).

Elasticsearch and Kibana

Elasticsearch is a free and open source real-time search and analytics engine with a REST API. With its distributed, lightning-fast Apache Lucene index at its core, Elasticsearch powers companies such as Foursquare, GitHub and StackOverflow. At Dynatrace, we use Elasticsearch with Kibana, a framework for building dashboards on top of Elasticsearch data, to analyze volatility of our configuration management and automated deployment scripts, that is, to see whether our Ansible, Chef and Puppet scripts produce deterministic results in all executions: did all services come up correctly? Are all ports being served?

Fault Injection Testing

Docker

Based on Linux Containers, Docker provides frameworks to build, ship and run distributed applications inside multiple containerized Linux systems on a single Linux control host. While we see a lot more potential in this technology, we currently use it in an experiment to analyze Collector load balancing and fail-over scenarios. The Dynatrace Collector is a critical component in every Dynatrace setup that continuously relays masses of data from the OneAgents (the actual data collectors) to the Dynatrace Server for real-time analyses. We use the ingenious D3.js data visualization engine for visualizing our fault injection testing experiments.

What about you? What’s your favorite stack of DevOps tools? Which interesting use-cases can you share with us? Let me know if you have a particular interest in any of the above use-cases and we might open-source our tools to you. Don’t forget to come back once in a while, as I will gradually update this list with more interesting bits.