Observability, OpenTracing, and OpenTelemetry. These are just a fraction of the technology buzzwords you’ll find as you Google your way around the internet. In fact, most (if not all) of those terms are present on any technology website you visit. It’s gotten to the point where these terms are so prevalent that anybody who may not know the full scope of the topic is afraid to ask. If you happen to fall into this category, fear not, this post has got you covered! More specifically, we’re going to take a look at OpenTelemetry and clear up any confusion about what exactly it is and the benefits it can provide.
What is OpenTelemetry
In order to fully understand the purpose of OpenTelemetry, we must first take a brief look at “Observability”. Loosely defined, Observability boils down to inferring the internal health and state of a system by looking at the external data it produces, which most commonly are logs, metrics, and traces. Here at Dynatrace, we take Observability to the next level, by displaying all information in context and adding in code-level details, end-user experience, and entity relationships all while feeding this data into our AI engine, Davis®, to produce actionable insights. If you’re interested in a deeper dive on Advanced Observability, check out our Observability page for more information.
At this point, you might be thinking, “what does Observability have to do with OpenTelemetry”? The answer is in the data collection, and more specifically, how the logs, metrics, traces are collected. While collecting this data is nothing new, having a common format for how this data is collected and sent is where OpenTelemetry comes into play. OpenTelemetry is currently a Cloud Native Computing Foundation (CNCF) sandbox project with the ultimate goal of providing a unified set of vendor-agnostic libraries/APIs for collecting and sending data to compatible backends. Since the OpenTelemetry project started just over a year ago, many vendors (including Dynatrace) have come on board to help contribute to the specification in order to make rich data collection easier and more consumable.
How does OpenTelemetry benefit me?
Now that we have a basic understanding of what OpenTelemetry is, let’s dive into the benefits it provides. As mentioned in the section above, collecting application telemetry is nothing new. However, the collection mechanism and format are almost never consistent from one application to another. And as you can imagine, this inconsistency is a nightmare for developers and operations personnel who are just trying to understand the health of an application.
OpenTelemetry provides a de-facto standard for adding observability to cloud-native applications. This means that companies don’t need to spend valuable time developing a mechanism for collecting critical application telemetry, and they can spend more time delivering new features. It is akin to how Kubernetes became the de-facto standard for container orchestration. This broad adoption has made it easier for organizations to adopt container deployments since they didn’t need to first build an enterprise-grade orchestration platform. Using Kubernetes as the analog for what OpenTelemetry can become, it is easy to see the benefits it can provide to the entire industry.
What happened to OpenTracing and OpenCensus?
OpenTracing became a CNCF project back in 2016, with a goal of providing a vendor-agnostic specification for distributed tracing, offering developers the ability to trace a request from start to finish by instrumenting their code. The OpenCensus project was made open source by Google back in 2018, based on Google’s Census library that was used internally for gathering traces and metrics from their distributed systems. Like the OpenTracing project, the goal of OpenCensus was to give developers a vendor-agnostic library for collecting traces and metrics.
As you can see, we now had two competing tracing frameworks, which were informally called “The Tracing Wars”. Usually, competition is a good thing for end-users since it breeds innovation. However, in the open-source specification world, competition can lead to poor adoption, contribution, and support. Going back to my earlier mentioned Kubernetes example, imagine how much more disjointed and slow-moving container adoption would be if everybody was using a different orchestration solution. In order to avoid this, it was announced at KubeCon 2019 in Barcelona that the OpenTracing and OpenCensus projects would converge into one project called OpenTelemetry and join the CNCF. Which brings us to today where OpenTelemetry released its first beta version in March 2020.
What are the components of OpenTelemetry?
OpenTelemetry consists of a few different components as depicted in Figure 1 below. Let’s take a high-level look at each one from left to right:
Figure 1: OpenTelemetry Components (Source: Based on OpenTelemetry: beyond getting started)
- APIs are one of the core components of OpenTelemetry. These are language-specific (ex. Java, Python, .Net, etc…) and provide the basic “plumbing” for adding OpenTelemetry to your application.
- The SDK is also a language-specific component and is the middleman that provides the bridge between the APIs and the Exporter. The SDK allows for additional configuration such as request filtering and transaction sampling.
- The in-process Exporter allows you to configure which backed(s) you want the telemetry sent. The Exporter decouples the instrumentation from the backend configuration. This makes it easy to switch backends without the pain of re-instrumenting your code.
- The Collector, while not technically required, is an extremely useful component to the OpenTelemetry architecture because it allows greater flexibility for receiving and sending the application telemetry to the backend(s). This becomes especially important in enterprise environments where there are firewalls aplenty. The Collector has two deployment models:
- An agent that resides on the same host as the application (ex. binary, daemonset, sidecar)
- A standalone process completely separate from the application
- Since OpenTelemetry is just a specification about collecting and sending telemetry, it still requires a Backend to receive and store the data. There are numerous backends available with different capabilities depending on your specific needs.
What are the plans for the future?
As of this writing, OpenTelemetry is still in its beta phase and currently only supporting traces and metrics, with logs still in the initial planning stages. The plan for the second half of 2020 is focusing on stabilization of the core components while marching down the path to General Availability, in addition to ensuring a smooth transition from OpenTracing and OpenCensus. If that sounds like a lot… that’s because it is! However, OpenTelemetry is one of the most popular projects in the entire CNCF ecosystem with a lot of support from the entire technology community.