What is keptn, how it works and how to get started!

“Keptn is an open source enterprise-grade control plane for cloud-native continuous delivery and automated operations.” That’s the high-level introduction you will find on www.keptn.sh. But what does this really mean? Let me explain it in my words – or – just watch my latest YouTube Performance Clinic Tutorial on “Getting Started with keptn and Dynatrace” I ran with Dirk Wallerstorfer who leads the keptn Dev Team.

In our performance clinic, we pushed a simple node.js based microservice app we had available in 4 different builds on Docker hub through a two-stage Kubernetes environment. Keptn is taking care of:

  1. Automated Deployments and Automated Tests
  2. Automated Quality Gates through Pitometer – a Quality Gate as Code Service
  3. Automated Operations by integrating with full stack monitoring such as Dynatrace
In our example keptn automates the deployment of 4 different versions of my app. It stops bad builds at a quality gate and rolls-back bad deployments in case they end up in production
In our example, keptn automates the deployment of 4 different versions of my app. It stops bad builds at a quality gate and rolls-back bad deployments in case they end up in production

If you want to see keptn pushing notifications to Slack as an artifact is promoted through its release lifecycle or see the integration with Dynatrace on how keptn pushes deployment, test execution, and quality gate information to the correct Dynatrace entities, then its time to click on the play button to watch our keptn Performance Clinic!

Why we must build keptn?

We must build keptn because we have seen too many organizations struggle to deliver the promise of cloud native: delivery business value, deliver it fast & reliable and deliver it with better quality!

And we have facts to prove this thanks to our Autonomous Cloud Survey conducted by my colleague Katalin. Less than 5% of surveyed enterprises that work on cloud native projects deliver fast & with good quality:

The Autonomous Cloud Survey tells us that most cloud native projects are not yet delivering on the promise of cloud native
The Autonomous Cloud Survey tells us that most cloud native projects are not yet delivering on the promise of cloud native

First: Keptn solves Enterprise-Wide Continuous Delivery!

The first problem keptn solves is enterprise-wide automated continuous delivery! What does that mean? Most of the teams we talked with told us that they spend too much time and resources on building pipelines, custom integrations between tools and managing their own data stores. In large organizations, we have seen MANY teams spread across labs or geographies, building their own CD pipelines and spend a significant amount of time maintaining and nurturing this “soon to become monolithic legacy pipeline code”!

Keptn provides automated orchestration of all Continuous Delivery & Operation Tools without any custom coding. All done through Configuration following a GitOps approach!
Keptn provides automated orchestration of all Continuous Delivery & Operation Tools without any custom coding. All done through configuration following a GitOps approach!

Second: Keptn is not just another CD Tool! It solves CD & CO!

After presenting the above slide, I often get to hear this comment: “How does this differ to JenkinsX, GitLab, Harness, Argo, Tekton, CodeFresh, … – isn’t this just another CD Tool?”

Keptn not only orchestrates Continuous Deployment, but it also orchestrates Continuous or Automated Operations. I really like the analogy that Alois Reitbauer uses when he explains the two core principles of keptn. He compares it with Launch and Mission Control at NASA when they launch a rocket:

Launch Control == Continuous Deployment

At NASA, Launch Control is responsible until the launch timer hits 00:00. Until that point, they can abort the mission based on the data they have!

With Continuous Deployment, the release team is responsible to safely bring an artifact (e.g: a new container image) into production. Along the way they have the chance to “abort” that deployment through different means, e.g: automated quality gates or by leveraging blue/green or canary releases in production. At any point though, they can stop an artifact from being rolled out to the whole user base.

Keptn, therefore, supports automated multi-stage unbreakable delivery pipelines with automated quality gates and the support for production-safe deployments!

Mission Control == Automated Operations

At NASA, Mission Control takes over once the rocket lifts off. They can’t abort the mission but need to handle any problem that comes their way to ensure that astronauts are safely getting up and down from space.

With Automated Operations, the operations team is responsible to do whatever it takes to minimize the impact to end users in case a problem arises. That will happen by e.g: scaling up in case more resources are needed, redirect traffic for certain users, clear log directories, … In order to react fast, it is mandatory to automate as many of these tasks.

Keptn, therefore, supports event-driven runbook automation to remediate problems detected in production as fast as possible to minimize the impact on end users.

Keptn supports both Continuous Deployment and Automated Operations as the life of an artifact doesn’t stop after deployment!
Keptn supports both Continuous Deployment and Automated Operations as the life of an artifact doesn’t stop after deployment!

I hope this answers the question that keptn is not just another CD tool!

Now – if you want to – you can use your existing CD tools and let them do the CD part. Then you can either pass control to keptn once CD is done – or – let keptn simply talk with your existing CD tool once a new artifact is ready to be deployed. The event-driven keptn architecture allows all these combinations – giving you the freedom to pick the tools that make the most sense for your organization, technology stack, and existing investment! We already have a team in Germany that is building a GitLabs keptn service – watch out for this and other contributions on https://github.com/keptn-contrib

Now – let’s talk a bit more about the architecture!

Keptn Architecture Explained

I have spent the past couple of weeks following the instructions on the keptn’s doc pages installing pretty much every version from 0.1.0 until the latest 0.2.2 (as of the writing of this blog). Since version 0.2.0 keptn builds on top of Knative, a platform for container-based serverless workloads. Knative allows keptn to interact with the actual DevOps tools to deploy, test, evaluate, promote, configure, notify, auto-remediate … as keptn pushes an artifact through your multi-stage delivery pipelines. This brings me to first slides I typically use explaining two core concepts of keptn: Uniform (=Tooling) and Shipyard (=Pipeline Definition):

Keptn uses knative to integrate tools into the orchestrated delivery pipeline. The set of tools used are defined through a Uniform. The pipeline is defined through a Shipyard!
Keptn uses knative to integrate tools into the orchestrated delivery pipeline. The set of tools used are defined through a Uniform. The pipeline is defined through a Shipyard!

Technically, uniforms are already available with keptn 0.2.x – but – it is not yet convenient to tell keptn which tools are part of your uniform. A more convenient way of defining a Uniform is coming in one of the next keptn releases (check out the keptns backlog on GitHub).

Uniforms are a VERY POWERFUL concept as it defines which tools you want to use for your pipelines without having to integrate these tools with each other as keptn is taking care of this. Even better is that you can update your uniforms and therefore add / remove tools as time goes by, e.g: add a security tool or replace your deployment tool with a single keptn CLI command. This for me is a very compelling feature and eliminates maintaining custom pipeline code!

In my presentations, I always show how the keptn CLI command for “wearing” a uniform will most likely look like (stay tuned and follow the progress keptn makes). The following animation shows how you will be able to define and apply a uniform and how you currently create a new project:

After installing keptn we can use the cli to define our uniform (=set of tools) and our shipyard (=pipeline)
After installing keptn we can use the cli to define our uniform (=set of tools) and our shipyard (=pipeline)

Now – let me give you a bit of background information and narrative to this animation:

  • Uniform: as of keptn 0.2.x you don’t have to define your keptn uniform. When following the installation instructions in the doc keptn comes pre-installed with all available keptn services such as GitHub, Slack, Jenkins, ServiceNow and Pitometer.
  • Keptn services will be installed in the keptn Kubernetes namespaces and will automatically be subscribed to the relevant knative channels so that these tools get triggered when keptn sends different events such as new-artifact, deploy, run-test, … If you want to write your own keptn service in order to integrate your own tools check out the documentation around Write your own keptn service.
  • Shipyard: The example above shows the shipyard file that is part of the sockshop sample application that the keptn team is using for all their hands-on tutorials. Simply follow the onboarding a service (keptn 0.2.2 link) use case in the keptn doc where you will use that same file creating a stage pipeline (dev, stage, prod) with different deployment and testing strategies.
  • Istio: Istio gets automatically installed by keptn. Keptn uses Istio for traffic routing to allow keptn to support the different deployment strategies such as blue/green, direct … keptn ensures that Istio is correctly installed and will also automatically create all helm charts for your services. All these helm charts and configuration files get automatically stored in your Git – following the GitOps approach!
  • Git: Keptn follows GitOps. What does this mean? When you install keptn you have to give it a GitHub Organization and GitHub credentials. When you create a new project keptn will create a new GitHub repository for that project storing the shipyard file in the master branch. Keptn will also create branches in that GitHub repo, one for each stage in your shipyard file, e.g: dev, stage, prod. Any configuration file created or updated by keptn will be stored in the respective branch. This ensures that Git always reflects the source of truth. And as you will see later on – keptn will be triggered when config file change and keptn will do changes to the Git repo in order to promote or rollback artifacts.

Keptn in Action: Give it a try

If you follow the instructions on the keptn website or the keptn GitHub repo, you end up creating a new project and onboarding a new service. Feel free to either do it with the carts service that the keptn doc uses, feel free to follow the example I used in my keptn Performance Clinic or simply use your own services. Once you have your service onboarded, you can tell keptn about a new artifact which will kick-off the full Continuous Delivery and Automated Operations workflow as shown in the following animation:

Keptn orchestrates the whole lifetime of an artifact including deployment, quality gates and auto-remediation
Keptn orchestrates the whole lifetime of an artifact including deployment, quality gates, and auto-remediation

Join the keptn Community

Last but not least: Our goal is to contribute keptn to CNCF (Cloud Native Computing Foundation) and for that we need to build a strong community. Help us with feedback, join the keptn Slack channel, build your own services and tell us about them so we can put them into our keptn-contrib GitHub org. Also, make sure to check out our community meetings as the team is regularly giving a LIVE update from the keptn headquarters! 😊

Stay updated