Progressive delivery done right with Dynatrace and OpenFeature

“Move fast and break things.”

How many times have you heard that? Know what? No one likes that. No one likes breaking things. It’s more like: “Move fast and break things—only because you don’t know how to avoid it.”

Granted, as a soundbite, that is nowhere near as catchy. But progressive delivery is essentially the solution to that problem: it enables you to move fast and avoid breaking things.

How does an organization, especially if they’re not yet doing continuous integration and continuous delivery (CI/CD), either at all or well, “move fast and not break things”? The answer: Progressive delivery with feature flags and observability.

Progressive delivery encompasses multiple methodologies where DevOps teams introduce new features to small user subsets (or cohorts) slowly or gradually in a controlled manner. By doing so, teams can closely observe new functionality and hopefully roll it out to end users more safely and with fewer errors.

Progressive delivery with feature flagging delivers the following benefits:

• Decreases risk
• Provides faster feedback loops
• Decouples the mechanics of deploying from the act of a release
• Increases business agility and confidence

Feature flagging is one method of progressive delivery but there are multiple accepted techniques, for example:

• A/B (or blue/green) releases
• Canary releases
• Feature flagging

You can deliver both canary and blue/green deployments without feature flags. Just copy and paste your code, make changes to the “green” version, and deploy “green” alongside “blue.” Then find some way to direct “some” of your users to “green” and leave everyone else on “blue.”

There are problems with this manual approach, though, for example, the following:

• Manual changes make the tech stack more complex: You also have to manually update routing rules and other changes.
• It’s also a blunt instrument. It’s hard to be selective with what constitutes “some” users. Usually, it ends up being a percentage-based thing.
• You need to pay for, manage, and maintain two copies of the application.
• If you do need to rollback, all new functionality in that release is revoked (even if it still works)

A better way is to organize features and functions using feature flags, which gives you fine-grained control over what to release, when, and to whom.

How feature flagging enables progressive delivery

At their most basic, feature flags are an if/else statement that can alter the application behavior in real-time, at runtime without redeploying an application.

This technical capability brings the following benefits:

• Realtime application behavior changes without code redeployment
• No additional infrastructure required: The application is the same; no additional infrastructure or configuration is required
• Lower cost: No additional infrastructure means no additional cost
• Faster experimentation: Enabling and disabling functionality with the flip of a switch means you can try out many more versions and hypotheses. You spend time experimenting rather than waiting for CI/CD pipelines to run.

Feature flag use cases

You can use feature flagging in lots of situations, but here are a few examples:

Limiting access: For your eyes only

• Most start their feature flag journey with scenarios like these:
• Users in Australia get the new features first, before other countries
• US users see a localized version of the content
• Staff get access to additional features
• Upon login, the “user level” badge is bronze, silver, or gold, depending on the logged-in user
• A subset of users is selected to receive a special offer. The special offer is placed behind a feature flag. The flag is enabled for those users. After a set amount of time, the flag is disabled – the offer is no longer available.

A “deployment” no longer means a “release”

Using feature flags, “deploying” code no longer means “releasing code to end users.” With feature flags, teams can place new features and functionality behind a feature flag that is disabled by default.

This makes deployments almost risk-free and can dramatically increase your code-to-production time.

After all, if you know that by default, no one gets the new code, your deployment is safe. Then when you’re ready, enabling new functionality for only those you choose means that even if something goes wrong, you have the following safeguards in place:

• The blast radius is limited; only a small subset of users are impacted
• You know who it went wrong for and can easily disable the functionality without a redeployment.

Feature flagging with the Dynatrace Platform

The Dynatrace Platform uses feature flags extensively:

• All new functionality is placed behind feature flags. Customers can opt-in to early access programs. Product managers then enable the flags for those customers only.
• Dynatrace clusters are feature-flagged as “stable,” “early,” or “developer” to denote a risk appetite. Updates are pushed first to “developer” clusters. If everything is OK, the “early” clusters receive the update until finally, the “stable” clusters are updated.
• Documentation for new features or “internal only” knowledge is placed behind a feature flag. When logged in, staff have access to documentation that no one else can see.

What if I’m not doing CI/CD yet?

As LaunchDarkly points out, if you’re not doing CI/CD yet, at first glance, feature flagging can seem daunting and “too difficult.”

However, if you can deploy knowing the new code is never seen or used unless intended, why not continuously deliver software?

CI/CD is scary for many precisely because a deployment means a release, and historically, that’s an “all or nothing” activity.

Feature flags give you a safety net, so CI/CD doesn’t have to be scary.

OpenFeature

OpenFeature is an open standard that describes feature flagging. It has already been adopted by many feature flag vendors and adopted by some names that you might know.

Companies normally start their feature flag journey by building an in-house solution. This could be backed by a database or something as simple as a JSON file.

Sooner or later though, companies invariably find they’ve outgrown (or no longer wish to maintain) their in-house solutions. The business decides on a feature flag vendor and the developers get to work.

The developers must:

1. Understand the APIs of the chosen vendor
2. Remove all existing integration code between the application(s) and the in-house vendor
3. Replace all the above code with new code to “speak to” the feature flag vendor
4. Repeat this process across every application in the enterprise

OpenFeature makes it much simpler for enterprises to adopt feature flags whether they’re using an in-house or commercial flag solution:

• OpenFeature requests flag values in a standard way, regardless of “backend”
• Teams can swap backends easily

To enable feature flagging with OpenFeature, all you have to do is change the following value from

OpenFeature.setProvider(“My-In-House-Provider”)

to

OpenFeature.setProvider(“My-Vendor-Provider”)

The importance of observability

Feature flags and unrestricted continuous delivery bring great benefits, but observability is a critical component of a progressive delivery system. After all, if you can change everything about the system using flags but can’t see what the effects of those changes are, you’re flying blind.

You can configure Dynatrace to capture feature flag values on every distributed trace, so the impacts of those flags can truly be understood at a transaction and individual user level.

Dynatrace makes it easy to see exactly what the issue is and who is impacted.

Example: API traffic with feature flags

Imagine an API endpoint that a service calls to perform an action. This action relies on an algorithm. You’ve enabled feature flags to decide which algorithm a user receives.

Your team wants to introduce and test a new algorithm (which is supposedly faster) and you implement a feature flag to target only a small percentage of logged-in users to test this new algorithm.

To determine whether the new algorithm is actually faster, you must measure its response time and compare it to the response time of the old algorithm. If you only split the response time by endpoint, the statistics for logged out and in would be grouped together – you’d get the average of both algorithms (user groups).

Dynatrace captures the flag values, so splitting by the logged-in or out status is easy.

Dynatrace shows that logged-in users have a feature flag enabled which gives a better response time.

Perhaps it is time to roll out the new algorithm to a higher percentage of logged-in users. By using feature flags, you can roll it out slowly and safely. Dynatrace is there every step of the way, observing and ensuring a safe progressive rollout.

Dynatrace ❤️ progressive delivery with feature flags

Start your progressive delivery journey today with Dynatrace. Sign up for a free trial and start realizing safety-guaranteed progressive delivery with feature flags.

When you have your trial tenant, head over to OpenFeature to get started.

To learn more about feature flags with Dynatrace, see my previous post, Feature flagging done right with Dynatrace and OpenFeature.