Next-Level Continuous Delivery Testing with an API Gateway

A modern API gateway like Edge Stack Kubernetes API gateway empowers organizations with a cost-effective solution to harness the full potential of their microservices architecture by streamlining application development and management in a Kubernetes ecosystem. One area where Edge Stack really shines is in continuous delivery testing on many levels.

For example, you can deploy a new service or an upgraded version of a service into production and hide, or “cloak,” this service from end-users via the gateway. This effectively separates the deployment and release process, allowing you to run acceptance and nonfunctional tests on the cloaked service, such as load tests and security analysis. You can also perform canary testing by allowing a small amount of user traffic to flow to this new deployment.

There is also potential to use a gateway to “shadow” (duplicate) real production traffic to the new version of the service and hide the responses from the user, and “shift” traffic around to focus load on a specific cluster of your system. Finally, you can use an API gateway to implement and control chaos testing. These techniques allow you to learn how your service will perform under realistic use cases, load, and failure scenarios, which are critical for continuous delivery testing.

Traffic Shadowing

Unicorn organizations regularly use the technique of shadowing traffic (or “dark launching”) features. For example, Facebook famously tested the release of the username registration service by directing real user traffic at the service and hiding the data that was returned. Twitter has also talked about its creation and use of the internal “Diffy” tool that acts as a proxy, multicasts requests, and then compares or “diffs” the response. GitHub used a tool called “Scientist!” that allowed them to carefully refactor critical code paths during a migration from one technology or architecture to another.

Traffic Shifting

Adrian Colyer wrote a great summary paper written by the Facebook team that talked about their “Kraken” load testing tool. In a nutshell, the Kraken tool integrates tightly with the Facebook gateways. It can “shift” (or route) part of its global traffic to systems (or data centers) under test and monitor the results — reverting the traffic shifting if monitoring systems show an error. For example, if Facebook wants to stress test a new data center that has just opened in Germany, they can shift all of the European traffic to this center in a controlled and gradual fashion and watch what happens.

This technique helps us think differently about how to utilize an application gateway for continuous delivery testing.

Traffic shifting, in the context of API gateways, involves directing a portion or all of the incoming traffic to different services or versions of services. This can be crucial for scenarios such as A/B testing, canary releases, blue-green deployments, and gradual rollouts.

Chaos Testing — Chaos Engineering: Break it to Make it Better

The final topic of testing that I can’t resist talking about is referred to by many names: chaos engineering, chaos testing, or “resilience testing.” This type of testing increased in popularity as teams built distributed systems and bumped into the realities and complex failure scenarios when working within this domain. Chaos testing allows a team to hypothesize how a system will react to failure, design and run the experiment, and monitor what happens.

The Netflix team has historically been the pioneer within this space, and I’m sure many of you will have heard (or even used) the Chaos Monkey and Simian Army. Their second evolution of these tools inspired “Failure Injection Testing,” where failure could be injected into specific requests (perhaps for a test user or cohort of tolerant end-users) and monitored the results. Target requests were identified and modified via an application gateway.

Data planes like the Envoy proxy allow header manipulation, so an API gateway like Edge Stack Kubernetes API Gateway can be used to generate and inject request IDs or “chaos headers.” Of course, internal services may need to propagate this header through the service call stack, but this would be a small price to pay for the ability to run failure injection tests as part of continuous delivery testing.

Enhance Your Continuous Delivery Testing

Implementing robust continuous delivery testing is essential for maintaining the quality and reliability of your cloud-native applications. Edge Stack API Gateway provides the advanced capabilities you need to enhance your testing strategies, from traffic shadowing and shifting to chaos engineering.