Voiceflow Story

From monolith to microservices

"We Chose Telepresence because it allowed us to run multiple services locally on our laptops and allowed us to deploy around 20 microservices within five minutes." shares head of Infrastructure, Xavi Portilla Edo.

Voiceflow’s architecture has become more complex over the years. We only had a monolithic application and two database engines when we started.

But as we adopted the microservice architecture, it grew to over 20 microservices, multiple databases, and queue engines. As a result, running the entire application on our developers’ laptops like we used to was no longer scalable nor practical for our developers.

Challenges with the existing setup

1. Applications became too large to run locally. Not only the number of services we had to run but also the external services we needed: DBs, queue engines, etc.
2. The development workflow was very slow. Building images and waiting for them to deploy isn't the fastest or most efficient use of a developer’s time.
3. Even if we had capacity, we couldn’t replicate the interdependency of cloud deployed apps locally, so our local testing was not comprehensive enough.
4. Devs didn’t know what other devs were working on. For example, a backend dev does not need to know about the frontend services: how to run, compile, and package them.

Since our transition from the monolithic to the microservice architecture, we’ve learned that mocking and testing an entire application on a laptop causes problems like excessive resource consumption on the dev’s local machines, unrealistic environments, complex setups, and lengthy onboarding. As the app’s architecture becomes more complex, it doesn’t scale.

As shown in the diagram above, every time a developer is meshing a service, every request to the service meshed needs to go to the traffic-manager deployed on the Kubernetes cluster to route that request to the service running locally on the developer laptop.

Looking at things from a networking perspective, every request that’s been service meshed will have this route:

Incorporating Telepresence commands into our CLI

At Voiceflow, we have developed an internal tool for our devs called vfcli. With the vfcli, our devs can create their ephemeral environments using the power of GitOps. We have three different types of environments:

• Development environments
• Review Environments
• Test Environments

In every single environment, we deploy all the services, databases, and queue engines mentioned above. When we designed the development environment and the inner loop, we had in mind that simplicity was our top priority. This is why we embedded telepresence on our CLI.

With the vfcli you can just run the vfcli mesh connect <service> command and it will run the telepresence intercept command under the hood. Additional pre/post meshing hooks can also be run for service setup/teardown. We also have vfcli mesh connect to start the telepresence connection with the remote environment. These hooks are useful if we have to run any script to prepare the service (install dependencies, build, etc.) or to execute post-scripts (cleanup things, revert changes, etc.):

Now that you know how our system works, this is what a typical day for a Voiceflow dev might look like using our ‘remocal’ dev environment:

1. A dev wants to add a new API call from the frontend to a backend service
2. The dev spins up a dev env to work in
3. They use vfcli to connect their local frontend and backend-API to their env, initializing Telepresence
4. With this, the developer can quickly iterate and test their changes in a prod-parity environment without needing to push any code or run all the services locally.

I finished my development. What's next?

Of course, while you are developing, you will probably run your source code in dev mode and, depending on the language, that code is not the final code that is going to be executed in a real environment; you will need to compile it and run a compiled code or a binary and also that code should be executed in a container in a cluster. For this, we have created a system called Track System. A track is basically a particular branch that gets continuously updated. Each developer can set what track wants to follow in each service. The devs can specify the track they want to follow using the vfcli. Then our GitOps system will update the remote environments.

The following steps are involved when you want to push a change to your branch:

1. The code is built.
2. The built code is containerized.
3. The container is published to a container registry.
4. The metadata of the track is updated.
5. All the remote environments that are following that track for that specific service are updated automatically by an internal reconciliation service.

We also have tracks that are not related to a branch. We called them “Virtual Tracks”: They are similar to the above but not directly corresponding to a branch: (update a track every XYZ commits or update a track every XYZ minutes/hours, etc.). These special tracks can be used on our development environments just in case our developers want to get updates for their environments that are not strictly following a git tree.

That is for our remote development environment, but what happens with a production cloud? For that, we have created a Release channel system. A channel is a set of services and tracks. In each channel, you specify the services you want to update and which track follows each. Then our GitOps system will update our clouds with the spec defined on that channel.

Implementing a hybrid development environment with Telepresence allowed us to develop faster compared to when we were running everything on our local machines.

According to our talk at KubeCon NA 2023, in the end— Telepresence improved Voiceflow’s developer loop from 15 minutes to only 10 seconds, and our developer onboarding has also significantly improved.

As more companies continue to make the monolithic to microservice transition, we want to share our experiences so others can learn from our mistakes, insights, and successes. I hope this article gives you an insider's view of what is possible and available with open source technologies and explains how having the right dev loop improves the developer experience.