Crafting Dev-Friendly Spaces via Telepresence & Automation

Timestamp Speaker Transcript

00:00.00 Kay James Introduction and then I'll throw it over to you to introduce yourself and then we could just start chatting so free to one action. Welcome to the living on the edge podcast. My name is Kay James I'm your host. Ah I'm a customer enablement engineer here at ambassador labs. Um, and I'm joined by shavi he's going to tell us a bit about ah the work they do with automation over at voiceflow and some of the stuff they do telepresence. So go ahead and introduce yourself.

00:32.30 Xavi Yeah, well thanks for for reminding me? Um, yeah I'm having I'm based in Spain but I'm working for a canadian company like company is called ah voice flow. And we are like buseller labs ah users since a while we were like early users of Teles and I am also on a ba lab advocate. So yeah, that's. That's my year with with ah ambassador lab so far. Ah so good and and looking forward for the for the the the feature and error. Basically.

01:15.70 Kay James We love that we love ah chatting with an advocate. Um, so why don't you tell us a little bit more about ah your role over at voiceflow and what are you kind of working on over there and and that type of stuff.

01:27.93 Xavi Yeah, sure So I'm working here as a head of Cloud and basically what we are doing under under. My team is like managing everything that is related to the to the Cloud of course. But you know like um. Ah, starting from the Cloud itself like managing all the terraform code um managing the cicd pipelines we have created some cool stuff. Ah there that I will mention in a bit. Thanks telepressence by the way. Um, what is where they were doing the bi and all the data warehousing everything related to yeah data modeling and all that stuff that is under a my team as well infoak so you know compliances.

02:22.14 Xavi Um, and the development environments where where we are using telepressence as well. So yeah, a small team but doing a lot of things and and yeah building a bunch of automations as well.

02:37.55 Kay James And how did you get into that type of field and environment like was it something you were always interested in in terms of like automation and that type of thing or did it kind of fall into your lap and you just really enjoyed it.

02:51.77 Xavi Yeah, so um, when I start working like you know like ten years ago something like that I was just like a classic Java developer and then over and over time we I was chaining myroll in more like kind of a. Integrat person like more working on automations and make things like easier for the teams and then a I joined when I joined my past company that was um, edixon we were like basically. My team. We were basically doing a bunch of automations and a bunch of testing automation as well like in in terms of pipeline like not just creating the pipeline itself like let's say just not managing Jenkins files and all the stuff like. Everything in the in the development lifecycle and make make it like fully automated with you know, like creating by creating clis um tools you know dev tools. So yeah. And that was something that that I really really love and and yeah, when I joined voiceflow that that it was what what I would what I was doing since ah since the period since the beginning starting as a senior Div and then scaling and and being promoted as ah as a head of cloud.

04:23.42 Xavi And yeah, my free dime like what I usually do as well is like um like creating a bunch of clis a bunch of developer tools and a bunch of ah articles I yeah I love to do like some technical writing. And and yeah like all the things related to depth tools like how to do that. How to improve a these development workflow financea. Let's say like I don't know like and developing a work cloud function or adol la how can ah how can. I you improve you know like the the development workflow. So I usually write about those things because because that that is something that I love and say in the Kubernetes world as well because you know like that's something that is training right now and developer tools for The highbree development. So yeah, that's that's kind of my journey and what I really like.

05:31.48 Kay James Yeah I could tell that you um, really like helping people to um, that's kind of the overall thing that I kind of got from that and also like I noticed that you have a book as Well. So It sounds like you know teaching people and and helping make their lives easier is kind of. Something that is important to you.

05:46.45 Xavi Yeah, yeah to dady the dahi.

05:52.95 Kay James So um, what are some of the challenges that your team is currently working on overcoming and and what solutions are you guys looking to implement.

06:00.88 Xavi Yeah, so right now. Um at at baseflow what you can see is basically that we have like bushflow.com which is let's say ah our public cloud. But. Um, when you get like the enterprise edition that means that we deploy a private cloud like full and full new tenant for you and that means that we have to deploy a bunch of resources. A. For the new customer. So that means that we are deploy like all the databases that we are using which are um, a um, you know like everything fully dominated using you know like data firm as I said and then the Kubernetes clusters. So because we are using Kubernetes to deploy our application and everything like um, the data layers and they compute and all the stuff that is a currently we you know uscp and same interest that go with that total. What you want to deploy. And yeah, once we a go to the Kubernetes phase a same using terraform we.

07:29.21 Xavi Deploy like all our hand charts. All our applications kind of like yeah automated and that is where we are seeing our one of major challenges. Um because you know like we are deploying applications. Into kubernetes and those obligations needs to be and need to be updated so we develop something like Keel which is you know like ah ah, kind of custom tool that just updates ah them. And they all creams in all like clusters currently, we are maintaining around 30 clusters and each cluster has around 5 nodes. So. It's quite a lot That's only the Kubernetes side. We talk about the databases. It's there are more and more nodes. So yeah, like we are not following any kind of like like pattern you know like a giubs pattern or something like that. It's kind of hackcky. So yeah, that is where our challenges are are. Are coming up and that what we are seeing and also deploying our hand charts using data form is creating like a bunch of boilerplate code a bunch of code that is kind of useless because you know if you are like deploying of this not whole application. You just need.

09:06.40 Xavi That Crd that jaml you deploy that and then within the clustersters where that reconciliation happens so that is ah um, one of our biggest challenges. We have successfully move a our 1 big piece of our infrastructure task goal.

09:27.32 Xavi Into um into an external computation management system called Doppler which is a is like aw secrets mannaier or or ask your secret manna or workclass managera or hasi or bold. With superpowers like it has a ah bunch of things. super super useful like for instance, you can create projects. You can create environments and once you have your changes in 1 environment. You can promote those values into the other environmentments with just one like Apis.

09:58.73 Kay James Are.

10:04.46 Xavi Cli that you can use it automate things and you can um, sync your doppler secrets with your kubernetes secrets. So we have removed a bunch of code was in the form into that system.

10:17.78 Kay James I.

10:23.60 Xavi But they're still missing the the the gitop solution.

10:27.34 Kay James So how do you guys? go about trying to find solutions for some of these challenges like is there a resource that you go straight to to look for an answer or maybe it's just like word of mouth.

10:44.26 Xavi Kind of both like um, we usually that's something that I always try to you know like empower my team like going and attending to confidences to local meetups. You know, like to see real scenarios like how. Um, and the other companies or or the other open source projects or or whatever are doing the things you know so people can Learn. So yeah and also reading a bunch of articles for instance on on infocu. On this own. Um non. So For instance, the Aws blog the world Cloud Block Those are really really good and they are like I think those are good resources. Um, but yeah, like basically is how we are. Doing that right now.

11:42.40 Kay James Um, and what are some like automation. Best practices that you think everyone should be doing these days.

11:52.68 Xavi So I think there's 1 thing that should be Mandatori which is um per team and like every single team they they should have like their their cli for their own.

12:08.91 Kay James Um.

12:10.31 Xavi Like daily tasks you know like even if you want have to domain like how to do a commit for instance instead of writing Git commit just like writing a cli that has like multiple steps you know so you don't you don't have to take care about these even this small. Pieces you know of course a C like to set up environments a C like to connect to somewhere a a c like to run like this kind of test you know like. That is how we usually approach and then you can reuse that cli quite easily or you should build that cli in the way that you can reuse that in your pipeline. So. There's the same code that you're running locally or the same optimizions that you're running locally you are gonna.

12:49.59 Kay James Your.

13:04.62 Xavi Execute those in the um in the in the cli so it is easier to reproduce and and easier to to the because we have seen a bunch of things you know like the classic ah on my laptop works and in the Ci is not working. and and yeah with for instance, using a so a common cli and also creating like good standards and good patterns like when instance when you are switching from one team to another and you know like there.

13:31.94 Kay James Are.

13:40.13 Xavi Automations that are already in place. You know, like because they are a standard for the a you know like in the in the company you know company wise which is which is important as well. So yeah, this this kind of things.

13:55.30 Kay James Yeah, that sounds like um I agree I think that's important too because then you know developers can just focus on actually developing instead of the extra steps to make things work or to get things started and rolling and whatever. Um, which you know it. In my head that sounds like having that devops platform engineering that type of team is kind of essential right for any company or whatever. So I'm curious like um I feel like platform engineering is kind of one of those buzzwords that's been a thing lately. What is your perspective On. You know the rise of platform engineering and and how that will evolve the whole devops. You know that role in that world.

14:39.48 Xavi Yeah, so I think platform is something like quite important like in the past like what we have seen is ah like teams doing their own you know like platformating or the dev ops teams doing their own platform engineering. But. You know, like creating um like a separate team you know doing just platform engineering doing just developer tools internal developer tools for the current request from the from the developers I think that's awesome and that's the. The the the way to ah you know because developers can just focus on their job which is develop new stuff and then when they need a automations and environment creations. These kind of things you can request that to the platform engineering and also. Or or the platform and in Anyfo instance can leverageate all these clis or or create like um, custom a automations for instance like using Spotify pack stage which is one of the most a platform. That are right now is is being used as a platform ind engineering a platform and because you know like is kind of like because with with ah Spotify backstage channel also with like.

16:12.57 Xavi With those crs that you can ah create all the automation. All these kind of pipelines and and just throw that to a kubernetes cluster is an easy way to create these kind of things and create like a quick graphic user interface with a bunch of. Plugins you know, like connecting that to your Jenkins connecting that to your cycle ci to your github actions to your grafana to your Argos cityd and then you know like deploy all these things by just creating those steps are is is is is really great I think and. And yeah, like a good balance between you know, like the platform depths and and the and the bos teams and working not as a silos you know, like working together towards you know like 2 main thing which is releasing faster and with better quality. That's the and the way to all that that is how I usually approach the the platform engineering and you know like all the the um and everything that is.

17:21.37 Kay James Are.

17:29.30 Xavi Between the teams you know in an engineering and in a bigger engineering team. Basically.

17:36.82 Kay James Cool and obviously earlier you mentioned that you guys use telepresence I'm curious to know how did you find telepresence and then what was the journey of beginning to adopt and implementing that what did that look like.

17:53.65 Xavi Yeah, so um, in the past like two years ago or three years ago what we were doing basically was you know like probably as. As all the companies or most of the companies were doing which is dogger compose app and running everything locally and that's great, but at certain point when you have like a really big. Um.

18:25.38 Xavi A really big stack. Um, it's becoming a problem because the laptop finances are I mean right now like there are good laptops but you know like the resources ah are limited. So.

18:40.60 Xavi And and also you have to maintain all these a local stacks and all that stuff so we were at um, a certain time that if you want to run like bushflow locally. You need like three d abytes of memory or something like that just running the docker compose. So. And what we did was like okay, let's try to do an hybrid approach and let's start developing using this hybrid approach. So all the heavy things are running in the cloud and then the developer because that depends. Also how is the. The company like manage. But at least 1 team should touch like 1 or 2 microservices. So at the end a developer should start working in 1 or 2 a microservices and running like 1 or 2 process locally which is good like. Those are are just good like even if you are running like node Js or Java that are consuming a lot. Um, that should be good so that is what we did we were and looking for you know, like first solutions that were doing this kind of hybrid approach. We are fine. Um. Some of them that were like for us too complex that are like syncing um local files with um with a cluster and and like revealing the the container every time you save so it was like kind of.

20:17.36 Xavi Annoying and at the end what we wanted is like something quick. You know like running something locally that is connected to the clusters and that is what telepressence does you have like you know everything a. In it resolve like the all the Dns and everything that you have on your cluster you have that on your laptop. So for like you want to connect ah to a postgres a database to a redis to a mon or even a queue you can just run like tele percents connect with your namespace. Because the solution that we that we took is basically we have ah multitenant cluster and each name space is a small voice flow per developer so you running telepressence connect with an endpace of that developer. You can connect to dull resources and the other services that are deployed there and are not running locally because you know you are developing just 1 or 2 um services which is great so and then at the end we we went from. Thirty gigabytes running locally to you know like one hundred or two hundred megabytes running in locally which is awesome and it's super fast. So yeah, that's our our telepressence journey is.

21:48.96 Xavi For us is like the the the simplest tool to develop or to have like that this hybrid approach you know of developing using Kubernetes and I think that should be the the main. Or like the the trend you know like doing acom years and start using like docker compose and these kind of things with local stack just running everything in a Kubernetes cluster because at the end is not that expensive to be honest and also. Ah know like you can even run like local cluster and use deliverrations as well. So yeah, these kind of things.

22:32.35 Kay James So what was the response from the actual developers who were you know and they ended up using telepresence like were they excited Did they was there like a learning curve or what was their actual reaction to adopting this.

22:49.63 Xavi Yeah, so we I think we did it properly because instead of like bringing to them the um. The tele presence cli and all the commands and everything what we developed was like an internal tool called bfcli like voice flow cli and we rub like all the telepresss commands and you know like we created also like um, Jarn. Like an mpm interface that is common to all the services. All the a node Js services and the bfc I interacts with that and and that is what is calling like teleppressence and with.

23:24.97 Kay James Are.

23:33.78 Xavi You know, like that that big command that is running the the Intercept. So Basically at the end what we did is basically you are just a developer you can create your development development environment with running a bfc like M create and then your Dev name. Um it. Well create your environmentment with the telepresss. How to inject like the the traffic traffic and agents are out injected in those services and then once that is done. You just need to run Jarn local in the service that you want to develop and that's it. So That's why I have like I am emphasizing you know like the the importance of you know, like writing automations creating standards across the the engineering team Because. Building things like this makes you or makes your developers life easier and they are like happier because the learning core was literally nothing. You know.

24:42.43 Kay James Yeah, that sounds very convenient from like a management of the ah you know the cli implementation that you guys have how do you guys manage like upgrading telepresence or have you had any.

24:57.70 Kay James Had to do that or anything like that.

24:59.63 Xavi Yeah, So what we is basically the telepressens version is man at a as well with the Bfcli. Ah so when we won't upgrade telepressen that the teleper's version. Um. The developers version itself in the cluster is managed by terraform so we just bomb the version and Boom the we upgrade the the the film chart and that's it and then um, we release a new version of the vfci like so the developers. And they are using telepresssence again. The new telepressence version and when they're using the bfc like the new telepressences version is being now loaded and and everything is working fine.

25:44.37 Kay James Cool and are there any other um, new tools that you found recently that you're excited about.

25:49.40 Xavi Well, we are on our journey of you know like gitub so Arlo City hundred percent and and and Spotify back a stage that's something that we want to a include in our in our tech stack because. Yeah, like fornesssa with Spotify backitch. We want to have like a graphic use and interface that we can develop to execute at the end like Bfc like commands Forsa like if anyone can and if someone wants to create um like I don't know like um. Ah, marment or ah development environment instead of running the cli you can go to Spotify backstage and check that like in entering all the the information that you want and click a button them and that's it this is more for like. Ah, stakeholders, you know like product manageras and all these kind of things. Um, because yeah, with with a bfc light. You cannot only create development environments. You can create like review environments you can create test environmentment. So that's the the idea and then. And the main goal is to fully automate the new like the new boy through like then in the whole new private cloud. So do that again with epodiify back stage like and when we close a deal ah person from the sales teams goes there.

27:21.90 Xavi And just enter the the metadata needed and then boom I like could rain off. For instance, we are deployed in around like five hundred or six hundred cloud resources per cloud. So if you can just add the the require metadata which is. <unk>ns is not allowed on spotify backstage then boom everything is good and with arago city you can see that as well within spottify back stage because it has some really cool clientance. So yeah, that's our goal for twenty twenty four.

27:59.31 Kay James Cool and I guess kind of in the same vein as that. It's new year. Um, is there any doesn't necessarily have to be a Kubernetes thing but is there anything exciting that you're looking at in tech that you're excited about them in this new year

28:12.65 Xavi Well everything related to you know like lllms I think um, last year was the lllm explosion and this year is gonna be like the kind of your like maturity or of this world. So yeah. We will see how that goes. Um I'm excited and yeah, like there there are there. There were a lot of startups that kick off you know, like on on top of lm solutions and all that stuff. So yeah I'm quite curious about. These things. So yeah, what we say what say.

28:52.16 Kay James Cool. Um I'm running out of questions this point. Um Bailey I'm a tap over to you is there anything else that I should mention or should we.

29:06.15 ambassadorlabs And um, anything that he'd like to promote and then we can wrap it up.

29:08.87 Kay James Okay, cool so to close things off is there anything that you would like to promote or shout out before we sign off today.

29:20.36 Xavi Ah, well I mean um, you guys can check my ah my blog is is shabbvid up dot me so you go there, you can check like all the articles. My books. Um, also my talks like everything I have a focus more on conversational ai but you will find some Kubernetes things and some telerescent stuff and and yeah I think that. That should be so we have also at voiceflow within and in Cubecon North america we did that talk about how we are using tele percents more in deep. So yeah, if you go I think that was uploaded in the kubecon a Youtube channel. So if you check like. Kubecon voiceflow Toru probably you will see that that was a run by by my director of engineer frango and yeah, quite quite quite useful if you are like thinking about introducing telepressence in your in your company. Yeah I think that's that's it. Thank you Kay for for having me. It was a pleasure and yeah, looking forward to next episode.

30:40.98 Kay James Awesome! Thanks so much. Saw you for joining us and thank you for listening to our listeners have a great day.