Edge Computing with Pandelis

Speaker 0: Hello, and welcome to learning things I love to hate. This is a show where I basically stop putting off learning things that I have avoided for one reason or another, inviting in my friends and experts around the topics that I'm trying to learn. And today's topic is edge computing, a topic which has got a little spicy in web circles in the last few weeks. I'm inviting my lovely dear friend, Pandellus, here today. Pandellus, would you like to introduce yourself?

Speaker 1: Yeah. Hi, Kevin. Nice to see you again. Yeah. I'm I'm Pandellus.

I've been an engineer for the last few years working in web, working in in deploying a lot of the things that I claim to write. I've worked in web agencies where delivering content to users was super, super important. Back in the jQuery CDN days where when edge computing was basically only CDNs into kind of our modern edge computing world now where we can put entire runtimes, entire databases onto the edge. There's all things I've experimented with at various companies. So I I'm so happy when I saw your tweet.

It's something I've been experimenting with a lot. And, as someone as a user of the web, I think it's something super important for for everyone to to to to to know about and experiment with.

Speaker 0: Awesome. And, and yeah. So I tweeted out a few days before this recording. Can anyone just, like, explain edge computing to me? Because I really don't understand it.

And you came in with a wonderful set of illustrations, which actually will just edit in here right now. These are the illustrations, to kind of explain a little bit about how Edge works. So I thought, hey, I'm just gonna invite you onto this show. This was already a topic I was planning on talking about. You preluded, I suppose, some of the things I wanted to discuss today.

So just to just to open with this, I wanted to explain kind of where my knowledge is now, and hope that you might be able to take me and anyone watching who's in a similar spot through that journey of understanding more. And perhaps if we have some time at the end, we can do a little demo of this abstract concept. So what I know of edge computing, I think, really is around content delivery network, CDNs. We can maybe start a little bit about talking about this. And I know that the concept of what can be stored, you know, and replicated in multiple regions and serve to people based on where they are in the world has just grown more complex.

We we're doing that with more things. And I suppose my biggest concern and the reason I haven't really done it yet is, 1, the idea that, you somehow need to replicate things really rapidly in lots of regions. And 2, unrelated, it just seems really complex, and I'm not sure how how much the complexity is worth, what might be a payoff that doesn't matter. I know there are some performance hardcore people who are like, of course, it matters. But, yeah, that's kind of my starting point.

So maybe if you take us back, talk a little bit about CDNs and what they are and kinda how that's evolved to where we are now, that would be really good.

Speaker 1: Yeah. Of course. Yeah. No. I mean, as engineers, we like to make our lives harder for sure.

It's it's it's we we we do sometimes like to like to overengineer, and it and it's all as with everything in engineering, there's a million ways to do them. Not all of them are right, and a lot of them are right. So your mileage may vary with your needs of edge whatever. A lot of people don't even need the CDN, but it comes down to your users. Right?

It always comes down to your users and the experience that you're trying to deliver. So let's let's start with CDNs, and what is the problem that that that CDNs solve? So if we go back to the mid 2000, when jQuery and, you know, serving libraries over the wire were were were all their age, we would, you know, blindly go to, CDN stack or whatever those CDN websites were and grab URLs for for jQuery and put them in our sites. Before build systems, before your web packs, before the the the modern JavaScript ecosystem back in days, CDNs were all their age because we wanted to deliver large JavaScript bundles to the user quickly. Internet was slower, and still is slow.

Or this is h t p one days, right, before streaming h before multi, you know, multiple h t p two streams, and all that type of stuff. So it was very important or we people saw a lot of gain in having these large JavaScript bundles close to the user, meaning the end users could consume that JavaScript faster. And in the end, what that means is when they get their HTML through, it becomes interactive faster.

Speaker 0: Could I ask you a question? Could I pause it? So when you're saying near to where people are, just to clarify, we mean that this one jQuery bundle or whatever is stored on multiple servers throughout the world. I request it via one URL, and then the application which will serve up that file will find the place nearest to where I'm making the request and serve it. And then the additional benefit of CDNs, I suppose, is if I visit 10 sites that all are trying to load the same large bundle, this might be more CDN specific, and I think it also might be a relic of the past, but it

Speaker 1: it would not It is a relic of the past. That's that's that's one of the topics I was gonna touch on. In in the early days, it was when there was less CDNs around, we we could make the ex we could make the excuse that, okay, by using a shared CDN or, say, a common example was also Google Fonts. Mhmm. By using Google Fonts, by using a common CDN, it means that there's a higher likelihood that a user would already have those assets on their computer.

Cached.

Speaker 0: Okay.

Speaker 1: Exactly. So it's a further cache. Ultimately, the the most edge location is the user's computer. So caching is always a tiering system and CDNs are a type of cache, and the ultimate caching is local. When you have a local cache, that's that's the ultimate edge.

And really with edge compute, with edge with with the topic of delivering on the edge, what we're trying to achieve is native like performance, and the most edge you can get is being on device. So, arguably, the the most edge computing delivery method today is an app. Apps are on your device, and there's no more edge than the the the device that you're touching. And everything that we're doing on the web or or in edge compute land is to try and achieve native like performance.

Speaker 0: Cool. That makes sense.

Speaker 1: So, yeah, those are those are sort of CDNs, and that is kind of the the most basic form of putting things on on the edge that we had. So that and that was, and there was a legitimate benefit to to to to CDNs. And what we've tried to do in or what has happened in in the modern era is CDNs themselves have become more and more capable, to the point where they can actually run and execute code. 2 of the most popular and, CDNs around are are CloudFront from AWS and, Cloudflare. And Cloudflare are one of the pioneers of of edge computing as well, and their CDN offering just over time naturally has become more has gained features, has had feature creep, and eventually, we wanted to do more complicated things over time.

Right? We we may wanna deliver a slightly different version of a bundle for certain users. Or if it gets hit with a with a certain URL parameter, we might wanna bust the cache or or run some sort of, run something on that server that's delivering it, maybe do an AB type test, where certain users got certain bundles or whatnot. So at

Speaker 0: some point transformations as well?

Speaker 1: File transformations became yep. Image image CDNs. So moving from kind of just JS bundles, image CDNs are, again, super popular because, on super useful because, the the 2 largest assets that get delivered are the bundle and the images. And images also very bandwidth intensive and benefit largely from being closer to the user. Sure.

And we're gonna we're gonna say closer to the user a lot throughout this conversation. And what that what that practically means is that you are spending less time in flight over cables. Your server could be very far away. And the further away it is, the longer it will take to navigate, you know, the the, the the sea of cables that live under the sea, to get to your user. So the closer you can be, the less hops we can make, the the the the the better for for the end user, but also for our bank, CDNs, specifically, not necessarily edge compute, but CDNs, specifically, also benefit benefit us in that we send less data out because it's also a caching mechanism.

Edge compute, not so much when we when we get when we get down to it, because one of those goals is to have as fresh content as possible. So when you're when you're when you're caching, you're limited by the freshness that you can deliver. So that that's that's sort of that that's sort of the niche that CDNs sold for a while. And, yeah, more and more doing more and more things on the edge or closer to the user or doing more compute at the CDN level means that we could do these kind of transformation things that you just mentioned with images. Right?

So say I wanted an image 30% smaller, we could deliver 1 all the way back from our original server, or we could use the kind of already existing cached asset and do some computation on it and then also cache that. And that's this kind of level of level of tiering that that that being closer to the user kind of offers us is is we don't wanna keep hitting our main server, and wanna keep delivering faster things to the user because, yeah, it's all about speed at the end with with with the goal of edge.

Speaker 0: Cool. So now we're in a space where it's not just about storing assets, whether that's bundles, images, you know, media, whatever. It's actually about running computational tasks at the edge. So I suppose, naturally, that moves us on to maybe what what is edge compute edge computing? What is it capable of now?

And then I suppose we're going back to my initial concerns or skepticisms. 1, how do you keep all of these nodes? I don't know if they're called nodes. I'm assuming they're called nodes in in sync, especially when you gave an example of, you know, doing transformations at the edge and then caching that. Well, is the cache just at a node level?

Do they propagate throughout a network? Stuff like this, I'm not too sure about. And then it sounds really complex. How how does this actually shake out to being something that's tolerable to developers?

Speaker 1: Yeah. Edge and the and this this edge cache or caching cash busting, in CDN land, it has always been a pain, especially in the days where I was working as at a web agency. So, basically, all all all that we deliver over the wire is is assets and and JavaScript and images, and it was definitely always always a battle of, okay, well, what is actually being delivered to the user and cache busting it and and and figuring out, oh, okay. Oh, they're on an older version of the of the site, and they're seeing old content and it and us getting rang up by our clients being like, the other site that we just published it. It's not gone live, and that's having to explain.

Okay. You're gonna wait for the cache to propagate. It's it's gonna it's gonna get to the your user eventually. No. But we've we've launched this because, some context here.

I used to work in, theater websites, where we had very peak peaky demand spiky demand and very, and certain requirements around, you know, if a certain show went live, they want it live now so that people can book it now.

Speaker 0: Which is not an unreasonable expectation if I hit publish that I mean, that's even the thing with, like, static website building. There's always a build period, and some people don't get that that build period, you know, is can be quite material.

Speaker 1: Yep. And that is also that's a huge challenge now with edge compute as well, but and also at the time with CDNs as well because we are we're still trying to we're managing demand, and we're managing speed. And the most fast asset we can deliver to someone is something that's cashed. So there's always a challenge on how you distribute or revalidate that cash. So in the CDA, it's so it's definitely I'm not gonna say it's anything that that's that's easy or perfect.

It's a challenge at every level no matter how simple, even at the simplicity of image and and bundle cache, but also to the level of, of full HTML page cache. Right? So during during go during, say, ticket sale go lives or or certain events, we may sort of cache entire web pages, and that becomes incredibly complicated to deliver to many, many users because we then have the challenge of say, okay. What happens when the tickets now run out? And everyone should be seeing a, you know, sold out banner.

And, depending on where you last were when that cache hit that node Yeah. And the the the way the way cash is generally work is, you know, it's a pass through cache, meaning, the CDN is the thing that will ask your server, make the request to your server for content, receive it, stream it back to the user, and also save a version on on that server. So depending on who access it at what time, there will be a cache with a certain time to live of 5, 15 minutes a minute. So meaning someone accessed it at some point and has a version maybe 5 minutes old that is sitting there. So and that and, using DNS, using domain name resolution, Based on where you're coming from, it will pick that server and that cache from that server.

So depending on where you are around the world, if no one's hit it in a in a few minutes, you may get stale content. Stale content, yes, huge challenge. And publishing new content, also a huge challenge because you will spike server resource, meaning it's you don't want to always necessarily be be live, because being live means you are sending a lot of data out.

Speaker 0: Sure. Sure.

Speaker 1: It's one thing that we was was common to do is or is like, okay. If I update, you know, if I update this article in WordPress or update this article in in whatever c CMS that I'm using to trigger like a full invalidation. Right? Like Directus. Thank you.

Speaker 0: I

Speaker 1: have used directors at at an organization as well. Don't be worried. In in in in in whatever CMS that you are using to just have, like, a blanket, like, asterisk invalidate everything.

Speaker 0: Yeah. Yeah.

Speaker 1: Which is a perfectly valid strategy for keeping things up to date if that's, you know, your your, your your your requirement. Yeah. Your But there and there's there's also the requirement then of, well, keeping things up and keeping things, keeping things efficient as well. C CDNs and edge computing are also about efficiencies. We talked a bit about sending requests to your server and that caching level being also a protection mechanism of you not actually hitting your server all of the time.

So, yes, it's very complicated, and it's kind of still on the engineer's side to figure that out. There are better frameworks to help with this. And as we get closer to real edge computing, there's less whole page caching that we have to do. And this is where we we I kind of wanna transition into well, okay. Well well, why?

Well, why edge compute? Edge compute means we can run more things closer to the user, meaning, hopefully, we need to cache less. And if we cache less, we could be more live or or we could be we we can have, less individual things that we cash, and the broader interactivity of the website or smaller subsections of it can be more individually individually controlled as to what level of, of staleness we accept within a web.

Speaker 0: But it's that trade off, isn't it, against, the or, you know, having non stale content ultimately delivered to users and the efficiency gained by being close to them.

Speaker 1: Exactly. That's it's that's the that's the balance that we're trying to strike. And, really, CDN caching, and at the level of caching whole HTML pages is kind of like is, you know, is the nuclear option. Right? If you can afford to have if if you're a a, you know, a news site, a very, very static type content website where you can you can do things over JavaScript on the client, or, the the content that you're delivering is is very static, then you can afford whole page caching.

But as as we get close as we get nearer the graph of, right, highly static versus highly interactive or highly, like, live content, like a ticketing experience where you before you go through the booking experience, you need to know you kind of wanna know, okay. Am I going into this? Am I gonna have to make it a whole account and then be told, oh, by the way, it's sold out as it is.

Speaker 0: Or even or even more, like, personalized experiences like a social media network where we're not gonna full page cache every one of the profiles and every post and every comment that exists. No. No. No. So there's some element of we need to get fresh fresh data as part of this load.

Speaker 1: Exactly. I mean, how many times are you, you know, just scrolling scrolling up to the top to get your fresh tweets all the time. Right? It's it's, there's a certain amount of stillness a user will accept, and there's also a certain amount of stillness that, the us as engineers or us as whatever company entity that we are will accept with with end users. And Yeah.

And, and that's that's that's the balance that we're trying to strike. And with edge computing, we are trying to deliver more things or cash listings, and that's achievable in how's that? How's how how do you describe this now? It's a very challenging to describe because we already have compute in in one location. And, moving it closer, just moving it doesn't fulfill, like, kind of the whole stack of requirements that we have, which is we need to deliver quickly.

We need to be efficient with our resource. We don't wanna just, you know, put things on the edge for the sake of it being on the edge and it costing an arm and a leg, which is one of the concerns you sort of raised as well, which is a a a perfectly normal, you know, objection to this.

Speaker 0: I didn't even think about the cost. I was thinking purely complexity. But, yeah, of course, there's cost. You're replicating nodes, basically. And the more that happens on those nodes, the more expensive it is.

I'd not even considered that, but, of course, that's part of it.

Speaker 1: Yep. And a lot of the the a lot of the current generation, edge compute are serverless pricing models, meaning you are paying for the, there's a there's a couple of pricing models in in this. It's called wall clock time and, and CPU time. One being CPUs, the actual CPU cycles that you consume, meaning you have to write also efficient code to execute. I'll do.

Otherwise, if you're writing code that is doing weird thing, and there's all sorts of limitations, that exist in terms of what you can do because the run times are also different. When we have a real server, we have any run time that we want. We have any c libraries that we want. We have any language that we want. And a lot of the modern, edge compute locations or edge compute run times because they're also designed to run very, very fast on and not in necessarily, I'll say in air quotes, real service like, real they're not actually real servers.

They are like a subset of the resources that a that these compute platforms have. So so, again, a couple of the popular ones I mentioned, CloudFlare and and CloudFront. Cloudfront is fronted by, Lambders, but not real Lambders. They're actually a subset of the Lambda runtime, that can only run certain limited amount of JavaScript.

Speaker 0: What? In an effort to be less computationally expensive?

Speaker 1: That, but also they've developed from this need of okay. I just need to transform an image very slightly. And over time, we've just kind of put trying to put more and more things in it. Cloudflare is is is a bit more is a lot more advanced. And also the CloudFront one, since I've used it back then, it has got a lot more complicated as well and has gained a lot more features.

CloudFront have again, it's a limited runtime. It's not node. It is a node compatible runtime, but you don't have access to say every single node library or you don't have access maybe even to the entire node modules, or the entire NPM registry.

Speaker 0: Oh, is this, is this isolates where it's like pure it's a pure JavaScript runtime with, like, some exposed additional functionality? Because we use those inside of directors, inside of our automation builder. But I didn't know that a lot of the JavaScript creature comforts, things even like console log, things like set time out and things like this, they're not they're not part of Node JS. They're not part of, sorry, the core JavaScript spec. They're part of Node and the browser implementation of JavaScript.

And you take for granted when these are the 2 really predominant runtimes that we use. So maybe it's something like that where they're using these isolate, environments with whatever additional functionality is exposed by the vendor.

Speaker 1: Yep. Yep. I I think isolates her was inspired by some of the some of the work that cloud flooded or or may even be a direct, a direct thing from from from Cloudflare. Yeah. Is it a direct descendant?

But, but, yeah, it's it's exactly that of of we're no longer in the browser. We're very much on a highly efficient, sub resource, that exists far out on on the edges of of these compute regions. And if we if we were to bring up a map, and I'll share share some assets after as well of we we know the general, you know, EU west 2 being London and, US east 2 from AWS being, you know, in Ohio somewhere and your EU West 1 or your or Google's, you know, EU Central 1 in Norway. These are all locations that they have massive, massive, huge data centers with further sub availability zones being, you know, eu west 1ab, e u west 1 c, etcetera. So huge, huge, you know, warehouse sized data centers with tons and tons of servers that have tons and tons of services on them.

When you additionally look at the map of of, what are the cloud front regions, which are regions where AWS or some other, you know Vendor. Vendor may have, caching locations for, which which are much the smallest subset of machines that they that they control that in partner data centers or in what are called crosslink or interlinked locations where, you know, many vendors, have their servers and, you know, pass cables all over to each other. That's sort of how how the Internet works in that. There's a whole bunch of data centers where eventually cables interlink with each other's data centers. But vendors will have also service in these locations, and these locations are, you know, a lot pricier, a lot closer to the user, and also run much less much much more controlled environments.

And these are these are these are what we actually refer to. This is what we really, really refer to when we talk about edge locations. We can be close to the user in terms of real locations also in terms of okay. Practically, you sort of want say, if you have a server in you e in US in the West Coast of the US in California, you would also really like a u a a server sort of in Europe ish to serve most of Europe. And then the next optimization is to put them super, super, super close to exactly where, you know, their ISP will interlink and hop over into AWS's domain.

And that that right there is where we where we actually talk about edge compute.

Speaker 0: Interesting.

Speaker 1: So we're putting them beyond server farms going all the way down to kind of the interchanges. And that's all, yeah, we kind of we we put up with lesser resource at our disposal, because also we're sharing with a lot more users or, like, the servers have to serve, you know, many other users, things like that. And, also, because we're so close to the edge and we're doing many other things like fetching and, you know, sending things over the wire as we're streaming. We're we're we're constrained to much more, much faster response times. So Cloudflow, for example, I think you you have to do whatever you whatever compute you do on that edge has to be within, like, half half a second or something or or a second, and you can't do anything more than that, and you get instantly cut off.

Speaker 0: That's fascinating. So this is really interesting because I thought edge computing was a concept that anyone could apply. It's just the idea that, hey. You know what? I'm gonna spin up resources on these different, you know, servers and do some complex, you know, routing of those.

No. It really is provided by vendors who are kind of more at that backbone layer of infrastructure. And it's uniquely enabled by being really close to what you're calling these interchanges. That's really interesting and not something I quite understood because it's never a marketing material because it's it's really getting in the weeds.

Speaker 1: Yep. Edge computing, unfortunately, is a very monopolistic type, very, type of operation because it's not anything that we can get involved with. I I cannot I cannot go and run around and just give people, just put some pies around in server files.

Speaker 0: Just that. Also, nice sticker. That is a retro sticker.

Speaker 1: Of mine. Yeah. Because there's there's even more stickers in Slack. I can't afford to go and put these servers around, really, really close to users because I do not own that infrastructure, and I would never have access to that infrastructure because you need a lot a lot of money to be put into these type of locations. Like, you need to be running a network.

But what you were describing there for just a moment as well of, yeah, just being able to put services kind of vaguely close to users, that is distributed computing, which is different from edge computing. It's both that's an versions of distributed computing are also very important to our goals of of, serving content to users really, really quickly. And this should be a computing and edge computing kind of go a little bit hand in hand as well. Because in in this day and age, at least today, we do have, and you raised at at the head of the show as well, your talk about databases or or data as well. This day and age, we sort of have, an inkling of kind of like the early CDN days.

We have an inkling of being able to put databases on the edge. But, again, because there's such limited sub resourced run times, it's very, very difficult. So the current generation of databases still lie on the distributed computing layer, which means we our our content can be really, really close to the user. And, hopefully, the content that that little edge computing box is requesting is is kind of like our users are over here, and then the the users of our database being the edge computing nodes, we wanna also be close to those users. So, we need to we need to both balance, where we put the user's content and also where you put the edge computing nodes content.

Yeah. Yeah. And more often than that, you know, at the end of the day, a website is kind of putting together a whole bunch of database queries. Right? Most websites.

So if if we can also put the data that is needed to construct that web page close to where it is being constructed, then we have further performance gain. If we can get it if we can get them collocated right next to each other, that's fantastic. And one of the arguments against edge computing always like and you've also mentioned it as well. Why go through all the hassle to need to have this, now we have less environment. Now we have we can't use all my nice libraries, and I have to I have to struggle with, you know, doing things in under 500 milliseconds and whatever.

Why do I have to go through all these struggles when it's good enough if they can just access my already distributed you know, I've already put a node in Europe, and I've already put a node in the US. That's good enough. And it comes down to what is the quality that what is the quality experience that we are aiming for? Are we aiming for near native, or, is, you know, is good enough good enough? I am a firm believer in it's the world wide web, and it's not the US web.

And there was a time, Guillermo Rauch, kind of pioneers a little bit now with with Vercel, and their efforts in in edge compute or or delivering good UX as well to developers for for engineering on these new new run times. I've read the tweet of him once of in the early days when he was setting up Vercel or, whatever it was called back in the day. Zite. Zite. Zite.zite.co.

Whenever he's setting up zite.co and he first went to the s went first went to San Francisco, he's like, oh my god. The Internet's so fast over here because that's just where that's just where your your data center starts is in the US. And then good luck when you finally if you decide to put a server somewhere in Europe and we get access to some slightly faster websites. And And at the end of the day, we are talking slightly faster here and there in a lot of cases because our undersea cables between, you know, Europe and the US are pretty fast. If I was to put a data center in Oregon, and we can do we can do this this is one thing we can do.

We could do a little, ping test as these sites that can ping across all the data centers. If you if if you do a ping test into Ohio, it's about 250 to 300 milliseconds of round trip time. Vaguely okay for most for most use cases. It depend

Speaker 0: it depends how resource intensive your application is, I suppose. If you're starting to build something that's really, really bandwidth intensive, multimedia site, actually, it starts to matter a lot more.

Speaker 1: Yep. Multimedia if just even just a normal website. That's that's that's kind of fine for Europe. Right? But, again, we're not the worldwide web of US and UK.

We're the world wide web. And imagine having being in Australia or being in South Africa or being in India or being in China, and having 800 milliseconds to 1.2 seconds of round trip time. And that is round trip time meaning on literally anything that you do. So first, there's and in modern web applications, that is alright. First load HTML.

1.2 seconds just to get the first request going with streaming HTML, then it hits some JavaScript that he has to go fetch. And that's another 1.2 seconds of doing that. And that's that's just the round trip time, not only you then have to be downloading through that entire time. So it's it really compounds the more that's going on. And when we're aiming for native like snappiness, the goal the goal is native like snappiness with edge.

And if the the the point is we need to get things really, really close to the user for that initial load. Humans, there's a psychological, element to slowness or to to delivering fast experiences. And if we can show things quickly, it doesn't matter as much how long it actually takes to then deliver things as long as something's happening. There's actually an interesting UX that exists today in AI because, there's this new UX pattern now of streaming text Oh, yeah. Which we didn't have a year ago.

Speaker 0: The the data physically doesn't exist when that when I start getting a response. It's yeah.

Speaker 1: And you you you imagine if if if we didn't have that u that UX only serves the purpose of AI being too slow today. Imagine us having to wait a full 10 seconds for us to get the full kind of page back of the AI response. That it that that would be an extremely frustrating human UX experience. But just by the fact that it's actually coming in character by character, you think, oh, shit. That was really fast.

Because you you actually you're reading you and you're consuming. And that is the UX that edge computing is trying to tackle of. And having having a risk response back instantly and having your content just there straight away.

Speaker 0: Could I ask a question? When you were talking about the, the concern I had around the effort that needs to go in, the effort you described was all about writing performant code in a more, in a more limited environment. But I ask, what is the actual work involved in setting up edge applications beyond just writing code that will run? Do I have to manage where stuff is, or do I basically just, like, fling it up to a vendor and they take care of the of the distribution of that? Like, is there is there work required to physically make it edge code that will run on the edge beyond the environment and beyond the, the performance requirements?

Speaker 1: There is a gradient. There there there is there is a gradient of of effort that can be spent here. And every sing every single day, there are new there is more and more tooling that makes it easier. As a baseline, anything you do in AWS will be hard because they are a platform. Right?

They're they're just they they are they are infrastructure of a service. They're not the platform, and they're not the tooling. They are they have other services on top, like, you know, your, whatever they call their their stuff.

Speaker 0: Amp Amplify.

Speaker 1: Amplify. AWS Amplify are sort of some of those toolings that do distribute compute. Google Cloud, also, it's gonna be hard, but they have other services built on top. They have things like Cloud Spanner is their distributed database. Really, really fun if you read their white paper on on Cloud Spanner.

It's it's a very googly white paper, and it's very very overcomplicated thing. And they talk about syncing satellites and time clocks in servers across the planet and how they use this network of satellites to have, like, microsecond time precision is very interesting read. Highly recommend. There's things like that. And there's Firebase on top, which also does distributed computing.

And so there's, so there's frameworks built upon these things that that that make it easier. And ultimately, it comes down to it to frameworks or other platforms that make it easier as we go up the stack. Vercel also are a platform, but they make underneath you know, they they use AWS Lambdas. They use Cloudflare edge workers.

Speaker 0: Mhmm.

Speaker 1: And that's sort of where if if you've ever experienced working with Nest, which is, again, kind of another superset on top of Vercel, because it just kinda uses all of Vercel features. When you are working in a an edge nest route, you have a limited amount you have a subset of things that you are able to sort of do from from the from the platform Sure. Because it's running it's no longer running on a Lambda, which is a 4 Node JS instance. It's actually running on a Cloudflare Worker, which is that isolate that you just described. So they are tooling.

There is tooling that makes it easier. And then in other realms, in databases or in CDNs, there's more tooling there as well. There's there there's such tools like, PlanetScale is becoming very popular these days with its easy distributed computing of or easy distributed computing of databases. And there are other things that are trying to actually put databases on the edge, and the current kind of front runner for that are, SQLite, which actually Cloudflare again, another pioneer in this area. KV, key value store, is a product that is built on SQLite.

Dino, have a KV, a key value store. Again, kind of built not on SQLite directly, but a fork of it that does this distributed computing. A whole bunch of startups doing SQLite type distributed databases. Edge actually, edge database. Not only distributed, but actual edge

Speaker 0: On those edge nodes. The how does syncing oh, sorry. What I'm basically hearing is vendors more or less will take care of the of the distributing of your assets, whatever those assets are across nodes that they make available. That's not something I necessarily need to do. If I don't care about edge, if I just care about distributed, it's something I could do with great pain.

Yep. I or I imagine there's tooling still. But a lot of these vendors will just take care of that. That's part of the offering. What about syncing stuff?

Surely, there's some trade off around, you know, even if it's the data store is what I'll call it over a database. The data store, the cloud, you know, the cloud functions that run the edge functions, and all the assets and stuff like that. How do they all stay in sync? Obviously, when I push an explicit update, I imagine there is an explicit, you know, propagation of that of the updates. But just day to day, I

Speaker 1: Mhmm.

Speaker 0: I'm a user. I add an item to a data store that happens near me. How does it get to everything else? And is there a risk involved that things will start to fall out of sync?

Speaker 1: Yep. Certainly. Now the for the KV stores, for the for the SQLite stores, I've really not read up enough about what's going on there. It's like mind blowing stuff that goes beyond my comprehension right now. So I have no idea how they sync because, also, SQLite is a binary data format.

So it's it's mostly if anything, I believe it's kind of for, like, edge caching or some sort of be able to write small things. I don't I don't know how it gets back to your main server. Crazy crazy black magic over there.

Speaker 0: Sorry. Can I pause you? Your main server. So, in all of this, there's still a main server running that

Speaker 1: You could. You could. Or at least, you you you definitely could write you kind of have to flip how you how you develop. If you wanna be entirely edge, you certainly can. And a lot of use cases don't really fit into just being entirely edge.

Mhmm. Even with, say, Vercel, for all their efforts in doing, distributed computing, or edge computing, when you actually log into your Vercel instance, there is a drop down that you could actually pick your main location that it that it puts. Interesting. It's, it still needs to run that Lambda somewhere in some AWS data center. So there's still a a a main place that you're actually doing real compute.

You can definitely, if put with a lot of effort, build entirely edge, services, but there will be points at which you need to escape from that to build, to have the full, you know, a company of of real runtime. Sure. And any other any other service that you might need to inter interconnect with, like any, like, queuing system or whatever or and any any real app that is doing something more complicated than just serving a website is gonna need other resource that cannot be run on the edge. And, honestly, it doesn't necessarily even need to run on the edge because we can get you know, there there is, like, the liveness of the website and your interactivity. And then there may be other things that you need to do in terms of, you know, workers or sending batch emails or, you know, do doing other compute or, like, you know, on YouTube scale, like, you know, upload a video and you have to transcode it and all that.

So there there there's there's always other things that you may need to do in a in a business to serve whatever use case that your business is serving, but the the, serving of a website can kinda be isolated down to, you know, this edge thing.

Speaker 0: That's but that's fascinating. I never thought about that. Edge is like a is like one of the tools in a wider application. It's not I it's not application that's very traditional or perhaps distributed or whatever, but it runs properly. And then elements of this application are run on the edge that benefit from the, from the characteristics of running on the edge.

That is something that has never quite I never knew that. That's really interesting. But is it fair to say that that core application has to coordinate the edge nodes? Is that what you're what you're thinking? Because you said because you were talking about stuff going back to your main application.

Tangent.

Speaker 1: You you you you could. And I and I think it's it's really clicking with you when, earlier when I was describing, you know, just delivering full page versus those small elements of it, and it's like and it's kind of exactly that. But in terms of coordination, you know, this this then gets into, you know, microservice distributed computing type deals of, like, okay. They they can be isolated in their own way and, more was more referring to when sending things back is kind of the data that the user may may have input. So user input or, you know, if they're chatting away, if they're interacting with if they purchased a ticket and writing that back to the source of truth that ultimately, there must be a source of truth somewhere.

Ultimately, some database needs to know that, look, the tickets were a 100, and now they're 99 available. Someone needs to know and be written back to, and that's what I what I'm referring to when I say, okay. Eventually, get back to

Speaker 0: Got it.

Speaker 1: That that source.

Speaker 0: Got it. Got it.

Speaker 1: And there's many strategies that we can employ on that source of truth because we can we can have a distributed source of truth. Kinda this Cloud Spanner database from Google, which kinda came out, like, 8 years ago, and that that white paper was making its rounds in distributed computing land, was really, really exciting because it sold it was trying to solve the problem of multi node writers. It's a challenging databases today to have multiple, multiple writer nodes. So current current database technology relies on having a single writer with multiple read rep. So you can you can you can replicate readers all day long because it's easy to, and sort of the the whole theme of this conversation has been it's really easy to replicate, caching content, so replicate reading content, but it's very, very hard to replicate writing or updating content or Absolutely.

Invalidating. So it's always been very easy to do low latency writes with low latency read replicas where, you know, you might have, you know, those, you know, basic basically, the the ping between 2 servers be your latency from from read write replicas. So you may say and it and depending on your use cases, it may be acceptable that, okay, Europe is, say, 500 milliseconds behind US. So if you were to go to a website, technically, you would be seeing 500 millisecond, stale content, and that may be acceptable. But it may in terms of fairness of the Internet or the World Wide Web and not the UK, US Web, It's unfair to set to always put an advantage on, your your geological, position.

So, technically, or, and I think that there there was an article once upon a time around when the Cloudspana or one of the Cloudspana customers originally was Ticketmaster. And their particular use case was this sort of fairness topic of well, okay. Australia will always be 1.2 seconds behind everybody. So if they go to book a ticket and someone literally just, you know, 1.2 seconds closer to the server clicks it, they will always win. So it's unfair to put people at this kind of dispute disadvantage.

So we're trying to build technologies that distribute these rights more evenly, and you can get kind of more fair distribution of of updates.

Speaker 0: Yes. Ticketmaster, the bastion of fairness. Sorry. Please continue.

Speaker 1: Well, in in in reality, what they actually want is, you know, more they can cash less and not have this this, queuing system, which which leads to more clicks, more purchases. Right? That's that's that's the end goal. That's the end goal with all of this. There's well documented stats about how Google goes about about web performance on the web.

And, you know, if you're x milliseconds slower than a competitor, you know, you're more likely to drop off and just go to the next site that is faster. So while us as engineers, we wanna over complicate our life and do fun complicated engineering things, Us as business people, we wanna deliver things fast because it yields greater returns and is a better native experience and will yield happier users. That's ultimately the goal that we're trying to trying to solve, while also doing it more cheaply, hopefully.

Speaker 0: Or at least over time. So a question for you then. In the current state of edge computing, what are people actually doing on

Speaker 1: the edge beyond what, you know, what was once possible with, you know, just CDNs? What people what the most trendy thing now is we are rendering HTML on the edge. We are rendering websites closer to users. And what I mean by rendering websites is we are rendering more interactive content next to the user. So no longer are we constructing the whole webs web page, you know, 800 milliseconds away from the user and then delivering that over the wire.

We are constructing it closer, meaning all the other subsequent calls that we might have to make to other services, are hopefully also calling services also closer to that edge region. So it's it's more and more, companies are putting more and more of their own services in more locations. Sure. So say, let's take an example of I wanna construct a web pay a Shopify web page. Shopify have a fantastic global API network.

I argue that actually Shopify have the most distributed database on the market because actually, you know, when you update something in Shopify, it's already distributed to their network of of of re replicas around the world. If I was to construct the data there, and then, okay, I also call out to another service that I control, or I also call out to, like, another partner API. If we if we request that, make a request there, and it goes 800 milliseconds to wherever it was being constructed, and then that that does its course and it comes 800 milliseconds back, that's, you know, 600 a 1.6 second round trip or whatever. If we take that request and say put it right next to the user and go, okay. As soon as it comes in, I'm instantly streaming back.

So it's close to the user, say, 20 milliseconds away. Generally, edge location is a sub 100 millisecond close to the user.

Speaker 0: Good to know.

Speaker 1: So they they hit it. Instantly, they get back, you know, they white page with loading things, and it goes, okay. Slows this. Okay. Now I need to grab some stuff from Shopify.

I've done a request to Shopify back and forth. Hopefully, they're also at edge location, 20 milliseconds back and forth on my edge. I've got my Shopify content. Oh, I'm doing an AB test. I've hit my AB test service.com, and I've got back the result of what needs to happen.

And, okay, I've loaded that content. And it's this it's this dynamicness that we wanna put closer and closer to the user because if we were

Speaker 0: to And it and it compounds. Yep. All of these 800 millisecond round trips are grueling, but a bunch of 20. I mean, of course, you still wanna be mindful of how many you're doing, but that's the same in any web application. But we're talking orders of magnitude quicker realistically when you're loading any modern web application.

Speaker 1: Exactly. Yep. So it's it's it's more it's more of this dynamicness because we could totally just off cache that entire page and delivered it to all the users equally. But what if one one user is logged in, one user has a basket where they, that that that we wanna also display and and render on the server? We could totally render it on the client, and this is where also this new modern trend of, you know, it was very it was perfectly fine to deliver entire huge bundle to the user and it be interactive and do API calls on the client side because the client side is also an edge location.

Sure. But we pay for the upfront cost of them receiving that bundle. So if we can

Speaker 0: That's interesting. Yep. So it's that trade off still of speed. We need to deliver a a reasonable experience as quickly as possible. Then, ideally, we want all subsequent requests to be as speedy as possible.

But if, you know, if this is where we're starting to pay a little bit, that can be more acceptable dependent on user and kind of, developer experience or, like, you know, vendor experience requirements. Exactly. That's really interesting. So it's just this trade off of of requirements. I feel like this, you know, edge computing starting to gain some traction is bringing a whole new realm of, I don't know what to call that requirement gathering, like, trade off decisions for developers who didn't need to care as much about infrastructure before.

And now suddenly, the same way front end developers now have to do a bunch of back end work with these kind of mixed hybrid frameworks that run everywhere. I feel like in the in the same way, anyone who does back end work is now having to care about infrastructure in a way where perhaps the the expectations in the past were less than. Yep. And then it's the job of these vendors who who run edge nodes to make that experience as contain as little pain as possible.

Speaker 1: I think I think you've hit the nail right right on the head there is, we we've we've this this trend is the flip side of is is the next evolution of SPAs, of single page web apps. So that was our edge in the past, and that was our limps limited subset runtime. We could only run JavaScript, then we could only run web JavaScript in that runtime. And, what we're having now is that it's a limited run time, yes, but there's we have more languages, more things that we can run closer to the user, and it's we're delivering them. We're not using the user's resource.

Right? So a a lot of SPAs or a lot of, mid 2000 in the Internet relied on, you know, fast devices or, you know, doing a lot of read writing of, you know, the DOM tree and all this, and then lower, lower quality devices with less CPU, like dumb phones or or or or basic Android devices had, you know, a worse experience on the web. And what we're trying to deliver is by doing doing the complexities of the HTML rendering and whatnot close to the user and delivering them, you know, the content that just needs to be rendered and device can become dumber again, and need to do less, you know, JavaScript execution on the client. Because, really, all we all we actually want at the end of the day is to deliver that HTML to the user. And that liveness, that interactivity used to come from or used to only be able to come from SPAs, client side JavaScript.

And what we're seeing today is that we're moving more and more of that client JavaScript onto the server, but also quite close to the user to be kind of that native like experience.

Speaker 0: And that and they're the parts they're the parts. It is we want to make the users do as little computation as possible while not, suffering based on their location. The fact that a load of that computation is going to happen elsewhere. And it's that specific pairing of of requirements, that makes edge com that makes distributed computing more powerful and even more so edge computing because they run right at those interchanges. That's that's great.

Now, for some of the other episodes of, learning things I love to hate, we went into a demo, but I feel like we've covered a lot here. I don't necessarily think we need to we need to do it so much. Plus, it's a really abstract concept where, you know, I'm sure there are ways, but it isn't gonna be now it's running on the edge, and we can really show that to its fullest. But I think I understand a little more why why edge computing is interesting now. Some more or less basic concepts about how it works when it also might not always be right.

And this new idea that, oh, no. It's just parts of your application you can delegate to the edge. Again, not something that is spoken about by the hype machine very often.

Speaker 1: Yep. It's just edge edge

Speaker 0: edge edge all the time.

Speaker 1: Just parts. Just parts. Right? And we we we have the we have the same mistaken view of serverless computing, in the past where it was like, okay. Now everything must be serverless.

Mhmm. No. It's a subset of use cases that are that that benefit from it, and that's the it's the same with the edge. It's not we don't wanna put we don't need to put the entire server close to the user. That's impractical.

But elements of it definitely can. And, and, yeah, and I think the the the the the reason it's so abstract in this way is it's hard for us to experience what other people are experiencing when we're in our, you know, our our nice modern MacBook on gigabit Internet in countries with good Internet. So it's it's hard to put us put ourselves in those shoes. And, the way we used to do that with SPAs and stuff is literally have a crappy old Android phone for dev that you would, you know, every so often look at and use. And we can replicate that if try try the web on a, on a VPN every so often.

Or if you're in a foreign country, try going to some of your favorite websites and just seeing how much slower they might feel. Because, yeah, it's it's

Speaker 0: conjured up a memory of a it's slightly off topic, but you've just conjured up a memory of in London, Mozilla used to have an open community space, and in that, they had what they called the Mozilla Device Lab. I imagine there

Speaker 1: were a few of them, and

Speaker 0: it was just a bunch of different devices. Or, like, they run they run browsers which are, you know, really, really, really limited with this exact idea that you could be testing on all these different devices. But, yeah, now we need to consider and I suppose that's the other part here. What edge computing enables is less computation happening on a user device, so it's lowering the needs on the user device, and it is making that computation closer. So we're kinda leveling the playing field.

It's this fairness, as you mentioned earlier, of you don't need the best hardware, and it doesn't necessarily matter where in the world you are. Both of those factors are handled by this new emergent technology.

Speaker 1: Exactly. Exactly. And it's it's it's all it's all right as rain for us to talk about fairness, but at the end of the day is we want to have users around the world because we're hitting we're hitting more ability for us to sell to more users. And at the end of the day, that's really what we're trying to solve with with edge computing because get more and more users with more and more happy users because we can have users that are unhappy, but, really, we wanna have more and more happy users.

Speaker 0: Yeah. Awesome. Thank you so much for joining me. This has been really, really interesting. I'm loving filming this series.

I'm learning tons. And hopefully people who have joined us for the ride also, are learning loads. Just before we go, is there anything else you wanna share, point people to where to find you?

Speaker 1: Sure. I mean, if you want if you wanna see more of, like, crazy drawings on my Twitter, I'm I'm at Pandeliszed, p a n d e l I s, zed, on Twitter. It's it's funny when I was drawing that in the office because we we kinda sit in a bit of a semicircle, and my colleagues turned around and were like, what? How are you doing? I'm like, oh, just drawing.

Just drawing some stick figures. I was like, okay. Fine. But, yep, that's that's that's where my normal antics are or on Twitter. And, and yeah.

No. I'm so so happy, that you had me, Kevin, and it was, I I saw your brain clicking at the end there. So I'm so glad that I could I could share that with you, and, hopefully, it's it's no longer anything that you that you hate.

Speaker 0: It is not something that I hate. I have, a newfound like other episodes, a newfound appreciation and more importantly, an understanding of the qualities. Right? Which mean that it may be something I reach for when appropriate. Whereas before, I just lacked the knowledge.

So I was never gonna reach for it. And, you know, in time, that could lead to me building things that aren't as good, as they could be. So, yeah, thank you ever so much. Thank you to everyone who has joined us, and, we will see you in another episode of learning things I love to hate. Bye.