AI in the Enterprise

Speaker 0: Excellent. Today, we have another episode of Bridging Bytes. It's been a minute, since we did our last one, but I'm extremely excited, to have two folks here to run through some exciting, topics around AI in the enterprise. So today, I'm joined with, with Holger Hammel, who joins us from HelloFresh. I actually think the intro came from Emma, our VP of marketing, from your time at Ivan.

But, really excited to have you here, Holger. And then also Peter Bell, CTO, founder, head, of AI over at Gather dot dev. And I think you're actually a little bit closer, Peter. You're in Absolutely. In New York, in the city?

Speaker 1: New York City. Just north of the city in Westchester.

Speaker 0: Nice. And, Holger, you're you're over in Berlin. Right?

Speaker 2: Yeah. Correct.

Speaker 0: Yeah. We have slightly different time zones and, and lighting probably in the background. But, Holger, do you wanna give a quick intro in terms of, like, what you're doing over there at HelloFresh?

Speaker 2: Yeah. I like to. So first, thanks a lot for having me. It's a super interesting topic, of course. And so I'm VP engineering at HelloFresh.

I'm leading there the consumer alliance. So it's, like, about three hundred three hundred engineers across different tribes and squads, and we're covering the, client side applications, right, the web and the app page, and first layer of back end, aggregation layer, and data science, data engineering a little bit, and customer care. And, basically, you know HelloFresh. Right? We have, we have, the meal kits in, The US and in European countries, 18 countries overall.

And we deliver fresh meals to everybody at home, and we have as well ready to eat meals. And it's a it's an interesting, setup where we have, physical products delivered to to customers, right, really, like, food products with their own kind of challenges in a sense, right, how to do this. And then we have the digital product on top, that we work on. And my key part here is to really, solve for personalization, right, to really create a digital experience that brings the best customer experience. And, and as well, and of course, the topic for today is, like, how to make this very effectively, and kind of make best use of AI internally for engineering departments, but as well of of introducing it into our products, to, for

Speaker 0: Love that. Very, very relevant. It's always good to hear, you know, anything on the hardware or the physical side, with what you're doing at HelloFresh. It's that that's the moat these days, in terms of, in terms of AI, and everything that that it's it's gobbling up. Thank you, Holger.

Peter, could you maybe tell us a little bit about Gather Dev, and what and everything else that you're you're doing on your side?

Speaker 1: Absolutely. I was an IC, wrote software for many years, then I became an engineering leader, CTO of a bunch of startups. I ran engineering at general assembly, built teams up to about 50, so fairly small scale. These days, I'm doing two things. I am writing the book, Scaling AI Adoption and Engineering.

Basically, AI for CTOs. And it sure. We're gonna cover software factories and verifications and all the tech, but the hard part is how do you manage stakeholder expectations? How do you do the change management? How do you teach people when the curriculum changes every three days?

Like, how do you do that at scale? And so we have these niche in person and online communities for founding CTOs. You're like an individual contributor or a team lead. You got a team of less than 10. Startup CTOs venture back to a scaling from 10 to about a 100, like, how do I hire my first DMs, and how do I do performance management, and how do I create a consistent hiring, process, things like that.

And then VPs and CTOs at scale who are running also, like, a 100 up to a couple thousand where it's generally good news is you have the systems in place. Bad news is they are now what is stopping you from getting the work done, that and the people. And so I spend a lot of time talking with engineering leaders about how do you scale adoption, not only what are the good technical practices, but how do you actually do the change management, which will be the hard part.

Speaker 0: Love that. Yeah. I mean, that is super relevant for some of the questions we'll get into today. I I, you know, built teams to 50 sounds small maybe, but, that's still where we are, and that's our whole team, over here at Directus. But, and I think you also do CTO hour, right, part of O'Reilly's Yep.

Speaker 1: So couple of the things I'll do is I do a quarterly CTO hour for O'Reilly. The most recent one, I got to interview Camille Fournier, the author of Manager's Path, and Ali Adasan, who's the CTO at Dropbox. It was a lot of fun. And then twice a year, I get to, go to KubeCon, the Kubernetes conference. And for CNCF, I get to facilitate the executive summit along with Kelsey Hightower.

So that's always a blast.

Speaker 0: I love that. Is it KubeCon or KubeCon? Because it seems like Kubernetes, it'd be KubeCon.

Speaker 1: Yeah. You know what? I don't even know how to pronounce it, but with my accent, people always think I'm saying Q con, which is great, but a whole other thing. I I was at QCon AI recently presented something there, but so I I just try to make it clear, which one I'm talking about because my my computer always gets it wrong for sure when I'm dictating.

Speaker 0: Yeah. Well, say it with confidence, and I guess it doesn't matter too much. So let's kick it off. I think sort of like just table stakes, you know, setting the foundation here. In terms of day to day AI, Holger, maybe you can kick this one off, with your thoughts.

Where where is your organization today with AI? Like, what does it actually look like for your employees, for your teams? You know, how's how's that actually operate?

Speaker 2: Yes. So I think we are in a crossroad. Right? In a we're in a in a in a in a situation where maybe some of the teams are as well, but, we are, very good, I think, already in adoption of AI on an individual level. So we have all engineers, most of, you know, using some form of AI.

We have everybody using some form of AI to kind of improve the documentation, their workflows, and so on. What we recently, what we started, beginning of, the year or a little bit before as introducing our framework for the whole product development life cycle. And that's the interesting part. That's where we wanna go to, right, is, really understanding how we solve for not just cycle time, like, making one engineer faster, but, like, getting the flow for the whole squad to a complete new level. Right?

It doesn't really make only sense to have one engineer being four times faster. Right? Not bad, but it's not enough. Right? And I think we're in the middle of we we as organization understood this.

We are in the middle of transforming this. I really look at this as an AI transformation. We had DevOps transformation. We have AI transformation. Now we are in the in all of that on steroids, right, which is the AI transformation, and really trying to understand the cultural, the technical, aspects of this is something we're in the middle of right now.

And it's something. Right? So you see engineers having, like, tons of agents running. You have designers creating pull requests. All of this is happening.

Right? Just really bringing it together and super effective, on scale is where we are at and trying to solve for.

Speaker 0: Yeah. Is there is there a specific team or department that was know, doing something different than you expected in terms of, you know, that day to day? You know, may either moving faster, maybe they're moving slower, or just kind of a different different vibe altogether?

Speaker 2: I think it was what what was striking was that we have a few, like, smaller businesses that, basically, we have, like, HelloFresh and and Factor, the established big businesses, if you want. And then we have smaller, separately managed, startups, kind of. Right? So GoodJob and Pets Table. And they are organized a bit more lean in a sense.

Right? They kind of kept the startup vibe, and they you saw picking up. They saw we saw them picking up very fast. Right? Because it's just out of pure necessity in a sense.

Right? So resources are scarce. So to get anything done, right, and they couldn't ask for, like, five more engineers, and they had to solve it. And they start solving it with AI, and they went very fast. Right?

And they were the first one adopting it. And there was maybe less red band as well. Right? So they just, you know, did it. Right?

And so they kind of act as the as the template or as the as the as the leaders down. Right? And where we got the inspire the whole organization are inspired by that speed and the that results.

Speaker 0: Yeah. I mean, that's a huge network effect to be able to inspire other teams, you know, by seeing how fast that you can move and, like, maybe be more efficient. Obviously, that startup mentality is huge right now. We're seeing a lot of, you know, constriction across headcount in different orgs, you know, huge orgs that are saying like, oh, we can not just do things more efficiently, but, let's kind of adopt this more agile, you know, startup way of thinking. I think that's good across the board, you know, and, of course, sort of, sped up by by AI.

Speaker 2: And the interesting thought I was just want to mention is really is the the constraints. Right? So the the the constraints of limited resources and still an ambitious motivated team that wants to run this, this was a key cut cut cultural aspect. Right? Because if you then have a big organization, very distributed, decentralized, like everybody does their part, this might not be there.

Right? And so thinking about how to create this urgency, is a big part of that transformation, I feel.

Speaker 0: Yeah. Yeah. I mean, the only you have these, like, enormous organizations, and it's almost like you you can only shrink the overall headcount so much. Like, how do you actually get that startup speed when you have hundreds, thousands, tens of thousands plus of, employees? And I think, you you know, that sort of had you eat an elephant, you know, one bite at a time.

If you kind of break up your teams and kind of make them very autonomous and give them that that skill individually, they can operate like many different, smaller agencies or smaller smaller, startups within a larger org. That's I love that. Peter, similar question to you. I guess, you know, what what does that AI look like day to day for you? And maybe more specifically, like, how has that changed, you know, over the past year or so?

Speaker 1: So for me personally, it's a very different thing. As an individual, I'm a solopreneur, and it's great. I have a team of eight named agents running twenty four seven. I built my own little orchestrator. I'm creating my own custom memory system, which, puts skills.

It puts, prompts and agents into a GitHub repo, and some high value shared informally structured context also goes there. Everything else goes into a Postgres database, which allows me to have things like I can have agents doing message passing through the database so that I can have multiple agents collaborating without having to go through a pull push cycle that I would have to if all of my tickets or work to be done was in some kind of Git based system. So personally, it's great. I'm just starting to dig into software generation. I'm looking to spin up in the next two weeks my own clone of monday.com so that I can basically have an interface for playbooks for running the business and projects for either running experiments, building new playbooks, or incrementally improving playbooks.

And the idea is deterministic workflows with small model and human of the loop steps. So that way I can firstly reduce the cost latency of the models by the like, I just need a simple classifier. I don't need to be running Opus four six to do that. And then on the other side, for the human in the loop steps, every single interaction is captured in a fresh session. So we keep them in the smart zone, very small percentage of context used, and it means that then I run a compounding loop so that every night, every agent reviews what it did and updates its own instructions and skills in a way that it would be able to do it a little better the next day.

And I'm already this system I'm building, I I started two weeks ago, and it's already running the entire business. And now I'm digging into software factories and the verification and looking at the Dan Shapiro, looking at what OpenAI did internally, looking at what Harper Reid is talking about about building rich software factories. When I see people who are blessed and cursed with 200 or 500 or 2,000 engineers, it's a slightly different story. The first thing I see is that, the widespread thing we're still seeing is augmented rather than agentic development, cursor in IDE mode or Copilot. It's fine.

And if you wanna be 30% quicker doing 10% of your job, which is actually writing the code, you're gonna get some incremental improvement there. I think the next step to go is agentic. So maybe you've got six or eight small agents doing that you trust to write code or review code. That's fine too, and that you know, maybe you can be one and a half, two x as fast depending upon the code base, how well you prompt it, how much context you provide it. It starts to get interesting then you need to compound.

I see people saying, oh, you know, like, I keep trying to do this, and it creates bad code. I'm like, have you told it what good looks like? Have you told it to remember what good looks like? And then if there's too much context where you're blowing the entire context window with all the rules, have you decomposed it so that you have one agent writing the code, one reviewing it? Maybe you have one reviewer that's looking for cyclomatic complexity, quality of naming, and architecture, and another one that's looking for security because you don't want one agent to have to solve for seven different dimensions in a single pass.

It starts to get dumb. And once you so that then becomes interesting both on an individual level, even I'm getting compounding out of my small, agentic army. But once you start to do interesting things, you generally have a DevEx or platform team that owns the repo with the skills, agents, and prompts, and the key context, what you then do is you have this mechanism where other people within the org can fork that, tweak it, try to make it better, and then if, say, three people in the org give it a thumbs up, you then say, okay. This now gets owned by the head of AI, the platform team, the DevX team, and then you can share good practices across the organization without a priority knowing what the good practices are gonna be. Because Hochul is absolutely right.

This is exactly the same as a cloud migration. You don't just give people a book on Kubernetes and say, are we there yet? Right? You actually want to have a program manager, and you want to have dashboards, and you want to have KPIs, and you want to have lunch and learns, and you want to have training sessions and Slack channels. You want to elevate people in your in your all hands.

But at the same time, while you're doing all of that stuff, the curriculum changes every day, so you actually need to have a bottom up mechanism to get the best ideas from whenever they come. The one other thing I would say is, create a little bit of space for solo players. You can run way faster with this stuff solo. And if you have a small number of fire breathers, especially if they're building something that's not it's not the main way you charge all your customers. Right?

This is the admin dashboard or this is the internal dev tooling where it goes down for an hour. It's not the worst thing. Those people will learn the practices that can then broadly be shared to the majority of your team. There's three buckets, and I promise to stop talking. The first bucket is the people who can't wait to do this more.

They play with Steve Yegi's Gastown on the weekend. They are killing it. They're like, if I don't have 60 agents running, if I'm not blowing through 12 max plans at $200 each, personally, I'm not doing it right. Then there's the vast then there's some people who are honestly like, this is destroying the planet. This is destroying my job.

This is miserable. I shall never do this. AI doesn't know how to write code as well as I do because I'm a Java developer. And they're gonna have a hard time. Hopefully, they will get AI infected like we got test infected for TDD.

And the vast majority of people are somewhere in the middle saying, dude, I would do this, but you're telling me to read 50 blogs. Don't I have, like, features to shit? Tell me how to do it, and I'll give it a shot. Yeah. And you need to figure out how you deal with each one of those populations.

Speaker 0: Yeah. No. And that's that's huge. And it's interesting, you know, hearing about, you know, minimum spend, like, you know, on on AI. Like, is that a motivator?

You know, you just kinda say, like, you're not spending enough. It's also interesting, you know, hearing the comparison of, like, a small team, a big org down to small teams or solopreneur. You know, at the end of the day, you can ramp up the the minions as it were, and get the the team that you need quickly. But you mentioned sort of like piping your data into a database, you know, into Postgres. Like, whether that's you doing that with with the minions or it's a bunch of teams, I think that kind of bridges into the next the next topic pretty well, which is either way, we're connecting data up to these these services, up to the tools that we're building, whether it's internal and maybe, like you said, it goes down for a second, Peter, and maybe that's not the end of the world.

You know, maybe it is, depending on the usage. But in terms of governance, like, how how do we find that balance? Holger, I'm gonna throw this back over to you. When we think about people moving fast, you know, getting that agility, we want people building things and experimenting, optimizing their workflows. How do you think about guardrails?

Because, obviously, you know, people are just kind of out there, you know, building, but, you know, data is is crucial. Data is the backbone of all of this. And if that gets leaked, if that, you know, isn't know, given given the proper RBAC or permissions, in these quick systems that are being built, you know, how do you how do you speak to that across your org?

Speaker 2: That's a very interesting question. And there's probably no final answer, but I think one, of course, we we have a high responsibility for for the data of our customers and our employees, and that's not gonna be sacrificed or, like, a change. Right? So we need central governance for for data, and we have it. So it's basically centralized through the existing teams that we have, like security or data privacy teams, reliability teams, ops teams.

And we have a central centralized Gen AI team, basically, infrastructure team that kind of, is owning owning those things. Having said this, traditionally, you have guard rates around cost or maybe, I don't know, who can who can access to this. And and this, we we deliberately said we wanna, while protecting all the all the, PII data and relevant data very clearly, we want to open up basically everything else and and make people just try it out. Like, remove every red tape that we can. Right?

And I think, you know, we were not looking at, should we have Corsa and Cloud Code and Gemini and whatnot. Right? We just take it all, cut it out, and see what sticks. Right? And if there's a new kid on the blog, we probably take this on as well and then see later, how we decommission it.

It's just more important to get people excited, to get people working, and then, there will be a time for consolidation. And, you know, it's it's kind of ongoing. It's ready. We know more about, like, how how to manage context, you know, how to have a memory that is not, tool dependent and agnostic and stuff like that. So that helps.

But I think that's the key part here. Right? It's removing red tape where we can while, while protecting the the data that we have to.

Speaker 0: Yeah. I mean and there's also I mean, shadow IT, obviously, and now we have, you know, shadow AI. Is that an issue for you? Like, people just bypassing, like, you know, the the research shows that, you know, when leadership is actually shaping all of this, AI governance, you're getting better value for your company. But you're still gonna have your ICs, your, you know, pretty much anybody go out and say, I'm gonna use my personal account, my personal service.

You know, is what is your solution to that? You know, is that that that's, like, sort of a sidecar risk. You know, do you just lock that down? You just make it easier to use the approved, like, internal services? Or, you know, how do you avoid that sort of issue where people are using things that just aren't even approved and they're just kinda going rogue for better or worse?

Speaker 2: Yeah. So the honest answer is you cannot fully fully, kind of mitigate this, I would say. Right? And I think, I think we have a very compelling offer for people. Right?

The you know, you have, you have, plans for for, for Copilot, for Gemini. There are. Right? So I think, if people now still choose to copy paste something in their own, Chativity, you cannot really, do much about it, I guess. But I think it's it's about training.

Right? Kind of, we have a policy for for JNI. Right? We very early on had a had a policy and a got got guidelines, of course. Right?

So that that people understand this. And I think especially if you're not in engineering, we want everybody to become a builder. Right? But not everybody in outside of engineering might have the context and might be security, sensitive also. So I think it's a big part of, like, training and making people aware of the risks and, what IP means, and, you know, what are the difference between private and enterprise accounts and stuff like that.

So that is very, very important. But I think we tend to be more encouraging of using the tools than kind of limiting for now.

Speaker 0: Yeah. Well, Well, I would hope that everybody's being security sensitive. You know, that's that's obviously the name of the game. It it only takes one incident, one issue, and everything goes down quickly.

Speaker 2: Wonderful.

Speaker 0: Yeah. Peter, I guess, you know, different different sort of, route to, you know, think about this. But, you know, are you doing are you kind of baking that into your process and how you're thinking of you know, if if we're saving data into a database, are you just piping it straight in? Are you piping it straight out? Is there are there any sort of things that you're thinking of when you're you're building out these these systems?

Speaker 1: So one of the things I like is that, in many ways, I can build, an engineering infrastructure that feels like a a bigger company. And in fact, I need to. I find that the guidance, the onboarding, the design systems, the less decisions an agent is allowed to make, the more processing it can bring to those decisions, and the less likely is it it is to make random bad ones. I I was just chatting with a a few CTOs running larger teams at breakfast yesterday, and one of the common threads was how can we go more heavily into design systems, into standard patterns, into processes, into minimizing the number of decent technical decisions that need to be made so that the agents don't get don't don't have too too much space to keep getting it wrong. So I I think, managing the number of decisions for the agents, I think being very thoughtful about data, you need to think through I think we're gonna see a lot more about agentic roles and permissions.

It turns out that if you just have a prompt that says never, never, never merge your work domain, It works almost all the time. If, however, you literally, I know somebody who has 50 agents and each one has their own GitHub account. And the nice thing is that you just put branch based permissions on main, and they are unable to merge it in until either a human or some other agent has done it. So I think you need to be very thoughtful. Don't assume that the agents will do what you say or even something reasonably close to what you say and have the guardrails.

And I think it also comes back to all the classic good engineering practices. You know, you should decouple release from production by using feature flags so that you can canary rollout. You can load test things. You can roll things back. All of these are important.

And I think to the biggest story is, like, whether it's shadow IT or, like, how you use this, I recommend picking a lane. If you think about it, technology adoption life cycle, crossing the chasm is like a 30 year old book, and it's still true. You can have innovators, early adopters, early late majority, and laggard. And, you know, that's okay. If you run a bunch of gyms, if you run grocery stores like physical plant, if you run ski resorts, maybe you can just wait till Microsoft figures it out and just tell everyone to go use Copilot.

And in a year or two, you'll be a little bit faster. That's actually perfectly okay. You're gonna lose anybody who wants to work with AI, so it's gonna be a negative selection in terms of the team you get, but you're gonna keep most of the people who know how your systems work, and it'll keep them happy. And shadow IT is gonna be a problem for a while, but, honestly, all the people who wanna use shadow AI are gonna leave your company anyway. So it just is what it is.

And then you can go to the other extreme. You can be like a Toby at Shopify. Right? You can be like, hey. Not only do I want to say we're AI first very early on, there was one time where he got the head of his, AI team to say, I want the 20 people using the most tokens.

Promote them. Is that a good business decision where those tokens being used usefully? Doesn't matter. That is a cultural concept that that creates a sense of we want to be innovators. We want the people who wanna be innovators, and that's what we wanna attract.

And if that costs, you know, 500 k in bonuses, it was totally worth it. So I think you need to figure out where you play on that. But then even if you're an f aider or early adopter, you still need education and enablement. Make it as easy as possible, like Holger said, for them to use. If you're just like, what do you mean we do AI?

We have Copilot licenses, and you can request one. You're gonna get a lot of shadow AI and lose a lot of great people.

Speaker 0: Yeah.

Speaker 1: If you work hard but have reasonable red lines that explain why sending PII to service in China may not be in your customers or your business's best interest, That kinda makes sense. And providing you teach people what's going on and especially for the nontechnical folks in the org, that you give them an understanding of why the rules exist and good ways to be a good corporate citizen and still actually get stuff done.

Speaker 0: Yeah. Oh, absolutely. It's interesting thing. I've never even heard of, you know, just finding who's using the most tokens and promote them. Like, I can imagine people starting to use their agents just to run other agents just to ramp up the token spend.

Over here, you know, building AI into our platform, it's all about, you know, how do we optimize this and get the tokens down. But I guess different different strokes for different folks. You had mentioned sort of nightly builds and sort of feature gating, you know, behind the flag. That's sort of kinda leads us into the next topic here, which is, you know, we we're seeing a million apps, you know, flood the market. We're seeing all these cool POCs and pilots and experimental, you know, things internal and external.

Like, what what gets that into production? Like, what makes that a viable application or something that, you know, you kinda mentioned it works almost all the time, you know, and, you know, we're gonna wrap it in the secure prompt, like, never ever ever do this, and it still does it, you know, on occasion. That doesn't fly in production when you're dealing with mission critical systems, when you're, you know, really building for the customer, and externally. Beyond sort of, like, the the point that you mentioned, Peter, maybe you can kinda is there anything else that you think is is critical? Like, we talked on governance and sort of maybe permissions.

But what else helps get you to that production scale and resilience?

Speaker 1: So there's a lot of things. A really good starting point is to remember we already have nondeterministic systems building software. They're called humans. There's just slightly different parameters when you deal with these new nondeterministic systems that are building software. It's it's an intern, and by the time you get to the end of the context window, it's an intern who's been on Adderall for two nights and is starting to forget stuff.

So you have to be thoughtful about how you engage with your guardrails. My assumption that to take the extreme example is that my only job is to provide training for my future robot overlords. Do I believe that's true? Do I believe there's no rule for humanity? Hopefully not.

But if I use that as an operating assumption, then there is no part of there's no time that I interact with an agent that I'm not doing it in a thoughtful way to improve their ability to give me a better outcome next time. So the first thing is you just rinse and repeat. You, like, you wanna build an entire software factory, right, where it goes from you just say the vision and it's in production with no human no required human gates between the two. All you do is you tell an agent to write some code, you tell it what was wrong, you make sure that it captures that context, and you tell it to go again. And you keep repeating that loop time and time and time again until you build more and more validators, adversarial reviews, and other elements that will reduce the likelihood.

And you know what? I've taken that in production database in my life. Like, humans get it wrong too, but it's much less likely the more revs you do around that cycle to figure out what does good mean, what does bad mean, and how do we put, both tooling and and also just prompts. You've got the prompts, so you've got the tooling, you've got the context, but also you do need, I think, permissioning systems. You need to think about the blast radius.

And then all the good stuff we've always been doing, observability, monitoring, alerting, I think there's great opportunities now for or self healing software is a fancy word for incident response where the LLM knows enough about the system to propose a a move forward. I think that's going to become mainstream. And, also, just stuff that we're doing fifteen years ago, game daying, resilience engineering. How can you make it so if your cart goes down, at least you can still see the products? If your payment provider goes down, it can still save your cart and send you an email once it goes up.

Assume that stuff's going to break more and create more resilient systems with less moving pieces so that even when things go wrong, the blast radius is constrained.

Speaker 0: Yeah.

Speaker 1: One final thing I'd point is and the only other part is you can also we're not gonna review every line of code at some point that's going away. However, you can take a statistical process control approach to that, which is it's like ball bearings. You don't check everyone, but but if one of them is out of tolerance, you start checking them more frequently. And you do it risk adjusted, you're probably gonna test a smaller percentage of the PRs for your admin dashboard than you are for your main financial production flow.

Speaker 0: Yeah. That's that's amazing. I I love that. It's interesting. You kinda bring up sort of like, oh, when this stops working, you know, you have this, like, graceful degradation, process.

I my first thought is, you know, you don't know what you don't know. And you have these sort of non engineers building out systems, and the system might work. But if they don't know to prompt in, you know, hey, let's let's make sure that we have this, you know, progressive enhancement, graceful degradation, then that just won't be included there, which is which is interesting to think of. Like, the creator of, you know, the software still needs to be architecturally aware of, you know, software engine, design and architecture.

Speaker 1: Two possible wonders I've seen for that. One model which I think is is good for now, which is I think we're going to see, you know, the two pizza team is not always going to be six to eight people now. I think you're gonna find triads in groups of four being able to get a lot of move a very long way and very fast when you have teams of twenty, fifty, 60 agents working with them. But there's going to be a combination of product and engineering skills. Somebody needs to know what we're building and why.

Somebody needs to be thoughtful about, wait a second, latency, queuing, retime, split brain problems, all this stuff we deal with with large distributed systems. Those awarenesses need to exist. And then you might have some design skills or some data skills depending upon where the group is in your org and what it's trying to do. So I think in the early days, you make sure there's an engineer in the room before something you care about goes to production. I think, eventually, if you build a sufficiently rich software factory, you build those architectural concepts into the review process.

Whether we get there and how close we get there, I don't know. So for now, I'm gonna keep a staff plus engineer having to look over the code just to be sure.

Speaker 0: Yeah. How and how quickly we get there. Exactly. Holger, I'm I'm really curious to hear your sort of, like, take on this. You know?

Obviously, Peter mentioned a lot of things that do get sort of these pilots up into production. But, you know, our platform direct us, for instance, you know, we power, you know, mission critical software where we're pumping out, you know, bandwidth at, like, nine nine thousand pizzas per second, you know, for for certain customers or whatever applications they might be, building. And it's funny to me thinking of, like, that that is, like, production grade, like, at scale. You can't just one shot, you know, or Vibe code your way to that. Like, there's there's a process.

So I'm curious, you know, when you take sort of that equivalent at, you know, HelloFresh, like, what are you what are you doing? What do you layer on top to make sure, that your applications, you know, if you're if you're building in that way, you know, through AI, that it can handle that.

Speaker 2: Yeah. That's a fantastic question. I think we, so so some some of the stuff, you mentioned, we do have to be one of the things, like, with the higher throughput, right, of of agentic AI or just, you know, teams speeding up. We see more, pull request reviews coming up with a new problem. Right?

So so we need to find more, ways of, I think it's a really good idea, right, to kind of, find another abstraction level, basically, to look at, you know, risk management and and reviews and and and in a sense. Right? So that's quite interesting. So I think there we need to invest more. Right?

This is something we need to do. What we have been doing is we have brought a few things to production, like AI products to production, which is kind of interesting. So we've been, maybe the most obvious case. We did go first as well. This is like customer care and having chatbots right there and experimenting there, a lot.

And there, it's it's really it was we were kind of mid mid class last year or so. Right? So we were learning how to do this as we go. Right? And then managing, basically, the technology was not so much of the problem.

It's more like managing the, really the the quality of it. Right? So all the different ways of of of how how a conversation with a human, the one deterministic, entity, right, can go, is, is, you know, not to underestimate. Right? And there's a lot of things that can go go wrong and how to how to design communication, the getting the intent right, and and keeping it safe, right, and and correct, is an is an ongoing task.

And we, as we invested into, you know, the teams, that build up this messaging and then as well, the the automatic test. We experiment with AI as judged. Right? So, basically, having LLMs checking those those those flows as well. I think this this is very, very promising.

Right? So this is one pattern that we see more is that you'll introduce, AI, maybe even different models, either for pull request reviews, having iterations on them, basically watching the manual testing themselves. Right? Like, having an eye look at, at how how it looks like and then doing pull requests by themselves. This actually works pretty, pretty well and has a very has a very high showed very high impact.

So these are a few examples of where I can see. Right? So I think my my main concern right now is really getting the the the review part out of the way and and and on the other side, getting more, I don't know, engaging more UX and so on to create more experiments. So that's the other thing. I would like to have more, more variants basically tested earlier with the idea of having just already a more winning variant, basically, created into production, have higher higher signal, higher, success with more expensive AB testing in production.

But why not having, like, 10 we we see this already, but I think we can do more 10 variations, test the prototypes, high high fidelity, bring it into the building, having tested before or with synthetic audiences. Right? Checking out quantitative analysis even on on scale if a business model works with a new hypothesis. So these are the things we're currently on and want to invest more into. So to get it on the front.

Right? So getting better better signals earlier in the process, having the winning variance going to production faster and solving the PR revenue bottleneck problem. There are a few topics that we're currently working on. And then a few other things about personalization, improving the models, and as well creation from automatic creation of videos and and images, and we're creating a lot of, like, meals. And we have some AI support tooling that helps menu chefs, right, the creators of recipes to improve the process of getting this from idea to production, significantly faster than before.

Speaker 0: Yeah. Yeah. Well, it it that's I I love that. How how big is your org, Holger?

Speaker 2: So my org is about 300, people right now. The overall HelloTech org. So how this is how we call this either, combination of all, like, tech technical staff, if you want, is, is about 1,000 people.

Speaker 0: Yeah. And I I it's interesting. Like, all this only happens if we actually you know, the governance, you know, getting to production, you still have to have the people to make this happen. You know, going back to early, like, we can't just make the agents have the other agents work. Like, there's still human in a loop as Peter keeps putting it.

It. You know, we think of, you know, Jack Dorsey and and kinda what happened over at Square. Like, switching to, like, the people side of this, you know, with an org of that size, like, how do you actually get the workforce to adopt, to use, to, you know, to actually have this happen in the first place. You know? Is I'm I'm assuming there's a fear piece.

You know, we've talked a little bit about, I think, Peter, you had mentioned sort of building your for the future AI overlords. You know? There's a perception, I think, across some people that, like, oh, am I just digging my own grave and sort of, like, going out and and sort of building these these systems? But, you know, we have to lean in. You know, we can't just ignore it.

So, you know, Holger, within your org, like, how does that work? Like, how do you make that happen?

Speaker 2: Yeah. So I like to to frame it in a way that, I think HelloFresh is a really great way to experiment with this, learn this, apply it on scale, and just make yourself, stay relevant if you want. Right? And and be on on on top of the game there. Right?

So we we really try we we try to make this work. We support people. We support the teams. And that's kind of the positive way of framing it on if you wanna play it a bit differently. If you don't do this, right, the gap between, what the what the what the industry demands or what where the where the top level is and and everybody else, it's getting bigger very fast.

Right? So you you you risk of falling behind in a sense, and that's not limited to engineering at all at all. Right? It's like every function and software engineering in and around it at least is, has the same kind of, opportunities and and challenges in a sense. But I don't, like, underestimate the, those factors.

They are they are real. Right? So and I think there's a system of incentivizing, helping people, making it, you know, as we talked about it, easy to figure to try out things. Right? But then as well, set up clear expectations, right, that, that we expect every team to onboard into the new process and to to adopt to this agentic AI, for instance, use cases or to to have more variance in production now that we can do it, right, fairly easily.

So there will be, the expectations grow steadily as well, to keep up with this. So that's the both of the side. Right? Enabling people, but as well-being very clear about expectations.

Speaker 0: Yeah. Oh, 100%. And it did like, Deloitte's latest report, I think, said something like only 20% of companies of orgs, actually, their their teams are actually ready, and prepared for AI. So, you know, getting that number up, hopefully, HelloFresh, you know, is in that 20%. But, but we're come we're coming up on time here.

So I wanna kind of at least be a little bit more forward looking, with with sort of wrapping up here in terms of what's next. Peter, maybe I'll send it over to you. If you're looking out the next, like, say, one, one and a half years, like, where do you think the most change is gonna happen within AI? With an enterprise AI? I I think you go anything beyond that.

I mean, even if you go out six months, the question mark start piling up so high. But if you were just kinda, you know, behind the sky, where do you think things are are heading?

Speaker 1: Within the engineering org, I think it's pretty straight this will be I'm not gonna put a timeline on it. I've got a very good friend, senior CTO. He's like, you can make whatever prognostications prognostications, but don't put a timeline on it. You you'll thank me later. And I I think he's right.

I have no idea. It will be much faster at smaller companies in greenfield environments, in places where you happen to have, an engineer that's just super passionate about this and pulls the team forward. It should happen more quickly in SaaS companies and in companies that are potentially have an existential threat from AI. As I said, you're on a chain of gyms, maybe it doesn't matter. So there's going to be uneven, speed of change, but I think, eventually, we're going to see a flattening.

What the year of efficiency continues to resonate many years after. You could imagine a world where you have triads or maybe teams of three to four doing something similar to what a two pizza team does now so you can split them out and have more of them capturing more initiatives. I I think it's possible you might have five to 15 of those reporting straight up to a senior director or a VP where you have a team lead in each. I think we're gonna continue to see the collapsing of that, and I think the VP is gonna be spending a lot of time on the keyboard reviewing detailed specifications, verifications, harness improvements, as well as research and product elements. So I think we're going to see leaner engineering orgs.

The good news is we need a 100 x the software, and, I think there are lots of opportunities for anybody who is passionate about either. And I think we need to be happy with this. Right now, we're all excited about AI engineers. And for me, that's basically harness engineers, people who are thinking about prompts and verifications and steps and gates. We need 8% of those.

That's our platform or DevX team. Everyone else is going to become a product engineer who gets better at proposing experiments, writing specifications, and creating rich verifications while also being thoughtful architecturally. And I think those are the two roles that are gonna continue to grow and be incredibly impactful in the future.

Speaker 0: Yeah. Oh, I love that. Slightly different lens, hold on. I wanna hear sort of your your sort of take on where things are heading. But, on the enterprise, something that's very sort of near and dear to my, my heart, my job is sort of the commodification of the front end.

Like, we're we're seeing that happen actively with the AI app builders. The value of software that's being built, where where does that value shift? As as the front end, as the facade becomes commoditized and and everybody can build that, where does it shift? I mean, down the stack to, you know, all the way back to the database or somewhere in between. Where do you feel, that's that's heading over the next year or so?

Speaker 2: Yeah. It's a it's a it's a very good question. So basically, it's, I I still think, a a creativity and and being able to create a mode, the connection to your customers, building up this relationship is probably more important than ever. So you can be if you double speed on the wrong side of the highway, right, it it will not help you necessarily. So, so and I think that the the productivity gains that we have, that we're seeing now and that are super exciting, they will become a commodity at some point.

I'm not putting a timeline on it either. But, you know, you you will have your great engineering teams, and product engineering teams. They will they will work on a completely different level. You still need to figure out what really works for a customer. And now the levels the field is, we're closer.

Right? So I cannot rely on, say, a company like HelloFresh being faster, to production with some features than a small start up. So I need to be really, really smart about this. Right? I need to be have the most creative people writing the best specs.

Right? So and, we have a lot of customers. We have a lot of customer relationship that is that is super valuable. That is not easy to copy, right, even with AI. So building on top of this and making sure that it stays this way and we stay on top of the innovation, that is what, I think, what still matters the most.

Speaker 0: Yeah. I mean, bring it back to the people, you know, the as a designer, you know, focusing on the creative. You know, you can always it's really exciting right now because AI is able to take, you know, this massive context and, like, pipe it in and make everybody creative, make everyone a developer. But at the end of the day, you know, you have to be able to break outside of, like, what's already there and sort of just rearranging those things. So I love hearing the answer kind of always coming back to the human, to the core of, you know, creativity and, you know, even going back to earlier.

You know, you can throw the the kill switch in there. You can do all the right things and still, AI will kinda find its way to work around it. So, it's really important to have people that understand the the proper architecture of of what we're building and and why those things are important. So, hopefully, we don't stray too far from from those first principles. Holger, Peter, it was really, really exciting, getting to just chat through, you know, and the enterprise with both of you.

You know, hopefully, we'll get to chat again soon, maybe again on a bridging bites episode. But for now, anything that you guys wanted to sign off with before we wrap this episode up?

Speaker 2: No. Just thanks a lot for for for having me. And, yeah. It was a pleasure talking to you. And, it's just exciting times, I must say.

Speaker 1: Likewise, Ben. Thank you so much for the invite, Holger. It was wonderful to hear your insights. So much fun and great conversations. Lots for all of us to learn.

Speaker 0: Awesome. Yeah. Well, absolutely. Thank you both for being here, and we'll see everybody, soon on the next episode. Alright.

Thanks, guys.