Building Advanced Custom Operations for Directus Flows - Berlin Meetup

Speaker 0: Hello, everyone. I'm Stefan. Sure. I'm gonna do that. Well

Speaker 1: Oh, there's a little panel. You can either kill the whole table. There's a little panel on the podium.

Speaker 0: On the podium.

Speaker 1: And I can mutate that.

Speaker 0: Hey. Okay. So, again, my name is Stefan. Hello. Thanks for having me.

I haven't prayed to the demo gods as well, so maybe we will have 2 comedy shows today. I don't know. I wanna talk to you today about how custom operations can take your flows to the next level. First of all, a quick introduction. I'm Stefan.

I'm a product owner since the beginning of this year, and before that, I was working for 20 years now as a software developer. Yeah. I do have a specific love for JavaScript and open source, but, actually, I'm into every web related technology. And I'm a proud father of 2 boys. I'm cheering for FC Koln, which is not so funny at the moment.

Well, at least we are playing in the Bundesliga for now, and also for the San Francisco 40 niners, which is more fun at the moment. I work for SunCinet. We are an award winning full service agency. Won the German brand award design awards, Red Dot Awards. You name it, we won it, several times.

Full service means, we do strategy consulting, digital marketing, but, of course, also development of all kinds of enterprise level solutions, which is, ecommerce, platform development, mobile application. You name it, we do it. And, also I wanna highlight, we are the 1st official directors partner agency in Germany, actually, and I think the only one right now. So, if you have any questions about implementing something in directors, there's a short link you can follow, or you can go to our website, or just shout out after the talk, and I'm glad to talk to you about that. We do have 170 colleagues working for us in 8 offices around 3 countries.

Our headquarters based in Cologne, but we have offices in Austria and Poland as well. Yeah. That's it. Now, before I want to, talk about flows, I want to show you a few more projects we've built upon, directors just because I thought if I would be here, I want to hear what people actually have built upon directors. So, I want to give you a wide range of what we've built.

Just a few slides. First thing, I can actually not drop any names here because of NDA reasons, but the project was pretty cool. We've built a complete live streaming and virtual event platform for a huge car manufacturer, based on directors. They, contacted us in the beginning of the corona crisis where all the live events were actually canceled, and wanted something digital to replace that, but wanted to have their own platform. So we built that based on directors, which was great fun.

Directors is actually used here to serve all the content and site configuration. We implemented Keycloak. We have a Go middleware in between, and the front end was built on Nuxt. The most cool thing we've built here was actually a page builder, which wasn't available at the time we built the project. Kind of is available now or will be available soon, is available already.

Great. So, yeah, just to give editors a more systems with a page preview, editor preview, and stuff like this. Another recent project, is a mobile application we've built for, the German Art and Exhibition Hall, the Bundeskonsthaler in Bonn, where we use directors to actually serve all the content app configuration. Also, user feedback that is provided through the app is posted into directors, so editors when they create surveys they get the responses just in directors, and they can also send out push notifications to mobile users. We actually misuse direct here.

We created a collection where they can just enter a title, a description of the push notification, a target group, and a few other things, and hit save. And we have a hook running afterwards to actually connect to the Firebase cloud messaging API to send the data, send out the push notification, and afterwards it just deletes the record. So, yeah. But, it's nice for the editor. They can use one system to do all they need within the app.

That's it. Now, let's go to the hot shit, directors flows. Any one of you have ever worked with directors flows, except for you guys working for directors? 1. Okay.

That's great because, otherwise, you can tell me something about flows. Now, for all those who haven't worked with flows, let me give you a quick introduction. Directus flows is pretty cool. You have a fancy low code drag and drop editor right in the data center, where you can create workflows to of several, yeah, of several types. Common use cases are data workflows, where you, for example, want to deploy as a website whenever content changes or gets created, or content workflows, where you might want to send an email to a chief editor, as soon as you want to publish an article, or scheduled workflows, like when you want to create automated reports on a nightly basis or stuff like this.

And, the cool thing is you don't need to code anymore. You can just go to this fancy drag and drop editor and create your flows right in the data center. Flows are being defined by 3 elements, A trigger, then you have operations, 1 or more operations, and a data chain. Now, Let's quickly forget about the data chain and talk about trigger and operations. When you create a new flow in the data center, you can choose from several predefined, triggers, which can be an event when data changes or something like this.

You can create a trigger when an incoming webhook, hits, the director's instance, scheduled scheduled triggers like what I said before, or manual triggers, or even another flow can trigger your flow. So the first thing is you create the trigger for your flow. You choose those from the predefined ones, and this actually defines what starts the flow. And then, we have 1 or more operations which follow the trigger, and if you chose the flow in directors in the data center, you also have multiple operations predefined you can choose from. Like you can implement conditional checks, or you can do all of the CRUD operations, create, read, update data, or whatever.

You can send out emails. Like what I said before, you want to send out emails to a chief editor, you can just use this predefined operation and don't need to configure anything. Directus takes care of it. You can, ping other web services, and stuff like this, or transform your payload. So, a lot of predefined operations you can choose from, very powerful.

You can connect them and create all your data flows. Now, there's one specific operation, predefined operation, which is the run script operation. So if those predefined operations doesn't do what you want, you can extend on this by choose run script operation, and then you get this code editor where you can put in your custom JavaScript code to be executed. So to give you an idea, I took this from a great video series about directors flows by Bryant, actually, from the directors team. I will share the link with you in the end of the talk.

And in this example, this is a flow to generate testing data, seed data. The flow is a manual flow. It is defined to show a button in the collection, and when you click on it, it will run this run script operation, and I I condensed this of course for this, talk here, but what it does is actually it will run a custom JavaScript code to generate the seed data and returns this data actually to the next step in the flow to the next operation, which is again using one of the predefined operations to then create those, data that you received from your run script operation. So this is super powerful. However, it has a few limitations, because this code is executed in a sandbox.

It has no network access. It has no file system access, and at least there's no easy way to include third party dependencies into this. So if you want to build more, it's kind of hard to put all your code into this. If you want to use the file system for whatever reason you cannot even do this, so this is where custom operations comes into play. Custom operations allow you to create new types of actions for your flows.

So, for example, you can expose an MPN package to, as a custom operation to extend the flow. In this case, we use the camel case function of lodash, to just, actually return, the camel case text, but we can also, let's say for example, taking the example from before, you can include Faker JS to generate your seed data instead of writing all your custom JavaScript codes to generate the seed data. And there's another cool thing actually about it. You see here, that we have the in our handler function, we get the data passed in from our flow, but we also have a second argument, a context argument object, which exposes actually all internal service APIs to us, which is super useful and which is not available in the run script operation. Now, that was the 101 on Calarectus flows and custom operations.

Now I want to talk to you about the wall arstvertreteliste. I'm sorry. I have no clue how to translate that into English. Maybe chief physician surrogate list. If you understand what I mean?

No. Okay. To give you a quick overview on the project, the client is the University Hospital of Cologne and they asked us to, digitize the Wallas Watriteliste. What is this? So let me quickly try to explain this in English.

If you have a private health insurance or if you go to the hospital and just choose to do so and pay for it on your own, you can choose to be seen by the chief physician. And for billing reasons hospitals are required to actually have a persistent documentation on which doctor was the chief physician or its surrogate at any given date? At least in Germany. At least in Germany. So the insurance company might call the hospital or they don't call them, they have access to the documentation, but they if they check the bills and they want to check if it's legit, they go to the system and check.

At the 1st May, was doctor Scholz really the the the chief physician for this specific department. So they they are required to have this persisted documentation. Now, the university hospital in Cologne, which is one of the biggest in Germany, has 33 clinics, and they have 235 departments within this 33 clinics. And in the past, this Wallas Watretelister was created manually. So each of the departments, they have a lead.

And whenever there's a surrogate for this lead, they were contacting by email the main administration office and say, hey, next week it will be doctor Schmidt's. And they had to create those from huge excel sheets, a PDF file, and put it in the documentation, in the persistent documentation storage. Now let me quickly show you I'll try to show you. Using PowerPoint. Yeah.

But So it stays

Speaker 1: on top of everything else. You'll probably need to just

Speaker 0: out. No. It's because YouTube is still in full screen. So this is the Waldorf Vaterlisten. So we have a huge list of, all the clinics which are on top here.

This is the chief physician of of the clinic, and these are the departments listed within the clinics. And if there is no surrogate for the departments then the chief physician for the department is also the chief physician of the clinic, and so on, but they can differ. And now this is the huge list of all the clinics and all the departments, and this was created manually. So and we were asked to actually digitize this. Now when we spoke to the client, we learned that we need a web based interface to do so, and we actually need this to do some data management and for asset downloads.

We were required to have a user lock in. We needed a system with fine granular roles and permissions, and we needed server side asset generation and email sent out because, I'll tell you later. So our solution to this was to build something on directors. Server side asset gen asset generation data management fits good. We chose a slightly different approach because, the external companies like the insurance companies, they shouldn't access the director's instance.

That was a requirement, so we had to build a custom view. It was built built on view now. We had to build a custom front end for those insurance companies anyway to download this assets. And then we decided to also let the people who work in the OniClinic and set those surrogates to also let them do everything indirect, in the this front end, so all the user groups they all interact with this front end. Directus is actually just running in the background and no one knows about it.

Now, fingers crossed, live demo time. So, let me show you how we implemented this using flows and custom operations. So first of all, this is the front end, and let's lock in with as if we were someone working for an insurance company. So this is the interface. They they just can enter, access the archive where they can either, select an individual, date range.

Let's say they want to have all the documentation from July August, and they see, okay, there are only 2 lists generated, and they can download it here. Or there's also yearly archives, where they can download a zip file of all the documentation, which were created within a year. Now, let's log out and log in with a user who is working in one of the clinics and want to request a change on the list. Now the user here is working for a specific clinic, and he can only see the departments which are listed in the clinics. Let me quickly show you this in direct us.

So we have all the clinics here, and we have the departments here, which are 235, but we can see that we only see 11, which are the departments connected to the specific clinic. And now, I want to change the surrogate for next week. So, if I open one of the items here, one of the departments, I get this calendar view and I can set a surrogate for the next week. So let's say instead of professor doctor Max Musserman, next week it will be me. And if we save this, they they they will also see which, in this calendar view, the ones which are actually requested and which are confirmed.

So Yeah. Now, they requested a change. The workflow now requires someone in the administration, and they get an email now. So this is why we required email sent out. They get an email now and say, hey, someone wanted to change a surrogate.

Please confirm. So, if they log in, they see all the departments. They can also enter the archive, and they have this new, section here where they see the change requests, and actually, product color for the changes. And now they can go here and see, okay, for this clinic, for this department, from date, start date, end date, he wants. So we can then say, okay, we want to accept this, or we can say we want to decline this, give it a reason.

If we decline this, it will also automatically send an email to the email address connected to the director's user who requested the change, and say, well, your change was declined. Please change something or whatever. But now, we want to accept this change. Okay. Now, before we continue with the real workflow, I need to change something.

Today is the 18th, so let's say 17th. Okay. Now, let's go to the super cool drag and drop editor of, directors flows. So we have our custom operation here, and this is the generate PDF operation, and we created a flow which actually has a scheduled trigger. It's running once a night, and it's reading the data of the requested changes, of the confirmed changes, in fact.

So, we have a filter here. Status equals 1 means so all confirmed changes within the last 24 hours, and then it will pass it to our custom operation. The custom operation actually just gets the data, last, which is an, quick access to get the latest data from the previous, operation, and we have some, yeah, extension paths we need for static asset generation, which are configured here. Now we do have 1956. Let's change this to oh no.

Damn. It just changed and now we need to wait 60 seconds. So let's change the crunch up here to run at 58. And while we are waiting, we can actually go back here. You can leave it in.

Speaker 1: When I'm testing, I generally change the trigger to just a a webhook, and then I grab the URL, and open it in a new tab, and then it immediately triggers.

Speaker 0: It was bad that it just switched to 57. Actually, in all the preparation, it was always like like maximum 10 to, now we got it, like 10 to 15 seconds we needed to wait. I think the lock should be available immediately. Okay. So, in the logs, we saw that our operation was running, and it actually found one change that is great, and it passed this to our custom operation.

Now what happened, it actually created it should have created a new PDF, So let's go back to the front end, and here we go. And now if we Demo guts. Let's access the file from here. Yeah, that's right. And have a look if it worked.

Great. So, we got the surrogates now here, and the cool thing is actually that we can now set start and end dates on those surrogates, and that not only it doesn't need to be created when on start date, but the main administration doesn't also don't need to care about changing it again when a surrogate expires. So yeah, that's actually it, and now, let's have a quick look at the code, which is probably more helpful if I go to Versus Code later. So this is how custom operations actually look. You can use, the SDK to create 1, and you will get this boilerplate here, which has a actually, this is the API JS file.

You have another entry file where you can configure the con, the user interface indirectors, which we saw before where you had this last and the paths, and this is the API JS file where you have your handler function. And as I said earlier, we have the second context argument where we can access all the internal services, like we get the services, we get the database, and all kind of stuff even more. There's the environment variables, the p no longer instance, our accountability information about the current user, so there's a lot of stuff we can access here. And, now, if you remember from the beginning of the run script operation the few limitations, we actually were not able to access the file system, and we were not able to include third party dependencies. I think it's a bit too too small, so I'm gonna switch to here and hope you can all read.

So what we do, you see, we import codes from third parties. So, not only will we enclose some note libraries, but we also include archive PDF, which is a wrapper just around JS PDF, which is defined as a dependency in our package JSON. So, we, implement third party codes. We also access the file system later on. You will see that.

And from the services we use the file services, and then we actually the steps are to get the data from the database that we need to create, this list, and then there's one keyword I actually need to point out. We cannot specify the encoding actually, in the director's file service, and the default is utf8, which in fact breaks for our custom PDFs. If we would just upload them to the file service, it breaks custom fonts and images. So what we need to do here is actually to save the PDF that we generate with JS PDF to the local file system as a binary file, and then create a read stream on that binary file, and then we can send this to the file service and upload it. And then there's a few more things going on, we actually save it to the archive of the database again, then we check if there's a yearly archive already existent.

If so, we delete the zip file that exists, otherwise we create, yeah, and create a new one, otherwise we create a fresh one, and so, yeah. It it's about 500 lines of codes, and yeah. I don't want to show everything here. I think it's enough to get you an idea what what else whatever what you can do with custom operations. You have no such limits that you have with a run script operation, so you can actually use all of the JavaScript power and do whatever you want.

So, really, custom operations take your flows to the next level, and I think that's it. Yeah. Quick recap. FlowOS lets you automate data tasks right from within the data center. Really powerful, especially the run script operation, but this one is limited, and we don't have such limits for custom operations.

That's basically it. If you want to learn more about directors flows or custom operations, I created, QR codes. I can created a short link which I can share with you to a YouTube playlist with all flows tutorials I found on YouTube. And there's also a quite new section with some cool guides, like how to send bulk emails with SendGrid, how to add item components, how to send SMS notifications with custom operations. Thanks.