October 2024

Speaker 0: Alright. Hello, everyone. Hopefully, you can all hear me, and I am gonna take a pause and just check that that is the case. So if someone could let me know, that would be great. Oh, I was excited.

Carmen's saying hello, and I'm still gonna wait. Loud and clear. Perfect. Thank you so much, Kevin. Welcome to October's version of the change log from Directus.

If you're new here or if you've, joined us before, welcome. I hope you're having the most perfect day. I hope your day is not too hot, not too cold. It's like the crispest, best version of the day because that is what you deserve for joining us. Thank you for for being here.

We have a great show for you planned. We've got some updates from Brian and myself from product and extensions, we've got Carmen with a docs update for what is upcoming and that's super exciting, we have Kevin with a quick connect tutorial. We've got the community showcase, and we've got a whole bunch of different things. I am joining from London. I would love to know, where you are all joining from.

I can see that the chat is already busy. That is great. Please continue. We have team members in there answering all your questions. So, yeah, if you've got anything that you are thinking about, let us know based on anything we are saying.

We also have the general chat in Discord for afterwards if it comes to you and you're like oh I should have said that feel free to keep the conversation moving there as well. Did anyone say where they're joining from so far? I see a lot of hellos. Oh, Netherlands, North Germany. Hello.

Welcome. Welcome. Granada, Spain, Ukraine, Austria, Egypt, Sardinia, Nigeria. Amazing. Welcome, France too.

Hello. Hello. Super happy for you all to be joining us. I am gonna hand over to you, Bryant, with some product updates for the first start of the show. So hope you enjoy them.

Speaker 1: Brian here for Directus, and I've got 2 quick updates for you. Version 11.1.1 dropped last week with a load of bug fixes, optimizations, and a few dependency updates. I just wanted to give a quick shout out to our engineering team and the community members who are on fire at improving the Directus experience for you, for me, and the entire Directus community. There is however one breaking change. The underlying SendGrid package we were using is no longer maintained.

So rather than risk your security or some type of crazy exploit, we've removed it. Now if you're using SendGrid as your direct to email provider, you'll want to switch up your config to use SMTP instead of their API. For the full list of fixes and updates, check the release notes in the Directus repository. Now on to my favorite update, probably because I had a little hand in shipping these. There's no bias here.

I'm super excited to announce Directus cloud templates. These are handcrafted Directus instances with data models, permissions, flows, all the niceties already built out for you. Now, whenever you spin up a new project for your organization or your client inside Directus Cloud, you can get to 80% done by clicking one button. There are currently 3 templates to choose from. Simple CMS, which is great for your next website build.

It's configured with the dynamic page builder, blog posts, menus, and even dynamic forms. Your content editors are going to love this one. Next up, we have simple CRM, which is the perfect starting point for your own custom Salesforce or HubSpot. It includes the standard fare, like organizations and contacts, deals and activities, and a configurable pipeline. Last but not least, we have simple ecommerce, which showcases a small online apparel or t shirt shop build out.

The data model includes products, orders, customers, and it even handles more complex scenarios like product variance, which account for all the color and size combinations automatically. So hop on to Directus Cloud, spin up an instance, check them out, and as always, send us your feedback because we'll be releasing more of these templates in the coming months. And back over to you, Beth.

Speaker 0: I'm going to demonstrate each of the new extensions in Directus Labs this month. As a reminder Directus Labs is our team's experimental org in GitHub which regularly publishes new extensions we think are cool. The Gantt chart layout displays items in a collection in a Gantt chart helping those of you who use Directus for project management and task management. When configuring the layout specify a label that you want displayed on each task, a start date and an end date, and optionally a dependency field which will draw dependency lines in a chart. You can specify the zoom in as granular as an hour and as broad as a year.

You can download it now in the marketplace. A calculated fields interface is the most upvoted open feature request in the directors project and today we are releasing it as an extension. It allows you to write a formula and the value of the interface will be automatically computed and shown. It supports the full set of functions provided by formula. Js and the majority of JavaScript operators that work for numbers and strings.

It also supports relational fields and we pass formulas to ensure they are only running allowed functions which is important for security. A couple of things that are important to note values here are only visible in the interface and not in API responses. We've had this extension in the works for a while, but just a couple of days ago, a community member, Judah Sullivan, released the Directus AutoGen Extension which you might also want to check out as it's similar Both are available in the marketplace The API metric panel can be used to display a value from an external API, for example, that could be the number of docker downloads or sales or followers on social media platforms As you can see here we are making a web request to get docker metrics. We could also provide custom headers or a request body if required. Then you specify the path of the value you want to display.

It's a nice responsive panel that allows you to get an overview of your business. You can download this extension directly from GitHub or NPM. Lastly, one of our core team members, Tim, in his spare time released a pair of flow operations which allows authentication and permission checking. The first operation allows you to check whether the flow was triggered by a user with admin access, user access, or public access. The operation will return true if authenticated or else throw an error.

The second operation allows you to select a collection and it will return true if a user has read permissions in that collection. You can check this out over on github and npm we hope you enjoy using all these extensions Gantt chart layout, calculated fields interface, the API metric panel and tims authentication and permission checks let's bring my camera back up and I'll see how you're doing. I brought my camera back up hopefully you can see me again. Yeah, as I said we are super excited for everything we've just announced. Out of the extensions we've just talked about or if there are any other extensions that you regularly use in the marketplace, now is a good time to let us know in the chat.

I mean for me as someone who really really likes project management efficiency the Gantt chart layout is super exciting and Kevin has just put in the chat as well the extension link for you to check them out for yourself. Tim, I did see you. Brace look also same person, same super ability. We are very excited about the operations as well. So, yeah, if you have any, extensions that you think you're like, this has really changed the game for me, this has changed the way I use direct us, or if you're just like, this is cool and I'm excited to hear more about it, let us know in the chat, and we can have a chat about this right now.

Kevin is saying that Florian's flexible editor interface is the one. Yes. I agree. That one is great too. And, yeah, as we said, shout out to you, Judah.

I'm not sure if you are in the chat, but we were talking about this in Discord earlier, too. So do make sure to check out their contribution as well. Yeah. So those are the, four ones for the extensions this month. Every month, we aim to have more.

And if you are looking at where, to go, the extensions link is the best place. We do also have the extensions channel in Discord too, and the director's extension field, actions from Atomic Media is also a great one. I'm gonna move us along to the next part of the changelog, but feel free to keep the conversation going, via the chats whilst we're there. So I'm handing over to Carmen with a docs preview, which is super exciting. So I will send over to her now.

Speaker 2: Hey, everyone. My name is Carmen. I'm a developer educator at Directus, and part of my role is documentation and that's what I'm going to be showing you today. We are about to launch a beta of our new documentation and I just wanna show you a little bit about what our motivations are, what we've done, and what's next. And, in order to do that, I'm gonna tell you a little bit about what three aspects of our documentation we're aiming to tackle.

1 is content, the other is structure, and finally, the platform. So let's talk about the content. Over the years, starting from a very minimal set of documentation that served our needs, not only have we gotten a lot of feedback and requests for new pieces of documentation, but we've also seen lots of our team, lots of hands touch that documentation. And over time there have been different voices, there have been different tones and we just wanted to completely rewrite that. So everything that you will see in the new documentation, whether it's been brought over or adapted or written from scratch, will have been touched not just by me but by other educators.

Next is structure. And so what we've done is sort of split it into 2 areas. 1 is a reference manual where you're looking at the exact parts of the documentation that you need just so that you can use Directus. But there's also an implementation side of it. You're not just using Directus in isolation.

So we are also bringing in a set of tutorials so that you can work with directives, for example, with a specific framework, with a specific use case, maybe with a specific SSO provider. What we're doing here is hoping that we can make this sort of separation bring a lot of clarity to our documentation. Finally, there's our platform. When we started out, we were using VitePress which fulfilled so many of our needs. It came with a template that we could use and when we hit those limitations, we would extend them, make those parts our own, and adapt them as needed.

But, with the opportunity of rewriting our documentation for scratch, we also wanted to have a brand new platform. We are rewriting our documentation using Nuxt. And I just wanna give a little shout out to one of our engineers, Connor, who's been spearheading this effort in a magnificent way. And finally, let's be honest, maintaining an API reference all in markdown can be quite cumbersome when manually written. So what we've set out to do is to have these API references also be automatically generated from code.

So cool. Those are the 3 parts that we're addressing with our new documentation. But what's next? As I mentioned earlier, later this month, we'll be launching the 1st beta of the documentation. And this is where y'all come in.

I'd love to ask you to please look over them, give us feedback, and tell us what we can do to improve this documentation for when we launch towards the end of the year. The other thing that we're going to be doing as well is to really look over all of our existing tutorials and audit them and bring them over. That does not mean that our old documentation is going to hide them our way. We're not gonna be hiding that old documentation or our tutorials. We're just going to be systematically bringing them over, making sure that they are correct, relevant, that they work with our current version of Directus and making sure that those tutorials stay in lockstep with new releases and new documentation.

So that's it for me. Thank you so much for listening and I'll see you later.

Speaker 0: Alright. As Kevin said in the chat, we are releasing that in the next couple of weeks. He's also put the link where you can find it. Super exciting. And I am also gonna be the 3rd person to say a massive big up to Connor and also a massive thank you to Carmen as well.

We've now come to the my personal favorite part of the changelog which is the community showcase. Love seeing what everyone is building and how they are making directors their own And this month, we have a community showcase from David who is gonna be talking about the form builder project they're currently working on.

Speaker 3: Hi. I'm David from. We are small digital studio based in Montreal. We've been working with directors for the last 2 years. And today, I'm going to talk to you about a small form builder extension that we've built for Dericus and that we're going to open source soon, hopefully.

So let's start. So, the main feature that our clients have been missing since we migrated away from WordPress has been the form builder. Not sure if you're familiar with gravity forms, which is one form, complex form builder that is labeled in their WordPress ecosystem, that allow end users to configure forms, with conditions, with advanced fields, with notification, even integrations with external systems. So all those portions are quite easy to build with Directus apart from the advanced form builder. We were able to build a basic form builder using native features from directives, but they were not as easy for a end user to configure.

In creating a complex form builders with conditions and wizards like features, is quite a big endeavor. So that's why we chose to, we actually started looking into open source project that could help us. And that's where we came across a library called survey gs. Survey so basically, survey gs has 2 main comp 2 main components. One is called survey creator.

The survey creator is what the end user use to create the form or survey. It's quite powerful. And the other important portion is form library, or should I say what libraries? Now form libraries, they basically take in the JSON coming from the survey creator to render a form. They have library for Angular, Vue.

Js, React, vanilla JavaScript, and probably other frameworks that for oh, yeah, and backbone, if they're still user of backbones. But anyway, yeah. So the only when we found that we found that it was basically doing everything we needed from a foreign perspective. So the only thing left was to integrate with Directus. And that's where we created our first extension, actually, our first or should I say first public extension for Directus?

So let me jump into that. I'll do a quick demo of what we've built. The main features that we wanted to implement were, well, of course, integrating the survey creator, the ability to save forms or the form data in any collection, and finally send notification to either internal users or external users after form was filled. So here, as you can see, we have our small form module. So I'll jump into contact us form.

I'll start with the form builder portion. Here, we have the basic form, 2 columns, first name, last name, or email. We have a drop down with some choices, subject of the the form. In here, we have our first condition. So, for example, I have a condition here that say if the subject is product information, then we will display what specific product information are you looking for.

If it shows customer support, then instead, it's going to be what's your issue. I want this I won't demo all the features of, sorry. Js. There's a lot of documentation. You can look it up on their website.

But, yeah, it's fairly powerful from builder. Then afterwards, we have the actual integration with Directus. So if you I go into the configuration of the form. Let me hide that for now. So, yeah, what we do have is the schema that was generated by the form builder.

Technically, you could edit the schema itself, but it's easier to use the form builder instead. Please note, though, that the form builder of Suri. Js does require a license. We we do have the configuration to input your license in the extension, so that's good. But just so you're aware, it does require a license from, Suri.

Js to use a survey creator. Anyway, so what I was saying is here, we have the configuration. So we could we can see that, it's submitting data to form submission collection. You can choose any collection that you want. You can create a new one.

You can have multiple forms submitting to the same collection. It's really depend on your requirements, basically. We also have notification templates. I'll show I'll explain how notification templates work afterwards. And did I forget anything?

No. I think that's pretty much it. So, yeah, actually, let's jump in into notification template first. So if somebody submit or fill in that form, as I mentioned, you can configure 1 or more notification templates. So let's jump into it.

Form templates use like just flows. You can use any fields that was part of the form as as the recipient. For example, if you want them to have more than 1, you can add other recipient. In this case, we're using the field work email. But you could also add an hard coded one for, suppose, someone on your team.

The template is itself is pure HTML, but same thing. There's basic templating, so you can use, variables from the from the form. In this case, the contact us form. So let's try and submit 13. So I'm gonna go back to my form, contact us form.

And I didn't build my preview. Here in this case, it's an XGS application. So it's a React component that we built that is basically reading the schema here and passing the schema to the to the sorry. Js library. And once you're done submitting the form, it's going to call a webhook from Dactors to save the data.

So let's try that. So of course, we do have validation. So let me fill the form. We based in North America. So we want to chat about product information, customer support.

Yes. I'm not sure if this form is going to actually send email. Yeah. This field is not required. So I don't not gonna fill it.

I'm just gonna complete the form. Thanks. No problem. So if we go into our contact us entries, we can see my entry, all the fields that were filled filled in with the data that was passed. And if we look at our notifications that were generated, we can see here we do have the new notification, the email that it was sent to, the actual content of the email.

So you have a and a status, so you know if it it was fully sent or not. It does link back to the entry if needed that generate that that generated that notification. And once the form is submitted, we can see in my email that I received the confirmation template. Yes. So yeah.

So that's pretty much how the, extension work. If we look at our flows, there's a couple of flows that need to be configured, but it's not that complex. Most of the features are built in feature from directors or survey GS. So most of the extension is gluing together the the 2 great products, basically. We do have some configuration that you can pass to the survey JS extension.

So, for example, if you want to show the designer tab, the JSON tab, depending on your users, you might not want them to be able to see the JSON. So first of all, you can disable that, save the settings. And if you go back to the actual form, you see I don't have access to the JSON editor here. So we made that we made sure that we could configure every settings from, survey survey creator directly from directors. So, hopefully, it's an extension that, some of you will find useful.

We're hoping to release the extension in the next month. We just wanna make sure that it come with clear installation instruction. But, yeah, it was the main feature for us that was missing from Directus. Everything else has been great. We're really happy having migrating from WordPress to Directus.

And with that last missing piece of the puzzle, we have we now have a complete toolkit to, finalize our projects. So thank you very much.

Speaker 0: Alright. Thank you so much to you, David, for taking, the time to take us through that. I'm super excited for it to become available hopefully soon. I think it will make a great addition and if you're keen for more community showcases I highly recommend checking out the 'I made this channel'. Specifically people post updates on their projects and how they're going along and I think that's really great to see exactly how things are changing all in the same thread which is super interesting.

Next up, we have Kevin with a quick connect tutorial on AI web scraping API with flows. So I'm gonna hand over to him.

Speaker 4: Hello there. I'm really excited about this tutorial. So on Directus TV, we already have a show called Quick Connect, which shows you how to integrate third party services with Directus using Directus Automate and Flows. And in the spirit of that show, today, I'm gonna show you how to integrate FireCrawl with Directus. Now here they say that they turn websites into LLM ready data, And what that means in practice is you can feed it a URL, provide some options if you want, and it will go and take a look at that web page and return some structured data for you like so.

This is their scrape endpoint, which will take a single web page and scrape some data from it. They also have a couple of other endpoints, crawl and map, but today we're gonna use scrape. Now I've already logged into Firecrawl Cloud and generated an API key, which I'll copy for later. You can also self host Firecrawl, but for ease, I'm just gonna use their cloud product here. Now I have this directors project over here with a new empty collection called companies.

In this collection, there are a few fields, URL, a name, a description, a mission, and a boolean, a true force value, is it open source. And our goal will be to provide the URL and then have FireCrawl automatically populate the rest with flows. So let's go ahead and create a new flow. So this is our automation builder if you've not seen it before. I'll call this one get company data, I guess.

And we are going to use a manual trigger which will add a button to the side of collection and item pages. So we're going to say we'll run this on the company's collection. What else matters here? We're going to not require selection, so the button always works, and we're going to require confirmation which will pop up a modal, and in that modal we will just add a URL. We'll make it a string input and we'll make it full width, and I think that's all we need to do here.

So just to see what happens here, if I go back to the, to the company's collection, we now see this button here, this manual flow, trigger. I click that. It pulls up the box, and we'll put in a URL and hit run flow. So, now, if we go back to our flow, we should immediately see that there is one log. And in here, there is a body and the URL is the value that we typed.

Fantastic. Now we need to actually do something with it. So, let's go ahead and add a new operation here. And, honestly, fire crawl is pretty sick. You can just make one web request.

Let's take a look at their docs. We're gonna use the llm extract endpoint here, and let's just take a look at the kind of construction of this API call. It's a post request to this URL. We're gonna pass in our our API key here as an authorization header, and then they give us this kind of JSON payload here. Here, it's telling it to go ahead and extract specifically these four fields, the company mission, does it support SSO, is it open source, and is it in Y Combinator?

And it's saying you must go get all 4 of these. So let's actually just turn this straight into a flow request request URL operation. So we're gonna do a post request to this URL, post request to this URL. I'm gonna go and copy my API key again here, and at the end of this, I'll, I'll destroy the key. Authorization authorization, bearer API key, save, and then there's the request body.

And, honestly, it contains a little more than we need but this contains everything we need so we'll just pop that in there. The only thing we wanna do, of course, is pass in the URL that we put in the box. So we'll replace this with trigger dot body dot URL. Fantastic. Let's save that and see what happens if we go over to content, press the button, and type in directors dot io.

We see that's running. That's running. That's a good sign. It means it's going off and making the request, waiting for the request. And then we see there is a second log.

And we get some data back. There was a 200, so it was successful. Inside of data, there is a property called data, and then there is this value called extract. Extract contains all of those custom keys we asked for, company mission, supports SSO, is open source, and is in YC. And then always when you scrape, you get this metadata object, title, description, language, Open Graph data, source URL, and so on.

So really all we want to do here now is we want to take this data and create a new company from it. So let's add a new, let's create a new operation on the resolve path of that web request. Let's call it create data. We're gonna create something in the company's collection, we'll give it full access, and then we just need to provide a payload. So let's go ahead and do that.

We have an object here, we have a name, we have a name, and we're gonna pass in the value of the last operation dotdata.data.metadata.title. Then I, for 1, am I'm just gonna copy this and edit it each time. So we have name, URL. So that's last.data.data.metadata.sourceURL. We could, of course, just take it from the trigger body URL, but this is properly formatted.

You'll notice I typed in directus.io, but when it came back in the payload, it came back with the with the, with the protocol HTTPS and so on. So we have name. We have URL. We have a description. Now this one is also from the metadata Description.

We have the mission. Now this was a custom piece of data we asked to be extracted. Company mission is what we called it. And, finally, we have open, open underscore source. Last data, data extract and then is underscore open underscore source and then remove that trailing comma.

So I believe that's the name of all of the fields. We'll figure it out in a moment when when it inevitably doesn't work. We'll hit the button again, directors dot io and hit run flow. Once again that's going off to Firecrawl using their endpoint and there we see there we see it straight here. URL name, description, mission and the boolean is open source.

Let's, let's try that once more. Let's go in here and say fire fire crawl dot dev run flow. So let's see. And in theory we should just give that a moment and there it is. So now you can go ahead and grab more data.

Now, of course, if we take a look at this endpoint here, you can provide custom properties and it will try its best to get data out from that. They have a couple of other interesting things which I'll draw your attention to even if I don't think it works in this context. They have, attention to even if I don't think it works in this context. They have extracting without a schema. So this extract here was us creating a a schema.

Right. You can give it just a text. You can give it a prompt. Extract the company mission from the page, but the thing I don't like about that is you're not explicitly saying what the name of the key is, so you don't necessarily know what it's going to be at the end. I like creating a schema personally.

They do something else that's kind of interesting. If I take a look at where is it, I think it's in their API reference here inside of scrape. They have this interesting thing called actions. So you can get it to wait, to take a screenshot, to click, write text, press a key and scroll and the combination of clicking and writing text means you can get it to interact with your web page. You see it here, actions, wait 2 milliseconds.

You could get it to, like, sign in to things perhaps, perform search terms. I think it's super interesting and then take screenshots, of course, and upload those to directors if you fancy. So there's a lot of flexibility in this. Having seen kind of how easy this API is, I think I'll go ahead and turn this into, an extension some point in the next few weeks which we can release as part of Directus AI. But, yeah, that's how to integrate FireCrawl with Directus using Directus Automate.

Hope you found this interesting, and by all means, if you have questions just reach out. Alright. Over to Beth.

Speaker 0: Welcome to The Reading List. Internally, we have a number of Slack channels where we share links from all around that we found educational and entertaining and we'd really like to share some of our favorites this month with you. First up, we have EIAIO games on scaling 1,000,000 checkboxes. This was a web experiment where every time anyone globally checks a box that change is reflected across everyone's browser and it goes in-depth into the scaling challenges of the unexpected popularity of the website leading to 650,000,000 checked boxes. Next, we have a recorded YouTube video from wasm.

Io24 featuring Luke Wagner giving a talk on a stream of consciousness on the future of async in the component model. This is a great talk from one of the original designers of WebAssembly on how async will work in WASI 0.3. Next, we have introducing Netflix's key value data abstraction layer on the Netflix tech blog. This takes us through some of their global backend infrastructure design powering one of the most popular streaming platforms around and their use of multiple online distributed databases such as Apache Cassandra. Next up, we have an interesting read introducing the tool Cambria which is used to help you describe in a structured way how schemas change and why they change.

This is from an independent research lab called Inc and Switch, definitely one for folks interested in the more academic side of things. The data visualization catalog provides a list of charts categorized by their data visualization functions or by what you want a chart to communicate to an audience. It's a really useful guide for selecting charts based on your analysis or communication needs. It starts with what you're trying to communicate and recommends which chart or tools to use. Lastly, a nice little tip from Vic instead of storing states as booleans, so like true or false, you could consider a date stamp so you can see when a state was updated with null meaning false and any date value meaning true.

Those are just a few of the links that we shared internally this month. If you've got your own interesting links we would absolutely love to see them. We would recommend you share them within the discord server so you can see them and you never know they might even be in next month's changelog! We like to take a moment towards the end of each changelog to thank our contributors and sponsors who help make Directus as great as it can be. A huge thank you to Florian for dropping support for the old SendGrid transport option in favor of SMTP directly.

Thank you to Shane for adding support for disabling terminal screen clearing on rebuild in watch mode. Thank you to Junhong for fixing SVG truncation issue in image interface preview, for fixing an issue that would cause custom role names to not be translated in the role navigation sidebar, for fixing WYSIWYG interface to be updated when interface options are changed via conditions, for improving behavior of the translations interface when setting use users language option, for ensuring data in sidebar is re fetched when switching pages while a sidebar pane is already opened, and for ensuring errors occurred within the card layout are always shown. Thank you to renovate for updating dependencies across multiple packages. Thank you to Joshua for fixing an issue that would cause log streaming to fail on directors installations that use a sub path in the URL. Thank you to for ensuring users are redirected to the intended page after login via LDAP.

Thank you to CASE for adding the optional ignore rules option to schema apply CLI, and thank you to Anton for making the output types of SDK requests better readable in IDEs. Thank you again to each individual you can see their specific pull requests inside of the full release notes on GitHub. Lastly, we also want to take the time to thank our GitHub Sponsors of September who financially contribute to Directus' development specifically a huge thank you to Clement that's dlxclem for sponsoring as our backer tier of $100 a month. The money we are given from our Github sponsors goes straight back to the community members who build tooling and extensions for the directors ecosystem. Thank you again for being part of that.

Alright, that is the last of the segments that we have for this month. If you are still watching thank you so much for sticking around we appreciate you And if you've watched the whole thing we'd love to know which segments you particularly like. That helps us plan for the next few months and to show, what you'd like to see so do let us know any feedback about the change log. The next change log is November 12th at 3 pm the same time and I do hope that it's going to stay the same time in discord because we are changing from British summer time to the non British summer time which is Greenwich Mean Time So if you see a different, time, it'll still be 3 PM GMT BST. It's all gonna be there.

It's fine. Yes, Kevin. Not quite connect. Weird presenter in my opinion. Anyway, thank you so much again for tuning in.

Have a great rest of your day, and hopefully catch you for the next one. Bye, everyone.