Multitenancy in Directus - Berlin Meetup

Speaker 0: Okay then. So welcome everyone. So I'm Mariana. And today, I'm going to be talking a little bit about, we handle multi tenancy in one of our projects. So basically, like a startup guide for if you need to set up multitenancy in your project, just from the director's point of view, but hopefully it'll help you, like, make the process a little less, painful.

Let's let's put it that way. So, yeah, I'm Mariana. I'm a software developer at Ivory Heroes. I've been at the company for, okay, a year 8 months, I think. And this is Joliel who is not here.

Unfortunately, he was going to be co presenting with me, but something came up. So it's easier in spirit. So, and without without any further delays, let's just get right get right into it. So basically, the structure of the talk will be like, defining multi tenancy. So what it is, then the advantages, how why you might need it, why it may make sense for your own projects.

Then basically, we'll do a small case study of 1 of our own projects where we had to implement multi tenancy. How we approached it, how we got started, and also the downsides of our approach and some other, like, general topics that also came up while we were doing this implementation. So, yeah, let's let's get right into it. So, basically, multi tenancy is an architecture where you have essentially one single instance to serve multiple clients. So let's say you want to set up a CMS to, for schools to set up their own block content.

With multitenancy, you can have you can also only one instance of, of directors and serve all of all of the schools you want, while having the data collocated but separated from each other. So each school would only see the information that pertains to their own school. But, like, why would you want to do that? Why not just set up individual instances for for each school that would like to to use your services? So maybe maybe you want to lower infrastructure costs.

So in some situations, and now we'll stress the some parts of the sentence because it's it's very dependent on the context. It may be cheaper to just have one single instance, if you don't, like, expect any crazy loads. So, the problem here is that sometimes scaling up just one instance may be more cumbersome than just having several single instances separated and deployed individually. So, yeah, as I said, this is very much it very much depends on on the context. For our case, it makes sense, but it may not make sense for for yours.

So another possible benefit is that it can lower the the, entry barrier for your your potential clients. So and this is actually one of the reasons why we also choose this. So let's say we have a client that wants to use your product, but they don't really want to worry about the infrastructure. So they don't want to want to deploy anything. They don't want to run any infrastructure.

They just want you to take care take care of all of it. And let's say you also don't really have the resources. We have multiple instances. Multi tenancy can help it can help with this. Like, a good compromise could be to just bear the costs of one single instance that can support multiple, tenants.

And so you can also support multiple clients and a lower cost. And finally, this isn't really a reason that we, that resonated with us for our context, but it may resonate with you. It can make it easier to kind of aggregate data. So let's say you want to see which features of your, CMS are being used the most often by the most clients. It's easier when you just have, your data collocated.

And, again, we don't really have a lot of experience with it, but it may be something that you are interested in. So let's let's say these these benefits, did convince you to to try out multi tenancy. I thought it would be the best to kinda look at a real life, example, And that's why we're going to use, our project that my team is currently working on as as an inspiration. So direct, which is kind of similar to directors and sometimes gets confusing when we talk about the project, in a company, is a platform that we're developing for Freunde visited Berlin. And it's basically focused on the management of, psychological studies.

So it's basically a one stop shop for, coordinating a study, creating content for a study, distributing that content to potential study participants. Like one example of of, possible psychological study could be, like, to analyze the impact of immigration of an immigration process in an asylum seeker, for example. So where does multitenancy come into play in in in our product? So the main goal of our clients of AIUN VISITED was to be able to offer the platform to other partners. So think other universities, without having to ask them to set up their own infrastructure.

So I feel would bear the cost of of the infrastructure and would offer the platform to other, to to other clients. But, Fine Infrastructure Debt also didn't really want to, push their own infrastructure cost too much. So the goal was to lower their own infrastructure costs and make it easier for other clients to adopt the platform. And so if, the these these requirements kinda match up nicely with with the advantages that I listed, above. So we kinda went ahead and implemented multi tenancy for this.

So before we get started actually looking at the implementation, a quick side note. So today we'll be focusing on multitenancy just from the CMS point of view. However, your project and as our own did, probably has more moving parts. So there are more places where you have to implement multitenancy and there are more places where you have to set up systems that guarantee that multitenancy is insured on all of, on your entire platform. So direct direct itself is composed of several different systems.

So we have a CMS to create the content. We have an app that is used by the study, participants. We have an admin panel that's used by the coordinators and then an API to kind of tie it all together. So we had to do a lot of changes in all of these, on all of these parts of the app. So today, we're gonna look at a small subset of, of multitenancy, obviously, focused on directors.

And, hopefully, it'll be at least, like, a starting point for you to to kinda go through with with a pullbacking multi tenancy on at least one part of your of your product. So, yeah, with that said, like, let's jump right into it. So when you Google direct us multi tenancy, there are not a lot of results. Like there's not like one major guy that pops up to to, as as being, like, super relevant for your use case. We have a couple of, like, GitHub, issues, a glossary entry in the Directus website.

We also have, like, more discussions, more GitHub discussions, the down further. So let's maybe look at the glossary entry for for multi tenancy. So, this entry provides a very similar definition to, the one that we looked at previously, but it adds some director specific details. So, basically, that's the two main ways that you can achieve multi tenancy, indirect. So either through project scoping or through wall scoping.

So we, for our product opted for wall scoping because project scoping was just not a viable option for us. We self host, and we weren't really, it wasn't really viable to deploy multiple instances of directors as a project scoping approach does. But maybe if if you happen to use Directed Cloud, then, yeah, this definitely could be an option. It's just a disclaimer that we opted for rollscoping because it's all flows and at the moment, at the time, it's was the most reasonable option. So in this presentation, we're going to I'm going to showcase like a slightly modified approach of the roll scoping.

So basically, instead of using, one word per tenant, we're gonna assign tenants to users. We found this approach to be a bit simpler, and it is also one that is, kind of promoted by direct. This is one of in one of their more or less recent videos, that talks about multitenancy. But both options are obviously, good, and it's just the context dependent. There's not much to one over the other.

So regardless of which approach you choose, either assigning tenants to roles or to users, the main concept to keep in mind is permissions. So, in in our approach, the tenant based permissions are the core of the logic. So, essentially, it's these permissions that allow us to filter which items a user has access to based on the, on the tenant of that item and that user. So let's maybe to to kind of make it easier to visualize this. Let's take a look at all the data structure.

So in in this approach, we have a tenant's collection, as a matter of guess, which allows you to to find the tenant. So in a real life scenario scenario, which is what we do, this creation, the credit permissions for the tenants are limited to the super admin role, and which is something that you also might consider because the tenants are a very, very privileged is a very privileged privileged collection. So after we create the tenants collection, we also need a way to actually assign the tenants to to things, to tenants and to users and other collections. So we create one to many, relations between the whichever collections you want to be, multi tenant and to the users, collection as well. So now we can assign tenants to things and to users, but we don't have data segregation at this point.

So a a user from one tenant can see data from another tenant. So, but first, let's maybe look at at permissions. So, users belongs to roles and roles can have many permissions. And these permissions basically dictate how the roles can interact with the collections. So let's say we have a customer role and we, we have an admin role.

We may want to prevent the customer role from creating in tenants while a site while allowing the admin role for correct permissions to to the tenant collection. So you may have noticed that in these two diagrams, the permissions, collection doesn't really have, a connection with the tenants. So how do we actually create the tenant based permissions that our system depends on? So to put it all together, we kind of need the last piece of the puzzle, which are rules, permission rules. Rules allow you to make this permission the permissions more specific.

So in this case, we're adding a rule that states that the user can only access, an item in a collection if their tenant matches a tenant of that item. So let's say, this is a permission for the customer role to to read from the product collection. So this is basically the gist of how the multi tenancy solution with, with user permissions, goes. So you may be aware that also, besides the data structure, there are there's also the question of actually creating sorry. I have a question about that.

Yes. Isn't it pretty tedious Exactly. We're gonna we're gonna we're gonna cover this right in the right in the next slide. So as I like like you said, it's it's very tedious to set up roles and permissions automatically, especially if you have like multiple deployments of, of the application. So and it's Stephenson is also error prone.

So, if you forget to add a permission to a credit collection, then you're gonna be using our app and then you're gonna be, like, scratching your head, trying to understand why isn't is this not returning the items correctly? Or when I when I quit because I know that I created a collection. Why are they not showing up? So, you know, it's a very, very prominent. It's very tedious.

So because we have a lot of collections and because we have several deployments, like in review apps, in staging and production environments, we kinda set up, automated permissions through, scripts. So we don't use it for this. We use external scripts, but the concept concept is is the same. So using the direct SDK, we fetch all of the collections that should support multi tenancy. So, in this case, we check the collect the collections that have a tenant field.

And then we update the director's permission, system table to to include all of the permissions, that we want to support, and ensuring that the permissions field specifies that the rule that we we looked at a few slides ago, is is enforced, which ensures that users can only access, can only do specified actions, either reading or writing or sharing on the items that belong to their own tenant. And in addition to our own collections, you can obviously do this also to system collections. We also do it with files and folders because they, say it should be tenant specific as well. And the the logic is fairly similar. You just have to change the the target of of that permission.

And so with this basic data structure and with these, automated, permissions, we can have a simple multitenancy to to start with. And as useful as we found this setup to be, we, this this approach kind of does come with some downsides, and I would highlight 2. So the first one would be complexity. So setting up a multi tenant system is inherently more difficult, especially if you have more than just a CMS. And in the multi tenancies system as is our our own case, we also have to add additional features like our clients, also wanted a multi tenant kind of work for a multi tenancy approach because they wanted customization.

They wanted their tenants to be able to customize the UI. They want the to customize the way that the study flows. So it adds it's more complexity to to wrap your head around. It's just overall a are the solution to it to implement. And secondly, because the data is is co located, at least in this approach, it's we pay need to pay even more attention to security, especially if you're handling, very sensitive data like medical data.

Like, we're placing a lot of trust in this in this multi tenancy system. So this comes at a cost. We need to implement more systems, more checks to make sure that the data release, segregated and, that we we can ensure our clients that, you know, no tenants can see sensitive information from other tenants. As with everything, basically, multi tenant is yes as trade offs. And there really isn't one clear cut rule that can tell you whether you should or should not implement a multitenancy for us.

At the time it makes sense and it still continues to make sense. We're still working on it. We're still integrating on it, and it may or may not make sense for your project. And but, hopefully, this this brief intro will kind of at least, settle some, give you some stronger ground to to to thread when you start in, when you start exploring this topic. And we're almost done.

But before before you go, I'd like to circle back on on what I said a few slides ago. So this this brief guide, let's put it that way, is meant as a starting point to multitenancy. So there are a lot of topics which I definitely did not cover today. Some things we have come across in our own product and we have solved them. Others which, we have encountered and the decided that we don't really have the resources or it doesn't really make sense to to tackle them right now.

And surely, like many other many other topics we are not even aware of. So I just I gather a few here, like, a few topics of discussions for for examples. So how do you handle uniqueness across tenants, for example? So let's say you have a books collection and in the scheme of that collection you have an ISBN. So the, unique code that is attributed to 2 books, field that is set to unique.

And then 2 of your tenants have the same book, and the first tenant tries to add a book to a collection and it's offline. And then the second the second tenant second user in another tenant tries to add a book and throw an arrow stone because the field is supposed to be unique. So how do you handle the situation? So you can maybe set up a composite index, so that the unique constraint applies only to the ISBN tenant pair and not just the ISBN. But as far as I know, this is not a natively supported indirect.

So you would have to do some workarounds. You'd have to implement some custom interface to show that. But it is a possibility. Or you can even remove the unique constraint if if you think that it doesn't really compromise the integrity of your data schema, it can be it can be easier to just remove the constraints if you can ensure that the system is still compliant with with your with your architecture. And what about tenancy outside of directors?

I did mention before that we have several other systems in our platform. So how how do you sync up the tenants collection with with external services? So we can you can set up scripts that run periodically and sync the necessary information, which is what we do, or you can also make use of direct to Saks, which is also something that we are considering, and something that possibly we will start adopting. And what about permissions? Like, how do you keep permissions up to date?

If a new collection is added, it's just as come or correctly. So in our use case, we, the automation for this, for this, that it wasn't really worth the effort, but you can also adopt a similar approach to the tenancy that I mentioned in the point above. And, finally, like I said previously, we are following a user based tenancy as opposed to a world based one. But as I said, both options are are viable. I've linked some discussions, in the appendices of of the, of this presentation that do go further into into this topic if you like, if you're interested.

But, yeah, I'm sure there are many, many more topics that we haven't really had the chance to to cover, but hopefully this is a good for the, for the discussion. And I do hope that this presentation kind of provided you with a solid, solid introduction to, to multitenancy and that it can make its implementation a little bit less, less painful. And, yeah, I think I think that's it. Thank you all for listening. I left my email there if you're interested in just asking whatever about the presentation, about the product or whatever you you can think of.

And, yeah, if you have any questions, then let me know. If you don't, then that's fine. I'm not, I won't get mad, but yeah. Thank you.