How to effiently add thousands+ of entries to `many_to_many` relations
I'm having a brain freeze at the moment. I have
Controls and Requirements with a many_to_many relation (ControlRequirement). I can set those using manage_relationship... however I now have the use case where I need to add new relations one by one ... thousands of them. My understanding is that manage_relationship with :append_and_remove expects a list ... so I'd need to fetch the list, add an item, remove duplicates and then provide this to manage_relationship, correct? I assume :append will be the better choice here. To save on DB requests I was thinking of creating the join ControlRequirement entries directly. Is this the "Ash-way" of doing this?2 Replies
If you use
append we should not fetch all the data, only create each item we need to create.
So if you provided 1k inputs it should be performant enough. But the Ash Way is essentially to try out the built in behavior and if it works for you then use it and if not then your own after action hook or before action hook that does the work is perfectly fine
Right now managed relationships don’t use bulk actions but eventually they will.
But you could use bulk create in your own hook to insert very effcientlyThat's what I was hoping for. Just recreated the issue in a playground project (the real one generates way to much logs to follow...) and it does indeed kind of upsert just fine. No fetching of records. So let's see how this works out in production where those 1000+ batches care coming in hourly.
Excellent idea ... that might be the next optimazation step (if needed at all ...)