Help me understand how you would approach this problem.
consider a huge Tree structure data ,which will be shown as roadmap to users .
when a user clicks on a node it expands 1 more layer to show all related paths + nodes.
each user can save part of the tree to their account and they can edit the saved data , add custom nodes to their roadmap etc.
they can even remove a relationship between 2 node.
Metrics
There are 10 lakhs nodes each represent a point in roadmap
there are 150+ node types , each type will have unique set of parameters along with common ones like ID, name and desc.
there are 200+mapping tables created to represent mapping between 2 unique node type.
there are 2 DB with same set of tables , one will act as master DB with all 10 lakh record which are shown in UI roadmap,
DB 2 will be used to save same master data but client specific , along with history tables and all the modification done so far.
in our current implementation :
from UI we can get all 10 lakh nodes to be saved for single client, we have to refer Redis which will contain all master table Data and migrate all the 150+ node table data and 200+ mapping table records into new set of node tables and mapping tables which will have extra column (clientID) .
we have written backend api in dotnet core C# , with input payload we search redis for related data with which we will create list of objects which will finally get inserted into postgresql DB. this process is way to slow since it usually takes 4 to 5 mins to migrate entire DB data into different DB under specific client.
What will be your approach to solve a requirement like this and if you can nudge me in right direction to resolve this requirement that would be helpful
when a user clicks on a node it expands 1 more layer to show all related paths + nodes.
each user can save part of the tree to their account and they can edit the saved data , add custom nodes to their roadmap etc.
they can even remove a relationship between 2 node.
Metrics
There are 10 lakhs nodes each represent a point in roadmap
there are 150+ node types , each type will have unique set of parameters along with common ones like ID, name and desc.
there are 200+mapping tables created to represent mapping between 2 unique node type.
there are 2 DB with same set of tables , one will act as master DB with all 10 lakh record which are shown in UI roadmap,
DB 2 will be used to save same master data but client specific , along with history tables and all the modification done so far.
in our current implementation :
from UI we can get all 10 lakh nodes to be saved for single client, we have to refer Redis which will contain all master table Data and migrate all the 150+ node table data and 200+ mapping table records into new set of node tables and mapping tables which will have extra column (clientID) .
we have written backend api in dotnet core C# , with input payload we search redis for related data with which we will create list of objects which will finally get inserted into postgresql DB. this process is way to slow since it usually takes 4 to 5 mins to migrate entire DB data into different DB under specific client.
What will be your approach to solve a requirement like this and if you can nudge me in right direction to resolve this requirement that would be helpful
