Performance Best Practices Question
Not a domain expert on cybersec unfortunately. But in general, the most "intuitive" model is the best way to express a complex schema - and being minimal is strongly correlated with being intuitive, so the 2nd model looks better at a glance.
Are you finding that similar sized schemas are just generally more performant when the queries operate on entities and attributes only, without relations involved? Because if so then there could be a scaling issue there. If you have a couple of sample schemas and queries - as minimal as possible - that look like they should run in similar times but don't, that could be valuable info.
9 Replies
Thanks very much for getting back to me on this - it makes sense, and I'll just have to re-think my schema to limit the Relations to only include things that wouldn't conceptually exist without two or more entities (e.g. a Hunt can have zero results and still be a valid search, so it should be an Entity, whereas a Resolution can't exist without a DNS Query and Answer, so it would make sense for it to be a Relation)
As for testing the performance of a Relation vs an Entity with many Attributes attached, while it seems Relations are ever so slightly slower to return, it does seem to be a more-or-less linear correlation.
Entity + 2000 datetime Attributes returns in .46 sec
Relation + 2000 datetime Attributes (and one Player) .79 sec
Entity + 4000 datetime Attributes returns in .72 sec
Relation + 4000 datetime Attributes (and one Player) 1.5 sec
Oh that's good information actually thank you! Intuitively it makes sense that you'd see an approximately 2x increase in query "difficulty" because a relation represents more info/facts than an entity with attributes
A (has B, has C)
vs
A (has B playing role C, has D playing role E)
Does this reasoning check out @Joshua ?
I think that loosely checks out!
hmm, thats interesting @TheDr1ver , are those numbers using Fetch, or the concept API?
Concept - Fetch doesn't fit my use case in its current state.
ok, then those times also resonate with my own experience, but mine are slightly slower
which is why i will end upswitching to fetch
As long as fetch eventually has the capability to retrieve all attached Things without having to explicitly define the types in my query I'll be switching as well - but there are too many types in my schema to make an efficient query with fetch as it currently is.
you can use optionals to conquer that
Haven't gone too far down that path yet, but thanks.