T
TypeDB•3y ago
David

Query slowdown help

it hangs in TypeDB Studio and with the Python client
25 Replies
James Whiteside
James Whiteside•3y ago
This sounds like either rules generating excessive reasoning pathways, as @Joshua has pointed out, or insufficient resources for the size of the data. Could you please provide an estimate of your data size (number of entities and relations) as well as a list of rules in the schema?
Joshua
Joshua•3y ago
might also be good for a forum post
krishnan
krishnan•3y ago
Just out of curiousity: How does it performs when you don't specify $t? (i.e., comment out $t isa ul-entity, has id "Cheese";)
David
DavidOP•3y ago
it also hangs without the $t by process of elimination, it's the ul-verb line that causes the problem
James Whiteside
James Whiteside•3y ago
Can ul-verb be inferred by a rule?
David
DavidOP•3y ago
yes
krishnan
krishnan•3y ago
Can you share all your rules? It'll let us see what's hitting what What would also be useful is a summary of counts, like James suggested. Maybe these queries (With inference off):
match $e isa ul-entity; count;
match $v isa verb; count;
match $ul-verb (verb: $v) isa relation; $v has id $vid; get $u, $vid; group $vid; count;
match $e isa ul-entity; count;
match $v isa verb; count;
match $ul-verb (verb: $v) isa relation; $v has id $vid; get $u, $vid; group $vid; count;
David
DavidOP•3y ago
7 entities, 2 verbs, 1 IsA relation, 1 IsSubclassOf relation
krishnan
krishnan•3y ago
Wow, that shouldn't be hanging TypeDB at all. What version of TypeDB are you on? Could you share your rules? Are there logs in the server output?
David
DavidOP•3y ago
i don't see any logs. is there any way to just get a dump of the rules?
Joshua
Joshua•3y ago
in console you can do database schema <db>
David
DavidOP•3y ago
define id sub attribute, value string; ul-verb sub relation, relates arg0, relates arg1, relates verb; ul-entity sub entity, owns id, plays ul-verb:arg0, plays ul-verb:arg1; verb sub entity, owns id @key, plays ul-verb:verb; rule rule_MyPassage: when { $x0 isa ul-entity; $x1 isa ul-entity; $IsA isa verb, has id "IsA"; (verb: $IsA, arg0: $x0, arg1: $x1) isa ul-verb; $x2 isa ul-entity; $IsSubclassOf isa verb, has id "IsSubclassOf"; (verb: $IsSubclassOf, arg0: $x1, arg1: $x2) isa ul-verb; $IsA isa verb, has id "IsA"; } then { (verb: $IsA, arg0: $x0, arg1: $x2) isa ul-verb; }; rule vIsSubclassOf-trans: when { $e0 isa ul-entity; $e1 isa ul-entity; $e2 isa ul-entity; $verb isa verb, has id "IsSubclassOf"; (verb: $verb, arg0: $e0, arg1: $e1) isa ul-verb; (verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb; } then { (verb: $verb, arg0: $e0, arg1: $e2) isa ul-verb; }; btw, when a match (like in my last rule) has:

$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;

$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;
is it implicitly assumed that e0, e1, e2 are all unique? if not, this can lead to infinite recursion right
krishnan
krishnan•3y ago
Could you remind us what typedb version are you using? With 7 entities and 2 verbs, the maximum number of relations you can infer is ~ 196. If you don't mind, could you also share your data so we can reproduce it locally? This is very confusing and I'd like to check if there's a bug. While we're here,you can optimise your rules by enforcing an order on how you compute transitivity. @James Whiteside might be able to help with this.
David
DavidOP•3y ago
i'm using the latest image off Docker what's the best console command to dump data?
Joshua
Joshua•3y ago
it's not built into console yet, but you can use the server's built-in dump command: (with server running elsewhere), you can call:
./typedb export --database=<db-name> --port<database's port> --data=data.typedb --schema=schema.tql
./typedb export --database=<db-name> --port<database's port> --data=data.typedb --schema=schema.tql
https://typedb.com/docs/typedb/2.x/admin/export-import
David
DavidOP•3y ago
here we go 🙂
krishnan
krishnan•3y ago
Thank you You've hit a case that makes it search too many plans. I'll file it as a bug so I get to fixing it soon. I can reduce the search space for it a little by updating your rules. tl;dr: We define ul-verb-inferred as a subtype of ul-verb so we can avoid triggering some rules (using isa! ul-verb) when we don't strictly need them.
define

ul-verb-inferred sub ul-verb;

rule rule_MyPassage: when {
$x0 isa ul-entity;
$x1 isa ul-entity;
$IsA isa verb,
has id "IsA";
(verb: $IsA, arg0: $x0, arg1: $x1) isa! ul-verb;
$x2 isa ul-entity;
$IsSubclassOf isa verb,
has id "IsSubclassOf";
(verb: $IsSubclassOf, arg0: $x1, arg1: $x2) isa ul-verb;
$IsA isa verb,
has id "IsA";
} then {
(verb: $IsA, arg0: $x0, arg1: $x2) isa ul-verb-inferred;
};
rule vIsSubclassOf-trans: when {
$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;
$verb isa verb,
has id "IsSubclassOf";
(verb: $verb, arg0: $e0, arg1: $e1) isa! ul-verb;
(verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb;
} then {
(verb: $verb, arg0: $e0, arg1: $e2) isa ul-verb-inferred;
};
define

ul-verb-inferred sub ul-verb;

rule rule_MyPassage: when {
$x0 isa ul-entity;
$x1 isa ul-entity;
$IsA isa verb,
has id "IsA";
(verb: $IsA, arg0: $x0, arg1: $x1) isa! ul-verb;
$x2 isa ul-entity;
$IsSubclassOf isa verb,
has id "IsSubclassOf";
(verb: $IsSubclassOf, arg0: $x1, arg1: $x2) isa ul-verb;
$IsA isa verb,
has id "IsA";
} then {
(verb: $IsA, arg0: $x0, arg1: $x2) isa ul-verb-inferred;
};
rule vIsSubclassOf-trans: when {
$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;
$verb isa verb,
has id "IsSubclassOf";
(verb: $verb, arg0: $e0, arg1: $e1) isa! ul-verb;
(verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb;
} then {
(verb: $verb, arg0: $e0, arg1: $e2) isa ul-verb-inferred;
};
@David , Sorry there was a mistake in the second rule. I've updated it just now. (verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb-inferred; -> (verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb;
David
DavidOP•3y ago
oh great, thanks! i'll give it a try it still freezes 😦
krishnan
krishnan•3y ago
😦 Now I can't reproduce it. Can you dump your schema again?
David
DavidOP•3y ago
define
id sub attribute,
value string;
ul-verb sub relation,
relates arg0,
relates arg1,
relates verb;
ul-verb-inferred sub ul-verb;
ul-entity sub entity,
owns id,
plays ul-verb:arg0,
plays ul-verb:arg1;
verb sub entity,
owns id @key,
plays ul-verb:verb;
rule rule_MyPassage: when {
$x0 isa ul-entity;
$x1 isa ul-entity;
$IsA isa verb,
has id "IsA";
(verb: $IsA, arg0: $x0, arg1: $x1) isa ul-verb;
$x2 isa ul-entity;
$IsSubclassOf isa verb,
has id "IsSubclassOf";
(verb: $IsSubclassOf, arg0: $x1, arg1: $x2) isa ul-verb;
$IsA isa verb,
has id "IsA";
} then {
(verb: $IsA, arg0: $x0, arg1: $x2) isa ul-verb-inferred;
};
rule vIsSubclassOf-trans: when {
$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;
$verb isa verb,
has id "IsSubclassOf";
(verb: $verb, arg0: $e0, arg1: $e1) isa ul-verb;
(verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb;
} then {
(verb: $verb, arg0: $e0, arg1: $e2) isa ul-verb-inferred;
};
define
id sub attribute,
value string;
ul-verb sub relation,
relates arg0,
relates arg1,
relates verb;
ul-verb-inferred sub ul-verb;
ul-entity sub entity,
owns id,
plays ul-verb:arg0,
plays ul-verb:arg1;
verb sub entity,
owns id @key,
plays ul-verb:verb;
rule rule_MyPassage: when {
$x0 isa ul-entity;
$x1 isa ul-entity;
$IsA isa verb,
has id "IsA";
(verb: $IsA, arg0: $x0, arg1: $x1) isa ul-verb;
$x2 isa ul-entity;
$IsSubclassOf isa verb,
has id "IsSubclassOf";
(verb: $IsSubclassOf, arg0: $x1, arg1: $x2) isa ul-verb;
$IsA isa verb,
has id "IsA";
} then {
(verb: $IsA, arg0: $x0, arg1: $x2) isa ul-verb-inferred;
};
rule vIsSubclassOf-trans: when {
$e0 isa ul-entity;
$e1 isa ul-entity;
$e2 isa ul-entity;
$verb isa verb,
has id "IsSubclassOf";
(verb: $verb, arg0: $e0, arg1: $e1) isa ul-verb;
(verb: $verb, arg0: $e1, arg1: $e2) isa ul-verb;
} then {
(verb: $verb, arg0: $e0, arg1: $e2) isa ul-verb-inferred;
};
oh i'm missing an exclamation mark what does it do?
James Whiteside
James Whiteside•3y ago
Good question! The syntax isa! is used to constrain a variable to the given type only, rather than subtypes as well. For example, if we have car sub vehicle; bike sub vehicle; and we query for $v isa! vehicle;, we'll only get instances of vehicle back and none of car or bike. In the rule, it reduces the number of redundant operations performed that lead to the same conclusion, and so improves performance. There's also a sub! syntax, which is used for querying proper subtypes only rather than the type itself and its subtypes as with sub.
krishnan
krishnan•3y ago
(You're missing two. One in each rule)
Joshua
Joshua•3y ago
@David if you're resolved we can consider closing this thread down 🙂
David
DavidOP•3y ago
hi - i didn't have time to look today; can we hold off shutting the thread?
Joshua
Joshua•3y ago
yep no probs 🙂

Did you find this page helpful?