TypeDB•2mo ago

Rules?

I am sure this has come up before, but your main features page shows that version three supports rules while your migration page says they have been removed before I spend my time looking at three. Can somebody tell me are rules gone and is there any plan to bring them back or did this turn into just another database?

12 Replies

Joshua•2mo ago

Hi Randy, we're in the process of replacing parts of our website so the features page is still WIP! Rules as they were are now replaced by functions, which execute very similarly under the hood, but are more familiar to users && more performant - we're unlikely to bring them back for the foreseeable future https://typedb.com/docs/core-concepts/typeql/queries-as-functions we're going to add a section that's going to explain rules vs functions as well cc @krishnan see @callum we should really push on the website!

RandyOP•2mo ago

Thank you. Is there a way to do A implies relation in one function then B implies relation C fifty levels deep with rules. My use case was rule systems so huge no other tool could handle them. Removing rules took that one of a kind tool away. I understand that it was unmanageable as a database feature. As a stand alone reasoning engine, who cared if a reply to a query takes days when you are getting answers that no other system could

Joshua•2mo ago

I'm sorry to say that it's gone away, but yes as much as i'd love to retain rules for this kind of use case, most database operations have low-ish latency requirements to power applications

krishnan•2mo ago

Hi Randy, I've not written the docs page comparing them yet, but the thread you found (here: https://discord.com/channels/665254494820368395/1407632218922422374/1414798873230184559) did outline my thoughts on them. As Joshua pointed out, functions can emulate rules to a certain extent. Notably, you can no longer infer types in your schema, but you can see function evaluation as inferring "instances of functions". So you can see

rule x: when {
  ($x, $y) isa p;
} then {
  ($x, $y) isa q;
};

rule x: when {
  ($x, $y) isa p;
} then {
  ($x, $y) isa q;
};

Would have to become:

fun q() -> { t1, t2 }:
match 
  ($x, $y) isa p;
return {$x, $y};

fun q() -> { t1, t2 }:
match 
  ($x, $y) isa p;
return {$x, $y};

So there's a forced separation between the stored types and the inferred types. The catch here is that this functions currently (until we implement "partial evaluation") must return all pairs of $x,y. For good performance, you may have to replace calls to q() with a different variant. e.g.

fun q_given_x($x: t1) -> { t2 }:
match 
  ($x, $y) isa p;
return { $y };

fun q_given_y($y: t2) -> { t1 }:
match 
  ($x, $y) isa p;
return { $x };

fun q_given_x($x: t1) -> { t2 }:
match 
  ($x, $y) isa p;
return { $y };

fun q_given_y($y: t2) -> { t1 }:
match 
  ($x, $y) isa p;
return { $x };

And then update match $x isa person, has name "John"; let $x, $y in q(); to match $x isa person, has name "John"; let $y in q_given_x($x); To clarify this: Functions are evaluated the same way rules were, so they'd be equally performant for most cases. Functions aren't going to magically be low latency, we just expect users to avoid accidentally triggering more computation than they need, or our planner not being able to pick the right plan (which wasn't too often in the later versions of 2.x) I'm actually thrilled to hear we managed to solve your problems. As Joshua pointed out, all inferred results were stored in memory and when the reasoner slowed down, it was typically because the GC was running all the time trying to find a few extra KBs it needed to run the program. This shouldn't happen in 3.x because: 1. We don't "table" every inferred answer 2. We can actually clean up answers if we find use-cases where we need to 3. There's no GC in rust, so when we'll quickly reach the point where we're out of memory and the malloc will panic 1 & 2 weren't possible in 2.x because execution was goal-driven, but not exactly top-down. In 3.x we've returned to stack-based executor, and only cyclic functions are tabled (to break infinite recursion and guarantee termination for "finite models"). It's difficult to freely parallelise this executor, but when we do get there, we'll preserve this memory footprint and not forget the lessons learnt from 2.x Ah, the other limitation we have at the moment is that we have no lists, nor inferred relations. So there are limits to what we can compute. I'm happy to go back and forth on this to see if 1. Functions as they are satisfy your use case 2. If not, Whether the addition of lists would. 3. Would partial evaluation make greatly improve the experience. And if all of these would end up being answered "no", We'll have a great example of what we've lost from the transition and try to find a way to bridge the gap.

davux•2mo ago

it would probably be helpful to have a documentation page guiding users to migrate what was previously addressed with rules as this thread shows, it's not trivial even for TypeDB experts

Joshua•2mo ago

Yes krishnan is writing a page up in it!

RandyOP•2mo ago

You have limited reasoning but it can be turned into a major feature with a minor syntax change let $bool = exist(match | fun) be syntactic sugar for let $bool = {match | fin} The one programmers think of as part of a lookup while the other is program logic built into a query. Now they no longer have to fetch a permission to find out if it exists As for implementations you create a specific type of function that only can return a boolean value and can declare a signature, which is just a match statement. Then you look it up using this signature the way you matched rules. It returns the true false through the functionality in the body of the function then nothing has to be persisted, except for the bullion variable.

krishnan•2mo ago

We only persist if there are functions which call themselves so that's not really a problem for most cases. I don't really follow the rest of the suggestions

RandyOP•2mo ago

In 2.x you determined which rules were relevant by searching if they had the desired output signature that same logic could be used to find the is it implicated type functions the question the function answers is can the match defined in this function, be implied by the data currently in the database match statement. You do not generate the information you just inform if believing the implementation is justified by the current knowledge and other implications

krishnan•2mo ago

This might work if you're not interested in the answers returned by the match, but you can have queries where you're looking for second degree friends who also work together. You still don't need to store any information, but you do need to stream out the answers to $y

match 
let $y in second_degree_friends($x);
let $y in colleagues($x);

match 
let $y in second_degree_friends($x);
let $y in colleagues($x);

The implication would use existing rule finding logic to find the function instead of a name lookup you do a signature lookup

Could you give an example of this? Or if your project is public, a full example we can look at so we can be better informed about what we're talking about @Randy and @davux . I've written a document which compares functions to rules, though I don't want to write "a migration guide" since following a recipe might lead to less-than-ideal ways of writing functions. I'd love to hear your thoughts: https://development--typedb-docs.netlify.app/reference/typeql/functions/functions-vs-rules

RandyOP•2mo ago

@krishnan your document did a very good job of describing the difference between functions and rules You did gloss over the fact that for a user to infer new data from existing data they have to get approval for some programmer to add a new abstract entity or relationship because full read authority does not allow that while a user with full read authority could implement rules as part of his quarry and receive back new relations and entities that didn’t already exist in the database. This gives managers far less access to unique knowledge which is one of your selling points @krishnan for an example With this, ($a , $b) isa membership , Infer, ($b, $a) isa member-of, sub relation; You define the function as an inference Inference (($g, $m) isa membership ) -> ($m, $g) isa member-of sub relation, match ($g, $m) isa membership, Return ($m, $g) isa member-of; The search engine would match the “with this” to the inference input signature and the “infer” section to the output signature. The output would be a temporary abstract entity or relationship. The body anything that is legal in a function. This is for clarification only, the syntax and the implementation could take many different forms.

krishnan•2mo ago

. We do have "preamble functions" that are specified at query time and excuse within the scope of the query. Functions defined in the schema can be viewed as the library functions that ship with the types. Preamble functions can be seen as the application specific logic that the person running the query has to write This sounds like what rules used to do, as you said here

The implication would use existing rule finding logic to find the function instead of a name lookup you do a signature lookup

I've slightly lost context over the weekend as well. Was this an example to show we can infer that the existence of a new instance of type " is justified by the current knowledge and other implications" without having to actually "generate new information". I'm sure it's possible, but the decision to move to functions and not have rules wasn't based on technical issues. It was to find the easiest model for our users to use. As the write-up indicates, we felt that separating out computation & data by separating inferred types from stored types would make the whole computation more transparent to the user. I still think the analogy from relations to functions should apply to most of the use cases: i.e., they're the same except (1) the role-players are partitioned into arguments and return values, and (2) the roles are implicit in their positions) If you'd like, you can post a subset of your schema and we can see if functions fit (or if there is indeed a pain point, adapt functions to see how we can solve it) Footnote: The subtlety when inferring a relation (which should be solvable) is that you need to be able to take the instance of the relation and use it in other constraints in a query (e.g. passing to a function which would retrieve the role-players).

Gaming

Programming

Rules?

Did you find this page helpful?