VertexProgram filter graph before termination
I have a VertexProgram that operates on vertices of type A and B.
B vertices are "below" A vertices.
The VertexProgram aggregates stuff about the underlying B vertices into their common parent A vertex.
I've successfully done the aggregation, but now I want to filter the A vertices that don't pass a predicate about the aggregated property they've accumulated based on messages from their underlying B vertices.
I was thinking of having a follow-up state machine for the VertexProgram like the ShortestPath vertex program does when it does its aggregations to filter any vertices that aren't of label A or are label A but fail the predicate check upon their aggregated value.
I tried doing this via
.drop() after returning true from the Feature's requiresVertexRemoval() . However it seems SparkGraphComputer doesn't support this feature.
I've been able to flatten the identifiers of A vertices using a follow-up map reduce job chained with the VertexProgram but was just wondering if maybe there's something else I'm missing?
Being able to return a filtered view of the graph following a VertexProgram's execution would be nice without having to flatten IDs via a trailing MapReduce job writing a Set to the Memory object.2 Replies
I imagine you may need to add something in SparkExecutor::executeVertexProgramIteration method.
That does look plausible. I was hoping I could hook into something without having to muddle into SparkGraphComputer's internals or duplicate its implementation, if I understand what you're suggesting.
But persisting nothing graph wise and re-retrieving the vertices based on the IDs collected into the return Memory may be the least lift to get what I was wanting to arrive at.