AT
Apache TinkerPop•15mo ago
zendorphins

Has anyone else encountered "NoSuchElementException" when calling getLeafTrees() on a tree object

Did a little bit of debugging and it seems that the issue has to do with a cast to Tree before calling the isLeaf() method inside getLeafTrees(). As far as my testing goes it seems to happen when a child of the root is a leaf. Here's an example dataset and the tree query: Data: 
g.addV('node::Dev').as('1').
  property(single, 'data', 22).addV('node::Dev').
    as('2').
  property(single, 'data', 16).addV('root::Dev').
    as('3').
  property(single, 'data', 9).addV('node::Dev').
    as('4').
  property(single, 'data', 5).addV('node::Dev').
    as('5').
  property(single, 'data', 2).addV('node::Dev').
    as('6').
  property(single, 'data', 11).addV('node::Dev').
    as('7').
  property(single, 'data', 15).addV('node::Dev').
    as('8').
  property(single, 'data', 10).addV('node::Dev').
    as('9').
  property(single, 'data', 1).addV('node::Dev').
    as('10').
  addE('left').from('1').to('2').addE('left').
  from('3').to('4').addE('right').from('3').
  to('6').addE('right').from('3').to('10').
  addE('left').from('4').to('5').addE('left').
  from('5').to('9').addE('left').from('6').
  to('8').addE('right').from('6').to('7').
  addE('right').from('7').to('1')
g.addV('node::Dev').as('1').
  property(single, 'data', 22).addV('node::Dev').
    as('2').
  property(single, 'data', 16).addV('root::Dev').
    as('3').
  property(single, 'data', 9).addV('node::Dev').
    as('4').
  property(single, 'data', 5).addV('node::Dev').
    as('5').
  property(single, 'data', 2).addV('node::Dev').
    as('6').
  property(single, 'data', 11).addV('node::Dev').
    as('7').
  property(single, 'data', 15).addV('node::Dev').
    as('8').
  property(single, 'data', 10).addV('node::Dev').
    as('9').
  property(single, 'data', 1).addV('node::Dev').
    as('10').
  addE('left').from('1').to('2').addE('left').
  from('3').to('4').addE('right').from('3').
  to('6').addE('right').from('3').to('10').
  addE('left').from('4').to('5').addE('left').
  from('5').to('9').addE('left').from('6').
  to('8').addE('right').from('6').to('7').
  addE('right').from('7').to('1')
Query: g.V().hasLabel("Dev").hasLabel("root").emit().until(out().count().is(0)).repeat(out()).tree().by(elementMap()).next() The edge lables are inconsequential, i just modified a binary tree. The elementMap inside the by is also there for debug purposes.
11 Replies
zendorphins
zendorphins•15mo ago
Also here's the stack trace: 
java.util.NoSuchElementException: null
    at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
    at java.base/java.util.HashMap$ValueIterator.next(HashMap.java:1539)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree.isLeaf(Tree.java:112)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree.getLeafTrees(Tree.java:87)
java.util.NoSuchElementException: null
    at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1513)
    at java.base/java.util.HashMap$ValueIterator.next(HashMap.java:1539)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree.isLeaf(Tree.java:112)
    at org.apache.tinkerpop.gremlin.process.traversal.step.util.Tree.getLeafTrees(Tree.java:87)
Somehow when the "tree" object goes into isLeaf() it gets casted to a tree with no value key which returns an exception when used inside the method. At least that what i understood, i'm not amazing at debugging tbh
spmallette
spmallette•15mo ago
well, given your data and traversal it's returning an empty tree: 
gremlin> g.V().hasLabel("Dev").hasLabel("root").
......1>   emit().until(out().count().is(0)).repeat(out()).
......2>   tree()
==>[]
gremlin> tree = g.V().hasLabel("Dev").hasLabel("root").
......1>   emit().until(out().count().is(0)).repeat(out()).
......2>   tree().next()
gremlin> tree.getLeafTrees()
java.util.NoSuchElementException
Type ':help' or ':h' for help.
Display stack trace? [yN]
gremlin> g.V().hasLabel("Dev").hasLabel("root").
......1>   emit().until(out().count().is(0)).repeat(out()).
......2>   tree()
==>[]
gremlin> tree = g.V().hasLabel("Dev").hasLabel("root").
......1>   emit().until(out().count().is(0)).repeat(out()).
......2>   tree().next()
gremlin> tree.getLeafTrees()
java.util.NoSuchElementException
Type ':help' or ':h' for help.
Display stack trace? [yN]
so in some respects i think a NoSuchElementException is expected. Perhaps the behavior should be different in the case of an empty tree. i'm not sure.
zendorphins
zendorphins•15mo ago
That's odd, here's the tree that's returned with my current setup 
{
    "{id=2ac37f32-f84e-84fa-1c9f-c491ae922edb, label=Dev::root, data=9}": {
        "{id=a2c37f32-f854-9b11-33ba-23e14be55452, label=Dev::node}": {},
        "{id=5ec37f32-f84f-a21e-a2f1-8f721fb8d4c8, label=Dev::node, data=5}": {
            "{id=7cc37f32-f850-71b9-4955-79c49e5f4c54, label=Dev::node, data=2}": {
                "{id=3ec37f32-f854-0a28-859a-f34a0cc5e9f0, label=Dev::node, data=1}": {}
            }
        },
        "{id=5ac37f32-f850-e16f-4421-62ef02f5d3ce, label=Dev::node, data=11}": {
            "{id=a2c37f32-f853-dfdb-4c28-a13d5d7beaee, label=Dev::node, data=10}": {},
            "{id=40c37f32-f851-2bcd-7487-6b1b168e5640, label=Dev::node, data=15}": {
                "{id=b0c37f32-f849-1caa-c668-fc2676e16f9c, label=Dev::node, data=22}": {
                    "{id=7cc37f32-f84c-e896-533b-ebb550d64de4, label=Dev::node, data=16}": {}
                }
            }
        }
    }
}
{
    "{id=2ac37f32-f84e-84fa-1c9f-c491ae922edb, label=Dev::root, data=9}": {
        "{id=a2c37f32-f854-9b11-33ba-23e14be55452, label=Dev::node}": {},
        "{id=5ec37f32-f84f-a21e-a2f1-8f721fb8d4c8, label=Dev::node, data=5}": {
            "{id=7cc37f32-f850-71b9-4955-79c49e5f4c54, label=Dev::node, data=2}": {
                "{id=3ec37f32-f854-0a28-859a-f34a0cc5e9f0, label=Dev::node, data=1}": {}
            }
        },
        "{id=5ac37f32-f850-e16f-4421-62ef02f5d3ce, label=Dev::node, data=11}": {
            "{id=a2c37f32-f853-dfdb-4c28-a13d5d7beaee, label=Dev::node, data=10}": {},
            "{id=40c37f32-f851-2bcd-7487-6b1b168e5640, label=Dev::node, data=15}": {
                "{id=b0c37f32-f849-1caa-c668-fc2676e16f9c, label=Dev::node, data=22}": {
                    "{id=7cc37f32-f84c-e896-533b-ebb550d64de4, label=Dev::node, data=16}": {}
                }
            }
        }
    }
}
Perhaps I messed something up when copying the dataset insertion query, I'll check it out tomorrow This is the exact line i use inside my application 
 return g.V().hasLabel("Dev").hasLabel("root")
        .emit().until(out().count().is(0)).repeat(out()).tree().by(__.elementMap()).next();
 return g.V().hasLabel("Dev").hasLabel("root")
        .emit().until(out().count().is(0)).repeat(out()).tree().by(__.elementMap()).next();
The insertion seems fine, i'm not sure why the query itself wouldn't return anything for you. Was there any error thrown during either of the steps? I'm using gremlin in Spring Boot with the java driver, connected to AWS Neptune if that matters in any way. The tree was sent on a route. Also the tinkerpop version is 3.5.2 'gremlin': {'version': 'tinkerpop-3.5.2'}
spmallette
spmallette•15mo ago
on closer inspection, your data doesn't quite align with your query is one issue I see. i guess you are using multi-label features of Neptune. i wasn't trying on Neptune, so I guess that is part of the problem, but other tests I've done to avoid the multi-label feature haven't replicated the problem. i guess i'd have to try on Neptune.
zendorphins
zendorphins•15mo ago
Thanks for the response, I forgot it was a Neptune feature, sorry about that, I added the Dev label because we only have one db and i wanted to easily dispose of the nodes after testing. The labels shouldn't matter in the code as I don't use them for anything, my guess is that perhaps it's in the java driver. I'll try setting some breakpoints and send some screenshots of where it dies inside the method. Oops turns out my assumptions about the structure of the tree leading to that error were wrong, it was giving a completely unrelated error. Here's a few screenshots of what happens with the big dataset i originally encountered this error on. I'll try making another dataset to get to the root of this problem. I'll post it if i succeed in finding out what the problem is
zendorphins
zendorphins•15mo ago
zendorphins
zendorphins•15mo ago
Turns out it was the query actually, I wrongfully dismissed the fact that inside the main application there was a union() step inside the by() of the tree because I assumed it gets sent as a string key for the map. I'm guessing this mashup of data structures causes undefined behaviour. Here's the modified query that results in an error: 
g.V().hasLabel("Dev").hasLabel("root").as("root")             .emit().until(out().count().is(0)).repeat(out()).tree().by(union(identity().fold(),select("root"))).next()
g.V().hasLabel("Dev").hasLabel("root").as("root")             .emit().until(out().count().is(0)).repeat(out()).tree().by(union(identity().fold(),select("root"))).next()
I'll have to think of another way of gathering data from the tree, thanks for your time Yep, the key of the tree has to be a LinkedHashMap, which isn't the case with a union step. If anyone stumbles upon this in the future, the project step works really well as a replacement. I was using union for space efficiency on big data but it was probably premature optimization anyway.
zendorphins
zendorphins•15mo ago
This time I have absolutely no clue. Wish i could provide more info but i'm going crazy with this. Just tested that same query with the union on the exact same dataset and now it's working. Even worse the query with project in the main app that was working yesterday now gives the error. All i can say unfortunately is that somehow a tree with size 0 ends up in the if statement and breaks the isLeaf() function For the time being i'll probably just parse the tree as a string and insert it into a general graph class in java
zendorphins
zendorphins•15mo ago
Grasping at straws really, but it seems that the isLeaf() method is not being used correctly inside getLeafTrees() since it only checks the first branch for child nodes instead of checking all of them, thus providing a reason for the inconsistent errors (because the nodes aren't ordered in any way and get sent from the db in an arbitrary order) Perhaps i've misunderstood what a leaf is under this definition, but under the general tree leaf definition it looks a bit wrong Unless I'm somehow ending up with malformed trees ofc, i'll have to do more debugging Yeah it's not the isLeaf() method itself, somehow when running getLeafTrees() there end up trees in there that have multiple roots. Maybe i misunderstood what a "leaf tree" is, can't blame myself too much for that since there's no doc. Also found this which seems pretty similar to my issue (https://github.com/JanusGraph/janusgraph/issues/1360) Here's the code I now use to extract child trees from a root 
List<Tree<T>> subTreeList = new ArrayList<>();
        for (Tree<T> t1 : tree.values()) {
            for (Map.Entry<T, Tree<T>> t2 : t1.entrySet()) {
                subTreeList.add(new Tree<T>(t2));
            }
        }
List<Tree<T>> subTreeList = new ArrayList<>();
        for (Tree<T> t1 : tree.values()) {
            for (Map.Entry<T, Tree<T>> t2 : t1.entrySet()) {
                subTreeList.add(new Tree<T>(t2));
            }
        }
spmallette
spmallette•15mo ago
I'd like to help but I really need a reproducible case for what you are seeing. Is that something you could provide yet?
zendorphins
zendorphins•15mo ago
Still working on getting some consistency with this error, it really threw me off that the last query suddenly started working the next day. I understood that getLeafTrees() is supposed to retrieve all child subtrees of the root, so if that's the case i wasn't misusing it. I suppose the method is basically those for loops i wrote with an isLeaf() check , but it's too low level for me to completely grasp and debug. I guess for now this can be considered solved since i've found a work-around, hopefully i'll manage to get a consistent reproducible dataset.
Want results from more Discord servers?
Add your server
More Posts
The query gets slower as the number of vertices that already exist in JanusGraph gets bigger and bigWhen i start JanusGraph and perform any query like : - g.V().has(properties).limit(10).toList() - g.Is there a limitation on Neptune HTTP API endpoint compatibility when using a proxy and IAM Auth?Hi, Got a weird one today. I'm working on bringing full compatibility for the use of proxies frontiPreventing Janusgraph crash on timeoutAccording to this: https://stackoverflow.com/questions/61985018/janusgraph-image-stop-responding-aftWay to update static vertexhttps://docs.janusgraph.org/schema/advschema/#static-vertices I read document about TTL vertex. And Dotnet best practice: converting Vertex properties to ModelA very common task in Dotnet is to convert a stored entity into a Model class. How is this best accoWhat is the use of adding type information to e.g ElementMap<type> in Gremlin.Net?Consider the query and output in the attached image: What TYPE could be placed inside the `ElemementHow can I find property with a certain data type?I have a situation where the same property has different type under the same label, kind of like theVerifying the count of ingested vertex and edges after bulk loading in Janusgraph.I have bulk loaded around 600k Vertices and 800k Edges into my janusgraph cluster backed with bigtabTraversal is propagating to further edges?I have node label A and B with edge between them ("Has") Also i have node B with edge to another nodHow to load url data into Neptune?I am trying to load a small dataset into Neptune and it seems to always error. I tried g.io("<file