G.V() IDE can't visualize path().by(valueMap()) query

Hi @G.V() - Gremlin IDE (Arthur) sorry if this is a duplicate question. I am playing around with G.V() IDE and run into a problem: if I run a path() query, G.V() IDE could properly display the nodes and the edges, but since a path query itself does not return properties, G.V cannot display the properties. If I add .by(valueMap()) to the end of the query, then the Gremlin query result would include both the path and properties, but G.V IDE cannot visualize it. Is this a known problem? Thx!
28 Replies
gdotv
gdotv11mo ago
hey, that's odd, the way G.V() works it should just automatically fetch properties on elements returned by queries regardless of the structure they're wrapped in (hashmap, path, list, etc), either by leveraging Tinkerpop's 3.7.0 bundling of props with elements or for tinkerpop < 3.7.0 by fetching the properties "in post" and affixing them to the elements. are you able to share a query perhaps just to check? also im assuming this is happening on a G.V() playground? the .valueMap() by modulator isn't showing a graph because technically valueMap isn't a vertex or an edge, and so G.V() doesn't know to offer a graph view for those so that part is expected behaviour, but properties not being included in your element in a normal path query that contains edges/vertices definitely aint rigth
Bo
Bo11mo ago
Interesting! I am not using playground - I am using a local TinkerGraph instance: The query is fairly simple:
g.V("1060.1").out().
limit(1).
path()
g.V("1060.1").out().
limit(1).
path()
I am using TinkerGraph 3.7.0. I do remember it worked for me in the past. Maybe I was using a different TinkerGraph version or G.V version back then. Let me try the latest ones.
gdotv
gdotv11mo ago
Yeah that should work. Any chance your tinkergraph is 3.7.0 but not allowing properties to be included with elements? I know that behavior can be configured, can't remember the setting's name off hand
Bo
Bo11mo ago
If I run g.V("1060.1").out().limit(1), then I can see vertex properties I just tried with TinkerGraph 3.7.1 but no luck My TinkerGraph configurations are fairly simple:
gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY
gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
gremlin.tinkergraph.vertexIdManager=ANY
I deleted the existing connection and established a new connection. It works now! So it seems to be there's something off with GdotV
gdotv
gdotv11mo ago
hey, weird, i wonder what went wrong there, ill investigate on my end, see if i can spot anything weird
Bo
Bo11mo ago
I am able to reproduce the problem. Every time I create a new connection,
g.V("1060.1").out().
limit(1).
path()
g.V("1060.1").out().
limit(1).
path()
shows the vertex properties ONLY for the 1st time. If I run the same query again, I get "no properties available" in the IDE.
gdotv
gdotv11mo ago
okay, that's definitely weird, lemme see if i can reproduce this with a standard gremlin server is it only occurring for path queries on any queries returning elements? also thanks for the additional context, this is helpful
Bo
Bo11mo ago
It's only for path queries If I remove path(), it shows the vertex properties
gdotv
gdotv11mo ago
wow, that's super specific, i guess that's how it slipped through the cracks, really interesting in the query editor on G.V(), it should show on the bottom left what the detected gremlin version is, is that showing as 3.7.1?
Bo
Bo11mo ago
yes, 3.7.1
gdotv
gdotv11mo ago
weird, so the version is definitely getting correctly detected
spmallette
spmallette10mo ago
was there ever a solution to this one? just a bug @G.V() - Gremlin IDE (Arthur) ?
gdotv
gdotv10mo ago
Haven't managed to reproduce the issue so unfortunately not at this time, though i have it tracked internally, could probably just close this and ill follow up directly with Boxuan if/when i get it fixed
Bo
Bo10mo ago
I created a playground and wasn't able to reproduce either. I can only reproduce it using a standalone tinkergraph server.
gdotv
gdotv9mo ago
Hey, bumping this with some recent-ish findings that may provide some further explanation, though I'm still unclear on the exact issue and whether it's TinkerPop related or not. I've seen one separate report on a different tinkerpop database provider using Gremlin >= 3.7.0 where the inVertex/outVertex fields of the Edge object had an empty "properties" field, which effectively resulted in what Boxuan initially reported of properties not showing in G.V() with that provider the issue occurred only under certain queries using the bothE, outE, inE and path steps. I don't know whether it's a framework level issue or provider specific, but I find it suspicious now having heard reports of it for 2 different providers. I wonder if it's at all possible under some circumstances the in/out vertex fields in Edge objects are missing properties when Edge result objects are returned @spmallette i think there's potential a gremlin-server bug here, ive now heard multiple reports from various implementations of the same issue
Lyndon
Lyndon9mo ago
Arthur mentioned this to me. I tried this very simple test with TinkerGraph behind gremlin-server.
final Cluster cluster = Cluster.build().addContactPoint("localhost").port(8182).create();
final DriverRemoteConnection drc = DriverRemoteConnection.using(cluster, "g");
final GraphTraversalSource g = traversal().withRemote(drc);
g.V().drop().iterate();
Vertex foo = g.addV("Foo").property("has", "bar").next();
Vertex bar = g.addV("Bar").property("has", "baz").next();
Edge knows = g.V(foo).addE("knows").to(bar).property("hi", "lo").next();
List<Edge> edges1 = g.V(foo.id()).outE().toList();
List<Edge> edges2 = g.V(bar.id()).inE().toList();
Edge edges3 = g.E(knows.id()).next();
final Cluster cluster = Cluster.build().addContactPoint("localhost").port(8182).create();
final DriverRemoteConnection drc = DriverRemoteConnection.using(cluster, "g");
final GraphTraversalSource g = traversal().withRemote(drc);
g.V().drop().iterate();
Vertex foo = g.addV("Foo").property("has", "bar").next();
Vertex bar = g.addV("Bar").property("has", "baz").next();
Edge knows = g.V(foo).addE("knows").to(bar).property("hi", "lo").next();
List<Edge> edges1 = g.V(foo.id()).outE().toList();
List<Edge> edges2 = g.V(bar.id()).inE().toList();
Edge edges3 = g.E(knows.id()).next();
What I see is that the inVertex/outVertex properties of edges1, edges2, and edges3 are all not populated. With 3.7.0 do we expect the vertices attached to edges to have their properties populated @spmallette? I read the docs but it wasn't clear to me about this specific case.
spmallette
spmallette9mo ago
for an edge it is just the id/label of the inV/outV. i think writing properties is meant to only apply to the main element being serialized, not ones incident to the element. like a VertexProperty doesn't have the parent Vertex with all its properties serialized with it (we might have even skipped even the id/label in that case) - @Valentyn Kahamlyk is the VertexProperty issue i just mentioned something to reconsider in terms of consitency in serialization?
Valentyn Kahamlyk
GitHub
tinkerpop/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/s...
Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.
spmallette
spmallette9mo ago
i think writing properties is meant to only apply to the main element being serialized, not ones incident to the element.
i guess Path falls under that rule, though i'm not sure it holds consistently like for Graph though it may hold for Tree. Graph may need to be is the exception, which seems sane to me
gdotv
gdotv9mo ago
If a path were to return the raw elements (vertex/edge), would it make sense then to include properties too?
spmallette
spmallette9mo ago
i think the the "element" being serialized there is the Path and that what is contained in the Path is incident to it, so i'd say "no" based on that thinking. to me Graphis the only exception to that rule. i suppose i could be convinced otherwise, which is why i ping'd @Valentyn Kahamlyk since he's thinking about such things right now.
gdotv
gdotv9mo ago
i see, I think this feels a bit inconsistent in terms of behaviour, i would expect serialization to behave the same for a given type of element regardless of context given that the data type it's contained within doesn't really exclude it from being serialized, i think in this case we're just saying it's getting serialized differently i was just checking the docs as well and couldn't find any reference to this behaviour (checked https://tinkerpop.apache.org/docs/current/reference/#_properties_of_elements specifically) in terms of use case it's been causing issues with some G.V() users running path queries and not getting properties fetched as part of the downstream graph visualisation it produces - obviously i can alter the way G.V() behave to try and identify elements that don't include properties because they're within a path, but it feels like unnecessary complexity for this behaviour
spmallette
spmallette9mo ago
i understand it feels inconsistent and it's bad that we don't document this well. here's what i tend to think about:
gremlin> g.V().outE().inV().path()
==>[v[1],e[9][1-created->3],v[3]]
==>[v[1],e[7][1-knows->2],v[2]]
==>[v[1],e[8][1-knows->4],v[4]]
==>[v[4],e[10][4-created->5],v[5]]
==>[v[4],e[11][4-created->3],v[3]]
==>[v[6],e[12][6-created->3],v[3]]
gremlin> g.V().outE().inV().path()
==>[v[1],e[9][1-created->3],v[3]]
==>[v[1],e[7][1-knows->2],v[2]]
==>[v[1],e[8][1-knows->4],v[4]]
==>[v[4],e[10][4-created->5],v[5]]
==>[v[4],e[11][4-created->3],v[3]]
==>[v[6],e[12][6-created->3],v[3]]
in this really simple example for a Path object, each one would have the Vertex serialized twice. It seems like a costly burden, especially if those vertices have a lot of properties. Also, if we're serializing incident vertices on an Edge, is it inconsistent to not also serialize incident edges on vertices? aside from the serialization costs, what a strange circular place that would be if we included that. Similary, should a VertexProperty serialize a parent Vertex which would then reserialize the same VertexProperty? What happens when we have EdgeProperty (i.e. meta properties on edges) ? My general feeling right now is that a Path and a Tree are containers like a Graph and in that sense should probably contain associated properties for contained elements. On the other hand, I feel like an Element should have an exception and treat references to other elements as references and only include id/label. That seems like a clean distinction and avoids circular, repetitive serialization. Again, just sharing some thoughts here and not a decision. I'd like to hear what others have to say.
gdotv
gdotv9mo ago
i get your point now, it's definitely not an easy line to draw - i think what made me feel path would be okay to have these elements serialized is that the Path object is effectively not dissimilar from a simple array. That being said now that I understand the logic it's something I can easily work around - it's very hard to tell how many would actually be impacted by this either way - I suspect not many!
Valentyn Kahamlyk
I agree that it makes sense to add properties on elements to containers like Path, Tree and Graph. We can skip properties by default , send only with explicit with('materializeProperties', 'all'), but this will add more chaos and confusion. Or maybe introduce other options like with('materializeProperties', 'top') or with('materializeProperties', 'nested'). @G.V() - Gremlin IDE (Arthur) what do you think is the best from your point of view as a user of this function?
gdotv
gdotv9mo ago
I think at this stage changing the serialization behavior from implicit to explicit would cause a lot of problems. I spotted this materializeProperties and gotta say it's a great option to have as an opt out. I reckon an all or nothing would do, it seems highly unlikely anyone would find much value in an in between behavior. Selfishly of course trees, graphs and paths being treated the same as arrays or hashmaps in terms of serialization would be ideal for G.V(). I suspect it would make serialization more consistent in the sense that these objects are essentially data structures first, not classes like a vertex property might be
spmallette
spmallette9mo ago
i'd suggested a more complex approach to configuring property serialization and i'm glad i was convinced that we just went with the all or nothing approach in "materializeProperties". I think we should stick with that for now and just have clear definition for where they should be expected and where they should not. the more i think about my thoughts around "containers" and "elements" the more i like what i proposed there. hopefully that can get implemented and straightened out soon once and for all.
gdotv
gdotv9mo ago
Quick update on my end, the upcoming version of gdotv (should be out next week) will include a fix that sorts this out, irregarding of how this is taken forward in apache Tinkerpop
Want results from more Discord servers?
Add your server