Unable to deserialize results with Gremlin-go client + JanusGraph

Hi all - I'm trying to set up a JanusGraph database and use the Gremlin-go client to run some gremlin traversals over it. I'm facing some serialization error. My docker-compose file to start a JanusGraph at localhost:8182:
version: '3'
services:
janusgraph:
image: janusgraph/janusgraph:1.0.0 # Use the latest JanusGraph image
container_name: jce-janusgraph
ports:
- "8182:8182" # Map the Gremlin server port to the host
environment:
JANUS_PROPS_TEMPLATE: cql
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.storage.hostname: jce-cassandra
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
networks:
- jce-network
restart: unless-stopped
depends_on:
- cassandra

cassandra:
image: cassandra:3
container_name: jce-cassandra
ports:
- "9042:9042"
- "9160:9160"
volumes:
- graph_data:/var/lib/cassandra
networks:
- jce-network

volumes:
graph_data: # Named volume to store JanusGraph data

networks:
jce-network:
version: '3'
services:
janusgraph:
image: janusgraph/janusgraph:1.0.0 # Use the latest JanusGraph image
container_name: jce-janusgraph
ports:
- "8182:8182" # Map the Gremlin server port to the host
environment:
JANUS_PROPS_TEMPLATE: cql
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.storage.hostname: jce-cassandra
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
networks:
- jce-network
restart: unless-stopped
depends_on:
- cassandra

cassandra:
image: cassandra:3
container_name: jce-cassandra
ports:
- "9042:9042"
- "9160:9160"
volumes:
- graph_data:/var/lib/cassandra
networks:
- jce-network

volumes:
graph_data: # Named volume to store JanusGraph data

networks:
jce-network:
My minimal example Golang code that is not working:
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

query := `g.V().limit(2)`

result, err := remoteConnection.Submit(query)
if err != nil {
panic(err)
}

resultList, err := result.All()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

query := `g.V().limit(2)`

result, err := remoteConnection.Submit(query)
if err != nil {
panic(err)
}

resultList, err := result.All()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
Solution:
For Python, you can also use JanusGraph-Python to get better support for JanusGraph types: https://github.com/JanusGraph/janusgraph-python (JanusGraph-Python only extends Gremlin-Python so you will still be using that. It just adds serializers for JanusGraph types)...
Z
zlfben20d ago
Output (at result.All()):
panic: E0408: unknown data type to deserialize 0x0

goroutine 1 [running]:
main.main()
/home/apr16-2024/main.go:30 +0x44d
panic: E0408: unknown data type to deserialize 0x0

goroutine 1 [running]:
main.main()
/home/apr16-2024/main.go:30 +0x44d
Some notes: - I didn't have the above issue if I used the Gremlin-go's bytecode APIs (i.e., not a string, but constructing steps within Golang). - The above code also didn't have any issue when connecting to Tinkerpop or Neptune instead of JanusGraph. - The same query under the groovy console works. Ran into the same error using Gremlin-go's bytecode APIs:
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

g := gremlingo.Traversal_().WithRemote(remoteConnection)

resultList, err := g.V().Limit(2).ToList()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

g := gremlingo.Traversal_().WithRemote(remoteConnection)

resultList, err := g.V().Limit(2).ToList()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
I noticed in gremlin_python, I faced the same issue, but I was able to fix it by setting message_serializer=serializer.GraphSONMessageSerializer() in the DriverRemoteConnection to fix it. The issue seems to come from the default binary serializer after reading some Github Issues and the general channel. I suspect the issue is with the serializer my Golang code uses (the binary one), but I have no idea how it could be set to another serializer (Graphson) like gremlin_python does. Does anyone have some clue how to change the serializer in gremlin-go? Thanks!
S
spmallette20d ago
can someone who knows @janusgraph help on this one?
YX
Yang Xia19d ago
One note on gremlin-go, it only supports the graphBinary serialization format, so there is no way to change the serializer. Not sure if folks from JanusGraph know any workaround?
S
spmallette18d ago
really need to get rid of the need for custom serializers in the future. it's too much burden on providers to have to build them and too complicated for users to figure them out. providers should make sure that their ids can be coerced to/from string or otherwise operate with the set of datatypes TinkerPop allows.
B
Bo18d ago
@zlfben As a workaround - could you try JanusGraph 0.6.3?
providers should make sure that their ids can be coerced to/from string
That will be a huge breaking change 😦
Z
zlfben18d ago
Thanks. Just tried JanusGraph 0.6.3. With out of box configuration it worked well with gremlin-go's graphBinary serialization. However, it doesn't support my old configuration (shown below). After setting these for the JanusGraph, it seemed to have a lot of issues setting any types of vertex ids (long, string, etc.). I do need to use UUID as my vertex IDs - is there any way to configure JanusGraph 0.6.3 to support that, or is it only available in 1.0 or later?
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
B
Bo18d ago
I do need to use UUID as my vertex IDs - is there any way to configure JanusGraph 0.6.3 to support that, or is it only available in 1.0 or later
That's only supported in 1.0 😦
Z
zlfben18d ago
Thanks for the replies! In the end I just built a small gremlin broker using gremlin_python + Graphson serializer and called the broker in my Golang code. Hopefully future updates of gremlin-go or JanusGraph can solve this issue.🤞
Solution
FH
Florian Hockmann18d ago
For Python, you can also use JanusGraph-Python to get better support for JanusGraph types: https://github.com/JanusGraph/janusgraph-python (JanusGraph-Python only extends Gremlin-Python so you will still be using that. It just adds serializers for JanusGraph types)
GitHub
GitHub - JanusGraph/janusgraph-python: JanusGraph Python Gremlin La...
JanusGraph Python Gremlin Language Variant (GLV). Contribute to JanusGraph/janusgraph-python development by creating an account on GitHub.
FH
Florian Hockmann18d ago
And if you want to continue Go, then maybe you are interested in implementing basic support for JanusGraph types yourself there? In that case you could check out the GraphBinary support for JanusGraph in JanusGraph.Net: https://github.com/JanusGraph/janusgraph-dotnet/tree/master/src/JanusGraph.Net/IO/GraphBinary to get an idea of what needs to be implemented. Most types are even optional, e.g., you don't need support for Geoshapes / Point if you aren't using those
GitHub
janusgraph-dotnet/src/JanusGraph.Net/IO/GraphBinary at master · Jan...
JanusGraph .NET Gremlin Language Variant (GLV). Contribute to JanusGraph/janusgraph-dotnet development by creating an account on GitHub.
Want results from more Discord servers?
Add your server
More Posts
Fulltext-search-like features without ElasticSearch, OpenSearch, Solr and such?I've read in multiple sources that Apache TinkerPop isn't optimized for text search operations like Conditionally updating a variable with choose()How do I create and update a variable with a conditional? I need a number to be calculated based on Systems Analysis Report on Apache TinkerPop - Where to Start?Hey all, I'm currently writing an alaysis on Apache TinkerPop for grad school and was just hoping thLambda example in TypeScriptDoes anyone know where I can find example code that demonstrates up-to-date best practices for writimergeE(): increment counter on matchHi, is there an easy way to increment an existing edge property based on its current value using `meSerialization IssueI have a weird error, when I am connecting with JanusGraph gremlin client using `conf/remote-graph-Design decision related to multiple heterogenous relational graphsI'm working with over 100k instances of heterogeneous, relational node-and-edge attributed graphs, eStackoverflow when adding a larger list of property values using traverser.property()Hey, we encounter a stack overflow: ``` Exception during Transaction, rolling back ... org.apache.tijava: package org.apache.tinkerpop.shaded.jackson.core does not existWhile trying to `mvn clean install` with jdk11, I ran into the above error using the master branch. Performance issue in large graphsWhen performing changes in large graph (ca. 100K nodes, 500K edges) which is stored in one kryo fileConcurrent queries to authentication required sever resulted in 401 errorHey guys, playing around with gremlin & encountered this very odd error where concurrent queries wilDiscrepancy between console server id conventions and NeptuneSo I'm working with my test server and on Neptune--and I'm noticing a difference in the type of the how to connect the amothic/neptune container to the volume?I need to know which directory needs to attach to containeer. so that the data is stored safely. eveDocker yaml authentication settings (gremlinserver.authentication) questionDoes anyone have any experience setting up authentication on Docker by using the supplied .yaml fileGremlin Injection Attacks?Is anyone talking about or looking into attacks and mitigations for Gremlin Injection Attacks? That Returned vertex properties (JS client)Hi, I've got a question regarding the returned vertex value when using the JS client. How come non-aAnyone using Tinkerpop docker as a local Cosmos replacementRunning into some random issues. Looking for tips and tricks.Configuring Websockets connection to pass through a proxy serverHey, I'm working on making G.V() fully proxy aware, but I can't seem to get websockets connection tpython goblin vs spring-data-goblin for interactions with gremlin serverI want an OGM to interact with my gremlin server. What would be a good choice?Is there any open source version of data visualizer for aws neptune?Is there any open source version of data visualizer for aws neptune. I'll need it since it essential