Unable to deserialize results with Gremlin-go client + JanusGraph

Hi all - I'm trying to set up a JanusGraph database and use the Gremlin-go client to run some gremlin traversals over it. I'm facing some serialization error. My docker-compose file to start a JanusGraph at localhost:8182:
version: '3'
services:
janusgraph:
image: janusgraph/janusgraph:1.0.0 # Use the latest JanusGraph image
container_name: jce-janusgraph
ports:
- "8182:8182" # Map the Gremlin server port to the host
environment:
JANUS_PROPS_TEMPLATE: cql
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.storage.hostname: jce-cassandra
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
networks:
- jce-network
restart: unless-stopped
depends_on:
- cassandra

cassandra:
image: cassandra:3
container_name: jce-cassandra
ports:
- "9042:9042"
- "9160:9160"
volumes:
- graph_data:/var/lib/cassandra
networks:
- jce-network

volumes:
graph_data: # Named volume to store JanusGraph data

networks:
jce-network:
version: '3'
services:
janusgraph:
image: janusgraph/janusgraph:1.0.0 # Use the latest JanusGraph image
container_name: jce-janusgraph
ports:
- "8182:8182" # Map the Gremlin server port to the host
environment:
JANUS_PROPS_TEMPLATE: cql
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.storage.hostname: jce-cassandra
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
networks:
- jce-network
restart: unless-stopped
depends_on:
- cassandra

cassandra:
image: cassandra:3
container_name: jce-cassandra
ports:
- "9042:9042"
- "9160:9160"
volumes:
- graph_data:/var/lib/cassandra
networks:
- jce-network

volumes:
graph_data: # Named volume to store JanusGraph data

networks:
jce-network:
My minimal example Golang code that is not working:
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

query := `g.V().limit(2)`

result, err := remoteConnection.Submit(query)
if err != nil {
panic(err)
}

resultList, err := result.All()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

query := `g.V().limit(2)`

result, err := remoteConnection.Submit(query)
if err != nil {
panic(err)
}

resultList, err := result.All()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
Solution:
For Python, you can also use JanusGraph-Python to get better support for JanusGraph types: https://github.com/JanusGraph/janusgraph-python (JanusGraph-Python only extends Gremlin-Python so you will still be using that. It just adds serializers for JanusGraph types)...
GitHub
GitHub - JanusGraph/janusgraph-python: JanusGraph Python Gremlin La...
JanusGraph Python Gremlin Language Variant (GLV). Contribute to JanusGraph/janusgraph-python development by creating an account on GitHub.
Jump to solution
10 Replies
zlfben
zlfben3mo ago
Output (at result.All()):
panic: E0408: unknown data type to deserialize 0x0

goroutine 1 [running]:
main.main()
/home/apr16-2024/main.go:30 +0x44d
panic: E0408: unknown data type to deserialize 0x0

goroutine 1 [running]:
main.main()
/home/apr16-2024/main.go:30 +0x44d
Some notes: - I didn't have the above issue if I used the Gremlin-go's bytecode APIs (i.e., not a string, but constructing steps within Golang). - The above code also didn't have any issue when connecting to Tinkerpop or Neptune instead of JanusGraph. - The same query under the groovy console works. Ran into the same error using Gremlin-go's bytecode APIs:
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

g := gremlingo.Traversal_().WithRemote(remoteConnection)

resultList, err := g.V().Limit(2).ToList()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
package main

import (
"time"

gremlingo "github.com/apache/tinkerpop/gremlin-go/driver"
)

func main() {
endpoint := "ws://localhost:8182/gremlin"
var remoteConnection *gremlingo.DriverRemoteConnection

remoteConnection, err := gremlingo.NewDriverRemoteConnection(endpoint, func(settings *gremlingo.DriverRemoteConnectionSettings) {
settings.TraversalSource = "g"
settings.KeepAliveInterval = time.Duration(15 * time.Minute)
})
if err != nil {
panic(err)
}

g := gremlingo.Traversal_().WithRemote(remoteConnection)

resultList, err := g.V().Limit(2).ToList()
if err != nil {
panic(err)
}

for _, item := range resultList {
println(item.String())
}

defer remoteConnection.Close()
}
I noticed in gremlin_python, I faced the same issue, but I was able to fix it by setting message_serializer=serializer.GraphSONMessageSerializer() in the DriverRemoteConnection to fix it. The issue seems to come from the default binary serializer after reading some Github Issues and the general channel. I suspect the issue is with the serializer my Golang code uses (the binary one), but I have no idea how it could be set to another serializer (Graphson) like gremlin_python does. Does anyone have some clue how to change the serializer in gremlin-go? Thanks!
spmallette
spmallette3mo ago
can someone who knows @janusgraph help on this one?
Yang Xia
Yang Xia3mo ago
One note on gremlin-go, it only supports the graphBinary serialization format, so there is no way to change the serializer. Not sure if folks from JanusGraph know any workaround?
spmallette
spmallette3mo ago
really need to get rid of the need for custom serializers in the future. it's too much burden on providers to have to build them and too complicated for users to figure them out. providers should make sure that their ids can be coerced to/from string or otherwise operate with the set of datatypes TinkerPop allows.
Bo
Bo3mo ago
@zlfben As a workaround - could you try JanusGraph 0.6.3?
providers should make sure that their ids can be coerced to/from string
That will be a huge breaking change 😦
zlfben
zlfben3mo ago
Thanks. Just tried JanusGraph 0.6.3. With out of box configuration it worked well with gremlin-go's graphBinary serialization. However, it doesn't support my old configuration (shown below). After setting these for the JanusGraph, it seemed to have a lot of issues setting any types of vertex ids (long, string, etc.). I do need to use UUID as my vertex IDs - is there any way to configure JanusGraph 0.6.3 to support that, or is it only available in 1.0 or later?
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
JANUSGRAPH_RELATION_DELIMITER: "@"
janusgraph.graph.set-vertex-id: "true"
janusgraph.graph.allow-custom-vid-types: "true"
Bo
Bo3mo ago
I do need to use UUID as my vertex IDs - is there any way to configure JanusGraph 0.6.3 to support that, or is it only available in 1.0 or later
That's only supported in 1.0 😦
zlfben
zlfben3mo ago
Thanks for the replies! In the end I just built a small gremlin broker using gremlin_python + Graphson serializer and called the broker in my Golang code. Hopefully future updates of gremlin-go or JanusGraph can solve this issue.🤞
Solution
Florian Hockmann
For Python, you can also use JanusGraph-Python to get better support for JanusGraph types: https://github.com/JanusGraph/janusgraph-python (JanusGraph-Python only extends Gremlin-Python so you will still be using that. It just adds serializers for JanusGraph types)
GitHub
GitHub - JanusGraph/janusgraph-python: JanusGraph Python Gremlin La...
JanusGraph Python Gremlin Language Variant (GLV). Contribute to JanusGraph/janusgraph-python development by creating an account on GitHub.
Florian Hockmann
And if you want to continue Go, then maybe you are interested in implementing basic support for JanusGraph types yourself there? In that case you could check out the GraphBinary support for JanusGraph in JanusGraph.Net: https://github.com/JanusGraph/janusgraph-dotnet/tree/master/src/JanusGraph.Net/IO/GraphBinary to get an idea of what needs to be implemented. Most types are even optional, e.g., you don't need support for Geoshapes / Point if you aren't using those
GitHub
janusgraph-dotnet/src/JanusGraph.Net/IO/GraphBinary at master · Jan...
JanusGraph .NET Gremlin Language Variant (GLV). Contribute to JanusGraph/janusgraph-dotnet development by creating an account on GitHub.
Want results from more Discord servers?
Add your server
More Posts
Fulltext-search-like features without ElasticSearch, OpenSearch, Solr and such?I've read in multiple sources that Apache TinkerPop isn't optimized for text search operations like Conditionally updating a variable with choose()How do I create and update a variable with a conditional? I need a number to be calculated based on Systems Analysis Report on Apache TinkerPop - Where to Start?Hey all, I'm currently writing an alaysis on Apache TinkerPop for grad school and was just hoping thLambda example in TypeScriptDoes anyone know where I can find example code that demonstrates up-to-date best practices for writimergeE(): increment counter on matchHi, is there an easy way to increment an existing edge property based on its current value using `meSerialization IssueI have a weird error, when I am connecting with JanusGraph gremlin client using `conf/remote-graph-Design decision related to multiple heterogenous relational graphsI'm working with over 100k instances of heterogeneous, relational node-and-edge attributed graphs, eStackoverflow when adding a larger list of property values using traverser.property()Hey, we encounter a stack overflow: ``` Exception during Transaction, rolling back ... org.apache.tijava: package org.apache.tinkerpop.shaded.jackson.core does not existWhile trying to `mvn clean install` with jdk11, I ran into the above error using the master branch. Performance issue in large graphsWhen performing changes in large graph (ca. 100K nodes, 500K edges) which is stored in one kryo file