Trying to run a local version for a test, what is the correct serializer?

Windows machine, local host. I can't find the Running this, https://github.com/bricaud/gremlin-server but updated the version to apache-tinkerpop-gremlin-server-3.7.0 I had issue with the serializers: section of the gremlin-conf.yaml so I commented out to see if the defaults would work. The docker image loads fine, but then I can't seem to connect to it. My simple test:
print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))
print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))
My error:
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected
So I'm trying to find out what serializer goes there, but all of my variants are failing. Anyone know what the correct serializer is?
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 , config: { serializeResultToString: true }} # application/vnd.gremlin-v3.0+gryo-stringd
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 , config: { serializeResultToString: true }} # application/vnd.gremlin-v3.0+gryo-stringd
[WARN] o.a.t.g.s.AbstractChannelizer - Could not find configured serializer class - org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 - it will not be available
[ERROR] o.a.t.g.s.AbstractChannelizer - No serializers were successfully configured - server will not start.
[WARN] o.a.t.g.s.AbstractChannelizer - Could not find configured serializer class - org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 - it will not be available
[ERROR] o.a.t.g.s.AbstractChannelizer - No serializers were successfully configured - server will not start.
GitHub
GitHub - bricaud/gremlin-server: A Docker container with a gremlin ...
A Docker container with a gremlin server. Contribute to bricaud/gremlin-server development by creating an account on GitHub.
24 Replies
ManabuBeach
ManabuBeach15mo ago
Hi @billmanh Serializer choice is often the target graph implementation dependent. Please let us know which server you are going to connect to.
billmanh
billmanh15mo ago
Yeah, I'm creating a server (hosting), then I want to connect to it. I'm just looking for the right ioserializer that's current per my gremlin version.
spmallette
spmallette15mo ago
it's not really relevant to your question i suppose but Is there any reason you are using that docker container? we do have an official image that we maintain that you wouldn't have to update: https://tinkerpop.apache.org/docs/current/reference/#gremlin-server-docker-image if you prefer that one, it should just work without having to mess around with the serializers. docker
billmanh
billmanh15mo ago
I think my reasoning for not doing that was that I wouldn't be able to customize the security or configuration if it was a pre-baked image. I'll want to be able to edit the config process. Using the pre-baked container, can I build it with my own config file? However, I did try that container and got a new error RuntimeError: Event loop is closed when running that image. Any idea? The server gave out a longer error [WARN] c.c.m.CsvReporter - Error writing to org.apache.tinkerpop.gremlin.server.GremlinServer.user-agent.Python/3.8 aiohttp/3.7.4 java.io.IOException: No such file or directory @spmallette , is it possible to see the repo that generates that dockerfile? If I could see the buildfile and config then I might be able to answer my own question. I'll bet it's just written right there.
spmallette
spmallette15mo ago
GitHub
tinkerpop/gremlin-server/Dockerfile at master · apache/tinkerpop
Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.
spmallette
spmallette15mo ago
in what context did you get that error? "event loop closed" sounds like a python error, so i guess you saw that on the client side trying to connect?
billmanh
billmanh15mo ago
Yeah, my validation is just running a script with the python line: graph = Graph() g = graph.traversal().withRemote(DriverRemoteConnection('ws://localhost:8182/gremlin','g')) print('Number of nodes {}, number of edges {}.'.format(g.V().count().next(),g.E().count().next()))
spmallette
spmallette15mo ago
i guess the server error isnt too relevant. just a metrics reporting issue. that said we shouldnt have that happening. we should fix that. @
billmanh
billmanh15mo ago
Looking at the dockerfile, I see that the config looks like a buildtime configuration. Can you tell me which config it runs to get the correct serializes? CONF_FILE=$1
spmallette
spmallette15mo ago
not sure why you test would fail offhand. perhaps its worth a test on our end
spmallette
spmallette15mo ago
i think it just runs the default config file if you dont specify one: https://github.com/apache/tinkerpop/blob/master/gremlin-server/conf/gremlin-server.yaml
GitHub
tinkerpop/gremlin-server/conf/gremlin-server.yaml at master · apach...
Apache TinkerPop - a graph computing framework. Contribute to apache/tinkerpop development by creating an account on GitHub.
spmallette
spmallette15mo ago
@colegreer @kennh couple of issues here regarding docker - (1) the csv metrics reporter probaly shouldn't throw that warning - i dont think it even needs to be configured for the docker image (2) a potential issue with python connecting in 3.7.0 - worth a quick check?
billmanh
billmanh15mo ago
Interestingly, was able to get the serializers settings from that, however still unable to connect. File "C:\Users\william.harding\anaconda3\envs\exoplanets\lib\site-packages\aiohttp\streams.py", line 604, in read await self._waiter aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected Unclosed client session client_session: <aiohttp.client.ClientSession object at 0x000001FD10DE6430> Sounds like a client issue, but is there an issue with how I'm connecting? aiohttp.client_exceptions.ServerDisconnectedError: Server disconnected this is a windows/linux issue regarding handling async is that right?
spmallette
spmallette15mo ago
i've tested the official 3.7.0 Gremlin Server Docker image with the 3.7.0 gremlinpython package and i'm not finding any problems connecting:
>>> g = traversal().with_remote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
>>> g.V().count().next()
0
>>> g.addV('person').property('name','bill').iterate()
[['addV', 'person'], ['property', 'name', 'bill'], ['none']]
>>> g.addV('person').property('name','bill').toList()
[v[2]]
>>> g.V().count().next()
2
>>> g.V().toList()
[v[0], v[2]]
>>> g = traversal().with_remote(DriverRemoteConnection('ws://localhost:8182/gremlin','g'))
>>> g.V().count().next()
0
>>> g.addV('person').property('name','bill').iterate()
[['addV', 'person'], ['property', 'name', 'bill'], ['none']]
>>> g.addV('person').property('name','bill').toList()
[v[2]]
>>> g.V().count().next()
2
>>> g.V().toList()
[v[0], v[2]]
that's with Python 3.8.10
billmanh
billmanh15mo ago
Is that on Windows? Yeah, I'm still getting the: ~\anaconda3\lib\site-packages\aiohttp\streams.py in read(self) 614 self._waiter = self._loop.create_future() 615 try: --> 616 await self._waiter 617 except (asyncio.CancelledError, asyncio.TimeoutError): 618 self._waiter = None ServerDisconnectedError: Server disconnected But that could be an issue with my cleint, not the server.
ColeGreer
ColeGreer15mo ago
@billmanh I'm not sure this is related to your issue here but I wanted to point out that in 3.7.0, the serializers were migrated from a gremlin.driver.ser package to gremlin.util.ser. Therefore the serializer in the config in your original question should be changed to org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1
billmanh
billmanh15mo ago
Yup! That is exactly what I was origionally looking for. I was able to get that from looking at the docker buildfile, but thanks for confirming. I'm able to build now. Now if I can just get the client issue (which isn't my OG issue) @colegreer , if the server sends a "GraphBinaryMessageSerializerV1" and I'm using bytecode, then the differences between the platforms shouldn't matter, right? I should be able to conenct on a windows machine. Is there annother step that I'm missing? Perhaps the async loop thing?
ColeGreer
ColeGreer15mo ago
Platform differences shouldn't matter here. I'm wondering if that server metrics error is somehow related to the server disconnecting. I believe I can reproduce that metrics error to confirm this.
billmanh
billmanh15mo ago
Yes, I was able to do this as well:
No description
billmanh
billmanh15mo ago
Windows. So I'm unblocked. Thanks for your help. However, @spmallette If I wanted to modify the credentials of the pre-baked container, how would I do that? Don't you have to build it in order to place the secruity file?
spmallette
spmallette15mo ago
sorry - i forgot to come back to this after not knowing the answer immediately. @triggan do you happen to know how to do what's asked above?
billmanh
billmanh15mo ago
I suppose there might be some way to take the pre-baked container, source that in a new build file and copy your config files to it. That should work. I haven't started working on that thought. However, I think that's the pattern I'm going to try.
triggan
triggan15mo ago
You can create your own Dockerfile that uses the TinkerPop-provided Gremlin Server image as a base and then customize it from there. Here's an example that I use to create a Gremlin Server container that behaves more like Neptune. I use RUN sed commands to modify the gremlin-server.yaml file.
FROM tinkerpop/gremlin-server:latest

# Add support for both websockets and http requests
RUN sed -i "s|^channelizer:.*|channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer|" ./conf/gremlin-server.yaml

# Allow for string based IDs
RUN sed -i "s|^gremlin.tinkergraph.vertexIdManager=.*|gremlin.tinkergraph.vertexIdManager=ANY|" ./conf/tinkergraph-empty.properties

# Remove ReferenceElementStrategy for use with graph-explorer - return all properties
RUN sed -i "s|^globals << \[g.*|globals << [g : traversal().withEmbedded(graph).withStrategies()]|" ./scripts/empty-sample.groovy

# Increase thread stack to 2m
ENV JAVA_OPTIONS="-Xss2m -Xms512m -Xmx4096m"
FROM tinkerpop/gremlin-server:latest

# Add support for both websockets and http requests
RUN sed -i "s|^channelizer:.*|channelizer: org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer|" ./conf/gremlin-server.yaml

# Allow for string based IDs
RUN sed -i "s|^gremlin.tinkergraph.vertexIdManager=.*|gremlin.tinkergraph.vertexIdManager=ANY|" ./conf/tinkergraph-empty.properties

# Remove ReferenceElementStrategy for use with graph-explorer - return all properties
RUN sed -i "s|^globals << \[g.*|globals << [g : traversal().withEmbedded(graph).withStrategies()]|" ./scripts/empty-sample.groovy

# Increase thread stack to 2m
ENV JAVA_OPTIONS="-Xss2m -Xms512m -Xmx4096m"
billmanh
billmanh15mo ago
Hey thanks for that @triggan . Not my origional question but it's a solid bonus.
Want results from more Discord servers?
Add your server