Implement new storage backend using Tablestore
Hello, thank you for allowing me to join. This is my first post so apologize if I post my question in the wrong channel.
So, I am planning to setup a new storage backend using Tablestore from AlibabaCloud https://www.alibabacloud.com/help/en/tablestore/tablestore-hbase-client, is there a doc on how to get started to integrate a new storage backend in Janusgraph?
Thanks in advance...
Is there a way to connect janusgraph to an existing opensearch instance as the index backend?
I have an opensearch instance running already which I want to also use as the index backend for janusgraph.
...
janusgraph.index.search.backend: elasticsearch
janusgraph.index.search.hostname: <my-opensearch-hostname>:9200
janusgraph.index.search.backend: elasticsearch
janusgraph.index.search.hostname: <my-opensearch-hostname>:9200
Janusgraph Servers start throwing Exceptions when load increases
We have 10 instances of Janusgraph servers with 1 CPU each and 10GB of memory limit.
It works fine when the number of requests are not much, but when we increase the request load, every instance of server starts throwing "Connection reset by peer" error.
Previously we were facing idle connection closing issue, for which we added below configuration in our servers to resolve:
evaluationTimeout: 0
idleConnectionTimeout: 0...
Granular configuration options for such as JANUSGRAPH_RELATION_DELIMITER on driver
Hi folks,
I'm curious about the possibility to make configuration options such as JANUSGRAPH_RELATION_DELIMITER more granular at the driver level. I know it can currently be set as an environment variable, but I'm wondering if there's thoughts of making this configurable on a per DriverRemoteConnection/Cluster basis client-side.
The reason I'm asking is to cater for potential use cases where users on G.V() might need to connect to multiple janusgraph databases with disparate setups.
Is that something reasonable/achievable as a potential new feature?...
Solution:
It’s unfortunately not viable, because driver package has no dependencies on config (core) package
Schema creation with groovy script does not work on inmemory setup
Hello, all. We are facing an issue while trying to create a schema by executing a groovy script with an inmemory setup. Below are the steps to reproduce:
Create container via docker compose -f docker-compose.yml up -d (see attachments).
Use the schema.groovy (see attachments) file to instantiate graph. In order to do this, we must:...
What would be the ideal way to set up deep learning on janusgraph data?
I have setup a simple knowledge graph on janusgraph and I wanted to use the data to train a deep learning model. I found DGL(https://dglke.dgl.ai/doc/index.html) library which seems to be used mostly. I am not sure how I should go about this.
DGL accepts training data as a simple comma seperated strings which defines the
from_vertex, relationship and the to_vertex
ex: "London", "isCapitalOf", "UK"
...How does remote gremlin server reacts to credential rotation.
I have are remote gremlin server setup with backend database as Cassandra. I need to rotate Cassandra Username/password once a week.
My Cassandra credential are stored in gremlin server properties file and are injected by side-car container at boot time.
How does the Janus graph reacts to if Cassandra credential are changed ?
How can I achieved a rotation of Cassandra credentials without any downtime for gremlin server? ...
Infinite recursion error when evaluating script on Inmemory setup
Hello all, in our usecase, we use a Cassandra backend for development and production, but use an inmemory setup for running integration tests. We run our tests from a python application, using Gremlin Python and JanusGraph Python. The issue we face when running the inmemory setup is that the following error is thrown when trying to submit a script to JanusGraph:
org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Infinite recursion (StackOverflowError)
Some background configuration on how we run our setup can be found below (setup.yml).
The python script and groovy code we are submitting to JanusGraph (script.py)....
Incremental schema changes - Indices get stuck in 'Installed' state
Hi All, in our use-case the graph schema is changing over time and we have issues to incrementally add indices to the schema as we get error in the below scenario:
1. We create a graph with an initial schema and fill it with data.
2. We try adding a new index and cycling through the recommended steps in the documentation document here:
https://docs.janusgraph.org/schema/index-management/index-lifecycle/
(Installed -> Registered -> Enabled (through Reindex action). ...
Is there a suitable JanusGraph visualizer where you can see your created graphs?
I am using Python (gremlin-python) to create JanusGraph graphs, and am using Docker to host the JanusGraph on my machine. Is there any software or plugin available that can display my created graphs?
MergeV "get or create" performance asymmetry
So I'm working on adding the mergeV step among others to the Rust gremlin driver. As part of that I took a pause and did a performance comparison to the "traditional" way of doing it.
So the Rust driver is submitting bytecode that's effectively doing:
"Traditional/Reference":...
Incremental schema changes - Property Key constraint does not exist
Hi All, in our use-case the graph schema is changing over time and we have issues to incrementally add properties to the schema as we get error in the below scenario:
1. day 1: we create a graph with an initial schema: person node with
firstName property.
2. day 2: we try adding lastName property to the person node. This still works and one can create persons with lastName
3. day 3: we try adding fullName property to the person node. This does not work any more as gp_traversal.addV('person').property('fullName', 'test full name') throws error: Property Key constraint does not exist for given Vertex Label [person] and property key [fullName].
...Solution:
Hello,
you need to close graph in order to take into account new schema constraint, no need to restart server.
You can close graph with ConfiguredGraphFactory.close, then it will automatically open again with GremlinExecutorGraphBinder in JanusGraphManager....
Text predicate not serializable (containsPhrase, notContainsX, etc)
Hello,
JanusGraphPSerializer is missing several predicates, this is causing a timeout exhausting all opened connexion when we're calling these predicates.
I fixed this locally, I can do a merge request, but i haven't found the related issue.
Here are the 8 missing predicates :...
How do we generate transaction logs ?
This is how I created Client
```
def loadClient(graphId: String): Client = {
val typeSerializerRegistry = TypeSerializerRegistry.build.addRegistry(JanusGraphIoRegistry.instance).create
val cluster = Cluster.build()...
RpcRetryingCallerImp while trying to connect JanusGraph with Hbase 2.2.7 under Cloudera Distribution
Hi,
I am new to using JanusGraph and I am trying to integrate JanusGraph version 0.6.4 with Kerberized HBase 2.2.7, which comes under the Cloudera Distribution 7.9.1. I had to use an earlier version of JanusGraph since the compatibility matrix specifies that JanusGraph is tested compatible with HBase 2.2.x.
I am using the following configuration to get a 3-node HBase cluster to interact with JanusGraph. However, when I try to start the server, I encounter the following loop, and as a result, the server does not start.
...
Potentially useless allocations when checking a field cardinality
👋🏻 Hey folks
In a service running JanusGraph embedded, I observed a number of sizeable allocations coming from
JanusGraphVertexFeatures#getCardinality (https://github.com/JanusGraph/janusgraph/blob/2c71b378339a3ab49b961eef29b5a042d018f513/janusgraph-core/src/main/java/org/janusgraph/graphdb/tinkerpop/JanusGraphFeatures.java#L161-L169).
From a profile (attached), it looks like...
Unable to load GraphSON file
Hi, I'm trying to load a very simple ...
.json graph into JanusGraph from the gremlin console, but I get this error:
gremlin> g.io("/opt/janusgraph/graphson-test1.json").read().iterate();
Label can not be null
gremlin> g.io("/opt/janusgraph/graphson-test1.json").read().iterate();
Label can not be null
Gremlin statement exceeds the maximum compilation size
Hi all, I'm trying to do some script-based bulk writes to a JanusGraph server, but I get the following error message
I have set
'status': {'message': 'The Gremlin statement that was submitted exceeds the maximum compilation size allowed by the JVM, please split it into multiple smaller statements ...}
'status': {'message': 'The Gremlin statement that was submitted exceeds the maximum compilation size allowed by the JVM, please split it into multiple smaller statements ...}
maxContentLength: 524288 in the server .yaml file to allow my client to sbmit larger scripts, because I found a significant performance improvement when submitting a larger set of queries at once. But now I'm hitting the JVM limit for compilation size, which I'd like to increase, if possible. ...Solution:
This error message means that the JVM internally threw a
Method too large error. At least from a quick search, it looks like this is a hard limit which can be configured.
That's at least what I got from this Stack Overflow question: https://stackoverflow.com/questions/3192896/how-to-circumvent-the-method-too-large-error-in-java-compilation...Comma Separated Config Options Values Via Environment Variable?
Hey @Bo & @porunov I was trying out the retry logic added by my PR (https://github.com/JanusGraph/janusgraph/pull/4409) by setting environment variables in my docker compose like so:
janusgraph.index.some_index_name_here.elasticsearch.retry-error-codes=408,429
Much like was done in the unit test I wrote (https://github.com/JanusGraph/janusgraph/blob/487e10ca276678862fd8fb369d6d17188703ba67/janusgraph-es/src/test/java/org/janusgraph/diskstorage/es/rest/RestClientSetupTest.java#L240). But to my surprise it doesn't seem to be received well during startup:...Solution:
Okay, after quite a journey of vetting the parsing behavior from environment variable into property files, into the various Confirmation implementations into ConfigOptions, etc etc I've found where this has gone sideways....and like the worst of these types of situations looks like this one was self inflicted 🤦♂️ .
A long long long time ago when I was starting on working with JanusGraph I was tired of JanusGraph deployments that silently failed and leaving my container up but dead inside the container, for reasons that weren't its fault (started up before Cassandra was ready timing out its storage wait time, etc).
Trying to find solutions for that I stumbled upon the Tinkerpop CheckGraphManager (https://github.com/apache/tinkerpop/blob/master/gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/util/CheckedGraphManager.java) that would automatically terminate the process if the specified graphs did not successfully open....