JanusGraphJ
JanusGraph•2y ago
aschwartz

Vertex ID collisions

Hello!
I'm following up on my questions from Migrating from Bigtable to Cassandra and questionsUsing custom vertex IDs for import/export

We want to migrate our existing JanusGraph from Bigtable to Cassandra.
One of the hypothesis was to try and leverage JG 1.0 custom vertex IDs, to import a GraphSON file while keeping the original IDs.

The experiment went like this:
  1. Export graph with JG 0.6, using
    graph.io(graphson()).writeGraph("/tmp/my-graph.json")
  2. Bootstrap a JG 1.0 insance against Cassandra, setting graph.set-vertex-id to 
    true
    .
  3. Importing the graph with graph.io(IoCore.graphson()).readGraph('/tmp/my-graph.json') (we make sure to call graph.tx().commit() at the end)
  4. Rebuilding our Elastic indices.
  5. Setting graph.set-vertex-id to false and shutting down the instance.
  6. Bring back JG 0.6.4 working with Cassandra, and resume operation.
The import worked and we were able to see the vertices with their original IDs.
The show-stopper came later, when we tried resuming our business logic, we started experiencing vertex lack-of-collisions.
New vertices would get IDs of existing vertices and overwrite them, resulting in a corrupteg graph.

Some thing we experimented with, that did not work

  • Explicitly setting "ids.authority.conflict-avoidance-mode") to GLOBAL_AUTO.
  • Increasing ids.block-size to a number slightly higher than the number of vertices we import (~26,000)
@Florian Hockmann @Boxuan Li would appreciate any ideas an insights on this.
I know that jumping between JG versions doesn't sound like a great idea, but we still working on migrating out python codebase to Tinkerpop 3.7.0

many thanks in advance!
Was this page helpful?