JanusGraphJ
JanusGraph•3y ago
deepak8347

Using Spark with JansuGraph on a Cassandra storage backend

Hello there!!

Hope you are having an awesome day. I am trying to use OLAP query for my janusgraph setup. I am using Cassandra for storing graph, but I am using our own defined custom storage backend class. I think that graph is created without using my storage backend.

Can you please help me figure out what is wrong ?

I am creating the graph using the following code.
graph = org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(executorConfig.getCassandraGraphConfiguration().getJanusGraphConfig());


cassandra:
  janusGraphConfig:
    gremlin:
      graph: "org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph"
      hadoop:
        graphReader: "org.janusgraph.hadoop.formats.cql.CqlInputFormat"
        graphWriter: "org.apache.hadoop.mapreduce.lib.output.NullOutputFormat"
        jarsInDistributedCache: true
        inputLocation: none
        outputLocation: output
        persistContext: true
    janusgraphmr:
      ioformat:
        conf:
          storage:
            backend: "com.company.department.diskstorage.companycassandra.CompanyCassandraStoreManager"
            cassandra:
              keyspace: "janusgraph-olap"
    storage:
      backend: "com.company.department.diskstorage.companycassandra.CompanyCassandraStoreManager"
      cassandra:
        keyspace: "janusgraph-olap"
    cassandra:
      input:
        keyspace: "janusgraph-olap"
        partitioner:
          class: "org.apache.cassandra.dht.Murmur3Partitioner"
        widerows: true
        columnfamily: "edgestore"
    spark:
      master: "k8s:https://kubernetes.default.svc:443"
      executor.memory: 1g
      serializer: "org.apache.spark.serializer.KryoSerializer"
      kryo.registrator: "org.janusgraph.hadoop.serialize.JanusGraphKryoRegistrator"
    query:
      batch: true
      batch-property-prefetch: true
      fast-property: true
      force-index: false
    graph:
      replace-instance-if-exists: true
    ids:
      block-size: 10000000
Was this page helpful?