AT

GraphSON mapper

DCDragos Ciupureanu10/19/2023
Hi, I'm trying to ingest some data into AWS Neptune and due to its size I'm forced to use a bulk data importer https://tinkerpop.apache.org/docs/current/dev/io/#graphson-3d0 (unless there's a bulk-insert functionality straight from Gremlin - I couldn't find this). Looking at the GraphSON schema/docs I see there are some IDs on the edges that I am not sure how/if I need to generate. https://tinkerpop.apache.org/docs/current/dev/io/#graphson-3d0 I'm doing this mapping in Python (but can be done in other languages if there's better support). Any recommendations/tips for this?
Solution:
Do you already have data in GraphSON format? Or do you just need to use a bulk importer? If the latter, Neptune has it's own bulk load feature: https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load.html
Jump to solution
Sspmallette10/24/2023
it appears we've missed this question on bulk loading on @neptune - anyone have any tips for this?
Solution
Ttriggan10/24/2023
Do you already have data in GraphSON format? Or do you just need to use a bulk importer? If the latter, Neptune has it's own bulk load feature: https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load.html
DCDragos Ciupureanu10/24/2023
Thanks for the suggestion. Yes, in the end I used the bulk loader as it's easy to export from Pandas to the Gremlin CSV format. 👍
MManabuBeach10/24/2023
Do we need to split up into edgesosns and vetexons right?
Ttriggan10/25/2023
@Dragos Ciupureanu - Neptune also has Pandas support through the AWS SDK for Pandas: https://github.com/aws/aws-sdk-pandas/blob/main/tutorials/033%20-%20Amazon%20Neptune.ipynb
DCDragos Ciupureanu10/25/2023
Nice, didn't know about this. Thanks @triggan Whilst on the same neptune topic, do you happen to know if I can get graph embeddings out of the graph? I see they use RotatE for link predictions but I just want the embeddings. From what I looked I couldn't find anything in their examples. Similarly, does a gremlin response hook up into something that can return embeddings for a subgraph?
Ttriggan10/25/2023
Neptune doesn't provide embeddings directly. Best I can say, for now, is "what this space" as there is a lot happening around this at the moment. At present, you would use Gremlin to fetch the subgraphs and then feed this into a separate library or model to generate the embeddings. Two popular ones used for this in the graph arena tend to be GraphSAGE and GraphStorm (https://graphstorm.readthedocs.io/en/latest/tutorials/quick-start.html#generating-embedding).
DCDragos Ciupureanu10/25/2023
That's very useful, thanks @triggan

Looking for more? Join the community!

Want results from more Discord servers?
Add your server
Recommended Posts
.drop() behavior confussionI have a basic java app and I'm learning hot to send gremlin queries to a JanusGraph from that java Can I name the result of an anonymous traversal without moving the traverser?I can currently do the following: ``` Graph graph = TinkerFactory.createModern(); GraphTraversalSCan GraphBinary be used to save a graph to file?Can GraphBinary be used to save graph in a file. Any example is welcome.How to get cardinality of property?I have a multi property and I want to find out its cardinality. How can I do that? valueMap/elementMinverted regex searchHey, In my vertices I store escaped regexp statements as labels (e.g: 'wh.' which in theory should Debug message spam from tinkerpop server 3.7Right now, when connecting to my local tinkerpop server, I am getting incredible amounts of debug loShould by() Modulator Work For More Types?This works. `gremlin> g.V().out().out().path().by("name") ==>[marko,josh,ripple] ==>[marko,josh,lop]InProcess GraphDB with Gremlin Support? (C# or NodeJS)Hello, is there any in process GraphDB out there in the world? Best would be c# or NodeJS and not JaEasiest Way to Get List Cardinality Properties As a List?What is the easiest way to retrieve the vertex properties that have list cardinality back as a list filter lambda in remote consolehi all, i’m trying to do filter on remote console to neptune server but keep getting MalformedQueryEwithin() and case insensitiveI need to check if a property of a node is in a list of values. How can I do that in a case insensitproject("p").by(__.values("a", "b") Only Outputs Single Property, Bug or Expected?I am curious why this does not behave in the way I expected. Not a problem - solution question. I Use of by()Can somebody explain the usecase of by() function in gremlin in very simple language.@GremlinDSL support in the GremlinLangScriptEngineHi, I recently sent a pull-request into the github ArcadeDB repository to add support binding custoRepeatStep does not appear to respect barriersI was digging into some traversal performance and had something similar to the following: ``` g.V(<Trying to find a Vertex using a variable injected earlier in the traversalI am trying to add a series of vertices and edges to an existing graph. The newly created Vertex wiDoes Gremlin support API for CRUD operations?Currently using g.V() for read and g.addV() for write.Individual Vertex per property or Vertex with grouped propertiesI'm building an identity graph that also stores User profile data - things like email address, phoneFilter out empty resultsgremlin> g.V().hasLabel('metadata').valueMap() ==>{} ==>{} ==>{oncall_roster=[oncall_schedule]} HowQuestion on running queries in windows env.I get an error `RuntimeError: Event loop is closed`, but after troubleshooting I notice that my scri