Gremlin traversal exceeded maximum compilation depth. Max: 400
Hi all, I'm using gremlin through cosmos and the python library and I'm getting this error when I try to add a lot of vertices to my graph at the same time. (A lot as in only 583 vertices). I'm guessing this refers to the actual traversal part, but considering my query is structurally just chained addV's:
g.addV("..").property(...).addV("..")....
I can't imagine it's that complex. I tried chunking them into smaller batches and submitting them individually but I still get the same error after the last batch.
The exception type is "GraphCompileException" and it's a 597 from cosmos. The graph interopstatuscode is QuerySyntaxError which seems strange.
I'm very new to gremlin, but I haven't found a better way to batch upload a lot of vertices at once. I see in cosmos the graph is represented as JSON, could I construct a graph in JSON then upload that to my DB?
Solution:Jump to solution
Hey, i've run into that one before - there's a maximum length of query Cosmos DB can handle and it looks like you're exceeding it. When writing a big big gremlin query that creates a bunch of vertices and their properties it can quickly exceed that limit.
I believe if I remember right from the CosmosDB docs that the solution here is to batch these smaller and run them in parallel (e.g. threading) to get the best performance...
5 Replies
Solution
Hey, i've run into that one before - there's a maximum length of query Cosmos DB can handle and it looks like you're exceeding it. When writing a big big gremlin query that creates a bunch of vertices and their properties it can quickly exceed that limit.
I believe if I remember right from the CosmosDB docs that the solution here is to batch these smaller and run them in parallel (e.g. threading) to get the best performance
Ah okay got it, let me try that. Thanks.
Just as a sanity check, chaining addV is the best way to add vertices to a graph right? No json uploading available or something like that?
There seems to be a tool developed specifically for batch loading https://learn.microsoft.com/en-us/azure/cosmos-db/gremlin/bulk-executor-dotnet
Ingest data in bulk in Azure Cosmos DB for Gremlin by using a bulk ...
Learn how to use a bulk executor library to massively import graph data into an Azure Cosmos DB for Gremlin container.
I can't remember exactly but options are definitely limited with Cosmos DB. Worth double checking exactly how the bulk loader does it to see if they use addV or something else, there might be some alternatives that leverage CosmosDB's multi modal aspects
Also if you're looking to boost your CosmosDB querying/visualization experience, check out gdotv.com 😁
Thats a great tool, thanks!