Apache TinkerPopAT
Apache TinkerPop•3y ago
Andys1814

Sequential IDs in Neptune?

@neptune

I'm attempting to implement sequential IDs for the vertices in our AWS Neptune graph. So far, we have added a new property called vertexNumber, which will store the numeric sequential ID for each vertex. Then, before saving a vertex to the database, I run a simple query to retrieve the current highest vertex number, increment it, and store the new vertexNumber to that vertex. Pseudo-code examples found below.

// Calculate the current highest vertexNumber
id = g.V().hasLabel('my_vertex_label').has("vertexNumber").values("vertexNumber").max()

// Increment result by 1 which will be for the next vertex we save.
vertexNumber = id + 1

// Add the new vertex
g.addV('my_vertex_label').property(Cardinality.single, "vertexNumber", vertexNumber).property(...etc)


My question is: How will Neptune handle this at scale? For context, we have a distributed architechture in which tens or hundreds (in super rare cases, maybe over a thousand?) new vertexes can be created per SECOND, meaning our db cluster probably sees a lot of concurrent transactions. We are looking for information on how Neptune will handle the initial read query with, for example, 10 or more concurrent transactions. Will all 10+ transactions return the same vertexNumber, or Will Neptune be smart enough to isolate these queries?

Thanks!
Solution
@andys1814 How much do you care about the ids being truly sequential or is having some gaps acceptable as long as they are human readable?

I ask as this was a common request when I was working with Cassandra. A common practice was to allot a range of ids to each client on connection versus getting a new one each time. When a client exhausts it's assigned range it then reaches out to get a new range.

This helps to minimize the single point of failure and additional overhead of having to go to a single coordinator to get an id value for each request. It does however means that inserts will not be in sequential order and that you may have gaps in the number. This may or may not be an issue depending on your use case.
Jump to solution
Was this page helpful?