AWS Neptune updating gremlin driver to 3.6.2 introduced many bugs to working queries

After updating Amazon Neptune engine version from 1.2.0.2 to 1.2.1.0 and the Gremlin.Net (C# nuget) driver from 3.5.2 to 3.6.2, suddenly queries started throwing exceptions, specifically exceptions about serialization errors. To pinpoint the cause, I downgraded the Gremlin.Net driver to 3.5.2 while leaving the engine version updated to 1.2.1.0, the queries started working like before. The problem is, according to the AWS documentation, the minimum required Gremlin.net version is 3.6.2 Will there be a problem keeping the Gremlin.net driver version to 3.5.2? Will there be any side effects? Resources: Aws documentation: https://docs.aws.amazon.com/neptune/latest/userguide/engine-releases-1.2.1.0.html#engine-releases-1.2.1.0-query-versions
No description
12 Replies
spmallette
spmallette•10mo ago
The preference would be to go with the recommended version of 3.6.2+ on 1.2.1.0. do you have other examples of the types of errors you are getting? or are they all like that one?
Haieketoun
Haieketoun•10mo ago
Thank you for your answer @spmallette, as of my last knowledge, all error types are like this. Specifically, these errors occurring when my query contains Enumerations and DateTime. A solution to this is converting these objects to strings with the ToString() function.
spmallette
spmallette•10mo ago
strange. i'm not aware of any changes to DateTime serialization that would cause a problem here with an upgrade. could you share an example of a failing query?
Florian Hockmann
Florian Hockmann•10mo ago
I guess you don't explicitly configure the serializer in Gremlin.Net? Because then you're getting the default serializer which was GraphSON before 3.6.1 and GraphBinary after that. That probably explains the changed behavior. You can now simply configure Gremlin.Net to continue using GraphSON instead of GraphBinary or you could use DateTimeOffset instead of DateTime as we have a serializer in GraphBinary for DateTimeOffset I guess we should also add one for DateTime, but the default is DateTimeOffset as that's the recommended data type for dates
Haieketoun
Haieketoun•10mo ago
C#
return DoWithRetry(async (conn, _) =>
{
var g = conn.TraversalSource;
var eightStagePostIds = await g.V()
.HasLabel(EightLabel.CreatorsContent)
.Has(GraphProperty.IsEightStage, true)
.Has(GraphProperty.PublishedDate, P.Lte(DateTime.UtcNow)) //This breaks 3.6.2 driver
.Where(__
.Not(__
.In(EightEdge.CreatedContent)
.Both(EightEdge.Blocks)
.HasId(citizenId.ToString()))
.Not(__
.In(EightEdge.CreatedContent)
.In(EightEdge.Mutes)
.HasId(citizenId.ToString()))
.Not(__
.Has(GraphProperty.ModerationDecision,
P.Within(ModerationDecisions.Rejected))))
.Order().By(GraphProperty.PublishedDate, Order.Desc)
.Limit<Vertex>(limit)
.ProjectBy(new Dictionary<string, ITraversal>
{
{ "id", __.Id() }
})
.Promise(x => x.ToList());

return eightStagePostIds?.Select(x =>
x["id"].ToString().ToGuid())
.ToReadOnlyList()
?? new List<Guid>();
}, cancellationToken);
C#
return DoWithRetry(async (conn, _) =>
{
var g = conn.TraversalSource;
var eightStagePostIds = await g.V()
.HasLabel(EightLabel.CreatorsContent)
.Has(GraphProperty.IsEightStage, true)
.Has(GraphProperty.PublishedDate, P.Lte(DateTime.UtcNow)) //This breaks 3.6.2 driver
.Where(__
.Not(__
.In(EightEdge.CreatedContent)
.Both(EightEdge.Blocks)
.HasId(citizenId.ToString()))
.Not(__
.In(EightEdge.CreatedContent)
.In(EightEdge.Mutes)
.HasId(citizenId.ToString()))
.Not(__
.Has(GraphProperty.ModerationDecision,
P.Within(ModerationDecisions.Rejected))))
.Order().By(GraphProperty.PublishedDate, Order.Desc)
.Limit<Vertex>(limit)
.ProjectBy(new Dictionary<string, ITraversal>
{
{ "id", __.Id() }
})
.Promise(x => x.ToList());

return eightStagePostIds?.Select(x =>
x["id"].ToString().ToGuid())
.ToReadOnlyList()
?? new List<Guid>();
}, cancellationToken);
spmallette
spmallette•10mo ago
i think @Florian Hockmann is on the right track with his reply @Valentyn Kahamlyk i know you are tracking some differences among the drivers/serializers/types.......here's another 😦
Haieketoun
Haieketoun•10mo ago
Thank you very much for your answer @Florian Hockmann, I will change to GraphSON serializer and get back to you with any updates.
Valentyn Kahamlyk
Valentyn Kahamlyk•10mo ago
in .net GLV both Date and Timestamp GraphBinary types mapped to DateTimeOffset, I don't know why it's done this way. DateAdd/DateDiff/asDate steps also works only with DateTimeOffset
Haieketoun
Haieketoun•10mo ago
Hello guys, indeed changing to GraphSON serializer fixed the issue, thank you all kindly for your help
Florian Hockmann
Florian Hockmann•10mo ago
in .net GLV both Date and Timestamp GraphBinary types mapped to DateTimeOffset, I don't know why it's done this way.
What else do you propose in .NET? The IO docs even state that both types are the same:
Timestamp Format: The same as Date.
https://tinkerpop.apache.org/docs/current/dev/io/#_timestamp_3
Valentyn Kahamlyk
Valentyn Kahamlyk•10mo ago
For Java Timestamp is A thin wrapper that allows the JDBC API to identify this as an SQL TIMESTAMP value GraphBinary type | Java type | .net type DATE(0X04) | java.util.Date | DateTime TIMESTAMP(0X05) | java.sql.Timestamp extends java.util.Date | DateTime OFFSETDATETIME(0X88) | java.time.OffsetDateTime | DateTimeOffset Looks like we need cross-GLV tests to verify all ~12 date related types handled in same way. Or keep only 1-2 most used types.
spmallette
spmallette•10mo ago
Or keep only 1-2 most used types.
must get rid of some types going forward
Want results from more Discord servers?
Add your server