Transaction Recovery not working as expected.
Hi all, I'm running into issues with JanusGraph's transaction recovery and log replay.
I'm using JanusGraph 1.1.0 with Cassandra + Elasticsearch, and have
tx.log-tx = true enabled. I can see that the txlog table in Cassandra is being written to whenever I insert data — for example, when Elasticsearch is down.
However, when I call:
(start_ms being a point from ~5 minutes ago to ~24 hours ago)
…it completes successfully, but no previously failed writes (e.g., to the index) are repaired. This is quite obvious as data inserted when elastic search is down, does not get pushed into the index and does not get returned via queries using that index. When I run it a second time, I get the error:
The only way to make this error disappear is to restart the janusgraph container.
It seems like JanusGraph thinks recovery is complete, even though the index (Elasticsearch) still lacks the data inserted during downtime.
I’ve also tried inspecting the txlog via graph.getBackend().getSystemTxLog() and registerReader(...), but I only see 1 message, which is null.
My config includes:
How can I debug why startTransactionRecovery() is not picking up/repairing failed index writes?
And why do I see no meaningful messages in the txlog via Groovy even though the Cassandra table updates?
Any help would be much appreciated! I can also provide the exact index and node insertion/query setup I used if that would be helpful.0 Replies