AWS Integration Issue: Action Await Behavior Not Working as Expected
Problem Summary:
We're experiencing an issue where AWS cannot properly await the completion of a specific Gadget action (Additional Observation - Return Value Inconsistency:
-
Questions:
1. Is there a specific pattern for long-running actions that ensures the Gadget client API waits for complete execution?
2. What's the recommended approach for AWS-Gadget integration with long-running processing requirements?
3. Why does
Any guidance on the proper pattern for ensuring AWS waits for complete Gadget action execution would be greatly appreciated!
updateRank) regardless of the action configuration pattern used. The AWS code continues to the next step before the Gadget action fully completes its processing.
Important Note: The shopifyMutationRank action works correctly - AWS properly awaits its completion. This suggests the issue is specific to updateRank rather than a general AWS-Gadget integration problem.
Gadget Client Details:
- AWS runs on ECS Fargate with concurrent processing (pMap concurrency: 10)
Expected Behavior:
AWS should wait for the complete updateRank action execution (including onUpdateRankSuccess() processing) before proceeding to the next step, similar to how shopifyMutationRank works correctly.
Actual Behavior:
- updateRank: AWS receives control back after the run function completes but before onUpdateRankSuccess() in onSuccess finishes, causing timing issues
- shopifyMutationRank: AWS correctly waits for complete execution (this action works as expected)
updateRank returns null to AWS despite having a record parameter
- shopifyMutationRank returns the customer record to AWS
- Both actions receive a record parameter but have different return behaviorsupdateRank return null while shopifyMutationRank returns the customer record? Is this related to the await timing issue?15 Replies
AWS Integration Code:
AWS calls:
- ❌
await gadgetApi.shopifyCustomer.updateRank(customerId, { taskId: taskId }) - AWS continues before onUpdateRankSuccess() completes
- ✅ await gadgetApi.shopifyCustomer.shopifyMutationRank(customerId, params) - This await works correctly
Action Configuration Patterns Tested:
(All in api/models/shopifyCustomer/actions/updateRank.js)
1. Pattern 1: transactional: false + onUpdateRankSuccess() in onSuccess
2. Pattern 2: transactional: true + onUpdateRankSuccess() in run
3. Pattern 3: transactional: true + onUpdateRankSuccess() in onSuccess (Current)
All patterns call onUpdateRankSuccess() from ../updateRank/onUpdateRankSuccess.js.
Result: All three patterns produce the same behavior - AWS receives control back before the onUpdateRankSuccess() processing completes.
- App URL: https://rw-ranky.gadget.app/
- Env: production
- The timeoutMS: 900000 (15 minutes) setting in updateRank was extended due to previous timeout errors encountered.
- We have plans for major structural changes after next month, so we'd prefer a short-term solution if possible.
Update: My earlier statement about updateRank returning null was incorrect. We actually don't use or log the return value in AWS, so the actual return value is unknown - the assumption of null was just speculation from our team member.
For this issue, we need to consider alternative approaches and make fixes if await doesn't work, so sorry to bother you when you're busy, but we'd appreciate a prompt response. 🙇♀️
@Chocci_Milk Since I'm not sure who to mention, I'll contact Antoine for now.🙏Hi Aya,
We are currently looking into this and will get back with more information as soon as we can.
Have you observed this issue again in the last day or two?
@Smelvin
Thank you Mark,
Yes, the issue is still occurring.
For example, in today’s log (Japan time, August 6, 2025), the updateRank process actually continues until 0:28:25, but AWS shows a completion log at 00:27:59, so it seems like it’s not being awaited properly.
Im currently investigating this with the team, sorry for the delayed response.
I will reply here once I have more information
The issues you were mentioning regarding the inconsistencies might have been a result of the increased latencies we experienced in the outage window.
Are you still seeing these issues today?
The person in charge is currently away, so I’ll follow up with you once they’re back!
Sounds good. If possible when you try the variations of your patterns and experience the inconsistency could you send a trace ID for the working pattern, and the one for the pattern you're having a problem with?
@[Gadget] Mark
It still seems to be the same situation. The person in charge reached out during their time off, so this isn’t from today’s run, but they shared the following:
the trace ID: aebf6573f21ac02de0075979d680e982
Also, the issue happens every time we run updateRank.
Looking at the updateRank.js action in shopifyCustomer, if you move
await onUpdateRankSuccess(context); from onSuccess to your run() it should solve the timeout issue with AWS.
run() will have AWS wait for it to complete entirely, and onSuccess() is running asynchronously after the API response is sent.
You would want actions that write data to the database in run() and then anything that should trigger after database work has completed successfully put within onSuccess()
Looking at onUpdateRankSuccess.js I see that there is a database read, then you apply the math, then write.
Is there a reason you have onUpdateRankSuccess.js in onSuccess()?
I dont know your average duration for your onUpdateRankSuccess.js but from the traceId I can see its around 1s and run() has a timeout limit of 5s so it should fit comfortably
Replying to your first message, your shopifyMutationRank works correctly because its in run(), where onUpdateRankSuccess() was in onSuccess()I actually already tried running it inside
Even then, it seemed like the function still wasn't being awaited properly. But I got the error in the screenshot when I used that setup, which is why I moved it to
run() — that's what I called Pattern 2: transactional: true + onUpdateRankSuccess() in run.Even then, it seemed like the function still wasn't being awaited properly. But I got the error in the screenshot when I used that setup, which is why I moved it to
onSuccess() instead. Looks like during the busy midnight hours, it can take more than 5 seconds to finish.
I still had the response from Gadget AI from when I was debugging, so I’ve translated and pasted it below for reference.
≡≡
Log Entry Analysis
This log shows a transaction timeout error. Here's the detailed sequence: 🔍
How the issue occurred:
15:02:48 - updateRank action started
15:02:51 - Skipped metafield update (no changes)
15:03:02 - Database update started (updating record)
15:03:03 - Transaction timeout occurred
15:03:06 - Rollback attempt also failed
⚠️
Error details:
Main error: GGT_TRANSACTION_TIMEOUT: Can't commit a transaction after it has timed out
Total processing time: about 17 seconds (responseTime: 17259.904472999973) 🚨
Root cause:
Gadget transactions have a 5-second timeout limit.
Your action has the following issues: - Long DB operation: the update took 11 seconds
- Heavy logic inside the transaction: e.g.,
💡
Solutions:
Option 1: Revert to transactional: false ```js export const options = { actionType: "custom", triggers: { api: true }, timeoutMS: 900000, transactional: false, // ← revert this }; Option 2: Move heavy logic to onSuccess export async function run({ params, record }) { // Only light DB updates applyParams(params, record); await preventCrossShopDataAccess(params, record, { shopBelongsToField: "shop" }); } export async function onSuccess({ params, record, context }) { // Heavy logic goes here await onUpdateRankSuccess(context); } Recommendation: Start by reverting to transactional: false to prevent the error.
This log shows a transaction timeout error. Here's the detailed sequence: 🔍
How the issue occurred:
15:02:48 - updateRank action started
15:02:51 - Skipped metafield update (no changes)
15:03:02 - Database update started (updating record)
15:03:03 - Transaction timeout occurred
15:03:06 - Rollback attempt also failed
⚠️
Error details:
Main error: GGT_TRANSACTION_TIMEOUT: Can't commit a transaction after it has timed out
Total processing time: about 17 seconds (responseTime: 17259.904472999973) 🚨
Root cause:
Gadget transactions have a 5-second timeout limit.
Your action has the following issues: - Long DB operation: the update took 11 seconds
- Heavy logic inside the transaction: e.g.,
rankJudge is being run within the transaction💡
Solutions:
Option 1: Revert to transactional: false ```js export const options = { actionType: "custom", triggers: { api: true }, timeoutMS: 900000, transactional: false, // ← revert this }; Option 2: Move heavy logic to onSuccess export async function run({ params, record }) { // Only light DB updates applyParams(params, record); await preventCrossShopDataAccess(params, record, { shopBelongsToField: "shop" }); } export async function onSuccess({ params, record, context }) { // Heavy logic goes here await onUpdateRankSuccess(context); } Recommendation: Start by reverting to transactional: false to prevent the error.
I see.
Since
onUpdateRankSuccess reads, computes, then writes you could do something like this:
- Update webhook triggers, onSuccess calls onUpdateRankSuccess
- onUpdateRankSuccess reads, computes, then calls api.internal.shopifyCustomer.update(computed data), the key thing being the internal which skips the onSuccess(), and just modifies the databaseThanks for the suggestion. I'm actually planning to fully switch this process over to a new implementation using
bulkOperationRunMutation next month, so I wasn't planning to make major changes to the current setup.
I just wanted to make sure the processing gets awaited for now — but if that requires putting everything inside run(), and it's currently taking more than 5 seconds during peak hours, is it correct to assume there's no way to make it await properly in this situation?
Also, just so I can explain it internally — was the reason it didn’t get awaited, even though we had it in run(), because transactional was set to true?by default
run() is set to transactional:true, and that enforces a 5 second timeout on the database transactionGot it — so with transactional:true the 5s timeout is the blocker.
Would switching to transactional:false and keeping the logic in run() be a safe workaround?
Any caveats or time limits we should know for non-transactional run()?
Would switching to transactional:false and keeping the logic in run() be a safe workaround?
Any caveats or time limits we should know for non-transactional run()?
transactional:true makes it so if one operation fails the entire transaction is rolled back to maintain consistency.
If the benefits of transactional aren't required for your actions then it could be a work around.
I have a link to the transactional tag here, as well this page is the documentation for the run() and onSuccess() functions: https://docs.gadget.dev/guides/actions/code#transactional