Topics

Gadget Developers•7mo ago

Using pub/sub with Gadget

I want to subscribe to the order webhook using google pubsub to handle the data since Gadget would be to expensive due to request time for me. In the Shopify docs is described to add the permission as well as the webhook to the toml file. But the existing permissions from gadget are not in that file so I need to dublicate them every time I do changes there. Is there an easier way / does the toml configuration change anything with the current Gadget Webhooks / other parts of Gadget? Thanks 🙂

# .toml
[access_scopes]
scopes = "read_orders, ..."


[webhooks]
api_version = "2024-04"

  [[webhooks.subscriptions]]
  topics = [ "orders/create" ]
  uri = "pubsub://...

# .toml
[access_scopes]
scopes = "read_orders, ..."


[webhooks]
api_version = "2024-04"

  [[webhooks.subscriptions]]
  topics = [ "orders/create" ]
  uri = "pubsub://...

51 Replies

Chocci_Milk•7mo ago

Could you please explain your use case and why you think it would be too expensive in Gadget?

RicardoOP•7mo ago

I want to save the order in another database and we have millions of orders coming through. Basically I need to just push the order to another api. I tried that out a while ago with gadget actions and it increased my request time a lot. So I need to use pubsub

Chocci_Milk•7mo ago

Do you just want to use Gadget as middleware? You don't need to use the data model if you're simply trying to push orders forward. Have you looked at these docs? https://docs.gadget.dev/guides/plugins/shopify/shopify-webhooks#adding-a-shopify-webhook-trigger-to-a-global-action

Working With Shopify Webhooks - Developer Docs - Gadget

Gadget Docs

RicardoOP•7mo ago

I have already used that for a similar use case in the past but it did cost me quite a lot. I had lot of those problems in the past with the request time so this time i went with pubsub. Like would be a few million orders per month coming through

Chocci_Milk•7mo ago

I'm not sure that I understand how it could have cost you that much. 10 million orders with extra logic only costs ~$185/month on the Pro plan. Are you sure that you were following best practices? If you would like me to help you get make your logic as performant as possible please let me know. On the other hand, I've never worked with pubsub. You would probably need to register new webhooks and point them to it. Take a look at this thread for HMAC validation: https://discord.com/channels/836317518595096598/1335229513122185317/1335992113871061036

No description

RicardoOP•7mo ago

Got it, just in the past every time i was using it I had to pay a lot and once I had to pay like 600usd for a weekend. Maybe i can try it again though

Chocci_Milk•7mo ago

I can share a cost optimization (work in progress) if you’d like to make sure that you’re doing best practices

RicardoOP•7mo ago

That would be perfect

Chocci_Milk•7mo ago

https://docs.google.com/document/d/1f5UN6eZD4WXUgYsIR2f4xYZiB9Ezxcyxs15fPUiUcWw/edit?usp=sharing

Google Docs

Copy of Cost opt docs

Separate account and billing into 2 sections (development-tools) Account → Account Management Billing – > Billing Cost optimization Cost optimization As your application grows it’s a good idea to begin to optimize it for to lower costs. BigCommerce Only store the data you need on models Do...

Chocci_Milk•7mo ago

Please note that its a work in progress

RicardoOP•7mo ago

Thanks I was trying to disabled the webhook (orders/create) for the global action for specific stores. But it doesn't seem to work.

// Giving error message: disabledWebhooks is not in update action
await api.shopifyShop.update(record.id, {
        subscription: {
          _link: selectedPlan.id,
        },
        confirmationUrl: null,
        disabledWebhooks: { shopifyCustomer: true, shopifyOrder: true };
      });


// Not changing anything
      record.disabledWebhooks = { shopifyCustomer: true, shopifyOrder: true };
      save(record);

// Giving error message: disabledWebhooks is not in update action
await api.shopifyShop.update(record.id, {
        subscription: {
          _link: selectedPlan.id,
        },
        confirmationUrl: null,
        disabledWebhooks: { shopifyCustomer: true, shopifyOrder: true };
      });


// Not changing anything
      record.disabledWebhooks = { shopifyCustomer: true, shopifyOrder: true };
      save(record);

ljspoor94•7mo ago

When trying to update with api.shopifyShop for disabledWebhooks you have to use internal API as far as I'm aware

RicardoOP•7mo ago

Thanks. But this doesn't change anything neither unfurtunately. Is this correct?

// Not changing anything
      record.disabledWebhooks = { shopifyCustomer: true, shopifyOrder: true };
      save(record);

// Not changing anything
      record.disabledWebhooks = { shopifyCustomer: true, shopifyOrder: true };
      save(record);

`

RicardoOP•7mo ago

But why isn't it changing anything? I get the correct response but it isn't changed in the db and also not filtering any incoming webhooks on the global action

No description

Chocci_Milk•7mo ago

Note that adding that disabledWebhooks field would deregister all webhooks for those models. I personally have your issue when testing it myself. The global action that I have didn't get triggered Could you please share the URL of the application so that I can take a look?

RicardoOP•7mo ago

https://keepcart-server.gadget.app env: ricardo-dev And yes, I want to disable all shopifyOrders webhooks since I only get the order/create webhook in the global action as a middleware

Chocci_Milk•7mo ago

Ok, just coming back to this now. Let me see what I can find On what action are you recieving the order webhook data? Do you also mind if I make an env off of yours end test on my end?

RicardoOP•7mo ago

sure 🙂 middleware/ordersToTinybird and then I am trying to change the disabledWebhooks on various places in the app (subscription change etc.) - but have deleted most of them again since it was not working

Chocci_Milk•7mo ago

Side note: I noticed that you're on 24-10 and you can change to 25-01 without making any changes at all. I would recommend 😛

RicardoOP•7mo ago

Thanks 🙂 I planned to change it and also change the gadget version soon.

Chocci_Milk•7mo ago

Sent you a message privately about a bug I just ran into How do I disabled the order webhooks from your app?

RicardoOP•7mo ago

What exactly do you mean? This is what I was trying to do. But didn't work

Chocci_Milk•7mo ago

My question is, do you have an action I should try using? I'm able to do it with the JS Playground but what to see if there's some functionlity broken on your end

RicardoOP•7mo ago

I was trying to do it in the subscribe action

RicardoOP•7mo ago

On your app I still only see the shopifyCustomer disabled. Not the shopifyOrder

No description

RicardoOP•7mo ago

idk if that's cause you changed it back...

Chocci_Milk•7mo ago

Interesting... I didn't Its almost like the update isn't working Even if I'm using the internal API

RicardoOP•7mo ago

exactly. This is the issue I am having. But the api request returns the correct value

Chocci_Milk•7mo ago

Let me see what can be happening Ok, something interesting just happened I changed it manually in the database viewer then ran the internal update and it changed back Almost like the call was not to the internal API somehow Nvm, it looks like it changes by itself

RicardoOP•7mo ago

hmmm, weird. Is that a bug you guys need to fix?

Chocci_Milk•7mo ago

Just got some information that I need to test Ok, so it looks like we're re-firing the action (with the public api) to change the registered webhooks field and the code that you put is changing the disabled webhooks value back Sooo, I think that you should add a check to skip the logic if the registeredWebhooks field is the only thing changing That would fix this for now I'm making a ticket to audit this functionality. Basically what's happening is the following: - internal api call to update disabled webhooks - Gadget internal logic to register/de-register webhooks - public api call from the Gadget platform to update registeredWebhooks - record.disabledWebhooks is changed to { shopifyCustomer: true } - triggers Gadget internal logic to register/de-register webhooks - public api call from the Gadget platform to update registeredWebhooks

RicardoOP•7mo ago

Thanks 🙂

RicardoOP•7mo ago

How would that work here?

No description

Chocci_Milk•7mo ago

Sorry, give me a moment. Writing a large ticket to explain a proposed change and whether or not making the change would break things for other users

RicardoOP•7mo ago

Sure, no stress 🙂

Chocci_Milk•7mo ago

I think that you should do something like this:

if (!record.changed("registeredWebhooks") && Object.entries(record.changes()).length != 1) {
  // ...
}

if (!record.changed("registeredWebhooks") && Object.entries(record.changes()).length != 1) {
  // ...
}

RicardoOP•7mo ago

This doesn't work unfortunately 😦

Chocci_Milk•7mo ago

How so?

RicardoOP•7mo ago

No description

RicardoOP•7mo ago

Running this and nothing changes in the db

Chocci_Milk•7mo ago

You need to only put the record.disabledWebhooks bit in there. Also not that what I wrote isn't perfect I asumed that you would have needed to change it The second bit of that if is the issue in this case You aren't hitting the logic in the if because changing disabledWebhooks is only one change

RicardoOP•7mo ago

Still not working.

RicardoOP•7mo ago

No description

RicardoOP•7mo ago

Basically trying to do that on the create action (and on others as well)

Chocci_Milk•7mo ago

You're not doing what I'm telling you to do

RicardoOP•7mo ago

Tried that as well

No description

Chocci_Milk•7mo ago

import { applyParams, preventCrossShopDataAccess, save, ActionOptions, ActionOnSuccess } from "gadget-server";

/**
 * @param { ActionRun } context
 */
export async function run({ params, record, logger, api }) {


  applyParams(params, record);
  await preventCrossShopDataAccess(params, record);

  if (!record.changed("registeredWebhooks") && Object.entries(record.changes())[0][0] != "registeredWebhooks") {
    if (!record?.customerTagsBlocked || record.customerTagsBlocked.length == 0) {
      record.disabledWebhooks = { shopifyCustomer: true };
    } else {
      record.disabledWebhooks = { shopifyCustomer: false };
    }
  }

  await save(record);
}

/**
 * @param { ActionOnSuccess } context
 */
export async function onSuccess({ params, record, logger, api }) {
  // Your logic goes here
}

/** @type { ActionOptions } */
export const options = {
  actionType: "update",
  triggers: { api: true },
};

import { applyParams, preventCrossShopDataAccess, save, ActionOptions, ActionOnSuccess } from "gadget-server";

/**
 * @param { ActionRun } context
 */
export async function run({ params, record, logger, api }) {


  applyParams(params, record);
  await preventCrossShopDataAccess(params, record);

  if (!record.changed("registeredWebhooks") && Object.entries(record.changes())[0][0] != "registeredWebhooks") {
    if (!record?.customerTagsBlocked || record.customerTagsBlocked.length == 0) {
      record.disabledWebhooks = { shopifyCustomer: true };
    } else {
      record.disabledWebhooks = { shopifyCustomer: false };
    }
  }

  await save(record);
}

/**
 * @param { ActionOnSuccess } context
 */
export async function onSuccess({ params, record, logger, api }) {
  // Your logic goes here
}

/** @type { ActionOptions } */
export const options = {
  actionType: "update",
  triggers: { api: true },
};

The create action can be whatever. Whats important is the update action

Chocci_Milk•7mo ago

What I put results in the correct outcome:

No description

No description

RicardoOP•7mo ago

thanks, seems to be working now

Chocci_Milk•7mo ago

Note that this code now checks the first entry of the object and looks to see what the key is called

RicardoOP•7mo ago

record.disabledWebhooks["shopifyOrder"] = false; record.disabledWebhooks["..."] = false; I am using this now. I think the issue was me overwriting it in my update code Thanks for helping me out with this 🙂

Gadget Developers Join

Build SAAS apps alongside thousands of other Gadget developers. Get help, and offer help to others in the community.

3.6KMembers

View on Discord

Did you find this page helpful?