Gadget Developers•2d ago

Can I sync data models for some of my users, not all? Can I fetch this data directly, without sync?

I'm almost at 400 compute hours this billing period, and the cost of it wipes out most of my app's revenue. The culprit is the orders/created webhook, which is firing nonstop. My app has a paid feature which sometimes needs to know the items in a particular Shopify order to be able to generate a custom receipt for that order. It doesn't need to do this often, and it only ever needs to do this for merchants on a paid plan (free users don't have access to the feature that requires this order data). The naive way I've gone supporting this paid feature is by: 1. Adding the "Orders" API scopes & data models to my Shopify app connection, which syncs all orders and order line items to my database (99.9% of which my app doesn't want or need). 2. In my api/models/shopifyOrder/actions/create.ts I handle the onSuccess event for orders/created webhook by deleting orders and order line items from my database that belong to merchants on a free plan (since these users can't access the feature that needs this data anyways). This is so so inefficient, and there has to be a better way. Here are some alternatives I am thinking about, and I want to know what's possible: - Instead of syncing orders data for my users and storing them in my Gadget database, is it possible to turn off the syncing and simply query the stores directly for their order data when needed? - Is it simply my webhook handler ( api/models/shopifyOrder/actions/create.ts ) that is using compute time? Or does the syncing of items to the database itself use compute time? I'm wondering if I could save money by just using a scheduled cleanup function to handle deletion of unneeded orders & order line items once a day, instead of using this webhook handler.` - Can I somehow turn on/off syncing of orders data for customers on the Free plan? Or can syncing only be turned on/off for all customers? I see an "Add Filter" beta feature in Gadget. Is that applicable here? Thanks!

5 Replies

Simon•2d ago

You deleting orders just adds even more compute time, (e.g. for deleting the order again) because on the next /create or /update from shopify gadget will then run the /create action (gadget runs /update if it finds a record and /create otherwise). If you want to delete data to clean up your database I'd maybe run cron jobs and delete actually old data, i.e. >14d ago or so. Anyways, in your case it sounds like you really do not need any of that order data in your gadget database, so you could throw away the shopifyOrder and lineItem model (and with that all the webhooks, syncing, compute time etc) and instead just do a direct query to shopify to get a specific order with the data you need. Direct call, i.e. something like

const shopifyConnection = await connections.shopify.forShopId(shopId); 

const res = await shopifyConnection.graphql(` your-query ...

const shopifyConnection = await connections.shopify.forShopId(shopId); 

const res = await shopifyConnection.graphql(` your-query ...

rayhanmemonOP•2d ago

This is super helpful! This does indeed work. However, now I want to disable the webooks for shopifyOrder and shopifyOrderLineItem for my existing customers, but it doesn't appear to be as simple as updating all theshopifyShop records disabledWebhooks column. The below fails with:

GadgetOperationError:"GGT_UNKNOWN: [GraphQL] Variable \"$shopifyShop\" got invalid value { disabledWebhooks: { shopifyOrder: true, shopifyOrderLineItem: true } }; Field \"disabledWebhooks\" is not defined by type \"UpdateRecordShopifyShopInput\"

I'm having a little trouble making sense of the docs for this. They seem to imply that updating the disabledWebhooks for an existing shop is problematic: https://docs.gadget.dev/guides/plugins/shopify/shopify-webhooks#disable-webhook-processing-per-shop

// DELETE AFTER RUNNING IN PROD!

/** @type { ActionRun } */
export const run = async ({ params, logger, api, connections }) => {
  const shops = await api.shopifyShop.findMany({
    select: { id: true },
  });
  for (const shop of shops) {
    await api.shopifyShop.updateRecord(shop.id, {
      disabledWebhooks: {
        shopifyOrder: true,
        shopifyOrderLineItem: true,
      },
    });
  }
  return { count: shops.length };
};

// DELETE AFTER RUNNING IN PROD!

/** @type { ActionRun } */
export const run = async ({ params, logger, api, connections }) => {
  const shops = await api.shopifyShop.findMany({
    select: { id: true },
  });
  for (const shop of shops) {
    await api.shopifyShop.updateRecord(shop.id, {
      disabledWebhooks: {
        shopifyOrder: true,
        shopifyOrderLineItem: true,
      },
    });
  }
  return { count: shops.length };
};

rayhanmemonOP•2d ago

Ah ok, so here's the way around it. In the install and reinstall actions for shopifyShop, mutate the record to set disabledWebhooks as required. Then, you can go to your "Installs" in the gadget dashboard and re-register webhooks. This will work

Simon•18h ago

What I meant is that you could just remove the models completely as well. If you don't actually use them.

Chocci_Milk•11h ago

@rayhanmemon, I would also recommend that you add webhook filtering for the order webhooks. Its hard to determine what you'd need to filter but you'd know best! Here are some docs: https://docs.gadget.dev/guides/plugins/shopify/shopify-webhooks#applying-a-webhook-filter

Working With Shopify Webhooks - Developer Docs - Gadget

Gadget Docs

Gaming

Programming

Can I sync data models for some of my users, not all? Can I fetch this data directly, without sync?

Did you find this page helpful?