API Extract not getting Up to Date / Live data - Lots of Lag Time

When using the extract API though N8N or direct using PostMan, it is taking hours before showing the most recent data. For example, if I change details of a product, create a new product etc. when I do a an Extract post and then a GET post with the POST ID, it keeps returning the old data. Not until an hour or so later when I do a POST and GET do I get the most recent data. I don't have any CDN or Caching installed on my server. I recorded a quick video here. Anyone else run into this issue? Any ideas? https://sammyt05-hotmail.tinytake.com/msc/MTA4MjA0MDdfMjQ1MzczODM
5 Replies
Samster
SamsterOP5mo ago
Here is an updated video with audio so to explain it better. https://vimeo.com/1086183174
Sam
Vimeo
FireCrawl Issue
Issue using FireCrawl Extract
Samster
SamsterOP5mo ago
I found the issue, it was related to including the following in my JSON body. "agent": { "model": "FIRE-1" } This needs to be included.
SoulBloxXer
SoulBloxXer5mo ago
Wait how/ why does using their agent suddenly solve the problem i dont get it
Samster
SamsterOP5mo ago
It forces the model to use FIRE-1
SoulBloxXer
SoulBloxXer5mo ago
I meant like why wasnt it able to do the job without the agent

Did you find this page helpful?