FIRE-1 agent is not able to scrape javascript dynamic content. It is erroring out every time.
code:
scrape_result = app.scrape_url(
'https://www.classmateshop.com/',
formats=['markdown', 'html'],
timeout=600000, # 60 seconds timeout (in milliseconds)
waitFor=5000, # Wait 5 seconds for page to load
blockAds=True, # Block ads for faster loading
agent={
'model': 'FIRE-1',
'prompt': 'Extract all content from this webpage. Click on buttons and explore navigation elements to find all content. Make sure to wait for the page to fully load before extracting.'
}
)
print(scrape_result)
I have spend so many credits already trying this with different public websites.
Is Fire-1 Agent the correct choice to get all the static and dynamic content?
0 Replies