Proper workflow for single page apps that don't retain URL state
Hey team 👋🏿 I'm curious on the suggested workflow for single page apps that don't retain URL state. Take this Airbnb UI for example: how best should I traverse through each category like
Beach, Cabins, etc. and then scrape what's in the DOM at that point?
My only thought is to take advantage of actions and make a request after each button click on the filter but this means that i'd make x amount of requests for however many filter tags there are.
5 Replies
Any thoughts from the firecrawl team? cc @thomas @Adobe.Flash
Hey, we have an internal action to scrape the page content but I am not sure if it's exposed on firecrawl. Will update you.
Yeah I realize in a case like this that i'd need to evaluate logic during a step, which doesn't currently seem possible with firecrawl – sorta like running a playwright script, if you will. Is that on the roadmap or capable to do now?
Hmm, it's something we are considering internally, but not currently in the road map
Okay would love to test this out if it becomes available