FirecrawlF
Firecrawl•11mo ago
Frnz

Need Help: Scrape Actions doesn't seem to work / apply

I'm trying to use scroll actions to navigate through a website (Kickstarter), but they aren't working at all. I confirmed this by taking a screenshot, which shows that the page remains at the top even after multiple scroll-down actions. To rule out a site-specific issue, I tested other websites, but the problem persists.

Clicking and text input actions also don't work. However, I do receive an error when a clickable element isn't found, which suggests that my syntax is correct and that the checks are being applied.

Additionally, the example actions from the documentation doesn't work out of the box ( did the website change? ): https://docs.firecrawl.dev/launch-week#example

Here is the code I used:

from firecrawl import FirecrawlApp

# Initialize the app
app = FirecrawlApp(api_key = "fc-XXXXX")


# URL to scrape
url = 'https://www.kickstarter.com/discover/advanced?category_id=35&sort=popularity&page=1'


# Define the actions to be performed
actions = [
    {"type": "wait", "selector": ".load_more"},
    {"type": "scroll", "direction": "down", "amount": 2000 },
    {"type": "click", "selector": ".load_more"},
    {"type": "scroll", "direction": "down", "amount": 2000 },
    {"type": "screenshot", "fullPage": True},
    {"type": "scrape"}
]


# Perform actions and scrape
result = app.scrape_url(url, 
    params={
        'formats': ['html', 'screenshot'],
        'actions': actions,
        'timeout': 999999
    })

# show screenshot
print(result['screenshot'])


The expected screenshot and html should contain more than 20 games, since we scrolled down and pressed on the "load more" button, but the screenshot shows only the top side of the page, and the html contains only the first 13 games.
Was this page helpful?