F
Firecrawl•3w ago
Jack74r

Jack74r - Hi , selfhosted it's possible to craw...

Hi , i have issue with selfhosted custom prompt, it"s work on cloud with exact same query ? I try to add in scrap option but it's return completed but empty data only when i set this option And sometime return : Only 0 result(s) found. For broader coverage, try crawling with crawlEntireDomain=true or start from a higher-level path like ... I dont understand the issue when i add custom prompt because its not get the full data on the page (and crawlentiredomain is set to true ...) After some tests, i tried exaclty same query on the cloud and local and the result is not working on local. json is empty , and on the cloud i get all perfect data . I'm using openrouter : Grok 4.1 Fast for the LLM but i'm not sure is the problem (log show another llm but on openrouter its grok not 4o) I'm using openrouter it's the issue ? OPENAI_BASE_URL=https://openrouter.ai/api/v1 OPENAI_API_KEY=mytoken MODEL_NAME=x-ai/grok-4.1-fast 
"scrapeOptions": {
    "parsers": [],
    "formats": [
      "markdown",
      "links",
      "images",
      {
        "type": "json",
        "prompt": "give me the informations, contact , socials, schedule (location, datetime, google map link) ",
        "schema": {
          "type": "object",
          "required": [],
          "properties": {}
        }
      }
    ],
    "onlyMainContent": false,
    "maxAge": 0,
    "waitFor": 2000,
    "timeout": 20000
  }
"scrapeOptions": {
    "parsers": [],
    "formats": [
      "markdown",
      "links",
      "images",
      {
        "type": "json",
        "prompt": "give me the informations, contact , socials, schedule (location, datetime, google map link) ",
        "schema": {
          "type": "object",
          "required": [],
          "properties": {}
        }
      }
    ],
    "onlyMainContent": false,
    "maxAge": 0,
    "waitFor": 2000,
    "timeout": 20000
  }
Result is : 
{
  "success": true,
  "status": "completed",
  "completed": 1,
  "total": 1,
  "creditsUsed": 5,
  "expiresAt": "2025-11-24T18:50:03.000Z",
  "data": [
    {
      "json": {},
      "links": [...
normal data ..
{
  "success": true,
  "status": "completed",
  "completed": 1,
  "total": 1,
  "creditsUsed": 5,
  "expiresAt": "2025-11-24T18:50:03.000Z",
  "data": [
    {
      "json": {},
      "links": [...
normal data ..
5 Replies
Jack74r
Jack74rOP•3w ago
Jack74r
Jack74rOP•3w ago
in openrouter logs, firecrawl call the LLM and LLM try using tool, it's strange because it"s only text extraction from the markdown no ? Nov 23, 07:49 PM Grok 4.1 Fast Unknown 2 139 1 049 0$ 70,9tps tool_calls
Gaurav Chadha
Gaurav Chadha•3w ago
@Jack74r The key issue is in the model selection. Your logs show Model: gpt-4o-mini | hasRef: false, which means the system is trying to use OpenAI's gpt-4o-mini model. It won't work with other LLM due to models
Jack74r
Jack74rOP•2w ago
ok i will try it thank ok you right thank, i see on firecrawl repo some one use openrouter with custom model It's not possible to use other models ? Because 4o & 4o-mini are not the best price / inteligence models compatible with firecrawl ? No reasoning model ?
Gaurav Chadha
Gaurav Chadha•2w ago
not atm

Did you find this page helpful?