I can't get it to work with openai's

TheCrazyBob · 2025-11-22T15:37:46.074Z

I can't get it to work with openai's models. Haven't tried gemini yet. I always get the following error: ```json { "success": false, "errors": [ { "message": "Unable to form JSON based on user prompt and/or webpage text. Please try with a different prompt try passing a response_format." } ] } ```

T

TheCrazyBobOP•11/22/25, 3:39 PM

Here is the curl request:

curl --request POST \
  --url 'https://api.cloudflare.com/client/v4/accounts/<account_id>/browser-rendering/json?cacheTTL=0' \
  --header 'Authorization: Bearer <cloudflare_token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://jobs.lever.co/1inch/0cb4295f-2a1c-4ee7-a56c-b093d25a8293",
  "prompt": "Extract the job title",
  "custom_ai": [
    {
      "authorization": "Bearer <openai_api_key>",
      "model": "openai/gpt-5-mini-2025-08-07"
    }
  ]
}'

curl --request POST \
  --url 'https://api.cloudflare.com/client/v4/accounts/<account_id>/browser-rendering/json?cacheTTL=0' \
  --header 'Authorization: Bearer <cloudflare_token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://jobs.lever.co/1inch/0cb4295f-2a1c-4ee7-a56c-b093d25a8293",
  "prompt": "Extract the job title",
  "custom_ai": [
    {
      "authorization": "Bearer <openai_api_key>",
      "model": "openai/gpt-5-mini-2025-08-07"
    }
  ]
}'

curl --request POST \
  --url 'https://api.cloudflare.com/client/v4/accounts/<account_id>/browser-rendering/json?cacheTTL=0' \
  --header 'Authorization: Bearer <cloudflare_token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://jobs.lever.co/1inch/0cb4295f-2a1c-4ee7-a56c-b093d25a8293",
  "prompt": "Extract the job title",
  "custom_ai": [
    {
      "authorization": "Bearer <openai_api_key>",
      "model": "openai/gpt-5-mini-2025-08-07"
    }
  ]
}'

curl --request POST \
  --url 'https://api.cloudflare.com/client/v4/accounts/<account_id>/browser-rendering/json?cacheTTL=0' \
  --header 'Authorization: Bearer <cloudflare_token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "url": "https://jobs.lever.co/1inch/0cb4295f-2a1c-4ee7-a56c-b093d25a8293",
  "prompt": "Extract the job title",
  "custom_ai": [
    {
      "authorization": "Bearer <openai_api_key>",
      "model": "openai/gpt-5-mini-2025-08-07"
    }
  ]
}'

T

TheCrazyBobOP•11/22/25, 3:40 PM

CC: @jonnyparris

T

TheCrazyBobOP•11/22/25, 3:40 PM

Tried different URLs, prompts and response_format

T

TheCrazyBobOP•11/22/25, 3:42 PM

The same request works if I remove the Custom AI parameter. Also no request is made to OpenAI as it doesn't show up in the OpenAI logs

T

TheCrazyBobOP•12/4/25, 11:43 PM

@jonnyparris Did this go live?

T

TheCrazyBobOP•12/4/25, 11:44 PM

Also, for some JS heavy pages the markdown endpoint returns nothing. Try this one for example:

https://fa-ewnx-saasfaprod1.fa.ocs.oraclecloud.com/hcmUI/CandidateExperience/en/sites/CX_1001/job/12358

https://fa-ewnx-saasfaprod1.fa.ocs.oraclecloud.com/hcmUI/CandidateExperience/en/sites/CX_1001/job/12358

https://fa-ewnx-saasfaprod1.fa.ocs.oraclecloud.com/hcmUI/CandidateExperience/en/sites/CX_1001/job/12358

https://fa-ewnx-saasfaprod1.fa.ocs.oraclecloud.com/hcmUI/CandidateExperience/en/sites/CX_1001/job/12358

The response is:

{
  "success": true,
  "result": ""
}

{
  "success": true,
  "result": ""
}

{
  "success": true,
  "result": ""
}

{
  "success": true,
  "result": ""
}

TTheCrazyBob Also, for some JS heavy pages the markdown endpoint returns nothing. Try this on...

K

Kathy•12/8/25, 6:08 PM

not able to duplicate. can you try again?

T

TheCrazyBobOP•12/8/25, 11:21 PM

@Kathy | Browser Rendering PM Try the /markdown endpoint instead of /content

T

TheCrazyBobOP•12/8/25, 11:22 PM

T

TheCrazyBobOP•12/8/25, 11:27 PM

another sample:

https://boards.greenhouse.io/andurilindustries/jobs/4794577007

https://boards.greenhouse.io/andurilindustries/jobs/4794577007

https://boards.greenhouse.io/andurilindustries/jobs/4794577007

https://boards.greenhouse.io/andurilindustries/jobs/4794577007

T

TheCrazyBobOP•12/8/25, 11:28 PM

and another one that works with /content but /markdown returns empty

https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/application

https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/application

https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/application

https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/application

T

TheCrazyBobOP•12/8/25, 11:40 PM

I figured it out.

waitUntil

waitUntil

needs to be set in the request body. I would recommend having this in the docs since more than half of the requests I was sending were returning empty markdown both for "/extract" and "/markdown" endpoints:

{
  "url": "https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/",
  "gotoOptions": {
    "waitUntil": "networkidle0"
  }
}

{
  "url": "https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/",
  "gotoOptions": {
    "waitUntil": "networkidle0"
  }
}

{
  "url": "https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/",
  "gotoOptions": {
    "waitUntil": "networkidle0"
  }
}

{
  "url": "https://jobs.ashbyhq.com/sagecare/395f5614-0223-4a17-acee-b6fca5f8e302/",
  "gotoOptions": {
    "waitUntil": "networkidle0"
  }
}

I can't get it to work with openai's

Similar Threads

Similar Threads

Similar Threads