CA
Crawlee & Apify•3y ago
genetic-orange

TypeError: __init__() got an unexpected keyword argument 'follow_redirects'

I am using Apify Client to scrape information in a cron job written in airflow dag. When i try to execute the task file manually, it works fine but when i do so from the airflow UI, it throws this error. I tried to upgrade the version of httpx to 0.24 but apify client support 0.23 
  APPIFY_CLIENT = ApifyClient(config('APIFY_CLIENT_TOKEN'))
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/client.py", line 126, in __init__
    self.http_client = _HTTPClient(
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/_http_client.py", line 59, in __init__
    self.httpx_client = httpx.Client(headers=headers, follow_redirects=True, timeout=timeout_secs)

TypeError: __init__() got an unexpected keyword argument 'follow_redirects'
  APPIFY_CLIENT = ApifyClient(config('APIFY_CLIENT_TOKEN'))
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/client.py", line 126, in __init__
    self.http_client = _HTTPClient(
File "/usr/share/pyshared/venv/airflow/lib/python3.8/site-packages/apify_client/_http_client.py", line 59, in __init__
    self.httpx_client = httpx.Client(headers=headers, follow_redirects=True, timeout=timeout_secs)

TypeError: __init__() got an unexpected keyword argument 'follow_redirects'
3 Replies
Alexey Udovydchenko
Alexey Udovydchenko•3y ago
its airflow issue, client not getting valid input and you need somehow to exclude follow_redirects
!!!Joefree!!! 👑
!!!Joefree!!! 👑•3y ago
Have you tried change to allow_redirects ?
genetic-orange
genetic-orangeOP•3y ago
@!!!Joefree!!! 👑 @Alexey Udovydchenko i removed the follow_redirects from the source code and installed on my system via pip. It works fine after that

Did you find this page helpful?