Crawler Help
Hi, I am new here. I was wondering if any of you know how to get the full url from the links on a website. This is the code I have. The only URL that this returns is the URL of the page I gave it. I need it to be able to give me the full URL like this(https://recruiting.paylocity.com/Recruiting/Jobs/Details/2480999) of each job link on the site. My other pieces of code that I have wrote have only been able to give me this "/Recruiting/Jobs/Details/2480999". Thank you for any help or advice.
app = FirecrawlApp(api_key='FIRECRAWLAPI')
Set up the crawl parameters to return only URLs
crawl_url = 'https://recruiting.paylocity.com/recruiting/jobs/All/d4447541-be32-4a64-81e4-fa735e6e2ec4/Icon-Vehicle-Dynamics-LLC'
params = {
'crawlerOptions': {
'returnOnlyUrls': True
}
}
Start the crawl
crawl_result = app.crawl_url(crawl_url, params=params, wait_until_done=True)
Print the URLs
print(crawl_result)
Icon Vehicle Dynamics LLC - Production Planning Manager
Summary:The Production Planning Manager is responsible for overseeing and coordinating the production planning and scheduling activities within the manufacturing department. This role ensures that production processes are carried out efficiently and effectively, meeting the company’s quality standards and customer demands. The ideal candidate wi...
Icon Vehicle Dynamics LLC - Job Opportunities
Icon Vehicle Dynamics LLC Careers Page - View all jobs and opportunities at Icon Vehicle Dynamics LLC and apply today. | Powered By Paylocity
0 Replies