CA
probable-pink
user_data is not working as expected
For some reason, when I push data to user_data (str type in this case), when I get user_data variables in another handler I get different values.
In this case the error is on tab_number. When I push tab_number to user_data the values seems to be good (values ranged from 1 to 100). But when I get tab_number through tab_handler I get a different value.
For example, for values from 1 to 19, I get tab_number 1, instead of the correct tab_number: tab_number pushed to user_data: "19", tab_number requested from user_data: "1".
I cannot find the error. Here is the code:
@router.handler('tabs')
async def tabs_handler(context: PlaywrightCrawlingContext) -> None:
tab_id = context.request.user_data["tab_id"]
await context.page.wait_for_selector('#tabsver > ul > li > a')
tabs = await context.page.locator('#tabsver > ul > li > a').all()
for tab in tabs:
tab_name = await tab.text_content()
tab_number = tab_name.replace("Tab number ", "").strip()
if tab_name:
await context.enqueue_links(
selector = f'#tabsver > ul > li > a:has-text("{tab_name}")',
label = "tab",
user_data = {"tab_id": tab_id,
"tab_number": tab_number
},
)
@router.handler('tab')
async def tab_handler(context: PlaywrightCrawlingContext) -> None:
tab_id = context.request.user_data["tab_id"]
tab_number = context.request.user_data["tab_number"]
2 Replies
Someone will reply to you shortly. In the meantime, we’ve found some posts that could help answer your question.
equal-jade•7mo ago
I think the issue might be related to the target site or selectors. Try debugging the value at each step to pinpoint the problem.