ideal input for `search` param in /map endpoint
hi @rafaelmiller @Caleb i was testing out the /map endpoint to fetch a given specific subset of links for a company's domain.
so i wanted to understand what would an ideal query to the
for eg. if i want a company's global offices information i would ideally go to the page where
I got the website where the locations are present however i got additional > 50 websites which were not required at all.
so i am hoping to narrow down the filtered list ..
i mean one potential solution i can think of is adding an additional llm call on top of these filtered queries to further filter the specific web pages but i believe it will add an overhead in the service time ?
another thing i noticed was given a
the frequency of this is less and i am sure u guys might have thought of this or considered but still any visibility would surely help !
potential solution that i can think of is maybe implement a caching mechanism with a specified
sharing this sheet for some reference : https://docs.google.com/spreadsheets/d/1EmIxn6sVOheugBu1RASAadJab5C0OqysLmaYjTpb-i8/edit?usp=sharing
so i wanted to understand what would an ideal query to the
search param should look like ? for eg. if i want a company's global offices information i would ideally go to the page where
locations , offices information is there, soo in the search query i passed location keyword as the param value. I got the website where the locations are present however i got additional > 50 websites which were not required at all.
so i am hoping to narrow down the filtered list ..
i mean one potential solution i can think of is adding an additional llm call on top of these filtered queries to further filter the specific web pages but i believe it will add an overhead in the service time ?
another thing i noticed was given a
url and search param value, if we run multiple iterations the total number of results is varied across iterations ( even though url and search param value is same ). This inconsistency might be an issue for enterprise solutions.the frequency of this is less and i am sure u guys might have thought of this or considered but still any visibility would surely help !
potential solution that i can think of is maybe implement a caching mechanism with a specified
ttl value ? - update : i noticed u guys have already implemented this .. but what if i trigger the same query after the ttl ? will i get the same number of results ?sharing this sheet for some reference : https://docs.google.com/spreadsheets/d/1EmIxn6sVOheugBu1RASAadJab5C0OqysLmaYjTpb-i8/edit?usp=sharing
