Architecture advice
Seeking architecture advice for a real estate matching service
Hi everyone,
I'm building a real estate notification service where users enter property requirements and get alerts when matching listings appear across multiple websites and social network groups.
I need to:
- Crawl 10+ real estate websites that have anti-scraping measures
- Process listings centrally, then match to user requirements
- Use LLM to evaluate complex matching criteria beyond basic filters
- Track which listings have already been processed for each user
My idea is to use a centralized crawler, store listings in a vector database, use RAG to retrieve candidates, and apply LLM evaluation only on promising matches. I'm considering n8n for orchestration.
What's the most effective tool combination for this use case? How should I structure this for scalability and handle crawler detection prevention? Any suggestions for efficiently tracking listing-user evaluation history?
Thanks for your insights!
0 Replies