Blockchain Big Data Engineer (Polygon-Focused)
I'm currently building on-chain analytics tools and real-time dashboards on the Polygon chain.
Tech stack:
- PostgreSQL, ClickHouse for analytics
- ELT pipelines, JSON/JSONB event handling
- Concepts like smart contracts, event logs, RPCs
But I want to know from real engineers who've done this at scale:
- What are the most critical things I should focus on to grow into this role?
- What skill gaps should I close?
- What real-world challenges hit you the hardest when scaling ingestion, querying, or infra?
I’m especially interested in:
Polygon-specific data patterns
Chain reorg handling
Event parsing at scale
Real-time dashboards and query optimization
Security in ingestion pipelines
Would love to hear war stories, or just your top tips.
Appreciate any insight!
4 Replies
Didn't even know "Blockchain Big Data Engineer" is a thing :spinning_think:
I'm using goldsky & polymarket API on GCP. posgresql on cloud sql, BigQuery. what can be weakness in combination of it?
Redis for caching, prevent duplication issue of data insert
out of curiosity, why polygon specifically?
cause of its low gas fee, capabilties to handle thousands of tx per second, we use polygon chain
also fully EVM compatible