Software Engineer - Web Crawling
Tavily, Israel
The role
- You'll build the systems that power how AI interacts with the live web.
- You'll design and implement large-scale, intelligent web acquisition pipelines, making sense of an ever-changing online ecosystem and transforming it into structured, high-quality data.
- This role blends technical understanding of browsers, networks, and automation with hands-on engineering.
- You'll work on adaptive systems that navigate modern websites, understand evolving web interaction ,detection systems, and gather meaningful information safely, efficiently, and at scale.
- Your work will directly shape Tavily's ability to provide AI agents with accurate, real-time knowledge of the world , bridging the gap between the open web and intelligent reasoning.
- This position is based in Israel and requires existing legal authorization to work in Israel. Tavily is not able to provide visa sponsorship for this role.
What you will do
- Build distributed data acquisition systems that capture and structure the live web