Software Engineer, Knowledge Engine – Data Infrastructure
Exa, San Francisco, California
The role
In this role, you'll build the knowledge construction pipelines: turning everything from a 100k docs/s web crawl to vendor datasets into a graph of what is true about the world, and keeping it correct as data changes.
This is greenfield work with a massive scope. There is no off-the-shelf system for building a model of the world from the open web, so you'll make foundational decisions, from how claims are represented and resolved to how trust is assigned and how the graph evolves over time, that everything built on top of it depends on.
What you will do
- Continuously synthesize the live web crawl, vendor deliveries and APIs, and other off-the-web sources into one typed graph: extraction that turns pages and records into ontology-shaped claims, each carrying its provenance, timestamp and confidence.
- Bring every new source in cleanly: versioned, append-only history with schema evolution, late-arriving and corrected records, and replayable cursors, so every claim traces back to the exact delivery that produced it.