Senior Software Engineer, Data Engineering
Distributed Spectrum, New York City
The role
DS collects radio data at a scale few organizations ever see — continuous, high-rate streams from sensors around the globe. We're hiring a Senior Data Engineer to design the platform that turns that firehose into an asset: a data lakehouse that serves researchers training models, production systems running inference, and agents retrieving context in real time.
You'll own the architecture from ingestion through storage, cataloging, vector search, and access, and you'll design it explicitly for AI/ML workloads, not just analytics.
What you will do
- Architect and build our data lakehouse: ingestion pipelines, open table formats, partitioning and compaction strategies, cataloging, and governance.
- Design and operate vector database infrastructure for embedding storage, similarity search, and retrieval at scale — choosing, tuning, and evolving the right systems for our workloads.