Software Engineer, Distributed Data Systems (US)
Onehouse, Sunnyvale, CA
What you will do
- As a foundational member of the Data Infrastructure team, you will productionize the next generation of our data tech stack by building the software and data features that actually process all of the data we ingest.
- Accelerate our open source <> enterprise flywheel by working on the guts of Apache Hudi's transactional engine and optimizing it for diverse Onehouse customer workloads.
- Act as a SME to deepen our teams' expertise on database internals, query engines, storage and/or stream processing.
A Typical Day
- Design new concurrency control and transactional capabilities that maximize throughput for competing writers.
- Design and implement new indexing schemes, specifically optimized for incremental data processing and analytical query performance.
- Design systems that help scale and streamline metadata and data access from different query/compute engines.
- Solve hard optimization problems to improve the efficiency (increase performance and lower cost) of distributed data processing algorithms over a Kubernetes cluster.