Job Summary
DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for specialists to join our engineering team and shape the future of accelerated data processing.
DataPelago Nucleus is a universal data processing engine that is designed to accelerate the processing of diverse data – structured through unstructured – with any parallel processing framework – e.g., Apache Spark – on any infrastructure – including vectorized CPU and GPU. DataPelago Accelerator for Spark (DPA-S), our product based on DataPelago Nucleus, is running large-scale production applications of many globally renowned customers.
As a Senior/Principal Data Processing Engineer - OSS, you will be a key individual contributor in adopting and advancing capabilities of open-source software (OSS) platforms such as Apache Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago’s data processing engine. You will enhance functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with community working on these platforms.
This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.
DataPelagoMountain View, California
Pay not posted
Posted Sep 25, 2024
DataPelagoMountain View, California
Pay not posted
Posted Oct 28, 2025
Pay not posted
Posted Apr 2, 2025
DataPelagoHyderabad, Telengana, India
Pay not posted
Posted Oct 28, 2025
Source: the company job board. We last checked this job 2 h ago. We first saw it 2 h ago. A job on a job board does not always mean that the team interviews people now. We found this job board in a public list. We did not yet compare it with the careers page of the company.
Pay not posted
Posted Oct 28, 2025
DataPelagoMountain View, California
Pay not posted
Posted Aug 13, 2025
No one has reported an interview at NetApp yet. Did you interview there? Add the questions.