Sr Data Engineer - Apache DolphinScheduler - Chennai
Photon, India
What you will do
- Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
- Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
- Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
- Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
- Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
- Develop custom plugins and extensions for DolphinScheduler as needed.
- Monitor and optimize job execution, resource allocation, and workflow efficiency.
- Maintain best practices for CI/CD, version control, and infrastructure automation.
What you need
- Strong experience with Apache DolphinScheduler in a production environment.
- Proficiency in Java, Python, or Scala for workflow scripting and automation.
- Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
- Understanding of workflow orchestration, DAG execution, and scheduling strategies.
- Strong problem-solving skills and ability to work in a fast-paced environment.
Similar jobs at other companies
Senior Tech Specialist, Cloud Engineering - R01573478
BrillioGuadalajara, Jalisco, Mexico
Pay not posted
Posted 7 days ago
Principal Data Engineer-R01570250
BrillioHyderabad, Andhra Pradesh, India