Data Engineer Manager
Artefact, APAC +2
The role
What you will be doing? As a Data Engineer Manager, your role involves crafting and maintaining robust data pipelines, utilising Python and SQL, to ensure efficient extraction, transformation, and loading (ETL) of data.
What you will do
- Data Pipeline Development: Building and optimising data pipelines to facilitate seamless data flow across systems and platforms.
- Database Management: Managing databases, ensuring their integrity, and implementing data storage and retrieval solutions.
- Cloud Services Integration: Leveraging cloud services such as MS Azure, GCP, and AWS to architect and deploy scalable data solutions.
- Machine Learning Integration: Collaborating with teams to integrate machine learning models into data pipelines for enhanced data processing.
- Utilising Spark & Kafka: Implementing and working with Spark and Kafka for real-time data processing and analytics.