Senior Data Engineer - USA
Coderio, Miami, Florida +6
What you will do
- Pipeline Architecture & Development: Design, build, and deploy scalable, decoupled, and high-performance data pipelines for batch and streaming data ingestion into Data Lakes and Data Warehouses.
- ETL/ELT Processing: Develop and optimize efficient data pipelines using Python, PySpark, and advanced SQL.
- Collaboration with ML & Data Science: Partner with Data Scientists and ML Engineers to build resilient pipelines, automate MLOps workflows, and gather model interaction data to support retraining loops.
- Performance Optimization & Monitoring: Continuously monitor, troubleshoot, and optimize pipeline execution, query performance, and storage resource utilization across cloud platforms.
- Data Quality & Governance: Implement data quality frameworks and testing controls to ensure data integrity, reliability, and governance across raw and curated datasets.
- DevOps & Standards: Apply MLOps and DevOps best practices, maintain CI/CD pipelines, use Docker for containerization, and document technical standards and architectures.