Senior Data Engineer
Shift, Sydney
The role
Want to put your stamp on a data platform? You’ll own our Azure Databricks platform, set its standards and roadmap, and lead the data side of a core lending system migration. You’ll be the senior engineer in a small team, with two junior engineers to develop and plenty of room to shape how data works at Shift.
What you will do
- Own the data platform roadmap and engineering standards, including CI/CD, separate environments, automated testing, monitoring and access governance through Unity Catalog
- Design and lead the build of robust, scalable ELT pipelines on Azure Databricks
- Lead the data workstream for our core lending platform migration: remap pipelines and models, and reconcile outputs before and after cutover alongside Engineering
- Build and extend a curated, dimensional data model that becomes the default place the business goes for trusted, self-serve data
- Lift data quality, freshness and alerting so issues are caught before the business notices them
- Keep the platform cost-efficient through cluster optimisation, job orchestration and smart use of compute