- Formal training or certification on software engineering concepts and 5+
years applied experience.
- Advanced experience in software engineering and Data Engineering, including
significant production delivery with Apache Spark on Databricks and/or AWS EMR.
- Demonstrated experience leading effective use of approved AI-assisted
software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
- Strong understanding of responsible AI use in engineering workflows,
including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices.
- Advanced hands-on Databricks expertise across Delta Lake, Unity Catalog,
Workflows, Repos/notebooks, and SQL Warehouses, including cluster configuration and optimization.
- Proven ability to architect, build, and operate reliable ETL/ELT data
pipelines (batch and streaming), including schema design/evolution, SLAs, and reliability engineering practices.
- Deep Spark performance tuning skills, with experience diagnosing bottlenecks
and optimizing jobs for scalability, cost, and runtime efficiency.
- Strong programming proficiency in Python for data processing, platform
tooling, and automation.
- Strong SQL and analytics data modeling expertise, including dimensional/star
schema design and Lakehouse best practices.
- Demonstrated experience leading effective use of approved AI-assisted
software development tools (coding, code review, test acceleration, troubleshooting), including setting team expectations and validation standards for correctness, performance, and security of AI outputs.
- Strong responsible-AI and security-first engineering mindset, including data
sensitivity awareness, secure handling of inputs/outputs, roles/instance profiles, secrets management, encryption at rest/in transit, network controls, and adherence to resiliency and security expectations; experience coaching teams on safe, compliant adoption within delivery practices.