Data Engineer III - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

• Architects and maintains large-scale data pipelines using Apache Spark and

related distributed computing frameworks, ensuring efficient processing of structured and unstructured data.

• Builds and optimizes both batch and streaming data workloads to support

analytical and operational use cases.

• Develops, optimizes, and productionizes notebooks, jobs, and workflows within

the Databricks environment to support analytical and machine learning use cases.

• Designs and implements data architecture and data modeling strategies,

including dimensional models, lakehouse structures, and schema design that balance performance, scalability, and governance requirements.

• Writes and optimizes complex SQL queries for data transformation, validation,

and performance tuning across large datasets.

• Supports ML model serving infrastructure, ensuring trained models are

deployed, monitored, and scaled reliably in production environments.

• Collaborates closely with data scientists and ML engineers on MLOps

practices, including model versioning, CI/CD pipelines for ML artifacts, monitoring, and automated retraining workflows.

• Partners with cross-functional stakeholders — including analytics, risk, and

platform engineering teams — to translate business requirements into technical data solutions.

• Maintains strong data quality, lineage, and security standards consistent

with data governance and compliance expectations.

• Leverages enterprise-authorized AI coding assist tools within the work

environment to improve code quality, delivery speed, and productivity across complex deliverables (e.g., code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standards; contributes learnings and reusable patterns to improve broader team effectiveness.

• Applies knowledge of tools within the Software Development Life Cycle

toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.

• Formal training or certification on data engineering concepts and 3+ years

applied experience.

• Demonstrates strong, hands-on proficiency with Apache Spark or comparable

distributed compute frameworks, including experience optimizing jobs for performance and cost at scale.

• Holds direct, production-level experience with Databricks, including

notebooks, Delta Lake, job orchestration, and cluster management.

• Demonstrates solid command of SQL, with the ability to write, debug, and

optimize complex queries against large and varied datasets.

• Brings practical experience supporting ML model serving infrastructure in

production environments.

• Shows familiarity with MLOps tooling and practices, such as model monitoring,

versioning, and deployment automation.

• Possesses a strong foundation in data architecture and data modeling,

covering both operational and analytical data store design.

• Has working knowledge of cloud data platforms such as AWS, Azure, or GCP.

Maintains proficiency with version control systems such as Git.

• Demonstrates hands-on exposure to distributed computing, cloud data

platforms, and production ML infrastructure.

• Hands-on experience using enterprise-authorized AI-assisted software

development tools within the work environment (e.g., for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.

• Understanding of responsible AI use in engineering workflows, including data

sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices

• Experience with workflow orchestration tools such as Airflow or Databricks

Workflows.

• Familiarity with containerization technologies such as Docker and Kubernetes.
• Exposure to CI/CD pipelines applied to data and ML workflows. Familiarity

with feature stores and model registries such as MLflow.

• Experience with real-time streaming architectures such as Kafka and Spark

Structured Streaming.

• Prior experience in financial services or other highly regulated industries,

with an understanding of data governance and compliance requirements.