Lead Data Engineer
3m, Remote - Minnesota
What you will do
The Impact You’ll Make in this Role
We are seeking a Lead Data Engineer with strong experience in AI/ML data pipelines, feature engineering, and scalable data architectures. This role will lead the design, development, and optimization of data platforms that power advanced analytics and machine learning applications across the organization.
Key Responsibilities
- Lead the architecture and development of scalable, secure data pipelines supporting AI/ML workloads.
- Own end to end data engineering processes: ingestion, transformation, storage, quality, and monitoring.
- Collaborate with data scientists and ML engineers on model features, training pipelines, and deployment.
- Drive best practices in data modeling, orchestration, versioning, and performance optimization.
- Mentor and guide junior engineers; contribute to technical roadmaps and solution patterns.
- Ensure data governance, lineage, and compliance standards are met across platforms.
- Support real time and batch processing frameworks in production environments.
What you need
To set you up for success in this role from day one, 3M requires (at a minimum) the following qualifications:
- Bachelor's degree or higher in computer science (completed and verified prior to start)
- Seven (7) years of data engineering experience, including leading technical initiatives in a private, public, government or military environment.
Nice to have
- Strong expertise in Python, SQL, and distributed data systems (e.g., Spark, Databricks, Synapse).
- Experience building AI/ML ready data pipelines, including feature stores and model training data flows.
- Hands on experience with cloud platforms (Azure preferred — Data Lake, Data Factory, Databricks, Cosmos DB).
- Strong understanding of ML concepts, lifecycle, and MLOps practices.
- Proven experience with workflow orchestration (Airflow, Data Factory, Synapse Pipelines, etc.).
- Strong communication and ability to translate business needs into technical solutions.
- Experience with streaming platforms (Kafka, Event Hub).
- Familiarity with vector databases, embeddings, or LLM oriented data pipelines.
- Background in DevOps, CI/CD, or infrastructure as code.
- Experience designing data solutions for AI applications or intelligent platforms.
Work location: Remote in the US
Travel: May include up to 5% primarily domestic travel