Staff/Principal AI/ML Engineer

What we’re looking for

Zus is hiring a Staff/Principal AI/ML Engineer to define how LLMs and AI agents power our products and the AI platform underneath them. You'll be one of the most seasoned AI/ML voices in the building, with real ownership of architecture, tooling and roadmap.

Zus aggregates clinical data from across the healthcare ecosystem. That data is messy, multi-modal and enormously valuable. Modern foundation models finally make it possible to turn it into structured, trustworthy insight at the point of care and across populations. Your job is to make that happen, safely and at scale.

This is a hands-on role. You'll write production code, ship prototypes to real users, set technical direction, and mentor the engineers who build alongside you. This work will require interactions with solution engineers, product managers, sales, and marketing.

What you bring

• 7+ years building and shipping AI/ML systems, including 2+ years putting LLM-powered products into production for real users.
• Hands-on depth in modern LLM engineering: RAG, agents and tool use, structured outputs, prompt and context engineering, and the trade-offs between frontier APIs and open-weight models.
• A track record of building ML infrastructure others depend on — serving, pipelines, model and prompt versioning, CI/CD, monitoring.
• A rigorous approach to evaluation: you've designed eval sets and metrics, and you don't ship on vibes.
• Strong Python and software engineering fundamentals; fluency in PyTorch and the Hugging Face ecosystem.
• Production experience on a major cloud (AWS preferred) with containers, and infrastructure as code.
• Sound judgment about safety, privacy and reliability when models touch sensitive data.
• Clear writing and communication; you can explain model behavior and risk to clinicians, product leaders and executives.
• BS in Computer Science, Statistics or a related field, or equivalent experience.
• Are in the Boston metro and excited to collaborate in person on a hybrid schedule, or are remote (EST/CST) but willing to travel for in person collaboration occasionally.

• Healthcare or clinical NLP experience, especially with FHIR, C-CDA, clinical terminologies or HIPAA-regulated data.
• Fine-tuning, distillation or preference optimization of LLMs at meaningful scale.
• Built MCP servers or multi-agent systems used in production.
• Inference optimization: quantization, speculative decoding, batching, GPU cost management.
• Designed and executed pre-training of transformer models.
• Open-source contributions, publications or talks in ML, NLP or LLM systems.
• Experience as the first or founding ML engineer at a growth-stage company.
• MS or PhD in a quantitative field.

• Architect our LLM platform. Design the shared infrastructure every AI feature at Zus runs on: model gateway and routing across frontier and open-weight models, prompt and version management, caching, cost and latency controls, and PHI-safe guardrails.
• Build agentic systems. Ship tool-using agents that reason over a patient's longitudinal record by summarizing histories, reconciling medications, surfacing care gaps and answering clinician questions with cited evidence. Use patterns like MCP servers, function calling and multi-step planning.
• Own retrieval over clinical data. Build RAG and hybrid search over FHIR resources and unstructured notes: chunking and embedding strategies for clinical text, vector and graph retrieval, reranking, and grounding so every answer traces back to its source document.
• Make evaluation a first-class system. Stand up offline and online evals: golden datasets with clinical reviewers, LLM-as-judge pipelines, regression suites in CI, and production monitoring for hallucination, drift and safety.
• Adapt models when prompting isn't enough. Fine-tune and distill (LoRA/PEFT, SFT, preference optimization) for clinical extraction, coding and normalization tasks where smaller, cheaper, specialized models win.
• Run AI/ML in production. Build CI/CD for models and prompts, inference serving, observability and human-in-the-loop feedback loops that turn user corrections into better models.
• Prototype with users. Work directly with providers, product managers and engineers to find high-leverage problems, ship prototypes fast, and iterate on real usage.
• Set the bar. Lead design reviews, write the docs and standards others build on, mentor engineers, and keep Zus current as the AI landscape moves.

• Competitive compensation that reflects the value you bring to the team, and a combination of cash and equity
• For full-time permanent roles, robust benefits that include health insurance, wellness benefits, 401k with a match, and unlimited PTO.
• Opportunity to work alongside a passionate team that is determined to help change the world (and have fun doing it)

Please Note: Research shows that candidates from underrepresented backgrounds often don’t apply unless they meet 100% of the job criteria. While we have worked to consolidate the minimum qualifications for each role, we aren’t looking for someone who checks each box on a page; we’re looking for active learners and people who care about disrupting the current healthcare system with their unique experiences.

We do not conduct interviews by text nor will we send you a job offer unless you've interviewed with multiple people, including the Director of People & Talent, over video interviews. Job scams do exist so please be careful with your personal information.

Zus Health is an AI-native healthcare data platform that brings fragmented patient information together to make it complete, current, and actionable. Through our nationwide data network, platform, and intelligent workflows, we help providers, value-based care organizations, payers, life sciences, and digital health companies drive business results, strengthen provider trust, and improve operational efficiency. Founded in 2021 by Jonathan Bush, co-founder and former CEO of athenahealth, Zus is building the foundation for a more connected, precise, and effective healthcare system.