Applied AI Engineer
Mpathic2, Remote
What you need
You'll Thrive in This Role If You
- Are a backend engineer first, and have carried LLM features through what comes after the demo: evals, regressions, cost, latency, outages, and audits.
- Enjoy building the platform other people build on, and measure success by how easily and safely the rest of the team ships.
- Treat a prompt change with the same rigor as a code change, and care deeply about reliability, reproducibility, and auditability.
- Understand modern AI evaluation concepts (LLM-as-judge, red teaming, regression evals) and how to turn them into automated release gates.
- Are comfortable with high-stakes, sensitive subject matter such as healthcare, safety, and wellbeing, and the data handling it demands.
- Can translate between researchers, clinicians, and engineers, turning a working prototype into a production service without losing what made it work.
- Communicate crisply through decision records, design docs, and diagrams, and can align people around a standard without relying on authority.
- Have shipped real LLM-powered products in production, shown through what you have built rather than years on a résumé.
Core experience
- 5+ years of backend software engineering, including multi-tenant SaaS services and APIs in production.
- Production experience building on commercial LLM APIs (for example Anthropic, OpenAI, AWS Bedrock, or Azure OpenAI), including prompt management, structured outputs, and tool use.
- Hands-on experience designing evaluation pipelines for LLM output and wiring them into CI/CD as release gates.
- Strong Python; NodeJS is a plus.
- AWS, containers (ECS or similar), and infrastructure as code (Pulumi, Terraform, or CDK).
- Observability tooling such as OpenTelemetry, Datadog, or similar, and the delivery toolchain (version control, feature flags, CI, and an issue tracker such as Linear).
- Working knowledge of application security: secrets management, access control, and handling of sensitive data.
Nice to have
- Experience in regulated domains such as healthcare or life sciences, including HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11 validated systems.
- Experience building an internal developer platform or paved road used by other teams.
- Experience with LLM-as-judge evaluation, red teaming, or conversational analytics.
- Experience with training or fine-tuning models from scratch.
In your first 60-90 days you'll
- Build a deep understanding of the platform, its LLM-backed services, and how a model or prompt change moves from prototype to production today.