AI Engineer
Momentic, San Francisco
What you will do
- Fine-tune models for long-running agents, including agents that investigate whether test failures come from infrastructure, recent code changes, or test authorship.
- Build and curate training and evaluation datasets from production workflows, using failure patterns to identify opportunities for improvement.
- Develop LLM evaluations and observability that connect offline results with production behavior and help the team verify improvements.
- Tune prompts, context, and model behavior to improve task success, reliability, latency, token usage, and cost.
- Design repeatable experiments and help publish benchmarks that demonstrate our core agents’ performance on testing and browser workflows.
- Own technical projects from problem definition through delivery, working with the engineering team to make practical tradeoffs and ship improvements.