Experience That Stands Out – Strong candidates may also bring experience in one or more of the following areas:
Blue Yonder is the AI company for supply chain. Our platform helps the world's leading companies plan, fulfill, deliver, and operate more resilient supply chains across complex global networks.
We are building toward the autonomous supply chain: intelligent systems that can understand operational context, reason through tradeoffs, use tools, collaborate with people, and take action across real supply chain workflows.
Autonomy Labs' mission is to find the fastest possible path to an autonomous supply chain.
We build LLM agents, learning systems, model training pipelines, evaluations, simulations, and decision-making systems for some of the hardest problems in global supply chain. The work spans LLMs, agentic workflows, tool use, software automation, evaluation, post-training, optimization, and production engineering.
In short, we are having a lot of fun.
Your Mission
We are looking for a deeply technical Director of Model Behavior & Evaluation Systems to own the behavioral quality system for Blue Yonder's LLM agents.
Our agents are not generic chatbots. They are being trained to operate supply chain software: querying state, calling APIs, interpreting operational context, proposing actions, handling exceptions, asking for missing information, and helping users make decisions in complex enterprise environments.
Your mission is to make model behavior a product-quality system, not a collection of dashboards. You will define what "good" means for agents operating supply chain workflows, establish the release gates that determine when behavior is ready to ship, and build the feedback loops that turn traces, customer feedback, SME review, telemetry, red-teaming, and eval failures into model improvements.
This is a director-level technical leadership role. You will lead through systems, standards, people, and decisions. You should be close enough to model traces, evals, post-training, tool use, and customer workflows to make strong technical calls, while operating at the level of ownership boundaries, launch authority, roadmap sequencing, and team building.
The stack is real and close to the work. You should expect to operate around Python, PyTorch, Hugging Face Transformers and Datasets, NVIDIA NeMo RL, OpenAI Agents SDK, Langfuse, LLM evaluation harnesses, tool-calling traces, model checkpoints, reward and preference data, synthetic scenarios, experiment reports, and production observability. You do not need to be the person implementing every pipeline, but you do need the technical depth to challenge designs, read artifacts, understand failure modes, and guide senior engineers toward better systems.
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, protected veteran status, or any other legally protected characteristic.
Our Values
If you want to know the heart of a company, take a look at their values. Ours unite us. They are what drive our success – and the success of our customers. Does your heart beat like ours? Find out here: Core Values
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.
Pay not posted
Posted 27 days ago
XplorRemote, Oregon
$275K–$290K / yr
Posted 11 days ago
Judgment LabsSan Francisco
Pay not posted
Posted Aug 13, 2026
$160K–$240K / yr
Posted Oct 13, 2026
Source: the JDA Workday careers site. We last checked this job 1 h ago. We first saw it 1 h ago. A job on a job board does not always mean that the team interviews people now. We found this job board in a public list. We did not yet compare it with the careers page of the company.
What Makes This Role Different? Model behavior is product behavior.
For supply chain agents, the difference between a useful system and an unsafe or frustrating one often comes down to judgment: when to ask for clarification, when to call a tool, when to explain a tradeoff, when to surface uncertainty, when to escalate to a human, and when not to act.
This role owns that judgment as an operating system: the standards, evidence, release gates, teams, and feedback loops that determine whether our agents are ready to operate in real customer workflows.
HumanoidUS, San Diego, CA
$300K–$450K / yr
Posted Oct 9, 2026
DatadogNew York, New York +1
$192K–$240K / yr
Posted Sep 18, 2026
No one has reported an interview at JDA yet. Did you interview there? Add the questions.