Data Scientist, Agent
Lovable, Stockholm +1
What you will do
- Define and own the metrics for agent quality: success, completion, error rates, and the behaviors that drive them.
- Build the eval systems and experiment framework that decide whether an agent change ships, like an A/B-tested rollout that catches a change increasing errors before it reaches everyone.
- Turn agent traces and telemetry into concrete fixes, working directly with the agent engineering team.
- Build the tooling and agents that produce these evaluations continuously as the agent evolves.
- Set the bar for how we judge agent behavior where there is no answer key to check against.
What you need
- A data scientist who wants to make an AI agent measurably better, not just report on it. You own agent quality metrics and drive them up.
- Experience or strong interest in LLM evaluation and observability: building evals, scoring outputs, tracing agent behavior, and catching regressions.