Research Engineer, AI Evaluations
Karya, Bengaluru, India +1
The role
As a Research Engineer, you will design, build and run these evaluations end to end, across eval design, model deployment, statistical analysis and engineering. The work spans ASR, TTS, machine translation and chat, with newer frontiers in agentic and multimodal evaluation.
What you will do
- Benchmark design: curate representative query sets and define rubrics and metrics that score reliably and separate models meaningfully. Choose between human, automated and hybrid evaluation based on reliability, cost and validity trade-offs.
- Expert input: design elicitation and adjudication processes that turn domain expertise into consistent, scalable judgments.