Linguist - Speech & Conversational AI

Besimple AI, San Mateo, California +1

What you will do

Evaluate speech and conversational quality. Assess pronunciation, intelligibility, prosody, pacing, tone, and turn-taking in human recordings and model outputs. Explain why a response sounds natural, awkward, overly scripted, or inappropriate for the situation.
Build quality standards and evaluation rubrics. Translate customer requirements and feedback into clear criteria, examples, and decision rules. Define what reviewers should accept, flag, or reject, including how expectations differ across conversational roles and tasks.
Screen and support speech contributors. Help design auditions, evaluate recordings, and give actionable feedback on delivery. Identify contributors who can sustain natural conversations while following a scenario’s requirements.
Create and review linguistic annotations. Develop guidelines and reference examples for transcripts, speaker turns, speech delivery, and conversational behavior. Review human and model-generated labels, resolve ambiguous cases, and improve consistency across reviewers.
Improve conversational tasks and model evaluation. Work with researchers to refine scenarios, instructions, and turn-level rubrics. Identify unclear wording or grading criteria, and distinguish real model failures from problems in the task itself.

Besimple AI, San Mateo, California +1

Evaluate speech and conversational quality. Assess pronunciation, intelligibility, prosody, pacing, tone, and turn-taking in human recordings and model outputs. Explain why a response sounds natural, awkward, overly scripted, or inappropriate for the situation.
Build quality standards and evaluation rubrics. Translate customer requirements and feedback into clear criteria, examples, and decision rules. Define what reviewers should accept, flag, or reject, including how expectations differ across conversational roles and tasks.
Screen and support speech contributors. Help design auditions, evaluate recordings, and give actionable feedback on delivery. Identify contributors who can sustain natural conversations while following a scenario’s requirements.
Create and review linguistic annotations. Develop guidelines and reference examples for transcripts, speaker turns, speech delivery, and conversational behavior. Review human and model-generated labels, resolve ambiguous cases, and improve consistency across reviewers.
Improve conversational tasks and model evaluation. Work with researchers to refine scenarios, instructions, and turn-level rubrics. Identify unclear wording or grading criteria, and distinguish real model failures from problems in the task itself.