Demonstrated improvements in product coverage, test signal quality, automation reliability, or testing efficiency within medium to large engineering teams Experience in leveraging data and quality metrics to identify trends, prioritize coverage gaps, and drive product quality improvements Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews) Experience in building reusable testing or evaluation frameworks adopted by multiple engineering teams Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements) Experience combining human evaluation with automated model-based evaluation and validating the reliability of the evaluation results Familiarity with frontier AI model evaluation practices, including weighted scoring frameworks for assessing model quality and performance Experience evaluating generative text, image, voice, or multimodal models, including nondeterministic behavior and model regressions Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies Experience partnering with AI research teams to translate model capabilities and limitations into product evaluation criteria
$138,000/year to $191,000/year + bonus + equity + benefits