QA Engineer

What you own

AI Evaluation. You personally design and execute the evaluation framework for LLM and agentic AI outputs across Foundry, Agent Builder, and client-deployed instances. You write the assertions, define the behavioral contracts, and own regression baselines for model behavior. Standard QA methods break down here: you cannot write a deterministic assertion for whether an AI accurately summarized a 200-page loan agreement. You need to think in distributions and confidence intervals, and you need to build tooling that does too.

Test Coverage. You write and maintain the automated test suite: end-to-end, integration, and regression coverage for backend APIs, document ingestion pipelines, AI inference workflows, and frontend surfaces. You own performance and load testing for latency-sensitive inference paths. You set up and enforce quality gates in CI/CD pipelines. When a bug surfaces in production, you are in the triage, you write the reproduction case, and you own the regression test that prevents it from coming back.