5+ years of experience in data and AI-related fields such as AI engineering, software development, ML engineering, data science, or QA roles. Eagerness and ability to learn new skills and solve dynamic problems in an encouraging and expansive environment. Strong Python skills. Hands-on experience with AI evaluation techniques, such as Golden datasets, LLM-as-a-Judge, or rubric-based scoring. Experience with different LLM ecosystems (OpenAI, Anthropic, Gemini, etc.), RAG pipelines, vector databases (e.g., Pinecone, FAISS, Milvus, PostgreSQL).
Proficiency in SQL and experience with at least one major data analytics platform, such as Hadoop, Spark, or Snowflake. Experience with CI/CD or release validation workflows. Experience working with data science teams on insights generation leveraging LLMs. Strong time management skills with the ability to collaborate across multiple teams. Able to balance competing priorities, long-term projects, and ad hoc requirements. Ability to work in a fast-paced, dynamic, constantly evolving business environment.
Hands-on experience with Langfuse or similar tools for LLM observability. Comfortable working with product/domain experts to translate fuzzy correctness criteria into measurable rubrics or metrics. B.S. degree in Computer Science/Engineering, or equivalent work experience