Technical Lead, AI Benchmarking and Evaluation Research (Remote, ROU)
CrowdStrike, Romania - Remote
The role
The CrowdStrike OCAIO team is looking for a senior technical lead to own how we measure AI models and agentic systems that perform cybersecurity tasks. This is a hands-on leadership role. You will lead a small team of researchers and you will personally help design, build and defend the benchmarks they ship.
Our mission is to establish rigorous, reproducible standards for how well AI and agentic systems perform real security work: malware analysis, reverse engineering, threat intelligence reasoning, alert triage, investigation and response. We build our ground truth with security subject matter experts, not with other models, and we score agents on what they actually did, not only on what they said.
As the Senior Lead, you define what "good" looks like for an AI system operating in a security workflow, you build the datasets, rubrics and judges that encode that definition, and you run the standing scorecards that engineering, product and research teams use to decide what ships. You set the technical bar and you meet it yourself.
What you need
- Hands-on cybersecurity expertise in at least one of: malware analysis and reverse engineering, incident response and threat hunting, detection engineering, SOC operations. You can label ground truth yourself and judge whether an expert's label is right.
- Demonstrated experience building evaluations, benchmarks or datasets for AI or LLM systems, with shareable artifacts (papers, repositories, internal benchmarks with measured adoption).
- Strong Python and data tooling skills. You write production-quality evaluation code, harnesses and analysis, and you review others' code.
- Practical experience with LLMs and agentic systems: tool use, multi-step planning, sandboxing, and how to instrument and trace agent behavior.
- At least 2 years leading technical people (as a team lead, tech lead or manager), with a record of growing researchers and engineers while remaining hands-on.
- Ability to define, design and standardize evaluation methodologies and reproducible testing pipelines, and to defend them under scrutiny.
- Broad knowledge of the cybersecurity landscape: attack techniques, defensive controls, and the analyst workflows that AI is meant to augment.
- Exceptional written and spoken communication. You can present measured findings and their limitations to executives and to researchers.
- Proven experience utilizing AI technologies to enhance decision-making, streamline workflows and processes, improve efficiency and drive business outcomes.