Member of Technical Staff, LLM & Production

• Build and own end-to-end evaluation suites for LLM capabilities: task specs, grading logic, and the dashboards researchers check every day.
• Run the eval harness as a production service — sandboxed execution, tool calling, multi-turn and agentic tasks, deterministic replay, and caching — so a full evaluation pass is cheap and gives the same answer twice.
• Own the data pipeline from source to training: collection, filtering, dedup, labeling workflows, synthetic data generation, and the quality gates that catch contamination and distribution drift before anything reaches a run.
• Build RL training environments – reward functions, simulators, verifiers, task generators — and keep them stable as they fan out across many workers.
• Work side by side with researchers: turn a hypothesis into a runnable eval or environment within days, then help make sense of what comes back.
• Pull capabilities out of the product layer and into the model. Find where product teams lean on long prompt instructions or hand-written skills, build the evals and environments that teach the model to do those things natively, and retire the workarounds once it can.
• Measure everything: eval regressions, environment health, throughput, cost — and make it obvious when a metric moved for the wrong reason.
• Raise the bar for research code across the team: tests, versioned datasets and evals, and clean boundaries between training, inference, and evaluation.

• 3+ years in software engineering, including 1-2 years hands-on with LLM training, evaluation, or data infrastructure.
• Deep Python fluency and real distributed-systems experience: you've chased failures across a fleet of machines, dealt with unreliable workers, and know where the compute budget actually goes.
• You've built evaluation infrastructure, not just run benchmarks — designing tasks, graders, and the plumbing around them.
• Working knowledge of SFT and RL for language models.
• Production instincts: you write tests, insist on reproducibility, and treat research tooling as something others depend on — because they do.
• Comfortable in ambiguity next to researchers: handed a half-formed idea, you can make it measurable.

• Agentic evaluation experience: tool use, code execution, sandbox environments, multi-turn tasks.
• Hands-on with training frameworks and inference stacks (e.g., PyTorch, JAX, vLLM, SGLang, or in-house equivalents).
• Experience running data-labeling operations or collecting human preferences at scale.
• Contributions to open-source evaluation frameworks or public benchmarks.
• Previous work on verifiers, code sandboxes, or simulation environments.

The US base salary range for this full-time position is between $180,000 - $450,000 annually.

The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.

Hark is an artificial intelligence company building advanced, personalized intelligence. One that is proactive, multimodal, and capable of interacting with the world through speech, text, vision, and persistent memory.

We're pairing that intelligence with next-generation hardware to create a universal interface between humans and machines. While today's AI largely operates through chat boxes and decade-old devices, Hark is focused on what comes next: agentic systems that interact naturally with people and the real world.

To get there, we're developing multimodal models and next-generation AI hardware together - designed from the ground up as a single, unified interface for a new era of intelligent systems.