Jobs Ask AI Pay Interviews Companies CompareSign in

Wait…

We read every job from the company's own job board. You apply on the company site.Jobs Job types Internships New graduate jobs Companies Pay Reported pay Check an offer Add your pay Interview questions Add your interview questions Compare with LinkedIn Job sites compared Pay transparency laws API for developers Monthly report Companies we cover Make a portfolio page For companies: find people About Privacy

Human Data Evals Lead (Remote/US/LATAM)

Anyone AI, Argentina - Fully Remote +5

Posted pay

Pay not posted

Location: Argentina - Fully Remote +5
Work type: Remote
Level: Senior
Experience: 5+ years
Posted: Jun 19, 2026
Last check: 41 min ago
Job type: ML / AI engineering

Apply on the Anyone AI site Make a résumé and a cover letter for this job Sign in to save

The Anyone AI site opens in a new tab.

AA

Anyone AI

About Anyone AI: funding, hiring and pay

Check my fit

Compare your résumé with this job. See where you match, what is missing and which lines to change.

Sign in to check your fit

Make a résumé and a cover letter for this job

Paste your résumé, or use the résumé on your profile.
Get the résumé as a PDF and the letter as text. We use only the facts in your résumé.
Apply on the Anyone AI site.

Open the apply kit

The role

Reports to: CEO

Owns: data proposals, sample development, quality, and pilot delivery

Location: Remote / Latam / US

You will own Anyone AI’s data initiatives and proposals to AI labs, from the data proposal or responding to requests, through pilot delivery. You own how we build proposals and develop the sample packages and benchmarks: frontier-grade packages across reasoning, coding, agents, and tool use, multi-modal and others, produced in collaboration with subject-matter experts, with expert-verified ground truth, multi-model headroom results, and QC that survives buyer-side scrutiny.

You are the person who designs the sample that demonstrates our quality, converts pilots into production engagements. On a small team, this is the operational center of the Human Data Division.

What you will do

Proposals & requests. Study public benchmarks and eval targets, and turn them into proposals and sample packages that demonstrate capability and win the work. Respond to lab data requests and pilots.
Sample & benchmark development. Design and build the sample packages, working with subject-matter experts. Every package meets the bar of our current sample set:
Expert-verified, exact-match-checkable ground truth and gold reasoning trajectories.
Multi-model evaluation showing real headroom, and proof the task discriminates the model, not just that it's hard.
Rigorous QC structure: calibration layers, severity-weighted rubrics, deterministic verifiers, evidence maps, etc.
Subject-matter experts. Recruit, brief, calibrate, and review a pool of experts across coding, agentic/tool-use, and STEM/reasoning. Raise their output to our standard and keep it there; be the arbiter of what "correct" and "frontier-difficulty" mean.
Lab relationships. Be a direct point of contact for lab partners on Slack and calls, with support from the CEO and the wider team. Keep senior lab contacts informed, surface what they actually need, and pull in the CEO and subject-matter experts when the conversation calls for it.
Pilot delivery. Own pilots end to end: scoping, SOW, staffing, production, QC, and delivery. Nothing ships before it's lab-ready, and nothing comes back rejected as "not frontier-level" without us already knowing why.

What you need

Experience

Originated data or benchmark proposals for AI labs, translated eval targets into sample tasks that demonstrate capability, and owned the engagement through delivery.
Deep evaluation and quality expertise: LLM benchmarking, with real strength in code-model evaluation.
Built QC processes and artifact standards that met enterprise or lab requirements, and set a quality bar a team of experts was held to.
Thrives in ambiguous, fast-moving environments where the rules are still being written, and delivers under pressure.

Qualifications

5+ years in technical delivery, quality, or program management, with recent experience in AI/ML data, model evaluation, or benchmarking.
Hands-on experience delivering data or evaluation work to AI labs or enterprise ML teams, scoping through delivery.
Working fluency with how frontier models are evaluated: benchmarks, rubrics, pass rates, headroom, and what makes a task discriminate a model.
Proven people/vendor leadership, you've recruited, calibrated, and held a team or expert pool to a quality standard.
Fluent English. Spanish is a nice to have.

Read the job posting on the Anyone AI site

Similar jobs at other companies

Human Data Manager
xAIPalo Alto, CA
- Software engineering
- Manager
$100K–$186K / yr
Posted Aug 12, 2026
WG
Project Perseus | Data Quality Analyst - Spanish Speakers (Human-in-the-Loop AI)
Welo GlobalCalifornia +11
- Onsite
- Data and business analysis
$38 / hr
Posted Apr 22, 2026
Senior AI Researcher
LatamCentBrazil
- Senior
- Hybrid
- Research
$7K–$8K / mo
Posted Oct 8, 2026
WG
Project Perseus | Data Quality Analyst - Italian Speakers (Human-in-the-Loop AI)
Welo GlobalCalifornia +10
- Onsite
- Data and business analysis
$32 / hr
Posted Apr 22, 2026
P
Human Data Quality Analyst, AI Business
Prolific Mexico +3
- Hybrid
- Data and business analysis
Pay not posted
Posted Jan 6, 2026
WG
Project Perseus | Data Quality Analyst - French Speakers (Human-in-the-Loop AI)
Welo GlobalCalifornia +11
- Onsite
- Data and business analysis
$32 / hr
Posted Apr 22, 2026

Source: the Anyone AI Ashby job board. We last checked this job 41 min ago. We first saw it 8 days ago. A job on a job board does not always mean that the team interviews people now. We found this job board in a public list. We did not yet compare it with the careers page of the company.

Interview questions

No one has reported an interview at Anyone AI yet. Did you interview there? Add the questions.

AA

Anyone AI

Open tech jobs: 49
With posted pay: 88%

About Anyone AI: funding, hiring and pay