Remote AI Researcher: Datasets, Benchmarks & Evaluation

Vetto; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Stock option plan

Remote-first work

Periodic on-sites

Job summary

Vetto, a remote-first AI data infrastructure company, is seeking an AI Researcher to design datasets, run experiments, and analyze model failures. You will work with top AI labs and enterprise partners to turn findings into data, benchmarks, and training insights.

You should be proficient in Python, enjoy empirical research, and thrive in a fast-moving team. You will publish results internally and contribute to reports, open benchmarks, and potential papers while staying comfortable with

Qualifications

• Experience doing empirical research or similarly open-ended technical work with machine learning systems.
• Hands-on experience designing and running agentic systems for complex work, such as coding agents and automated research pipelines.
• Good experimental judgment: you can form a hypothesis, choose useful baselines and metrics, run a careful experiment, and make sense of noisy results.
• Strong Python and data analysis skills, plus enough software engineering ability to prototype your own research workflows.
• The independence to take an ambiguous question about a model and turn it into a dataset, benchmark, experiment, or other concrete contribution.
• Rigorous analytical thinking — you look at the underlying evidence, notice subtle failure modes, and question results that seem too simple or too good to be true.
• Excellent written and verbal communication with researchers, engineers, customers, and non-specialists.
• Comfort with ambiguity, fast iteration, changing priorities, and occasional forward-deployed work with partner teams.

Responsibilities

• Design and run experiments to understand model behavior, test new ideas, and determine whether a dataset or training intervention actually works.
• Use fine-tuning and post-training experiments when useful to validate research hypotheses and measure improvements in the capabilities we care about.
• Create datasets, benchmarks, agent environments, rubrics, and evaluation methods for problems that are not well measured today.
• Work directly with AI labs and enterprise partners to turn open-ended model-development goals into concrete research and data projects.
• Dig into model outputs and trajectories to understand systematic failure modes and separate model limitations from problems in the data, grader, or evaluation setup.
• Write analysis code and build lightweight tools or prototypes for your own work, with support from engineers when a workflow needs to become reliable infrastructure.
• Share what you learn through clear technical reports, internal discussions, benchmarks, and research papers.

Skills

Empirical research

Agentic systems design

Experimental judgment

Python

Data analysis

Software engineering

Independent work

Analytical thinking

Communication

Ambiguity tolerance

Job description

Vetto, a remote-first AI data infrastructure company, is seeking an AI Researcher to design datasets, run experiments, and analyze model failures. You will work with top AI labs and enterprise partners to turn findings into data, benchmarks, and training insights.

You should be proficient in Python, enjoy empirical research, and thrive in a fast-moving team. You will publish results internally and contribute to reports, open benchmarks, and potential papers while staying comfortable with