Machine Learning Researcher

Recrew AI; Hyderabad; On-site; INR 1,800,000 - 3,400,000; Full time

Benefits offered by this job

Direct impact on frontier AI research

Cutting-edge research collaboration

High-growth environment with strong MO

Job summary

Recrew AI in Hyderabad, India is seeking an ML Researcher to advance how frontier AI systems are measured. You will design novel benchmarks and evaluation datasets that go beyond current standards, translating cutting-edge research into scalable evaluation frameworks used by leading labs.

The role collaborates with the ML Lead and project managers to deliver implementable research outputs, publish insights, and drive continuous improvement in benchmark design and model assessment.

Qualifications

• Experience designing AI benchmarks and evaluation datasets from scratch.
• Expertise in deep learning, NLP, or computer vision with evaluation methods.
• Ability to translate academic research into practical frameworks.
• Experience critiquing existing industry benchmarks for gaps.
• Proficient technical writing and documentation skills.

Responsibilities

• Track AI research to identify gaps in evaluation practices.
• Design novel benchmarks and datasets across coding and non-coding domains.
• Produce benchmark proposals with metrics and testing guidelines.
• Translate insights from papers into practical designs.
• Collaborate with the ML Lead and project managers to ensure usability.
• Iterate benchmarks based on feedback and new findings.
• Deliver monthly research summaries on AI trends.

Skills

AI benchmarks design

Deep learning / NLP / CV

Literature reviews

Benchmark critique

Technical documentation

Job description

Job Description:

Role: ML Researcher

Function: Machine Learning Research / AI Evaluation

Location: Hyderabad, India

Type: Full-time

Industry: Information Technology & Services, Computer Software, Artificial Intelligence

About Company

The company is a Series A GenAI data startup headquartered in Mountain View, CA. It runs the post-training and production layer for the world's leading AI labs and enterprises.

Founded in 2024, it has grown rapidly — 10x last year — and serves a majority of the Magnificent 7 and top frontier labs globally. Its global network of 1M+ domain experts powers proprietary platforms including Helix (a hybrid eval suite) and EnterpriseOS (a scaled ops automation platform).

Backed by A91 Partners, Susquehanna International Group, and Prosus Ventures, with clients including Google DeepMind and Snowflake.

Position Overview

This role sits at the core of the company's research function, responsible for advancing how frontier AI systems are measured and understood. The ML Researcher will design novel benchmarks and evaluation datasets that go beyond existing industry standards, translating cutting-edge academic research into actionable frameworks used by the world's top AI labs. The work directly influences how leading models are trained, tested, and improved at scale.

Role & Responsibilities

• Continuously track AI research publications, conferences, and emerging model architectures to identify gaps and opportunities in existing evaluation practices
• Design novel AI benchmarks and evaluation datasets across coding and non-coding domains, defining metrics that provide actionable insights into model capabilities and limitations
• Produce high-quality benchmark proposal documents covering problem definition, design structure, evaluation metrics, novelty rationale, and testing guidelines
• Translate insights from the latest AI papers into practical benchmark and dataset designs aligned with organisational and client priorities
• Collaborate with the ML Lead, pipeline-focused MLEs, and project managers to ensure research outputs are implementation-ready and meet project goals
• Iterate on existing benchmarks and datasets based on post-implementation feedback, new research findings, and evolving client use cases
• Deliver monthly research summaries on emerging AI trends impacting evaluation methodology and benchmark design

Must Have Criteria

• Proven experience designing AI benchmarks or evaluation datasets from the ground up, covering benchmark structure and metric definition
• Deep expertise in at least one of: agent frameworks, deep learning, NLP, or computer vision — with working knowledge of evaluation methodology in that domain
• Demonstrated ability to conduct in-depth literature reviews and translate academic research into practical, implementable frameworks
• Experience assessing and critiquing existing industry benchmarks for relevance, coverage gaps, and methodological limitations
• Strong technical documentation skills — prior experience producing requirement documents, research reports, or whitepapers that are implementation-ready

Nice to Have

• Familiarity with reinforcement learning environments or RLHF pipelines and their evaluation challenges
• Prior work at or in collaboration with AI labs, frontier model teams, or AI data/evaluation companies
• Published research or conference contributions in AI evaluation, benchmarking, or related ML domains
• Experience designing datasets for LLM post-training, fine-tuning, or model alignment

What We Offer

• Direct impact on how the world's leading AI labs measure and improve frontier models
• Collaboration with a research and engineering team working at the cutting edge of LLM post-training and evaluation infrastructure
• High-growth environment backed by leading investors with significant runway and momentum
• Access to a global network of 1M+ domain experts and proprietary evaluation platforms (Helix and EnterpriseOS)