AI Benchmark Architect & Evaluation Researcher

Deccan AI Automation Private Limited; Mountain View (CA); Hybrid; USD 150,000 - 200,000; Full time

Job summary

Deccan AI is seeking an ML Researcher to drive end-to-end AI research, identify emerging directions, and design novel benchmarks and evaluation datasets aligned with organizational priorities. This role bridges theory and practice by translating recent papers into actionable benchmarks for agent frameworks, deep learning, NLP, and other AI paradigms, with strong documentation and clear success criteria.

Collaborate with ML leads and project managers to deliver high-impact research outputs,

Qualifications

• Deep expertise in AI/ML domains including agent frameworks, deep learning, NLP, computer vision, and emerging AI systems.
• Proven experience designing benchmarks and evaluation datasets from the ground up.
• Excellent documentation and communication abilities.

Responsibilities

• Research & literature review: track the latest AI research, conferences, and publications.
• Design novel AI benchmarks and evaluation datasets aligned with organizational priorities.
• Provide clear problem statements, design overviews, evaluation metrics, and testing guidelines.
• Collaborate with ML leads and project managers to align outputs with goals.
• Produce high-impact research outputs with actionable insights.

Skills

AI/ML expertise

Agent frameworks

Deep learning

NLP

Computer vision

Job description

Deccan AI is seeking an ML Researcher to drive end-to-end AI research, identify emerging directions, and design novel benchmarks and evaluation datasets aligned with organizational priorities. This role bridges theory and practice by translating recent papers into actionable benchmarks for agent frameworks, deep learning, NLP, and other AI paradigms, with strong documentation and clear success criteria.

Collaborate with ML leads and project managers to deliver high-impact research outputs,