Staff Applied Scientist — AI Agent Interfaces & Evaluation

Datadog; New York (NY); On-site; USD 276,000 - 345,000; Full time

Benefits offered by this job

Stock Options

Professional Development

Competitive Benefits

Job summary

Datadog is seeking a Staff Applied Scientist in New York to own evaluation strategies for AI agent integrations. The role involves defining metrics, building datasets, and collaborating with AI engineers to enhance tool selection and retrieval relevance.

The ideal candidate will have 10+ years of experience, a high level of expertise in machine learning, and a proven track record in product-driven environments. Competitive salary and benefits package offered.

Qualifications

• 10+ years of relevant engineering or applied science experience.
• Proven track record of leading ML initiatives in product-driven environments.
• Experience with measurement of ML systems at scale.

Responsibilities

• Own the evaluation strategy for AI agent integrations.
• Build eval datasets and regression harnesses.
• Drive improvements to tool-selection accuracy and retrieval relevance.

Skills

Machine Learning

Evaluation and Measurement

Technical Leadership

Cross-Functional Collaboration

Education

BS/MS/PhD in a Scientific Field

Tools

Data Analysis Tools

Job description

Datadog is seeking a Staff Applied Scientist in New York to own evaluation strategies for AI agent integrations. The role involves defining metrics, building datasets, and collaborating with AI engineers to enhance tool selection and retrieval relevance.

The ideal candidate will have 10+ years of experience, a high level of expertise in machine learning, and a proven track record in product-driven environments. Competitive salary and benefits package offered.