Staff Applied Scientist, AI Evaluation & Metrics

Datadog; New York (NY); Hybrid; USD 140,000 - 180,000; Full time

Job summary

Datadog is seeking a Staff Applied Scientist to enhance their AI-driven Dashboards product, focusing on quality evaluation and improvement of AI systems at scale.

The role involves leading teams, defining metrics, and driving initiatives that ensure optimal performance and relevance of tools and insights. Candidates should possess a solid background in machine learning and significant experience in applied science.

Join Datadog's hybrid workplace culture that promotes collaboration and creativity!

Qualifications

• 10+ years of relevant engineering or applied science experience, including time as a technical lead.
• Proven track record of leading ML or GenAI initiatives from research through production.
• Significant experience with evaluation, experimentation, or measurement of ML systems at scale.

Responsibilities

• Own the evaluation strategy for Dashboards and sister teams.
• Build the eval datasets, golden traces, and regression harnesses.
• Drive improvements to retrieval relevance and tool-selection accuracy.
• Provide technical leadership through design reviews and mentorship.

Skills

Machine Learning (ML)

Data Evaluation

Technical Leadership

Data Analysis

Education

BS/MS/PhD in a scientific field or equivalent experience

Job description

Datadog is seeking a Staff Applied Scientist to enhance their AI-driven Dashboards product, focusing on quality evaluation and improvement of AI systems at scale.

The role involves leading teams, defining metrics, and driving initiatives that ensure optimal performance and relevance of tools and insights. Candidates should possess a solid background in machine learning and significant experience in applied science.

Join Datadog's hybrid workplace culture that promotes collaboration and creativity!