Data Scientist II, M365 Copilot

DataJobs; Washington (District of Columbia); On-site; USD 102,000 - 219,000; Full time

Job summary

Microsoft in Washington, DC is seeking a research-focused role on the Turing Team to build model-driven capabilities for Copilot, emphasizing orchestrator reasoning, evaluation, and A/B testing for LLM and agentic product performance. This onsite role focuses on deep learning, NLP, multimodality, and model optimization.

You’ll work on evaluation frameworks, model experiments, diagnosis, prototype AI in containerized environments (AI Foundry, LangChain), and retrieval-augmented generation

Qualifications

• Bachelor’s degree in relevant field plus 2+ years of data-science experience.
• Master’s degree plus 1+ year data-science or consulting experience.
• Doctorate in the same fields.
• Equivalent experience accepted.

Responsibilities

• Define evaluation metrics and frameworks for AI/LLM systems and agentic product performance.
• Experiment with models before selection for tasks within product domains.
• Diagnose issues in AI/LLM systems and identify opportunities to improve performance.
• Prototype and develop AI models in containerized environments (AI Foundry, LangChain).
• Develop and deploy retrieval-augmented generation (RAG) workflows.
• Collaborate with senior members to refine LLM systems and models.
• Take model-driven features from concept to production across development, evaluation, metrics, and A/B testing.

Skills

Data Science

NLP

Machine Learning

AB testing

LangChain

Education

Bachelor’s degree in Data Science/Related field

Master’s degree in Data Science/Related field

Doctorate in Data Science/Related field

Tools

LangChain

AI Foundry

Job description

As part of the Microsoft Turing Team, you will help build and deploy model-driven capabilities for Microsoft 365 Copilot, with an emphasis on improving orchestrator reasoning, evaluation, and A/B testing for LLM and agentic product performance. This onsite role in Washington, DC is focused on deep learning, NLP, multimodality, and model optimization, including retrieval-augmented generation workflows.

What you’ll work on

• Define evaluation metrics and frameworks for somewhat complex AI and large language model (LLM) systems, including agentic product performance.
• Experiment with models before selection (for example, comparing LLMs and prompting approaches) for tasks within product or solution domains.
• Diagnose basic issues in AI and LLM systems, identify opportunities to improve performance within your domain area, and support iteration.
• Prototype and develop AI models in containerized environments, including AI Azure Foundry and LangChain, to operationalize and validate scalable solutions for the team.
• Develop and deploy systems for retrieval-augmented generation (RAG) workflows, including assisting with the selection of data sources, retrieval systems, and generative models to help ensure output quality.
• Collaborate with senior members to assess and refine LLM systems and models.

How the role drives impact

• Take model-driven features from concept to production across model development, evaluation, metrics, and A/B testing.
• Contribute to evolving the orchestrator so it can reason more effectively and respond faster, including support for new capabilities in multi-modality and agent-based systems.

Required qualifications

• Bachelor’s degree in Data Science, Mathematics, Statistics, Econometrics, Economics, Operations Research, Computer Science, or a related field, plus 2+ years of data-science experience (for example, managing structured and unstructured data, applying statistical techniques, and reporting results).
• OR Master’s degree in the same fields, plus 1+ year(s) of data-science experience or consulting experience.
• OR Doctorate in the same fields.
• OR equivalent experience.
• Ability to meet Microsoft, customer and/or government security screening requirements for this role.
• Required to pass the Microsoft Cloud background check upon hire or transfer, and again every two years thereafter.

Preferred qualifications

• Bachelor’s degree with 5+ years of data-science experience.
• Master’s degree with 3+ years of data-science experience.
• Doctorate with 1+ year(s) of data-science experience.
• OR equivalent experience.

Technologies: AI Azure Foundry, LangChain

Salary range: USD 102,100 - 219,200 per year