Data Scientist II: LLM/RAG for Copilot

DataJobs; Washington (District of Columbia); On-site; USD 102,000 - 219,000; Full time

Job summary

Microsoft in Washington, DC is seeking a research-focused role on the Turing Team to build model-driven capabilities for Copilot, emphasizing orchestrator reasoning, evaluation, and A/B testing for LLM and agentic product performance. This onsite role focuses on deep learning, NLP, multimodality, and model optimization.

You’ll work on evaluation frameworks, model experiments, diagnosis, prototype AI in containerized environments (AI Foundry, LangChain), and retrieval-augmented generation

Qualifications

• Bachelor’s degree in relevant field plus 2+ years of data-science experience.
• Master’s degree plus 1+ year data-science or consulting experience.
• Doctorate in the same fields.
• Equivalent experience accepted.

Responsibilities

• Define evaluation metrics and frameworks for AI/LLM systems and agentic product performance.
• Experiment with models before selection for tasks within product domains.
• Diagnose issues in AI/LLM systems and identify opportunities to improve performance.
• Prototype and develop AI models in containerized environments (AI Foundry, LangChain).
• Develop and deploy retrieval-augmented generation (RAG) workflows.
• Collaborate with senior members to refine LLM systems and models.
• Take model-driven features from concept to production across development, evaluation, metrics, and A/B testing.

Skills

Data Science

NLP

Machine Learning

AB testing

LangChain

Education

Bachelor’s degree in Data Science/Related field

Master’s degree in Data Science/Related field

Doctorate in Data Science/Related field

Tools

LangChain

AI Foundry

Job description

Microsoft in Washington, DC is seeking a research-focused role on the Turing Team to build model-driven capabilities for Copilot, emphasizing orchestrator reasoning, evaluation, and A/B testing for LLM and agentic product performance. This onsite role focuses on deep learning, NLP, multimodality, and model optimization.

You’ll work on evaluation frameworks, model experiments, diagnosis, prototype AI in containerized environments (AI Foundry, LangChain), and retrieval-augmented generation