Senior Staff ML Engineer - Production LLMs & Evaluation

present of AI; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Ambience Healthcare in San Francisco is seeking a senior AI/ML engineer to shape the AI behind our clinical documentation and coding products. This hands-on role requires deep experience with production-grade large language models, robust evaluation design, and agentic systems, alongside strong Python and ML engineering skills.

You will lead cross-functional teams, translate vague quality problems into measurable tests, and contribute code while digging into failures to improve system

Qualifications

• Deep experience with large language models in production.
• Strong evaluation design and measurement.
• Experience with agentic systems and post-training methods.
• Solid Python and ML engineering fundamentals.
• Experience shipping consequential AI systems.

Responsibilities

• Set direction for AI behind clinical documentation and coding products.
• Lead across teams while writing code and debugging failures.
• Turn vague quality problems into measurable tests.
• Design and improve evaluation pipelines and feedback loops.

Skills

Production LLMs

Evaluation design

Agentic systems

Python ML engineering

Tools

Python

PyTorch

Job description

Ambience Healthcare in San Francisco is seeking a senior AI/ML engineer to shape the AI behind our clinical documentation and coding products. This hands-on role requires deep experience with production-grade large language models, robust evaluation design, and agentic systems, alongside strong Python and ML engineering skills.

You will lead cross-functional teams, translate vague quality problems into measurable tests, and contribute code while digging into failures to improve system