Staff AI Scientist

Fiddler AI; Palo Alto (CA); Hybrid; USD 175,000 - 260,000; Full time

Benefits offered by this job

Equity

Premium health, dental & vision

Open PTO

401(k)

Monthly fitness reimbursement

Parental leave

Job summary

Fiddler AI is hiring in Palo Alto to advance trusted AI through observability and safety. You will lead applied AI research for core models and guardrail classifiers used in production, partnering with engineering, product, and customer success to drive measurable value.

The role emphasizes training/evaluating LLMs, building data pipelines, and translating research into scalable services while mentoring peers on the team. Comfortable with hybrid work 2–3 days in our Palo Alto office.

Qualifications

• 7+ years of applied AI experience taking models from research to production.
• Experience in LLM or agentic evals and guardrailing.
• Strong background training/fine-tuning classifier models and encoder architectures.
• Hands-on dataset development: labeling, synthetic data, adversarial augmentation.
• Excellent communication and ability to translate research to product.

Responsibilities

• Lead applied AI research for models and datasets central to product.
• Collaborate with engineering, product and customer success to deliver value.
• Design, train and ship production classifiers for safety, security, quality.
• Develop synthetic/adversarial data pipelines exposing failure modes.

Skills

Applied AI experience

LLM evals

Python

Model deployment

Research to production

Mentoring teammates

Education

MS/PhD in Computer Science or ML

Tools

PyTorch

Hugging Face

Docker

Job description

Our Purpose

At Fiddler, we understand the implications of AI and the impact that it has on human lives. Our company was born with the mission of building trust into AI. The rise of Generative AI and Agents has unlocked generalized intelligence but also widened the risk aperture and made it harder to ensure that AI applications are working well. Fiddler enables organizations to get ahead of these issues by helping deploy trustworthy, and transparent AI solutions.

Fiddler partners with AI-first organizations to help build a long-term framework for responsible AI practices, which, in turn, builds trust with their user base. AI Engineers, Data Science, and business teams use Fiddler AI to monitor, evaluate, secure, analyze, and improve their AI solutions to drive better outcomes. Our platform enables engineering teams and business stakeholders alike to understand the \"what\", “why”, and \"how\" behind AI outcomes.

Our Founders

Fiddler AI is founded by Krishna Gade (engineering leader at Facebook, Pinterest, Twitter, and Microsoft) and Amit Paka (product leader at Microsoft, Samsung, Paypal and two-time founder). We are backed by Insight Partners, Lightspeed Venture Partners, and Lux Capital.

Why Join Us

Our team is motivated to help build trust into AI to enable society harness the power of AI. Joining us means you get to make an impact by ensuring that AI applications at production scale across industries have operational transparency and security. We are an early-stage startup and have a rapidly growing team of intelligent and empathetic doers, thinkers, creators, builders, and everyone in between. The AI and ML industry has a rapid pace of innovation and the learning opportunities here are monumental. This is your chance to be a trailblazer.

Fiddler is recognized as a pioneer in the field of AI Observability and has received numerous accolades, including: 2022 a16z Data50 list, 2021 CB Insights AI 100 most promising startups, 2020 WEF Technology Pioneer, 2020 Forbes AI 50 most promising startups of 2020, and a 2019 Gartner Cool Vendor in Enterprise AI Governance and Ethical Response. By joining our brilliant (at least we think so) team, you will help pave the way in the AI Observability space.

About The Team

You’ll join a tight-knit, highly collaborative AI Science team that partners closely with some of the world’s leading enterprise AI organizations. We operate in a hybrid capacity, but stay deeply connected through constant communication, collaboration, and shared purpose. Our team thrives on knowledge sharing, peer learning, and collective problem solving – no one works in a silo.

We celebrate each other’s successes, support one another through complex challenges, and take pride in helping our customers achieve real-world impact with Fiddler’s AI Observability platform. Every project is a team effort, and every win is shared. If you love working alongside smart, driven peers who genuinely care about both customer success and each other’s growth, you’ll feel right at home here.

What You’ll Do

• Lead applied research and development. Lead applied research and development for the models and datasets at the core of Fiddler's Trust Service and suite of guardrail classifiers and evaluators that customers depend on to keep their LLM and agentic applications safe, accurate, and compliant in production.


• Partner closely with other engineering teams, Product, and Customer Success. You’ll build strong relationships with customer data science and ML engineering teams, supporting their AI observability journey and ensuring they realize measurable value from Fiddler.


• Design, train, and ship production classifiers for safety, security, and quality detection (e.g., prompt injection, jailbreaks, PII, hallucination, faithfulness) under strict latency and cost constraints.


• Lead the development of synthetic and adversarial dataset pipelines, including novel methods for generating, filtering, and validating data that exposes failure modes our models need to learn.


• Drive the technical direction of generative insights – the LLM- and agent-powered analysis layer that helps customers diagnose what's going wrong in their AI applications and why.


• Contribute to the evaluation and experimentation infrastructure that lets the AI Science team and our customers reliably measure model quality, regression, and drift across rapidly evolving model populations.


• Explore reinforcement learning and preference-based methods where they offer real leverage over supervised baselines.


• Collaborate with Backend and Platform engineers to take research prototypes from notebook to a hardened, scaled, observable service.


• Partner with Product, Solutions Engineering, and Customer Success to translate enterprise customer needs into research problems and translate research results back into product.


• Mentor AI Scientists on the team, raise the technical bar through code review and design review, and represent Fiddler externally through publications, talks, or open-source contributions when appropriate.

What We’re Looking For

• 7+ years of applied AI experience, with a strong track record of taking models from research to production


• Experience in LLM or Agentic Evals, Guardrailing


• Deep expertise training and fine-tuning classifier models, including modern encoder architectures (BERT-family, ModernBERT, etc.) and LLM-as-classifier approaches; clear understanding of the tradeoffs between them


• Hands‑on experience with dataset development as a first‑class engineering discipline: sourcing, labeling, synthetic generation, adversarial augmentation, and quality control


• Strong applied experience with LLMs and agentic systems – prompting, fine‑tuning, and evaluation


• Proficiency in Python and the modern ML stack (PyTorch, Hugging Face, common training/serving frameworks)


• Comfortable working in production environments and partnering with backend and platform engineers on real‑time inference, monitoring, and rollout


• Excited by the prospect of using AI coding tools in your own workflow to push the limits of what one engineer can ship – responsibly, with a clear eye on quality, security, and the failure modes these tools introduce


• Excellent written and verbal communication; able to explain research tradeoffs to engineers, PMs, and customers


• Ability to work in our Palo Alto office 2-3 days a week

Even Better

• M.S. or Ph.D. in Computer Science, Machine Learning, Statistics, Physics, or a related quantitative field


• Published research at top ML or NLP venues (NeurIPS, ICML, ICLR, ACL, EMNLP, etc.)


• Experience with reinforcement learning, RLHF, RLAIF, or preference‑based fine‑tuning


• Experience with synthetic data generation pipelines at scale


• Background in AI safety, red‑teaming, or adversarial ML


• Experience working with enterprise customers in regulated industries (finance, healthcare, government)

Compensation

For Bay Area, Seattle & New York City: $220,000 - $260,000 + equity & benefits

Other cities $175,000-$230,000 + equity & benefits

Benefits & Perks

• Competitive pay + equity


• Premium health, dental & vision (100% premium coverage for employees)


• 401(k) plan


• Open PTO


• Monthly fitness reimbursement


• Paid parental leave


• Team and company events and offsites (this year was Las Vegas)

Palo Alto HQ Vibes

• Annual Caltrain pass


• Monthly in-office massages


• Fastrak reimbursement


• Lunch provided Mon–Thurs


• Happy hours and ping pong tournaments


• Fully stocked snacks and drinks

Fiddler is proud to be an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. If you require special accommodations in order to complete the interviews or perform job duties, please inform the recruiter at the beginning of the process.