Sr. ML Infrastructure Engineer - Kleiner Perkins backed Stealth Startup

Day One Partners; South San Francisco (CA); On-site; USD 140,000 - 200,000; Full time

Job summary

An innovative firm is seeking a Machine Learning Infrastructure Engineer to develop LLM evaluation systems that will revolutionize drug development. This exciting role involves designing frameworks and ensuring data quality for a stealth AI company backed by top-tier VCs. Your contributions will directly impact the efficiency of biopharma companies in bringing new drugs to market. Join a passionate team in a fast-paced startup environment where your expertise in machine learning and software engineering will be pivotal. If you're eager to tackle big challenges and make a difference in healthcare, this opportunity is for you.

Qualifications

• 4+ years of experience in machine learning infrastructure with a focus on LLM evaluation.
• Proven experience designing and implementing evaluation frameworks for LLMs.

Responsibilities

• Design and build robust LLM evaluation frameworks from scratch.
• Collaborate directly with customers to understand data needs.

Skills

Machine Learning Infrastructure

LLM Evaluation Systems

Software Engineering

Data Quality Assurance

Classical ML Methods

Data Analysis

Job description

This range is provided by Day One Partners. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$140,000.00/yr - $200,000.00/yr

Day One Partners is working with the founders of a stealth AI company backed by Tier 1 VCs. They’re building an AI-powered platform that helps biopharma companies bring new drugs to market faster and more cost-effectively. The company already has paying customers and strong interest from major industry players.

We’re hiring a Machine Learning Infrastructure Engineer to lead the design and development of their LLM evaluation systems. This role is onsite in South San Francisco and works closely with both the founding team and their first customers.

Why This Role Matters:

Developing and launching new drugs is expensive and slow—over $300M per drug, with high failure rates. This team is solving that by centralizing industry data and automating complex workflows. Your work will directly impact the accuracy, quality, and reliability of the data systems that support these critical decisions.

What You’ll Do:

• Design and build robust LLM evaluation frameworks from scratch
• Develop methods to measure LLM performance, identify errors, and ensure high data quality
• Own the infrastructure for extracting structured data from large volumes of unstructured documents
• Run experiments to improve data retrieval and extraction accuracy
• Collaborate directly with customers to understand data needs and explain evaluation methodologies
• Work with the founding team to shape the future of their AI-powered platform

What You Bring:

• 4+ years of experience in machine learning infrastructure, with a focus on building ML or LLM evaluation systems
• Proven experience designing and implementing evaluation frameworks for LLMs
• Strong software engineering skills
• Familiarity with classical ML methods (logistic regression, random forest, XGBoost)
• Excitement about working with large datasets and uncovering new signals
• Interest in life sciences, healthcare data, or similar domains is a plus
• Comfortable working onsite in a fast-paced, early-stage startup

Bonus Points For:

• Experience fine-tuning LLMs or deep learning models
• Previous startup experience, especially in high-growth companies
• Background in biology, biotech, or healthcare ML applications

If you have a strong track record building LLM evaluation systems and want to join a small team solving big problems, reach out to Day One Partners to learn more. For immediate consideration, email resumes to [e-mail] with the subject Day One <> ML Infra

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Industries: Biotechnology Research, Technology, Information and Media, and Software Development