Forward Deployed Machine Learning Engineer

Raydar; United States; Remote; USD 170,000 - 270,000; Full time

Benefits offered by this job

Competitive equity

Remote within the United States

High autonomy

Job summary

Raydar is hiring a Machine Learning Engineer to lead benchmarks and evaluations, partnering with customers, researchers, and a GM to establish the evaluation foundation for foundation models across domains. The role blends ML evaluation, backend infrastructure, and high-ownership customer delivery.

You will design production benchmarks, own data pipelines and execution environments, and create sandboxed setups for agentic evaluations.

Qualifications

• 4+ years of engineering experience, including hands-on ML model evaluation work.
• Experience deploying end-to-end ML evaluation or benchmark systems to production for enterprise timelines.
• Proficiency with ML evaluation frameworks and benchmark design (e.g., LLM-as-judge).

Responsibilities

• Define, design, and build production benchmarks and evaluations with customers and internal researchers.
• Build and own backend infrastructure, including data pipelines, execution environments, storage, and orchestration.
• Create sandboxed environments for agentic evaluations involving tools, code execution, and multi-step tasks.
• Own the engineering portion of customer engagements from technical scoping through production delivery.
• Identify repeatable evaluation patterns and infrastructure gaps that can become scalable products.
• Move quickly through ambiguity while maintaining strong technical judgment and clear written communication.

Skills

Machine learning evaluation

Backend infrastructure

Data pipelines

Customer-facing engineering

LLM evaluation frameworks

Education

Degree in CS/Physics or related field

Tools

Python

GPU/Compute resources

Job description

About the company

Our client is a fast-growing AI data infrastructure company building a secure marketplace and data lab for high-quality model-training data. The company helps data holders license sensitive, real-world datasets to vetted AI teams while protecting governance, privacy, intellectual property, and security. It has raised $65 million, including a $55 million Series A backed by leading venture firms, employs approximately 80 people, and is scaling quickly after reaching its full-year growth target early.

The role and why it matters

This is the first Machine Learning Engineer dedicated to the company’s Benchmarks and Evaluations vertical. You will partner directly with the general manager, researchers, and early enterprise customers to establish the technical foundation for evaluating foundation models across domains and modalities. The role combines hands-on ML evaluation, backend infrastructure, and customer-facing delivery in a high-ownership environment.

What you’ll do

• Define, design, and build production benchmarks and evaluations with customers and internal researchers.
• Build and own backend infrastructure, including data pipelines, execution environments, storage, and orchestration.
• Create sandboxed environments for agentic evaluations involving tools, code execution, and multi-step tasks.
• Own the engineering portion of customer engagements from technical scoping through production delivery.
• Identify repeatable evaluation patterns and infrastructure gaps that can become scalable products.
• Move quickly through ambiguity while maintaining strong technical judgment and clear written communication.

What we’re looking for

• 4 or more years of engineering experience, including hands-on machine learning model evaluation work.
• Experience deploying end-to-end ML evaluation or benchmark systems to production against demanding customer timelines.
• Strong proficiency with ML evaluation frameworks and benchmark design, including approaches such as LLM-as-judge.
• Ownership of backend and infrastructure systems such as large-scale data pipelines, execution environments, storage, and orchestration.
• Customer-facing engineering experience managing enterprise stakeholders.
• A degree in computer science, physics, or a related technical field.
• Current unrestricted U.S. work authorization without visa sponsorship.

Bonus points

• Experience building evaluations or human-data pipelines for large language models.
• Experience working directly with AI researchers or foundation-model labs.
• Published work or meaningful open-source contributions in ML evaluations or benchmarks.
• Early-stage B2B startup, forward-deployed engineering, or high-ownership generalist experience.

Compensation and benefits

• $170K-$270K base salary.
• Competitive equity.
• Comprehensive benefits provided by a well-capitalized, high-growth company.
• High autonomy and the opportunity to build a new technical vertical from the ground up.

Location and work model

• Full-time and remote within the United States.
• Strong independent ownership, customer responsiveness, and cross-functional collaboration are expected.