Evals Research Engineer — Backend for Agentic AI Systems

NeoSigma

San Francisco (CA)

On-site

USD 140,000 - 210,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive salary plus meaningful equ
Comprehensive medical benefits and PTO
Direct impact on company direction
High ownership and career-defining

Job summary

NeoSigma is seeking a Backend Engineer to own ingestion, evaluation, and serving of agent behavior data at production scale. You’ll work across telemetry ingestion, platform reliability, sandbox infrastructure, and the API surface used by customers and integrations.

You will design evaluation frameworks, research reward modeling, and build infrastructure to ship features end-to-end, including open-source contributions and technical blogs.

Qualifications

  • Strong foundations in machine learning, deep learning, and computer science.
  • Hands-on experience with LLM evaluation, agentic systems, reinforcement learning, or post-training.
  • Strong programming skills in Python and experience building reliable ML research infrastructure.
  • Comfortable navigating open-ended research problems and translating ideas into working implementations.

Responsibilities

  • Design and build evaluation frameworks for agentic systems.
  • Research methods for agent evaluation, reward modeling, reward-hacking, post-training, and self-improving systems.
  • Develop rubrics, verifiers, reward signals, and benchmarks to measure performance and reliability.
  • Own research streams end-to-end and build infrastructure to ship working systems.
  • Publish research findings through technical blogs, papers, and open-source contributions.

Skills

Machine learning
Deep learning
Computer science
Python
Reinforcement learning

Tools

Python

Job description

NeoSigma is seeking a Backend Engineer to own ingestion, evaluation, and serving of agent behavior data at production scale. You’ll work across telemetry ingestion, platform reliability, sandbox infrastructure, and the API surface used by customers and integrations.

You will design evaluation frameworks, research reward modeling, and build infrastructure to ship features end-to-end, including open-source contributions and technical blogs.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff - Evals Research Engineer
Member of Technical Staff - Evals Research Engineer

NeoSigma • San Francisco (CA)

On-site
USD 140,000 - 210,000
Competitive salary plus meaningful equ
Comprehensive medical benefits and PTO
Direct impact on company direction
+1
Senior Backend Engineer, Agentic AI & Telemetry (Hybrid)
Senior Backend Engineer, Agentic AI & Telemetry (Hybrid)

We Love AI Jobs • California

Hybrid
USD 180,000 - 230,000
Hybrid work model
Ownership of product area
Staff Backend Engineer, Scalable AI Platform
Staff Backend Engineer, Scalable AI Platform

FDE Times • San Francisco (CA), New York (NY)

Hybrid
USD 405,000 - 485,000
Equity donation matching
Generous vacation & parental leave
Flexible working hours
+1
AI Agent Safety & Evaluation Engineer
AI Agent Safety & Evaluation Engineer

Syntic AI, Inc. • United States

Remote
USD 120,000 - 160,000
Senior Backend Engineer - Distributed Systems & AI Platform
Senior Backend Engineer - Distributed Systems & AI Platform

Precision Labs • San Francisco (CA)

Hybrid
USD 405,000 - 485,000
Competitive compensation
Equity donation matching (optional)
Generous vacation and parental leave
+1
Senior AI Platform Engineer - Evaluator & Agentic Systems
Senior AI Platform Engineer - Evaluator & Agentic Systems

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 357,000
Staff AI Engineer — Agentic Systems & Evaluation
Staff AI Engineer — Agentic Systems & Evaluation

SIEL AI • United States

Remote
USD 150,000 - 210,000
Senior Backend & Distributed Systems Engineer – AI Platform
Senior Backend & Distributed Systems Engineer – AI Platform

Anthropic Limited • New York (NY)

Hybrid
USD 405,000 - 485,000
Senior Backend & Distributed Systems Engineer – AI Platform
Senior Backend & Distributed Systems Engineer – AI Platform

Anthropic Limited • New York (NY)

Hybrid
USD 405,000 - 485,000
Equity donation matching
Generous vacation
Parental leave
+2
AI Backend Engineer for Everyday AGI
AI Backend Engineer for Everyday AGI

AGI, Inc • San Francisco (CA)

On-site
USD 41,000 - 55,000
Equity
Relocation support
Immigration support