Research Engineer — Coding Agent Optimization

Entelligence.AI

San Francisco (CA)

Presencial

USD 170.000 - 210.000

Jornada completa

hace 30 horas
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Ventajas ofrecidas por este puesto de trabajo

Healthcare
Equity
Compute budget for experiments

Descripción de la vacante

Entelligence.AI is seeking a Research Engineer to optimize the Coding Agent Optimization Engine in San Francisco, in person. You will advance failure attribution, build robust prevention rules and judge checks, and own end-to-end evaluation harnesses that balance quality and cost across agent sessions.

You will also design and implement a comprehensive data layer linking sessions, PRs, projects, errors, and incidents, enabling scalable world-model governance.

Formación

  • 3+ years building ML or LLM systems in production or equivalent research with shipped code.
  • Strong Python and comfort reading a large production codebase.
  • Experiment design with held-out sets, variance and leakage checks.
  • Hands-on experience with LLM APIs, agents or tool-use loops, and their failure modes.
  • Experience with agent evaluation, failure analysis, LLM-as-judge or automated code review.
  • Experience with routing, cascades, bandits or learning-to-defer.
  • Experience with LLM inference economics: prompt caching, batching, token pricing.
  • Built or maintained public evals (SWE-bench, Terminal-Bench style harnesses).
  • Published research or well-known open-source work in ML systems or developer tools.

Responsabilidades

  • Failure attribution by linking transcripts and PRs to find harness errors.
  • Decide which lessons become rules, cards or judge checks and validate with held-out data.
  • Own the benchmark harness: replay tasks and measure quality vs. cost.
  • Build data layer connecting sessions, PRs, projects, errors and incidents.
  • Model cost and outcomes to continuously refresh harness parameters.

Conocimientos

Python
LLM APIs
Experiment design
Routing
Cost modeling
Code comprehension
Research engineering
Public evals

Herramientas

CLI hooks

Descripción del empleo

Entelligence.AI is seeking a Research Engineer to optimize the Coding Agent Optimization Engine in San Francisco, in person. You will advance failure attribution, build robust prevention rules and judge checks, and own end-to-end evaluation harnesses that balance quality and cost across agent sessions.

You will also design and implement a comprehensive data layer linking sessions, PRs, projects, errors, and incidents, enabling scalable world-model governance.

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Software Engineer, AI Agent Optimization
Software Engineer, AI Agent Optimization

Remote Genie • EE. UU.

A distancia
USD 110.000 - 325.000
Healthcare benefits
401k with employer match
Global benefits
+2
Lead AI Agent Optimization & Evaluation Engineer
Lead AI Agent Optimization & Evaluation Engineer

Reddit, Inc. • San Francisco (CA)

Presencial
USD 180.000 - 300.000
Comprehensive Healthcare Benefits
Income Replacement Programs
401(k) with Employer Match
+5
AI Research Engineer - Coding Agents Evaluation (Remote)
AI Research Engineer - Coding Agents Evaluation (Remote)

Joblet-AI • EE. UU.

A distancia
GBP 151.000 - 197.000
Equity compensation
Performance-based bonuses
Health-insurance reimbursement
+2
Software Engineer — AI Agent Optimization & Evaluation
Software Engineer — AI Agent Optimization & Evaluation

Reddit • EE. UU.

A distancia
USD 180.000 - 240.000
Comprehensive healthcare
401(k) match
Global benefit programs
+5
Research Engineer, Agentic AI & Tools
Research Engineer, Agentic AI & Tools

Rebash • San Francisco (CA)

Presencial
USD 180.000 - 260.000
Agentic Coding Research Scientist
Agentic Coding Research Scientist

Bison • San Francisco (CA)

Presencial
USD 400.000 - 500.000
Health, dental, and vision benefits
Unlimited PTO
Paid parental leave
+1
AI Engineer — Scalable LLM Systems & Agent Workflows
AI Engineer — Scalable LLM Systems & Agent Workflows

Introspection • San Francisco (CA)

Presencial
USD 140.000 - 210.000
Coding Research Evaluation Engineer
Coding Research Evaluation Engineer

Remotebridge • EE. UU.

A distancia
USD 200.000 - 260.000
Equity compensation
Bonus eligibility
Health-insurance reimbursement
+2
Lead AI Data & RL Engineer for Coding Agents
Lead AI Data & RL Engineer for Coding Agents

Turing • San Francisco (CA)

Presencial
USD 250.000 - 350.000
Competitive compensation
Equity opportunities
Collaborative culture
Research Engineer: AI Tooling & Agent Systems
Research Engineer: AI Tooling & Agent Systems

Composio • San Francisco (CA)

Presencial
USD 100.000 - 150.000