Staff Engineer: AI Agents, Evaluations & Environments

Asymmetric Security

New York (NY)

Híbrido

USD 180 000 - 240 000

Tempo integral

Há 5 dias
Torna-te num dos primeiros candidatos

Recebe uma resposta deste empregador

O teu currículo e carta de apresentação, adaptados exatamente ao que estão a contratar.

Ultrapassa os filtros ATS

Resumo da oferta

Asymmetric is seeking a Member of Technical Staff to advance agents, evaluations, and environments in London/New York. You will design longer, harder investigations, build scalable evaluation suites, and work with responders to ground truth tasks and scoring.

This role emphasizes rigorous, privacy-preserving environments and measurable model improvement. You will contribute to architecture, tooling, and experiments to push state-of-the-art AI cyberdefense capabilities and help scale our

Qualificações

  • Experience shipping LLM agents or LLM-powered systems beyond prototypes.
  • Ability to design robust evaluations and understand leakage, reward hacking, and overfitting.
  • Fluent in Python and able to build the infrastructure your experiments need.
  • Strong ability to reason about experiment design and statistical significance.
  • Ability to interpret model trajectories to form hypotheses from what you see.
  • Deep motivation by our mission.

Responsabilidades

  • Own the design and iteration of our investigation agents, from prompts and tools to architecture and orchestration.
  • Build and maintain the evaluation platform: suite governance, regression tracking, trajectory review, and fair comparisons across models and agent versions.
  • Build environment pipelines that convert real cases into graded tasks, and ensure graders reward the right behaviour and can’t be gamed.
  • Run experiments and write them up clearly: what changed, what it bought, and how sure we are.
  • Work closely with incident responders to capture expert judgement as tasks, rubrics and ground truth.
  • Shape the research direction alongside the founding team.

Conhecimentos

LLM agents
Evaluation design
Python
Statistics
Model analysis
Research direction

Descrição da oferta de emprego

Asymmetric is seeking a Member of Technical Staff to advance agents, evaluations, and environments in London/New York. You will design longer, harder investigations, build scalable evaluation suites, and work with responders to ground truth tasks and scoring.

This role emphasizes rigorous, privacy-preserving environments and measurable model improvement. You will contribute to architecture, tooling, and experiments to push state-of-the-art AI cyberdefense capabilities and help scale our

Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Staff AI Agent Architect: Evaluations & Environments
Staff AI Agent Architect: Evaluations & Environments

Asymmetric Security • New York (NY)

Híbrido
USD 140 000 - 190 000
Member of Technical Staff: Agents, Evaluations, and Environments
Member of Technical Staff: Agents, Evaluations, and Environments

Asymmetric Security • New York (NY)

Híbrido
USD 180 000 - 240 000
Remote Applied AI Staff Engineer — Agent Evaluation
Remote Applied AI Staff Engineer — Agent Evaluation

Confidential • EUA

Teletrabalho
USD 200 000 - 250 000
Equity compensation
Performance-based bonuses
Health insurance reimbursement
+2
Senior AI Agent Architect & Production Evaluator
Senior AI Agent Architect & Production Evaluator

People in AI • New York (NY)

Presencial
USD 225 000 - 275 000
Member of Technical Staff – RL Environments & Evals
Member of Technical Staff – RL Environments & Evals

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 170 000 - 250 000
AI Evaluation Engineer: RL Environments & Agents
AI Evaluation Engineer: RL Environments & Agents

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 140 000 - 210 000
Staff Software Engineer - AI-Driven Cyber Defense Platform
Staff Software Engineer - AI-Driven Cyber Defense Platform

Asymmetric Security • New York (NY)

Híbrido
USD 180 000 - 230 000
Staff Engineer: AI Agent Security & Red-Teaming Lead
Staff Engineer: AI Agent Security & Red-Teaming Lead

Artificial Intelligence Underwriting Company • San Francisco (CA)

Híbrido
USD 140 000 - 210 000
Equity in compensation
Competitive benefits
Visa sponsorship and relocation stipen
Staff Engineer, AI Agent Platform & Systems
Staff Engineer, AI Agent Platform & Systems

getbasis.ai • New York (NY)

Híbrido
USD 210 000 - 320 000
Health & Wellness benefits
Unlimited PTO & holidays
Daily meal stipends
+2
AI Evals Engineer – Reward Design & Agent Environments
AI Evals Engineer – Reward Design & Agent Environments

MaxIT Consulting - Max Corporate Group • San Francisco (CA)

Presencial
USD 120 000 - 180 000