Research Engineer, Safety Evaluation

Meta Careers; Menlo Park (CA); On-site; USD 219,000 - 301,000; Full time

Job summary

Meta is seeking a Research Engineer to advance Safety Evaluation for its frontier AI systems. You will set the strategy across model families and modalities, design novel safety evaluations, and build a distributed platform to run extensive checks against training checkpoints and production traffic.

You will translate policy and regulatory standards into concrete measurement criteria while mentoring researchers and communicating results to leadership.

Qualifications

• Experience designing and validating evaluations or benchmarks for ML systems, including metric reliability and failure modes.
• Hands-on work with LLMs, multimodal models, or NLP across production or research environments.
• Ability to communicate technical results to non-technical stakeholders and drive decisions across teams.

Responsibilities

• Set the technical strategy for safety evaluation across multiple model families and modalities.
• Design, implement, and validate novel evaluations for safety-critical behaviors (policy adherence, adversarial robustness, agentic risk).
• Build and harden distributed evaluation platforms to run hundreds of evals at scale during training.

Skills

Python

PyTorch

ML/AI

LLMs

Education

Bachelor's degree in CS/Engineering or equivalent

Tools

PyTorch

Job description

Meta is seeking a Research Engineer to join the Safety Evaluation team within Meta Superintelligence Labs. Our mission is to make the safety of Meta's frontier AI systems measurable — turning ambiguous notions of \"safe\" into rigorous, defensible metrics that model developers, product teams, and company leadership rely on to make launch decisions.Safety evaluation is the ground truth for every safety claim Meta makes. This role owns that ground truth: designing the evaluations that detect emerging risks in text, image, voice, video, and agentic systems; building the infrastructure that runs them continuously against training checkpoints and production traffic; and setting the technical direction for how safety is measured across Meta's AI portfolio. You will define measurement standards that outlast any single model generation, and your results will directly gate what ships to billions of people.Research Engineer, Safety Evaluation Responsibilities:Set the technical strategy for safety evaluation across multiple model families and modalities, and drive it to execution across teamsDesign, implement, and validate novel evaluations for safety-critical behaviors — policy adherence, adversarial robustness, agentic risk, jailbreak resistance, and emerging harm categories — including for capabilities with no established benchmarkBuild and harden the distributed evaluation platform so that hundreds of evals run reliably and continuously against checkpoints throughout large-scale training runsOwn the measurement quality bar: signal-to-noise, statistical power, saturation, contamination, and construct validity — and establish when an eval is trustworthy enough to gate a launchCreate, curate, and analyze high-quality safety datasets, including adversarial, borderline, multilingual, and long-tail casesconvert real-world incidents and red-team findings into durable, repeatable safety signalsDiagnose anomalous eval results mid-training-run, determine whether the cause is a model change or an infrastructure artifact, and communicate a clear answer under time pressureOwn the dashboards and reporting that researchers, product partners, and leadership use to monitor safety during training and post-launchTranslate evolving global safety policy and regulatory standards into concrete, testable measurement criteria, partnering with Policy, Legal, and IntegrityInfluence the roadmaps of partner research and product teamsmentor engineers and researchers and raise the evaluation bar across the orgRepresent Meta's safety evaluation methodology to internal leadership and, where appropriate, to external audiences and the research communityMinimum Qualifications:Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experienceBachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience3+ years of industry research or research-engineering experience in ML/AI, including hands-on work with LLMs, multimodal models, or NLPDemonstrated experience setting technical direction for a large, ambiguous problem area and driving it to delivery across multiple teamsExperience designing and validating evaluations or benchmarks for ML systems, including reasoning about metric reliability and failure modesExperience building production-grade or research infrastructure that must be reliable at scale — distributed systems, data pipelines, or evaluation harnessesProgramming experience in Python and hands-on experience with frameworks such as PyTorchExperience communicating complex technical results to non-specialist stakeholders and decision-makersPreferred Qualifications:Experience translating regulatory or policy requirements into technical measurement criteriaExperience evaluating LLMs across multiple languages and modalities (text, image, voice, video, reasoning, tool use)Experience operating in an on-call or production-support capacity for live training runs or safety-critical systemsExperience evaluating agentic systems — multi-step tool use, autonomy, and oversight mechanismsPublications at peer-reviewed venues (e.g. ICLR, NeurIPS, ICML, ACL, CVPR, ICCV, FAccT) with a track record in evaluation, alignment, or AI safetyExperience with large-scale distributed training (hundreds/thousands of GPUs) and evaluating models in-flight during trainingExperience with adversarial evaluation and red-teaming, including automated attack generation and jailbreak robustness measurementexperience with observability, monitoring, or experiment-tracking systemsPhD in Computer Science, Machine Learning, or a relevant technical fieldBackground in statistics and experimental designAbout Meta:Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].$219,000/year to $301,000/year + bonus + equity + benefitsIndividual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.