This role supports Meta Superintelligence Labs (Safety) by establishing analytical measurement and evaluation foundations that help advance personal superintelligence safely and securely. The work spans telemetry, scalable evaluation, safeguards measurement, experimentation, and operational monitoring through signals, pipelines, dashboards, and reporting.
Responsibilities
- Measure abuse: build statistical telemetry and measurement frameworks to detect and monitor policy violations or emergent harms
- Scale model evaluation: design a scalable framework that adapts to evolving agentic model capabilities and the safety and risk landscape, grounded in statistical methods
- Evaluate and optimize safeguards: scale offline and online performance evaluation of safeguards, using human-in-the-loop approaches and active learning
- Design experiments and analyses: run controlled experiments and rollout studies to assess the impact of changes to policy or risk definitions and safety mitigations
- Build intelligence and reporting: create monitoring signals, data pipelines, dashboards, and reporting to enable real-time monitoring that informs interventions and safety roadmaps
Requirements
- 12+ years of experience with data querying languages (e.g., SQL), scripting languages (e.g., Python or R), and statistical or mathematical software for large-scale data analysis
- Experience applying advanced statistical methods, including predictive modeling, causal inference, experimentation design, and time series forecasting in a product context
- Experience translating data into product strategy and executive decision-making through data-driven narratives, written analytical frameworks, and stakeholder alignment across multiple cross-functional teams
- Experience establishing analytical standards, data infrastructure best practices, or measurement frameworks that are adopted broadly across an organization or business domain
Technologies
Preferred Qualifications
- Experience adhering to and implementing responsible, ethical AI practices, including risk assessment, bias mitigation, and quality and accuracy reviews
- Experience defining and scaling self-service analytics platforms or data explorability tools that reduce time-to-insight for non-technical stakeholders
- Demonstrated use of AI tools to accelerate analytical workflows, including responsible validation of AI-generated outputs and adherence to ethical AI practices
- Background working with ambiguous and sparse data to operationalize harm prevalence measurement, causal inference, and root-cause analysis rooted in quantitative and statistical foundations
- Demonstrated ability to integrate AI tools to optimize or redesign workflows and deliver measurable impact, such as efficiency gains or quality improvements
- Experience in fast-paced, high-ambiguity, cross-functional environments, including the ability to move from agentic trace deep-dives to explaining risk dimensions in plain English for policy stakeholders
- Demonstrated ongoing AI skill development, such as prompt or context engineering and agent orchestration, and staying current with emerging AI technologies
Location and Compensation
Location: Menlo Park, CA (onsite)
Compensation: USD 210,000 - 281,000 per year
Minimum experience: 12 years