Product Operations Manager, Model Quality

Meta; Seattle (WA); On-site; USD 123,000 - 179,000; Full time

Benefits offered by this job

Bonus

Equity

Benefits

Job summary

Meta is seeking a Product Operations Manager to join our Product Operations Foundations team and drive model quality across Meta surfaces. You will own end-to-end quality programs for AI-driven workflows and work with cross-functional partners to advance accuracy, drift prevention, and real-time observability.

This role involves setting strategy for LLM models and advising leadership on risks while shaping roadmaps and staffing for impactful AI tooling.

Qualifications

• Bachelor's degree in a directly related field or equivalent practical experience.
• 7+ years of experience in strategy, operations, consulting, or data analysis.
• Experience using data to influence product direction with SQL.
• Experience building or deploying AI/ML solutions in production workflows.
• Strong communication skills with cross-functional stakeholders and leadership.
• Ability to break down complex issues into actionable components.
• Experience designing AI workflows within enterprise data constraints.

Responsibilities

• Defines the technical direction for model maintenance (retraining cadences, drift mitigation, performance recovery) and evolution (new capabilities, architecture improvements).
• Provides cross-product context, defines what good looks like at the model level, and informs evaluation methodology.
• Ensures consistency across the portfolio and surfaces performance patterns to unblock evals leads.
• Defines what leadership needs to see, how model health should be measured, and thresholds for escalation.
• Provides thought partnership to evals managers on model health narratives and classification strategies.
• Drives alignment with cross-functional teams on tooling needs and investment decisions for AI model quality.

Skills

AI/ML strategy

Model quality

Cross-functional leadership

SQL

Data storytelling

Communication

Problem solving

Strategic planning

Enterprise workflows

Education

Bachelor's degree in a directly related field

Job description

Meta is seeking a Product Operations Manager to join our Product Operations Foundations team and drive model quality across all Meta surfaces. We are in the middle of a transformation, becoming an AI-driven, IC-led organization that scales through orchestration, deep product expertise, and technical excellence. Our team is building and operating autonomous agents that handle end-to-end workflows (triage, bug resolution, launches, dogfooding, evals) with minimal human intervention. If you're energized by owning complex quality programs end-to-end, building and operating AI-driven workflows, and driving measurable product improvements in a fast-paced environment, this role is for you. You will be responsible for managing and evaluating our AI solutions and infrastructure to improve precision, prevent drift, and maintain real-time observability. This role is expected to set strategy for LLM models, determine areas of investment for increasing accuracy, advise leadership on impending risks, define roadmaps and reporting strategy to shape the future of our AI work. As part of this work, you will be expected to build and maintain industry-wide expertise, develop effective cross-functional relationships, advise engineering and cross-functional partners on areas of investment, determine staffing needs, and solution against critical bottlenecks.

Product Operations Manager, Model Quality Responsibilities:

• Defines the technical direction for model maintenance (retraining cadences, drift mitigation, performance recovery) and evolution (new capabilities, architecture improvements, multi-modal expansion). Translates cross-product performance patterns into investment recommendations for evaluation leads
• Provides cross-product context, defines what good looks like at the model level, and informs evaluation methodology. Evals owners own execution of verification pipelines within their products
• this role ensures consistency and identifies gaps across the portfolio while building institutional competence by surfacing performance patterns and proven methodologies, enabling evals captains' ability to execute and unblocking them as needed
• Defines what leadership needs to see, how model health should be measured and reported, and what thresholds trigger escalation
• Provides thought partnership to evals managers on narrative of model health, provides visibility into our classification strategy and accuracy measurement process
• Works with evaluation managers to drive cross-app taxonomy alignment in alignment with cross-functional needs and advises on a strategy for the migration of LLM accuracy assessment to judges
• Owns the consolidated view of all production model performance, identifies systemic patterns and emerging risks, and ensures leadership can verify model health on demand
• Partners with AI Implementations, operational systems teams and the Metrics & Measurement team to build and maintain the infrastructure that surfaces this information
• Establishes performance guardrails that evals captains implement. Continuously scans industry developments and best practices to incorporate into org-wide approach
• Maintains a tight feedback loop with product and eng teams across apps to ensure alignment on production priorities and deployment risks
• Deploys deep SME expertise to diagnose, unblock and directly resolve technical bottlenecks to complex model quality problems (atrophy, accuracy regressions, performance plateaus) when evaluation leads encounter blockers they cannot resolve independently
• Drives alignment with cross-functional teams (quality and reliability partner teams) on tooling needs to support Product Operations classification strategy (ML classification tooling for initial-tier classification, user voice, breakdown graphs). Advocates for investment, flags risks, influences direction

Minimum Qualifications:

• Bachelor's degree in a directly related field, or equivalent practical experience
• 7+ years of experience in strategy, operations, consulting, or data analysis
• Analytical experience using data to tell a story and influence product direction using intermediate to advanced SQL
• Experience building or deploying AI/ML solutions, LLM model quality or automation in production workflows
• Strong communication skills with ability to influence multiple cross-functional stakeholders and senior leadership
• Experience breaking down ambiguous issues into component parts to develop solutions
• Ability to design AI workflows that operate effectively within enterprise data sensitivity constraints, balancing quality and privacy principles

Preferred Qualifications:

• Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
• Experience operating in flat, IC-heavy org structures with high individual autonomy
• Demonstrated history of evaluating industry best practices and providing organizational recommendations on approaches to AI models and development
• Experience in product quality, QA, or technical program management
• Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
• Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)
• Familiarity with LLMs, AI agents, or ML evaluation frameworks
• Experience working with global/remote teams

About Meta:

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.

Meta is proud to be an Equal Employment Opportunity and affirm... We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.

Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].

$123,000/year to $179,000/year + bonus + equity + benefits

Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.