AI Agent Platform Manager for Scalable LLM Systems

Socket.dev; Mountain View (CA); On-site; USD 180,000 - 300,000; Full time

Job summary

Google seeks a Software Engineering Manager to lead multiple teams across applied AI initiatives, including low-latency scoring, automated evaluation, and RAG integrations. You will guide engineering managers, coordinate with product and data science, and help scale agent-based commerce platforms globally.

The role emphasizes building reliable LLM tooling, Ground Truth datasets, and dashboards while managing budgets and multi-site deployments.

Qualifications

• Bachelor’s degree in CS or related field or equivalent practical experience.
• 8 years of experience coding in Python, Java, C++, or Go focused on backend architecture.
• Experience building LLM evaluation pipelines, AutoRaters, or reliability metrics.
• Experience with backend, distributed systems, or AI/ML infrastructure.
• Experience managing software engineers including mentoring and performance management.

Responsibilities

• Manage and mentor a Software Engineering team, owning the roadmap and architecture for applied AI, low-latency scoring, and automated evaluation.
• Lead Knowledge Library and merchant control engineering, building advanced RAG integrations (PDF ingestion, auto-learning) for custom agent behavior.
• Oversee autonomous execution tooling and Universal Commerce Protocol (UCP) integration to enable active user journeys.
• Advocate prompt tuning and loss diagnosis while scaling LLM-based AutoRater frameworks to reduce hallucinations and measure factuality.
• Partner with Product, Data Science, UX, and core Google platform teams to define Ground Truth datasets, dashboards, and model upgrade standards.

Skills

Backend system architecture

Team leadership

Mentoring

Performance management

LLM evaluation pipelines

AutoRaters

Reliability metrics

Education

Bachelor’s degree in CS or related field

Master’s degree or PhD (preferred)

Job description

Google seeks a Software Engineering Manager to lead multiple teams across applied AI initiatives, including low-latency scoring, automated evaluation, and RAG integrations. You will guide engineering managers, coordinate with product and data science, and help scale agent-based commerce platforms globally.

The role emphasizes building reliable LLM tooling, Ground Truth datasets, and dashboards while managing budgets and multi-site deployments.