Staff Software Engineer, Agentic SDLC Foundations

Socket.dev; San Jose (CA); On-site; USD 207,000 - 300,000; Full time

Benefits offered by this job

Equity

Bonus target (20%)

Benefits

Job summary

Google in the United States is hiring a Staff Software Engineer to lead architecture and evaluation for the developer agent ecosystem as part of the Agentic SDLC Foundations team. You will drive HITL guardrails, automated evaluation harnesses, and KPI standardization across engineering workflows.

You will shape the technical strategy for scalable AI solutions and oversee end-to-end quality engineering for reliable autonomous systems in Google's Core stack.

Qualifications

• Bachelor’s degree or equivalent practical experience is required.
• 8 years of software development experience.
• 5 years testing and launching software products; 3 years in design/architecture.
• 5 years ML design/infrastructure experience (deployment, evaluation, data processing, debugging, fine-tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.
• Preferred: advanced degree and leadership experience in tech.
• Experience designing AI/ML evaluations, LLM benchmarks, or quality pipelines.

Responsibilities

• Design and scale production developer agents with trajectory planning and tool selection.
• Implement HITL approval gates and safety-conscious execution of actions.
• Architect evaluation pipelines for multi-step trajectories and tool fidelity.
• Create closed-loop data from failures for dataset refinement and model tuning.
• Develop benchmarks to evaluate KPI across complex developer tasks.

Skills

Software development experience

ML design & infrastructure

Generative AI integration

Technical leadership

AI/ML evaluations planning

LLM benchmarking

Prompt optimization

LLM fine-tuning

Automated dataset generation

Trajectory-based evaluation

Autonomous agent deployment

Education

Bachelor’s degree or equivalent practical experience

Master’s degree or PhD in Engineering, CS, or related field

Job description

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.

Preferred qualifications:

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
• 3 years of experience in a technical leadership capacity (e.g., leading technical roadmaps, designing multi-system architectures, and guiding cross-functional engineering efforts).
• Experience designing and implementing AI/ML evaluations, LLM benchmarking frameworks, or agentic quality measurement pipelines.
• Experience in prompt optimization, LLM fine-tuning, automated dataset generation, and trajectory-based evaluation methodologies.
• Experience taking generative AI solutions or autonomous agent loops from prototype to high-reliability production systems.

About the job:

As part of the Agentic SDLC organization in Core, our mission is to transform how engineers build, deploy, and manage software at scale by creating an end-to-end agentic platform for the modern software development lifecycle.

As a Staff Software Engineer on the Agentic SDLC Foundations team, you will serve as the technical lead driving the architecture, evaluation, and quality engineering of Google’s developer agent ecosystem. Bridging the gap between experimental generative AI prototypes and dependable, production-grade autonomous systems, you will lead the end-to-end technical strategy for building AI solutions with robust human-in-the-loop (HITL) guardrails. You will establish automated evaluation harnesses, benchmark suites, and methodologies that systematically elevate agent reliability, prevent regressions, and standardize key autonomy metrics across Google’s most critical engineering workflows. The Core team builds the technical foundation behind Google’s flagship products. We are owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure at Google. These are the essential building blocks for excellent, safe, and coherent experiences for our users and drive the pace of innovation for every developer. We look across Google’s products to build central solutions, break down technical barriers and strengthen existing systems. As the Core team, we have a mandate and a unique opportunity to impact important technical decisions across the company. Individual pay is determined by factors including job-related skills, experience, and relevant education or training.

US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits

Learn more about benefits at Google.

Responsibilities:

• Design and scale production developer agents, establishing patterns for trajectory planning, tool selection, and state recovery.
• Implement intelligent HITL approval gates, intent disambiguation, and graceful to ensure safe execution of actions.
• Architect evaluation pipelines that grade multi-step trajectories, tool-call fidelity, intermediate states, and reasoning paths beyond static input/output matching.
• Establish closed-loop pipelines converting execution failures and human corrections into golden datasets, prompt/tool-schema optimizations, and fine-tuning cycles.
• Curate end-to-end benchmarks replicating complex developer tasks to evaluate agent capabilities and establish standardized Key Performance Indicator (KPIs) across workflows.