Senior Software Engineer, Search Platforms, LLM Evaluation Infrastructure

Google; Bengaluru; On-site; INR 4,500,000 - 7,000,000; Full time

Job summary

Google is seeking a Senior Software Engineer to lead the architecture, design, and implementation of LLM evaluation platforms and benchmarking pipelines within the Search Platforms team. This role focuses on transforming complex evaluation workflows into a unified, scalable backend for worldwide deployments.

You will drive modernization of pipelines, integrate AI tooling, and optimize throughput while ensuring robustness for production-grade models across Google Search.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 3 years of experience working with embedded operating systems.
• 3 years of experience testing, maintaining, or launching software products.
• 1 year of experience with software design and architecture.
• Master's degree or PhD in Computer Science or a related technical field.
• 5 years of experience with data structures and algorithms.
• 1 year of experience in a technical leadership role.
• Experience developing accessible technologies.

Responsibilities

• Architect, build, and maintain high-throughput, low-latency platform services and orchestration pipelines that execute automated evaluation, benchmark suites, and metric computations for Generative Artificial Intelligence (GenAI) Search experiences.
• Design robust evaluation systems to detect metric drift, optimize evaluation throughput, and scale model-assisted rating (e.g., LLM-as-a-judge/Autorater) workflows while maintaining compute efficiency.
• Drive the modernization and consolidation of fragmented, bespoke evaluation pipelines into a unified core platform, identifying capability gaps and creating reusable, extensible frameworks.
• Optimize system throughput, execution turnaround times, and resource efficiency for continuous, high-volume evaluation workloads supporting production-critical Search models.
• Integrate modern AI tooling, automated validation harnesses, and AI-assisted workflows into daily engineering practices to boost developer velocity, evaluation excellence, and system robustness.

Education

Bachelor’s degree or equivalent practical experience.

Master's degree or PhD in Computer Science or related field

Job description

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 3 years of experience working with embedded operating systems.
• 3 years of experience testing, maintaining, or launching software products.
• 1 year of experience with software design and architecture.

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 3 years of experience working with embedded operating systems.
• 3 years of experience testing, maintaining, or launching software products.
• 1 year of experience with software design and architecture.

Preferred qualifications:

• Master's degree or PhD in Computer Science or a related technical field.
• 5 years of experience with data structures and algorithms.
• 1 year of experience in a technical leadership role.
• Experience developing accessible technologies.

About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

At Google, our mission is to organize the world's information and make it universally accessible and useful. The Search Platforms Large Language Model (LLM) Evaluation Infrastructure team is at the forefront of this mission, building the next-generation core infrastructure and benchmarking platforms that power robust, automated, and scalable LLM evaluations across Google Search.

As a Senior Software Engineer, you will lead the architecture, design, and implementation of LLM evaluation platforms and continuous benchmarking pipelines. You will transform complex, bespoke evaluation workflows into a unified, high-throughput platform capable of assessing quality, factual accuracy, safety, and latency across advanced model architectures. You will be responsible for creating and evolving scalable, reproducible back-end evaluation infrastructure for systems operating at a massive global scale.In Google Search, we're reimagining what it means to search for information – any way and anywhere. To do that, we need to solve complex engineering challenges and expand our infrastructure, while maintaining a universally accessible and useful experience that people around the world rely on. In joining the Search team, you'll have an opportunity to make an impact on billions of people globally.

Responsibilities

• Architect, build, and maintain high-throughput, low-latency platform services and orchestration pipelines that execute automated evaluation, benchmark suites, and metric computations for Generative Artificial Intelligence (GenAI) Search experiences.
• Design robust evaluation systems to detect metric drift, optimize evaluation throughput, and scale model-assisted rating (e.g., LLM-as-a-judge/Autorater) workflows while maintaining compute efficiency.
• Drive the modernization and consolidation of fragmented, bespoke evaluation pipelines into a unified core platform, identifying capability gaps and creating reusable, extensible frameworks.
• Optimize system throughput, execution turnaround times, and resource efficiency for continuous, high-volume evaluation workloads supporting production-critical Search models.
• Integrate modern AI tooling, automated validation harnesses, and AI-assisted workflows into daily engineering practices to boost developer velocity, evaluation excellence, and system robustness.

Google is proud to be an equal opportunity workplace and is an affirmative action employer. We are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. See also Google's EEO Policy and EEO is the Law. If you have a disability or special need that requires accommodation, please let us know by completing our Accommodations for Applicants form .