Senior GenAI Evaluation Engineer Scale & ML Infra (Equity)

Google LLC; New York (NY); On-site; USD 174,000 - 252,000; Full time

Job summary

Google LLC is seeking a Senior Software Engineer for GenAI Evaluation in New York, NY. You will design and maintain evaluation infrastructure for large-scale models, and collaborate with teams across product and AI platforms.

The role requires strong Python and C++ coding, experience with distributed systems, multithreading, and GenAI workflows. In-person interviews are typically part of the process and compensation includes a competitive package in USD.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 5 years of experience with coding in Python.
• 3 years of experience developing large-scale infrastructure, distributed systems or networks, or experience with compute technologies, storage or hardware architecture.
• 3 years of experience working with Generative AI agents.
• 2 years of experience with multithreading.

Responsibilities

• Design, build, and maintain reusable eval infra blocks, such as streaming evaluation, autorater pipelines, and metric aggregation.
• Make large-scale eval workflows more scalable, reliable, and efficient across compute, storage, and serving backends.
• Onboard product teams, debug their issues, and turn recurring needs into self-serve features.
• Build secure per-customer data isolation and access control for eval data and results.
• Help teams move smoothly from offline evals to online evals and to live experiments and production.

Skills

Python

C++

GenAI

Multithreading

Distributed systems

Large-scale infra

Education

Bachelor's degree or equivalent

Master's degree or PhD in CS

Job description

Google LLC is seeking a Senior Software Engineer for GenAI Evaluation in New York, NY. You will design and maintain evaluation infrastructure for large-scale models, and collaborate with teams across product and AI platforms.

The role requires strong Python and C++ coding, experience with distributed systems, multithreading, and GenAI workflows. In-person interviews are typically part of the process and compensation includes a competitive package in USD.