GenAI Research Engineer: Evaluations & Hillclimb

Socket.dev; Cambridge (MA); On-site; USD 174,000 - 252,000; Full time

Benefits offered by this job

Equity

Bonus plan (15% target)

Benefits

Job summary

Google DeepMind is seeking engineers focused on research-driven software development to set up large-scale experiments and deploy promising ideas quickly. You will design evaluations, implement modular systems, and work with product teams to ensure effective feedback loops, aiming for efficiency at Google scale with routing and multi-agent capabilities.

The role emphasizes collaboration across engineering, research, and product teams, along with publishing and contributing to the broader

Qualifications

• Bachelor's degree in CS/ML or related field or equivalent practical experience.
• 3 years of software development experience in Python or C++.
• Experience with ML frameworks (PyTorch, TensorFlow, JAX) and evaluating models.

Responsibilities

• Build rigorous evaluations to measure model performance across workloads and drive hillclimbing.
• Develop modular, adaptable solutions for multiple Google product areas.
• Collaborate cross-functionally with Eng, PM, and Data Science to translate requirements into model improvements.

Skills

Python

C++

ML frameworks

Bachelor's degree

Education

Bachelor's degree in CS/ML

Advanced degree preferred

Tools

LLM orchestration

Benchmark suite design

Job description

Google DeepMind is seeking engineers focused on research-driven software development to set up large-scale experiments and deploy promising ideas quickly. You will design evaluations, implement modular systems, and work with product teams to ensure effective feedback loops, aiming for efficiency at Google scale with routing and multi-agent capabilities.

The role emphasizes collaboration across engineering, research, and product teams, along with publishing and contributing to the broader