Research Engineer/Research Scientist, Pre-Training Evaluation and Validation, UK

DeepMind; London; On-site; GBP 60,000 - 90,000; Full time

Job summary

A leading AI research organization in London seeks a Research Scientist/Engineer to spearhead the development of large scale validation datasets for Gemini models. The ideal candidate will possess a PhD in Computer Science and a strong publication record. Join a collaborative environment focused on driving advancements in AI technologies while ensuring safety and ethical considerations are prioritized.

Qualifications

• Proven track record of publishing in major machine learning conferences.
• Ability to work collaboratively across organizations and disciplines.
• Experience with evaluation of large scale machine learning models.

Responsibilities

• Develop validation datasets for large scale pretrained models.
• Provide actionable insights to model developers.
• Drive improvements to technical infrastructure.

Skills

PhD in Computer Science or related field

Strong publication record in top machine learning conferences

Experience with modern deep learning frameworks

Education

PhD or equivalent practical experience

Tools

JAX

PyTorch

Job description

Snapshot

Artificial Intelligence could be one of humanity's most useful inventions. At Google DeepMind, we're a team of scientists, engineers, machine learning experts and more, working together to advance the state of the art in artificial intelligence. We use our technologies for widespread public benefit and scientific discovery, and collaborate with others on critical challenges, ensuring safety and ethics are the highest priority.
The Role

We are seeking a Research Scientist/Engineer to join the Gemini Large Scale Pre-training Evaluations team in London. This is a high-impact role with direct influence on decisions made for designing our large scale Gemini models. You will join a team working on evaluating and building large models, exploring their capabilities and improving the infrastructure used by a large team of researchers and engineers working on Gemini models.

Key responsibilities

• Spearhead and own the development of large scale validation datasets (text, image, audio, video) for our state of the art large scale pretrained models.
• Collaborate closely with model developers to provide them with actionable insights and trustworthy metrics.
• Own and drive improvements to their technical infrastructure.
• Drive the development of novel techniques and analyses to provide insights into the relationship between data, scaling laws, leakage and memorisation.
• Continuously develop new methods and insights to push the frontier of our understanding of model abilities and capabilities.
• Perform deep assessments of data and eval coverage to help drive future model decisions.
• Improve methods for data leakage detection and leakage prevention.

About you

In order to set you up for success as a Research Scientist at Google DeepMind, we look for the following skills and experience:

• PhD in Computer Science or related field, or equivalent practical experience.
• Strong publication record in top machine learning conferences (e.g., NeurIPS, CVPR, ICML, ICLR, ICCV, ECCV).
• Experience in at least one modern deep learning framework (e.g. JAX, PyTorch).

In addition, some of the following would be an advantage:

• Experience with large scale models evaluation and statistical analysis of model capabilities.
• Deep knowledge in scaling laws and the relationship between data, model abilities and emergent abilities.
• Experience with data leakage detection and prevention.
• Excited to collaborate across orgs and disciplines to leverage diverse perspectives and expertise and find creative solutions.

Application Deadline: 15th August

At Google DeepMind, we value diversity of experience, knowledge, backgrounds and perspectives and harness these qualities to create extraordinary impact. We are committed to equal employment opportunity regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or related condition (including breastfeeding) or any other basis as protected by applicable law. If you have a disability or additional need that requires accommodation, please do not hesitate to let us know.