Edge AI Data Scientist: Multimodal Model Evaluation

Amazon Inc.; Cambridge; On-site; GBP 65,000 - 95,000; Full time

Job summary

Amazon's Edge AI Product Science team in Cambridge is seeking a Data Scientist to advance multimodal model evaluation for devices, including audio, vision, and language components. You will work with cross-functional engineers to build robust assessment methods.

You will curate datasets, develop annotation pipelines, and prototype evaluation strategies that guide model improvement and deployment at the edge.

Qualifications

• Master's degree in a quantitative field such as statistics, mathematics, data science, business analytics, economics, finance, engineering, or computer science.
• Experience with machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance.
• Experience applying theoretical models in an applied environment.
• Experience with data scripting languages (e.g. SQL, Python, R etc.) or statistical/mathematical software (e.g. R, SAS, or Matlab).

Responsibilities

• Collaborate with cross-functional engineers and scientists to advance the state of the art in multimodal model evaluations for devices, including audio, images, and videos
• Invent and validate reliability for novel automated evaluation methods for perception tasks, such as fine-tuned LLM-as-judge
• Develop and extend our evaluation frameworks to support expanding capabilities for multimodal language models
• Analyse large offline and online datasets using Python or similar to understand model gaps, develop methods to interpret model failures, and collaborate with training teams to enhance model capabilities for product use cases
• Work closely with other scientists, compiler engineers, data collection, and product teams to advance evaluation methods

Skills

Statistical analysis

ML modeling

Data visualization

Education

Master's degree in a quantitative field

Tools

Python

SQL

R

Matlab

SAS

Job description

Amazon's Edge AI Product Science team in Cambridge is seeking a Data Scientist to advance multimodal model evaluation for devices, including audio, vision, and language components. You will work with cross-functional engineers to build robust assessment methods.

You will curate datasets, develop annotation pipelines, and prototype evaluation strategies that guide model improvement and deployment at the edge.