AI Model Evaluation Engineer — Benchmarking & Validation

SpreeAI; San Francisco (CA); Hybrid; USD 120,000 - 160,000; Full time

Job summary

A fast-growing AI company seeks a Software Engineer to focus on Model Evaluation & Benchmarking. This role involves building evaluation systems for multimodal AI, ensuring reliable performance. The ideal candidate will possess strong Python programming skills, familiarity with machine learning workflows, and experience in automation frameworks. Join us to redefine fashion and e-commerce with cutting-edge AI solutions. Passion for innovation in tech is a must. Enjoy a dynamic environment where creativity and technology drive real impact.

Qualifications

• Degree in Computer Science, AI, Engineering, or comparable combination of education and practical experience.
• Strong programming skills in Python.
• Familiarity with object-oriented programming (C++, Java, Python, or similar).
• Strong data structures and algorithms fundamentals.
• Understanding of machine learning experimentation workflows.

Responsibilities

• Build automated evaluation pipelines for multimodal AI models.
• Benchmark diffusion models, vision systems, and generative workflows.
• Validate model checkpoints and detect regressions across versions.
• Develop evaluation metrics for realism, consistency, and performance.
• Integrate evaluation tooling into CI/CD workflows.
• Collaborate with ML researchers and infrastructure teams.
• Analyze failure modes and propose evaluation strategies.

Skills

Programming skills in Python

Familiarity with object-oriented programming

Data structures and algorithms

Understanding of machine learning experimentation workflows

Education

Degree in Computer Science, AI, Engineering or equivalent

Tools

NumPy

Pandas

Job description

A fast-growing AI company seeks a Software Engineer to focus on Model Evaluation & Benchmarking. This role involves building evaluation systems for multimodal AI, ensuring reliable performance. The ideal candidate will possess strong Python programming skills, familiarity with machine learning workflows, and experience in automation frameworks. Join us to redefine fashion and e-commerce with cutting-edge AI solutions. Passion for innovation in tech is a must. Enjoy a dynamic environment where creativity and technology drive real impact.