AI Systems Evaluation Engineer - Vision & Agents

Socket.dev; Sunnyvale (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Apple DAQ team seeks an AI Engineer specializing in algorithm evaluation and agentic systems design for advanced computer vision and video understanding. You will lead benchmarking, develop evaluation frameworks, and build autonomous multi-modal workflows bridging experimentation with production.

The role requires a strong foundation in ML/CV, PyTorch, and Python, with experience in evaluating vision-language models and building scalable pipelines.

Qualifications

• MS and at least 3 years of industry experience.
• 3+ years applied ML/CV/AI system evaluation.
• Strong ML foundations: probability, statistics, data distributions, and bias/variance.
• CV expertise: ViTs, spatial-temporal modeling, image/video processing.

Responsibilities

• Lead benchmarking and integration of state-of-the-art models for image and video understanding.
• Design robust evaluation frameworks and metrics.
• Develop autonomous multi-modal workflows bridging experimentation and production.
• Collaborate to define metrics from first principles and communicate findings clearly.

Skills

Python

PyTorch

Computer Vision

ML Evaluation

Statistics

Edge-case analysis

Metrics design

System-level thinking

Communication

Education

MS in Computer Science / ML

Tools

Jupyter

Linux

Job description

Apple DAQ team seeks an AI Engineer specializing in algorithm evaluation and agentic systems design for advanced computer vision and video understanding. You will lead benchmarking, develop evaluation frameworks, and build autonomous multi-modal workflows bridging experimentation with production.

The role requires a strong foundation in ML/CV, PyTorch, and Python, with experience in evaluating vision-language models and building scalable pipelines.