Platform Engineer, AI Data Flywheel Pipelines

Necessary Ventures; Sunnyvale (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Necessary Ventures in Sunnyvale, CA seeks a software engineer to design and scale data curation pipelines, transforming fleet data into high-signal training data for foundation models. The role emphasizes production-grade Python and distributed systems, with collaboration across teams to accelerate AI development.

You will help build evaluation, training, and serving platforms for AI models, enabling the company's AI data flywheel and scalable data infrastructure.

Qualifications

• Requires strong production software engineering experience in Python and distributed systems, with a track record of shipping high-throughput data or ML systems.

Responsibilities

• Build and scale the data curation and enrichment pipelines that transform world-scale fleet data into high-signal training data for foundation models.

Skills

Python

Distributed Systems

Data Pipelines

System Design

SQL

Ray

Spark

Kubernetes

MLOps

Vector Search

Data Curation

Distributed Training

API Development

Workflow Orchestration

Lakehouse Architecture

Performance Optimization

Job description

Necessary Ventures in Sunnyvale, CA seeks a software engineer to design and scale data curation pipelines, transforming fleet data into high-signal training data for foundation models. The role emphasizes production-grade Python and distributed systems, with collaboration across teams to accelerate AI development.

You will help build evaluation, training, and serving platforms for AI models, enabling the company's AI data flywheel and scalable data infrastructure.