Software Engineer, Data Flywheel Platform

Necessary Ventures; Sunnyvale (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Necessary Ventures in Sunnyvale, CA seeks a software engineer to design and scale data curation pipelines, transforming fleet data into high-signal training data for foundation models. The role emphasizes production-grade Python and distributed systems, with collaboration across teams to accelerate AI development.

You will help build evaluation, training, and serving platforms for AI models, enabling the company's AI data flywheel and scalable data infrastructure.

Qualifications

• Requires strong production software engineering experience in Python and distributed systems, with a track record of shipping high-throughput data or ML systems.

Responsibilities

• Build and scale the data curation and enrichment pipelines that transform world-scale fleet data into high-signal training data for foundation models.

Skills

Python

Distributed Systems

Data Pipelines

System Design

SQL

Ray

Spark

Kubernetes

MLOps

Vector Search

Data Curation

Distributed Training

API Development

Workflow Orchestration

Lakehouse Architecture

Performance Optimization

Job description

Build and scale the data curation and enrichment pipelines that transform world-scale fleet data into high-signal training data for foundation models. Develop the evaluation infrastructure and training/serving systems to support the company's AI data flywheel.

Requirements

Requires strong production software engineering experience in Python and distributed systems, with a track record of shipping high-throughput data or ML systems. Candidates should have roughly 6 or more years of experience and strong computer science fundamentals.

Key Skills

Python, Distributed Systems, Data Pipelines, System Design, SQL, Ray, Spark, Kubernetes, MLOps, Vector Search, Data Curation, Distributed Training, API Development, Workflow Orchestration, Lakehouse Architecture, Performance Optimization