Senior AI Platform Engineer — Scalable Agent & LLM Infra (Hybrid)

Soter Insure

New York (NY)

Hybrid

USD 180,000 - 280,000

Full time

10 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Snorkel AI is seeking Senior/Staff Software Engineers for the AI Platform to design and build scalable infrastructure that powers AI development at scale—covering synthetic data generation, evaluation pipelines, simulation environments, and LLM infrastructure.

You'll work on non-deterministic workloads, build distributed systems for experimentation and production execution, and collaborate with research, product, and engineering teams to turn experimental workflows into reliable platform

Qualifications

  • 5+ years building production software systems, with AI/ML infrastructure or distributed systems experience.
  • Experience operating non-deterministic AI/ML workloads in production or at scale.
  • Experience building infrastructure for experimentation, evaluation, model development, synthetic data, or production ML systems.
  • Strong proficiency in Python and production-quality APIs/services.
  • Background in distributed systems and cloud platforms (AWS preferred).
  • Experience with workflow/distributed execution frameworks like Prefect, Airflow, Dagster, Ray, Kubernetes.
  • Understanding of observability, telemetry, reliability, and cost management.
  • Ability to reason about AI system quality, experimental reproducibility, and model variability.
  • Track record of leading complex engineering initiatives and measurable impact.

Responsibilities

  • Design and build infrastructure for running large-scale agentic workloads, including multi-step agents interacting with tools, external services, sandboxes, and simulated environments
  • Build scalable synthetic data generation and automated labeling systems for high-quality training/evaluation datasets
  • Design evaluation infrastructure to measure AI system behavior across models, prompts, tools, environments, and trajectories with reproducible experiments
  • Build orchestration and distributed compute systems for running thousands to millions of AI experiments and simulations
  • Develop infrastructure for agent simulation environments, including provisioning, isolation, lifecycle management, and scalable execution
  • Build and operate LLM infrastructure for routing, rate limiting, retries, caching, provider failover, and cost attribution
  • Instrument agent and model workloads to capture traces, interactions, tool calls, environment state, results, latency, reliability, and cost
  • Design systems to make non-deterministic workloads reproducible and measurable, enabling comparison of experiments and diagnosis of regressions
  • Improve developer experience for AI experimentation with APIs, SDKs, workflows, and tooling for moving from local to large-scale production
  • Collaborate with research, product, and eng teams to turn experimental workflows into reliable platform capabilities

Skills

Python
Distributed systems
Cloud platforms
APIs and dev tooling
System design
Observability
Experimentation/AB testing
Team leadership / impact

Tools

Kubernetes
Airflow
Dagster
Ray
Prefect
Terraform

Job description

Snorkel AI is seeking Senior/Staff Software Engineers for the AI Platform to design and build scalable infrastructure that powers AI development at scale—covering synthetic data generation, evaluation pipelines, simulation environments, and LLM infrastructure.

You'll work on non-deterministic workloads, build distributed systems for experimentation and production execution, and collaborate with research, product, and engineering teams to turn experimental workflows into reliable platform

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Platform Engineer – Scale & Observability
Senior AI Platform Engineer – Scale & Observability

Fdetimes • New York (NY)

On-site
USD 180,000 - 240,000
Senior AI Engineer / Staff — Production-Scale AI Infra
Senior AI Engineer / Staff — Production-Scale AI Infra

Snorkel AI • San Francisco (CA)

On-site
USD 180,000 - 260,000
Senior AI Systems Engineer — Remote, Scalable Platforms
Senior AI Systems Engineer — Remote, Scalable Platforms

ContentBuffer • San Francisco (CA)

Hybrid
USD 208,000 - 315,000
Senior Platform Engineer — Infra & Data Pipelines
Senior Platform Engineer — Infra & Data Pipelines

Soter Insure • New York (NY)

On-site
USD 220,000 - 300,000
Senior Software Engineer — Remote AI Systems
Senior Software Engineer — Remote AI Systems

Snorkel AI • Oregon

Hybrid
USD 208,000 - 315,000
Senior Software Engineer — Remote AI/ML Systems & Eval
Senior Software Engineer — Remote AI/ML Systems & Eval

JoinStartup • San Francisco (CA)

Remote
USD 208,000 - 315,000
Senior Software Engineer - AI Data Systems (Remote)
Senior Software Engineer - AI Data Systems (Remote)

Snorkel AI • United States

Remote
USD 208,000 - 315,000
Platform Engineer: Scale Data Pipelines & Infra
Platform Engineer: Scale Data Pipelines & Infra

Snorkel AI • New York (NY)

On-site
USD 180,000 - 230,000
Senior Software Engineer, AI Foundations & Scale
Senior Software Engineer, AI Foundations & Scale

Snorkel AI • United States

Remote
USD 180,000 - 230,000
Senior Manager, Forward Deployed AI Engineering — Remote
Senior Manager, Forward Deployed AI Engineering — Remote

JoinStartup • New York (NY), San Francisco (CA)

Hybrid
USD 260,000 - 350,000
Equity
Benefits