System Software Engineer - AI

Delos Data Inc

United States

Hybrid

USD 140,000 - 220,000

Full time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
401(k) plan
Health benefits

Job summary

Delos Data Inc. seeks an experienced engineer to develop system software that helps large AI models run efficiently across distributed GPU clusters.

You will connect AI frameworks with low-level infrastructure, design communication primitives, and explore performance across hardware and software. The role offers hybrid work in the United States, with a base salary between $140,000 and $220,000 plus equity and benefits.

Qualifications

  • At least five years of relevant experience.
  • Strong C++ and Python skills; memory management and concurrency.
  • Experience in Linux environments and OS internals.
  • Familiarity with networking protocols and concepts.
  • Bachelor's or master's in computer engineering/computer science or related field.

Responsibilities

  • Develop system software for distributed AI model training and inference.
  • Analyze performance of distributed workloads and identify bottlenecks.
  • Investigate issues at software-hardware boundary.
  • Benchmark on multi-node clusters and refine primitives.
  • Bridge AI frameworks with low-level infrastructure.

Skills

C++
Python
Memory management
Concurrency
OS internals
Linux
Networking

Education

Bachelor's/Master's in CS/CE

Tools

CUDA programming
PyTorch
JAX
DeepSpeed
vLLM
SGLang
TRT-LLM
Cluster orchestration

Job description

Role overview

Develop system software that helps large AI models run efficiently across distributed GPU clusters. This role connects AI frameworks with low-level infrastructure by designing communication and execution primitives, investigating performance across hardware and software, and shaping foundational technology for distributed training and inference.

Responsibilities
  • Work across the software stack to shape infrastructure around the needs of emerging AI models.
  • Analyze distributed training and inference workloads to find performance bottlenecks.
  • Investigate issues at the software and hardware boundary.
  • Benchmark and characterize performance on multi-node clusters.
  • Help build and refine communication and execution primitives for large-scale workloads.
Requirements
  • At least five years of relevant experience.
  • Strong C++ and Python skills, with knowledge of memory management, concurrency, and operating system internals.
  • Proficiency developing in Linux environments.
  • Familiarity with networking protocols and concepts.
  • Bachelor's or master's degree in computer engineering, computer science, or a related field.
Nice to have
  • CUDA programming or performance optimization for parallel architectures.
  • Experience with distributed AI frameworks such as PyTorch, JAX, or DeepSpeed, or inference engines such as vLLM, SGLang, or TRT-LLM.
  • Hands-on experience with cluster orchestration and telemetry at scale.Experience deploying production systems and managing releases.
Benefits and work setup
  • Hybrid work based in Palo Alto, California, or Charlottesville, Virginia.
  • Advertised target base salary of $140,000–$220,000 per year, plus equity, benefits, and a 401(k). The range depends on role level, experience, and location.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

System Software Engineer - AI
System Software Engineer - AI

Delos Data Inc • Palo Alto (CA)

Hybrid
USD 140,000 - 200,000
Equity
401k
Benefits
AI Infra Systems Engineer — Hybrid, GPU & Scale
AI Infra Systems Engineer — Hybrid, GPU & Scale

Delos Data Inc • Palo Alto (CA)

Hybrid
USD 140,000 - 200,000
Equity
401k
Benefits
Software Development Engineer in Test - AI
Software Development Engineer in Test - AI

Delos Data Inc • United States

Hybrid
USD 140,000 - 220,000
Equity
401(k)
Hybrid work arrangement
AI/ML Engineer – (Next-Generation AI Platforms & Workloads)
AI/ML Engineer – (Next-Generation AI Platforms & Workloads)

Veear Projects Inc. • Sunnyvale (CA)

On-site
USD 140,000 - 210,000
Applied AI/ML Engineer
Applied AI/ML Engineer

7Seventy • United States

Hybrid
USD 175,000 - 250,000
Health insurance
Dental insurance
Vision insurance
+4
Member of Technical Staff, ML Systems
Member of Technical Staff, ML Systems

true-talent-solutions-partners • Menlo Park (CA)

On-site
USD 180,000 - 230,000
AI Engineer
AI Engineer

CVS Health • United States

Remote
USD 79,000 - 159,000
Senior AI / ML Engineer
Senior AI / ML Engineer

Dataklid AI Services • United States

Remote
USD 130,000 - 180,000
Remote work
SF option
Equity included
+1
Senior AI Engineer
Senior AI Engineer

Goliath Partners Inc. • San Francisco (CA)

Hybrid
USD 270,000 - 325,000
Member of Technical Staff (AI Inference Engineer)
Member of Technical Staff (AI Inference Engineer)

Kindredventures • Palo Alto (CA)

Hybrid
USD 190,000 - 250,000
Comprehensive health insurance
Dental insurance
Vision insurance
+1