Senior Software Engineer, AI Infrastructure

3Y Health; San Francisco (CA); Hybrid; USD 175,000 - 205,000; Full time

Job summary

A leading company in healthcare technology seeks a Senior Software Engineer to architect and scale its AI infrastructure. You will be instrumental in building and optimizing robust systems that power machine learning models, collaborating with cross-functional teams to enhance deployment workflows and ensure efficiency. Your expertise with distributed systems and a deep understanding of ML frameworks will be crucial in this hybrid role, primarily located in San Francisco.

Qualifications

• 3+ years of software engineering experience, 2+ focused on ML infrastructure.
• Deep familiarity with ML frameworks and tools.
• Production experience with Kubernetes and scaling workloads.

Responsibilities

• Design and maintain distributed systems for ML model training and evaluation.
• Optimize compute performance and data throughput using technologies like CUDA.
• Develop infrastructure abstractions for standardized model experimentation.

Skills

Python

Go

C++

Rust

GPU optimization

Distributed systems

ML frameworks

Container orchestration

Education

Bachelor's degree in Computer Science or related field

Tools

Kubernetes

PyTorch

Apache Spark

TensorFlow

CUDA

Job description

Senior Software Engineer, AI Infrastructure

Senior Software Engineer, AI Infrastructure

1 week ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

About Us

At 3Y Health, we are building AI-driven software to empower healthcare providers and solve the overwhelming administrative complexity that consumes 40% of the industry’s revenue. Our end-to-end platform unlocks opportunities for clinician entrepreneurs, enabling medical professionals to launch, run, and grow private practices. By supporting these independent practices with the latest AI and automation, we’re helping providers reclaim their time, build thriving businesses, and deliver better outcomes for their communities. 3Y Health is backed by over $200M from top-tier investors including Founders Fund, General Catalyst, Softbank, and 8VC.

About Us

At 3Y Health, we are building AI-driven software to empower healthcare providers and solve the overwhelming administrative complexity that consumes 40% of the industry’s revenue. Our end-to-end platform unlocks opportunities for clinician entrepreneurs, enabling medical professionals to launch, run, and grow private practices. By supporting these independent practices with the latest AI and automation, we’re helping providers reclaim their time, build thriving businesses, and deliver better outcomes for their communities. 3Y Health is backed by over $200M from top-tier investors including Founders Fund, General Catalyst, Softbank, and 8VC.

About the Role

We're seeking a Senior Software Engineer, AI Infrastructure to architect and scale our internal machine learning platform. Unlike traditional ML Engineers, your focus will not be on developing models—but on building and optimizing the infrastructure that supports them. You’ll collaborate closely with researchers, engineers, and product teams to support training, evaluation, and deployment workflows, ensuring our systems are robust, scalable, and efficient. This role will be hybrid with the expectation of 3 days in the office.

Responsibilities

• Design and maintain distributed systems for ML model training and evaluation.
• Build and scale ML pipelines using frameworks like PyTorch, Apache Spark, and Kubernetes.
• Optimize compute performance, GPU utilization, and data throughput using technologies like CUDA, NVIDIA Triton, and low-level tuning.
• Develop infrastructure abstractions to standardize model experimentation and deployment.
• Partner with ML researchers and product teams to ensure the infra stack aligns with evolving ML needs.
• Establish robust monitoring, profiling, and logging tools for model performance and infrastructure reliability.
• Contribute to infrastructure codebases and tooling libraries used across the ML lifecycle.
  
  

Qualifications

• 3+ years of software engineering experience, with at least 2+ focused on ML infrastructure or distributed systems.
• Deep familiarity with ML frameworks and tools (e.g., PyTorch, TensorFlow, Spark, Ray, NVIDIA stack, etc.).
• Production experience with Kubernetes, container orchestration, and scaling training workloads across clusters.
• Strong understanding of GPU architecture and performance optimization with CUDA or similar technologies.
• Practical experience in building end-to-end ML pipelines, from training to serving, but with an infra-first mindset.
• Comfortable writing clean, scalable code in Python, Go, or a systems language (e.g., C++, Rust).
• Not focused on DevOps/SRE work—this is a deeply technical infra role rooted in ML systems.
  
  

Compensation

The estimated salary range for this role is $175,000 - $205,000. Total compensation for this position may also include stock options. Note that total compensation for this position will be determined by each individual’s relevant qualifications, work experience, skills, and other factors.

Seniority level

• Seniority level

  Not Applicable

Employment type

• Employment type

  Full-time

Job function

• Industries

  Real Estate, Financial Services, and Capital Markets

Referrals increase your chances of interviewing at 3Y Health by 2x

Get notified about new Senior Software Engineer jobs in San Francisco, CA.

Senior Software Engineer, Handshake Plus - Monetization

San Francisco, CA $180,000.00-$220,000.00 2 weeks ago

San Mateo, CA $90,000.00-$150,000.00 11 months ago

Sr. Software Engineer - Investment Manager

Senior / Staff Software Engineer - Computational Chemistry / Molecular Dynamics

San Francisco, CA $204,000.00-$259,000.00 2 days ago

Sr. Software Engineer - Payments Support

San Francisco, CA $138,400.00-$173,000.00 2 weeks ago

Principal Software Engineer - Computational Chemistry / Molecular Dynamics

Senior Software Quality Engineer (Non-Product)

Alameda, CA $98,000.00-$196,000.00 2 weeks ago

Staff Software Engineer / Tech Lead Manager

Founding Team Senior Software Engineer Manager for a Social Travel Platform targeting Digital Nomads

San Francisco, CA $150,000.00-$200,000.00 3 months ago

Senior Robotics Software Engineer, Planning and Control

San Francisco, CA $150,000.00-$240,000.00 4 days ago

San Mateo, CA $448,990.00-$515,210.00 2 days ago

Senior Software Engineer, Simulation Tooling

Software Engineering Technical Leader (Hybrid Remote- San Jose, CA)

San Francisco, CA $198,600.00-$282,900.00 2 weeks ago

Senior Software Quality Engineer (I, II, III)

Redwood City, CA $79,800.00-$178,100.00 3 weeks ago

Manager, Software Quality Engineering in Test

San Mateo, CA $195,500.00-$293,300.00 1 month ago

Senior Engineering Manager - Backend Developer User Experience

Senior Engineer - Warehouse Management System

Brisbane, CA $140,000.00-$180,000.00 3 days ago

Principal Software Engineer- Input & Controls

San Mateo, CA $283,780.00-$331,640.00 1 day ago

Redwood City, CA $140,000.00-$198,000.00 5 days ago

Lead Software Engineering Program Manager, AI

San Francisco, CA $205,000.00-$240,000.00 2 weeks ago

San Francisco, CA $180,000.00-$300,000.00 3 months ago

San Francisco, CA $95,000.00-$230,000.00 4 months ago

Senior Data Scientist - Credit & Lending

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.