Staff + Senior Software Engineer, Inference

FDE Times; Ontario (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Competitive Compensation

Equity Donation Matching

Generous Vacation

Parental Leave

Flexible Working Hours

Job summary

FDE Times is designing, building, and operating distributed inference systems to serve Claude at scale. This role focuses on intelligent routing, load balancing, autoscaling, accelerator orchestration, and reliable model deployment to improve compute efficiency and researcher infrastructure.

You will work on production-grade ML infrastructure, optimizing routing, deployment pipelines, and multi-region deployments, with a strong emphasis on performance and observability.

Qualifications

• Requires significant software engineering experience, particularly with distributed systems.
• Preferred qualifications include large-scale distributed systems, production machine learning, routing or traffic management, LLM inference optimization, Kubernetes and cloud infrastructure.
• A bachelor's degree or equivalent education, training, or experience is the stated minimum.

Skills

Distributed Systems

Machine Learning Systems

High-Performance Computing

Inference Optimization

Request Routing

Load Balancing

Traffic Management

Autoscaling

Fleet Orchestration

Deployment Pipelines

Kubernetes

Cloud Infrastructure

Python

Rust

Observability

Multi-Region Deployments

Education

Bachelor's degree or equivalent

Tools

Kubernetes

Cloud Platforms

Python

Rust

Job description

Design, build, and operate distributed inference systems that serve Claude at scale, including intelligent routing, load balancing, autoscaling, accelerator orchestration, and reliable model deployment. Improve compute efficiency and performance while providing researchers with infrastructure for developing and serving new models and architectures.

Requirements

Requires significant software engineering experience, particularly with distributed systems, along with flexibility, a results-oriented approach, and interest in machine learning infrastructure and its societal impact. Preferred qualifications include large-scale distributed systems, production machine learning, routing or traffic management, LLM inference optimization, Kubernetes and cloud infrastructure, and proficiency in Python or Rust; a bachelor's degree or equivalent education, training, or experience is the stated minimum.

Key Skills

Distributed Systems, Machine Learning Systems, High-Performance Computing, Inference Optimization, Request Routing, Load Balancing, Traffic Management, Autoscaling, Fleet Orchestration, Deployment Pipelines, Kubernetes, Cloud Infrastructure, Python, Rust, Observability, Multi-Region Deployments

Benefits

Competitive Compensation, Equity Donation Matching, Generous Vacation, Parental Leave, Flexible Working Hours