Senior Inference Engineer, AI Infrastructure & Production

Hamilton Barnes

United States

On-site

USD 225,000 - 275,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Full Benefits

Job summary

Hamilton Barnes is seeking an Inference Engineer to establish the technical foundation for inference, define architecture, and deploy production-ready systems. You will optimize model execution, memory management, and scheduling across machines, collaborating with Platform, Fleet, and Network Infrastructure teams.

The role covers kernels, compilers, inference engines, and serving APIs, balancing latency, throughput, model quality, reliability, and cost.

Qualifications

  • Production inference engines, GPU compute software, or distributed ML systems experience.
  • Strong C++ or Rust systems programming with Python proficiency.
  • Understanding transformer inference, including attention, batching, KV caches.
  • Experience with GPU programming, memory hierarchies, and performance analysis.
  • Distributed systems fundamentals: scheduling, concurrency, networking, and fault tolerance.
  • Experience delivering software from architecture to production deployment.
  • Ability to diagnose bottlenecks and implement measured improvements.
  • Willingness to own technical direction and communicate tradeoffs.

Responsibilities

  • Build and optimize the inference engine with model execution and memory management.
  • Write performance-critical kernels and runtime code across hardware and runtimes.
  • Design for high-performance hardware and profile on real devices.
  • Build distributed inference and its communication layer with model sharding and data movement.
  • Own inference scheduling, routing, and autoscaling to meet latency targets.
  • Ship a production inference service with loading, deployment, APIs, streaming, and backpressure.
  • Develop benchmarks to measure latency, throughput, and cost per token.
  • Set engineering direction and align with customer workloads and open-source projects.

Skills

Production inference engines
C++/Rust
Transformer inference
GPU programming
Distributed systems
Production deployment
Problem solving
Technical leadership

Job description

Hamilton Barnes is seeking an Inference Engineer to establish the technical foundation for inference, define architecture, and deploy production-ready systems. You will optimize model execution, memory management, and scheduling across machines, collaborating with Platform, Fleet, and Network Infrastructure teams.

The role covers kernels, compilers, inference engines, and serving APIs, balancing latency, throughput, model quality, reliability, and cost.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Distributed Systems Engineer — AI Inference Platform
Senior Distributed Systems Engineer — AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Equity
Comprehensive medical for US
Flexible PTO
+4
Inference Platform Engineer — Production-Scale AI
Inference Platform Engineer — Production-Scale AI

Designworks Talent • Bellevue (WA)

Hybrid
USD 150,000 - 230,000
Medical, dental, and vision insurance
401(k) with company match
Paid holidays
Senior AI Inference Engineer, Enterprise Platform & GPUs
Senior AI Inference Engineer, Enterprise Platform & GPUs

Uniphore • Palo Alto (CA)

On-site
USD 159,000 - 199,000
Annual incentive
Stock options
Medical, dental, vision
+3
Performance Engineer, Inference Systems
Performance Engineer, Inference Systems

Anthropic Limited • Seattle (WA)

Hybrid
USD 350,000 - 850,000
INFERENCE ENGINEER
INFERENCE ENGINEER

Makermaker • San Francisco (CA)

On-site
USD 120,000 - 160,000
Performance Engineer, Inference Systems
Performance Engineer, Inference Systems

Anthropic Limited • San Francisco (CA)

Hybrid
USD 350,000 - 850,000
Backend Platform Engineer - AI Inference & Scale
Backend Platform Engineer - AI Inference & Scale

present of AI • San Francisco (CA)

On-site
USD 140,000 - 200,000
Engineering Manager, Scalable Inference Platform
Engineering Manager, Scalable Inference Platform

Anthropic Limited • San Francisco (CA)

Hybrid
USD 405,000 - 625,000
Competitive compensation
Benefits
Equity donation matching
+4
Senior Backend Engineer - AI Inference & Low-Latency
Senior Backend Engineer - AI Inference & Low-Latency

Fdetimes • San Francisco (CA)

On-site
USD 200,000 - 290,000
Startup equity
Health insurance
Staff Engineer, AI Inference & Performance
Staff Engineer, AI Inference & Performance

Reflection • United States

Remote
USD 150,000 - 230,000
Top-tier compensation
Stock options
Health & wellness
+5