Senior Inference Engineer, AI Infrastructure & Production

Hamilton Barnes; United States; On-site; USD 225,000 - 275,000; Full time

Benefits offered by this job

Full Benefits

Job summary

Hamilton Barnes is seeking an Inference Engineer to establish the technical foundation for inference, define architecture, and deploy production-ready systems. You will optimize model execution, memory management, and scheduling across machines, collaborating with Platform, Fleet, and Network Infrastructure teams.

The role covers kernels, compilers, inference engines, and serving APIs, balancing latency, throughput, model quality, reliability, and cost.

Qualifications

• Production inference engines, GPU compute software, or distributed ML systems experience.
• Strong C++ or Rust systems programming with Python proficiency.
• Understanding transformer inference, including attention, batching, KV caches.
• Experience with GPU programming, memory hierarchies, and performance analysis.
• Distributed systems fundamentals: scheduling, concurrency, networking, and fault tolerance.
• Experience delivering software from architecture to production deployment.
• Ability to diagnose bottlenecks and implement measured improvements.
• Willingness to own technical direction and communicate tradeoffs.

Responsibilities

• Build and optimize the inference engine with model execution and memory management.
• Write performance-critical kernels and runtime code across hardware and runtimes.
• Design for high-performance hardware and profile on real devices.
• Build distributed inference and its communication layer with model sharding and data movement.
• Own inference scheduling, routing, and autoscaling to meet latency targets.
• Ship a production inference service with loading, deployment, APIs, streaming, and backpressure.
• Develop benchmarks to measure latency, throughput, and cost per token.
• Set engineering direction and align with customer workloads and open-source projects.

Skills

Production inference engines

C++/Rust

Transformer inference

GPU programming

Distributed systems

Production deployment

Problem solving

Technical leadership

Job description

Hamilton Barnes is seeking an Inference Engineer to establish the technical foundation for inference, define architecture, and deploy production-ready systems. You will optimize model execution, memory management, and scheduling across machines, collaborating with Platform, Fleet, and Network Infrastructure teams.

The role covers kernels, compilers, inference engines, and serving APIs, balancing latency, throughput, model quality, reliability, and cost.