Model API Engineer - High-Performance Inference

Baseten; New York (NY); On-site; USD 180,000 - 360,000; Full time

Benefits offered by this job

Competitive compensation including equity

100% medical, dental, and vision insurance

Generous PTO including Winter Break

Paid parental leave

Company-facilitated 401(k)

Job summary

A leading AI platform provider in New York seeks a Model API Engineer focused on infrastructure for hosted AI models. Responsibilities include optimizing performance, building benchmarking frameworks, and collaborating on robust model serving solutions. Ideal candidates have 3+ years in distributed systems and a strong background in backend services. With a competitive compensation range of $180K - $360K and a comprehensive benefits package, this role offers a unique opportunity to shape the future of AI within a diverse and inclusive team.

Qualifications

• 3+ years experience in building and operating distributed systems or large-scale APIs.
• Proven track record with low-latency, reliable backend services.
• Comfortable debugging complex systems, with strong written communication skills.

Responsibilities

• Design and operate the Model APIs surface with advanced inference capabilities.
• Profile and optimize TensorRT-LLM kernels for performance.
• Build benchmarking frameworks to measure real-world performance.

Skills

Distributed systems

Low-latency backend services

Performance profiling

Debugging complex systems

Strong written communication

Tools

Kubernetes

API gateways

Job description

A leading AI platform provider in New York seeks a Model API Engineer focused on infrastructure for hosted AI models. Responsibilities include optimizing performance, building benchmarking frameworks, and collaborating on robust model serving solutions. Ideal candidates have 3+ years in distributed systems and a strong background in backend services. With a competitive compensation range of $180K - $360K and a comprehensive benefits package, this role offers a unique opportunity to shape the future of AI within a diverse and inclusive team.