Model API Engineer — High-Performance Inference

The Consensus; New York (NY); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Competitive compensation including equity

100% coverage of medical, dental, and vision insurance

Flexible PTO policy

Paid parental leave

Fertility and family-building stipend

Company-facilitated 401(k)

Job summary

The Consensus is seeking a skilled engineer to join our Model Performance team, focusing on building and optimizing APIs used in AI applications. You will contribute to enhancing the performance and reliability of our model serving infrastructure, ensuring it meets the needs of developers.

This role requires experience with distributed systems and a strong foundation in backend services, allowing you to work effectively within our small, high-impact team. Enjoy competitive compensation, health insurance, flexible PTO, and an inclusive workplace at The Consensus.

Qualifications

• 3+ years of experience building and operating distributed systems or large-scale APIs.
• Proven track record of low-latency, reliable backend services.
• Comfortable debugging complex systems from runtime internals to GPU execution traces.

Responsibilities

• Design, build, and operate Model APIs with advanced inference capabilities.
• Profile and optimize TensorRT-LLM kernels and analyze CUDA kernel performance.
• Productionize performance improvements across runtimes with deep understanding.
• Build comprehensive benchmarking frameworks for real-world performance measurement.
• Collaborate closely with teams to deliver robust model serving experiences.

Skills

Distributed systems

Large-scale APIs

Profiling and tracing

Debugging complex systems

Strong written communication

Tools

TensorRT

CUDA

Kubernetes

Job description

The Consensus is seeking a skilled engineer to join our Model Performance team, focusing on building and optimizing APIs used in AI applications. You will contribute to enhancing the performance and reliability of our model serving infrastructure, ensuring it meets the needs of developers.

This role requires experience with distributed systems and a strong foundation in backend services, allowing you to work effectively within our small, high-impact team. Enjoy competitive compensation, health insurance, flexible PTO, and an inclusive workplace at The Consensus.