Head of Edge AI Inference

Montauk Capital; New York (NY); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Competitive compensation + equity

Studio support from Montauk Capital’s network

Job summary

Montauk Capital seeks a Head of Inference to define and lead the technical architecture for Edge AI. This role involves building a viable proof of concept and optimizing GPU utilization for production workloads. The ideal candidate has hands-on experience in production inference systems and is comfortable making foundational decisions rapidly. Join us to solve the AI inference bottleneck and participate in a massive market opportunity in AI technology.

Qualifications

• Hands-on technical builder with experience in production inference systems.
• Deep knowledge of model serving and practical engineering.
• Capacity to quickly translate concepts into viable proof of concepts (POCs).

Responsibilities

• Define inference strategy and architecture for Edge AI.
• Build credible POC that proves platform functionality.
• Optimize GPU utilization and manage production workloads.

Skills

Production inference serving — vLLM, TensorRT‑LLM, Triton Inference Server

GPU utilization and CUDA kernel optimization

Quantization, SGLang, containerization

Technical leadership experience

Batching, KV‑cache, speculative decoding expertise

Tools

Kubernetes

Ray

Job description

Montauk Capital seeks a Head of Inference to define and lead the technical architecture for Edge AI. This role involves building a viable proof of concept and optimizing GPU utilization for production workloads. The ideal candidate has hands-on experience in production inference systems and is comfortable making foundational decisions rapidly. Join us to solve the AI inference bottleneck and participate in a massive market opportunity in AI technology.