Staff AI Platform Engineer — Inference & Scale Leader

Greenhouse Software, Inc.; New York, Northern (NY, KY); Hybrid; USD 220,000 - 330,000; Full time

Job summary

Nscale is seeking a Staff AI Engineer (Specialised) to steer technical direction for parts of the inference platform at the core of our AI cloud. You will influence latency, throughput, and cost of tokens, guiding standards across serving, post-training, and platform teams.

We run state-of-the-art GPU systems and push to close gaps with open-source tooling, mentoring engineers as we go. This role demands deep expertise and hands-on leadership across complex AI workloads.

Qualifications

• 8–12 years of engineering experience, with significant depth in production AI systems or ML research at scale.
• 4+ years of hands-on work with LLMs in at least one focus area, in production or research.
• Deep, demonstrated expertise in at least one focus area.
• Enough working knowledge of the full serving stack to debug across API, router, scheduler, engine, kernel, and cluster.
• A track record of setting technical direction and creating standards or tools adopted beyond your own team.
• Strong Python and PyTorch, with production-grade systems.

Responsibilities

• Set technical direction for your focus areas and turn it into work that multiple teams can deliver.
• Lead the resolution of systemic performance and reliability problems across the serving stack, from kernel bottlenecks to fleet-level capacity and multi-tenant isolation.
• Own the trade-offs between cost, latency, throughput, and model quality in your area, and back them with measurement.
• Build reusable frameworks, benchmarks, and tooling that make other AI engineers at Nscale more effective.
• Evaluate emerging serving engines, kernel libraries, RL frameworks, and accelerators, and make clear build/adopt/contribute recommendations.
• Coach and grow engineers across teams, and raise engineering quality broadly.
• Work with research, product, and infrastructure leadership so the platform tracks customer demand.
• Represent your area in cross-team technical reviews and planning.

Skills

Python

PyTorch

LLM Expertise

System Design

Tools

Kubernetes

CUDA

Triton

CUTLASS

Job description

Nscale is seeking a Staff AI Engineer (Specialised) to steer technical direction for parts of the inference platform at the core of our AI cloud. You will influence latency, throughput, and cost of tokens, guiding standards across serving, post-training, and platform teams.

We run state-of-the-art GPU systems and push to close gaps with open-source tooling, mentoring engineers as we go. This role demands deep expertise and hands-on leadership across complex AI workloads.