Principal AI Product Engineer

nscaleoperationsukltd

Georgia

On-site

USD 260,000 - 360,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Nscale is seeking a Principal AI Engineer to lead the inference and post-training pillar of our AI systems engineering group. You will shape the multi-year roadmap for how models are served, evaluated, and post-trained on Nscale's GPU cloud, including bring-your-own-model deployments.

You will mentor 20–50 engineers, set engineering standards, and drive cost, latency, and reliability targets to stay competitive with leading AI providers.

Qualifications

  • 10-15 years of engineering experience with pillar-level impact on production AI systems.
  • 4+ years hands-on work with LLMs in inference, GPU performance, evals, or post-training/RL.
  • Demonstrated ability to define multi-year technical strategy for large AI systems organizations.
  • Deep understanding of hardware/software boundary for AI accelerators: CUDA/ROCm, memory bandwidth, interconnects.

Responsibilities

  • Define and own the multi-year technical roadmap for inference, evals, and post-training platform.
  • Lead company-scale architectural initiatives across serving, GPU, and benchmark frameworks.
  • Establish engineering standards: API design, evals methodology, training stability norms.
  • Own cost, latency, throughput, and model quality trade-offs across the pillar.
  • Identify systemic risks early and align leadership on multi-team technical strategy.
  • Mentor Staff and Senior AI Engineers and grow the next generation of leaders.
  • Represent Nscale externally: open-source leadership, publications, conferences, partnerships.

Skills

LLM inference
GPU performance
Evals frameworks
Post-training RL
Technical leadership
Architectural design
Staff mentorship
Open source leadership

Tools

CUDA
ROCm
NVFP4

Job description

About Nscale

Nscale is taking on the hyperscalers by building a vertically integrated GenAI cloud platform. We own the data centers, software, and applications that power today's AI stack using sustainable technology solutions. We thrive on a culture of relentless innovation, ownership, and accountability, where every team member takes pride in their work and drives it with excellence and urgency. As a Nscaler, you'll build trust through openness and transparency, where everyone is inspired to do their best work. Collaboration is key, and we work together swiftly and respectfully, embracing adaptability and resilience in all we do.

About the Role

Nscale is looking for a Principal AI Engineer (Specialised) to lead the inference and post‑training pillar of our AI systems engineering organization. You'll define the multi‑year technical roadmap for how models are served, evaluated, and post‑trained on Nscale's GPU cloud, across dedicated and serverless inference and bring‑your‑own‑model deployments. You'll lead the most consequential architectural programmes in that space and set the engineering standards that 20-50+ engineers build to.

As a Principal engineer, you are one of the deepest technical authorities in the company on AI systems. Your decisions set the cost, latency, and reliability at which Nscale serves tokens and runs post‑training workloads, and those numbers have to compete with the world's leading AI infrastructure providers. The problems span the full stack: kernel efficiency on state‑of‑the‑art GPU systems, fleet‑level KV cache and serving architecture, the evals that prove model quality, and RL loops where inference and training share hardware. You frame the solutions the organization executes against, including the API contracts customers see.

How We Work

Dog years. We move quickly and compress a lot of learning into a short time.

Don't let perfect be the enemy of good. Ship, measure, iterate.

Be relentless. Own the problem end to end and see it through.

One team, one mission. Outcomes over process, and no "not my job".

Responsibilities
  • Define and own the multi‑year technical roadmap for Nscale's inference, evals, and post‑training platform, and translate it into architecture that multiple teams can execute against
  • Lead company‑scale architectural initiatives in the pillar, such as next‑generation serving (disaggregated prefill/decode, KV cache orchestration across GPU, host, and storage tiers, speculative decoding, multi‑tenant scheduling), GPU kernel and model efficiency work (custom kernels, FP8/NVFP4/INT8/4 quantization, sparsity, distillation, MoE serving), evals and benchmarking frameworks, and post‑training and RL infrastructure
  • Establish engineering standards adopted across all AI teams: API design and compatibility guarantees, benchmarking and evals methodology, training stability norms, and performance testing practices
  • Own the framework by which cost, latency, throughput, and model quality trade‑offs are made and measured across the pillar
  • Identify long‑horizon systemic risks early (serving engine and framework bets, accelerator support, capability gaps) and resolve them before they block the organization
  • Align AI engineering, research, product, and infrastructure leadership on multi‑team technical strategy; frame technical trade‑offs in product and commercial terms
  • Mentor and develop Staff and Senior AI Engineers, and grow the next generation of inference technical leaders at Nscale
  • Represent Nscale's technical approach externally: open‑source leadership in the frameworks we depend on, publications, conference talks, and partnerships with GPU vendors and AI labs
Requirements
  • 10‑15 years of engineering experience, with a clear track record of pillar‑level impact on production AI systems
  • 4+ years of hands‑on work with LLMs in inference, GPU performance, evals, or post‑training and RL, in production or research
  • Demonstrated ability to define multi‑year technical strategy for complex, multi‑team AI systems organizations
  • World‑class depth in production LLM inference, GPU performance, evals, and/or post‑training and RL infrastructure, with strong working knowledge across the rest
  • Demonstrated ownership of the architecture of a large‑scale production inference or training platform
  • Proven ability to create architectural frameworks and engineering standards adopted across large engineering organizations
  • Deep understanding of the hardware/software boundary for AI accelerators: CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms
  • Strong history of growing technical leaders (Staff and above) and multiplying technical capability across teams
  • External recognition in the AI systems community through research, open source, or industry contribution
Preferred
  • Prior experience at a top‑tier AI lab or major hyperscaler AI infrastructure team
  • Maintainer or core contributor to a foundational inference, kernel, or RL framework (vLL
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff AI Product Engineer
Staff AI Product Engineer

nscaleoperationsukltd • Georgia

On-site
USD 180,000 - 240,000
Principal AI Product Engineer
Principal AI Product Engineer

Nscale • Houston (TX), New York (NY), San Francisco (CA), Seattle (WA)

On-site
USD 290,000 - 520,000
Medical insurance
Dental insurance
Vision insurance
+3
Staff AI Product Engineer
Staff AI Product Engineer

Nscale • New York (NY)

On-site
USD 220,000 - 330,000
Staff AI Product Engineer
Staff AI Product Engineer

Nscale • Seattle (WA)

On-site
USD 220,000 - 293,000
Principal AI Systems Architect - Inference & Post-Training
Principal AI Systems Architect - Inference & Post-Training

Nscale • Houston (TX), New York (NY), San Francisco (CA), Seattle (WA)

On-site
USD 290,000 - 520,000
Medical insurance
Dental insurance
Vision insurance
+3
Principal Front-End Network Engineer
Principal Front-End Network Engineer

nscaleoperationsukltd • United States

On-site
USD 180,000 - 260,000
Principal Site Reliability Engineer, Infrastructure Architecture- AI Infrastructure Operations
Principal Site Reliability Engineer, Infrastructure Architecture- AI Infrastructure Operations

Nscale • New York (NY), San Francisco (CA), Seattle (WA), Houston (TX)

Remote
USD 150,000 - 250,000
Competitive base + equity
Principal Network Engineer
Principal Network Engineer

nscaleoperationsukltd • Georgia

On-site
USD 180,000 - 260,000
Highly competitive package
Principal Technical Product Manager, Hardware & AI Infrastructure
Principal Technical Product Manager, Hardware & AI Infrastructure

Nscale • Bellevue (WA)

On-site
USD 180,000 - 240,000
Principal Technical Product Manager, Hardware & AI Infrastructure
Principal Technical Product Manager, Hardware & AI Infrastructure

Aimlroles • Seattle (WA)

On-site
USD 180,000 - 240,000