Principal AI Systems Architect – Inference & Post-Training

AI Pulse; New York (NY); On-site; USD 190,000 - 240,000; Full time

Job summary

Nscale is seeking a Principal AI Engineer to lead the inference and post‑training pillar of our AI systems engineering organization. You will define the multi‑year technical roadmap for model serving, evaluation, and post‑training on our GPU cloud, across specialized and BYOM deployments.

You will guide architectural programs, set engineering standards, and own cost/latency trade‑offs across the pillar while mentoring senior engineers and representing Nscale in external forums.

Qualifications

• 10–15 years of engineering experience with pillar‑level impact on production AI systems.
• 4+ years hands‑on work with LLMs in inference, GPU performance, evals, or post‑training and RL.
• Demonstrated ability to define multi‑year technical strategy for complex, multi‑team AI systems organizations.
• World‑class depth in production LLM inference, GPU performance, evals, and/or post‑training and RL infrastructure, with strong working knowledge across the rest.
• Demonstrated ownership of the architecture of a large‑scale production inference or training platform.
• Proven ability to create architectural frameworks and engineering standards adopted across large engineering organizations.
• Deep understanding of the hardware/software boundary for AI accelerators: CUDA or ROCm, memory bandwidth and interconnect constraints, and distributed compute paradigms.

Responsibilities

• Define and own the multi‑year technical roadmap for Nscale's inference, evals, and post‑training platform, and translate it into architecture that multiple teams can execute against.
• Lead company‑scale architectural initiatives in the pillar, including next‑generation serving and GPU kernel optimization.
• Establish engineering standards adopted across AI teams: API design, benchmarking, training stability norms, and performance testing.
• Own the framework by which cost, latency, throughput, and model quality trade‑offs are measured across the pillar.
• Identify long‑horizon systemic risks and resolve them before they block the organization.
• Align AI engineering, research, product, and infrastructure leadership on multi‑team technical strategy; frame trade‑offs in product terms.
• Mentor Staff and Senior AI Engineers and grow the next generation of inference leaders.
• Represent Nscale externally: open‑source leadership, publications, talks, and partnerships with GPU vendors and AI labs.

Skills

LLM inference

GPU performance

Multi-team leadership

AI systems architecture

Cost/latency modeling

External engagement

CUDA

ROCm

Triton

Megatron-LM

Tools

CUDA

ROCm

Triton

Megatron-LM

Job description

Nscale is seeking a Principal AI Engineer to lead the inference and post‑training pillar of our AI systems engineering organization. You will define the multi‑year technical roadmap for model serving, evaluation, and post‑training on our GPU cloud, across specialized and BYOM deployments.

You will guide architectural programs, set engineering standards, and own cost/latency trade‑offs across the pillar while mentoring senior engineers and representing Nscale in external forums.