Senior AI Systems Architect - GPU & Platform Internals

Accellor; San Francisco (CA); On-site; USD 210,000 - 320,000; Full time

Job summary

Accellor seeks a Senior Technical Architect to lead AI systems, inference runtimes, and platform internals. You will shape architecture across GPUs, distribution, and safety while balancing performance and cost.

The role requires hands-on leadership, deep expertise in AI infrastructure, and collaboration with research, product, and deployment teams to deliver scalable, reliable AI platforms.

Qualifications

• 10-12 years of experience in software engineering, systems architecture, ML infrastructure, distributed systems, platform engineering, inference systems, cloud infrastructure, or large-scale backend engineering.
• Strong hands-on engineering experience with Python and at least one systems/backend language such as C++, Go, Rust, Java, or TypeScript.
• Deep understanding of distributed systems, production infrastructure, reliability engineering, scalability, observability, and fault-tolerant architecture.
• Experience designing or operating large-scale systems involving APIs, microservices, distributed compute, orchestration, job scheduling, caching, high-availability infrastructure, and production monitoring.
• Strong understanding of AI/ML systems, especially model serving, inference workflows, context engineering, retrieval systems, evaluation pipelines, and production model deployment.

Responsibilities

• Design and evolve large-scale AI systems that support ChatGPT, OpenAI API, Codex, agentic workflows, multimodal models, and research workloads.
• Define architecture across inference runtime, model serving, request routing, batching, KV-cache handling, GPU scheduling, distributed execution, observability, release gates, and production rollout.
• Own technical trade-offs across latency, throughput, reliability, correctness, safety, scalability, cost, and infrastructure efficiency.
• Architect high-throughput, low-latency inference systems across large-scale GPU clusters.
• Work across inference engines, serving layers, scheduling systems, caching, streaming, deployment pipelines, and runtime optimization.
• Partner with engineering teams to improve model-serving efficiency, tail latency, GPU utilization, memory efficiency, correctness under load, and cost per request.

Skills

Python

Distributed systems

Architecture design

Communication

Team leadership

Tools

PyTorch

JAX

TensorFlow

Triton

vLLM-style serving

Kubernetes

CUDA

NCCL/RCCL

Job description

Accellor seeks a Senior Technical Architect to lead AI systems, inference runtimes, and platform internals. You will shape architecture across GPUs, distribution, and safety while balancing performance and cost.

The role requires hands-on leadership, deep expertise in AI infrastructure, and collaboration with research, product, and deployment teams to deliver scalable, reliable AI platforms.