Senior AI Systems Architect, Inference & Platform

Accellor; Mountain View (CA); On-site; USD 190,000 - 240,000; Full time

Job summary

Accellor is seeking a Technical Architect — AI Systems, Inference & Platform Internals to design, scale, and optimize the AI systems that power enterprise workloads. You will own the end-to-end stack from GPU-level performance to production reliability, with emphasis on safety, cost efficiency, and observability.

The role calls for senior hands-on leadership across distributed systems, model serving, and complex context engineering, partnering with multiple teams to deliver scalable AI platforms

Qualifications

• 10–12 years of experience in software engineering, systems architecture, ML infrastructure, distributed systems, or platform engineering.
• Strong hands-on engineering experience with Python and at least one systems/backend language.
• Deep understanding of distributed systems, reliability, and scalable architectures.
• Experience with APIs, microservices, model serving, and production monitoring.

Responsibilities

• AI Systems Architecture: design large-scale AI systems for ChatGPT/OpenAI API, tooling, and research workloads.
• Inference Runtime & Model Serving: build high-throughput, low-latency inference pipelines across GPU clusters.
• GPU, Kernel & Distributed Performance: optimize CUDA, Triton kernels, memory, and scheduling.
• Context Engineering: design context passing, retrieval, and cost-aware context management.
• Cost Optimization: implement token budgeting, caching, and efficient inference paths.
• Training & Research Infrastructure: support large-scale model training and experiments.

Skills

Python

Distributed systems

ML infrastructure

System architecture

Cloud infrastructure

Technical leadership

Communication skills

Tools

PyTorch

JAX

TensorFlow

Triton

vLLM-style serving

Kubernetes

Docker

CUDA

NCCL

RCCL

Job description

Accellor is seeking a Technical Architect — AI Systems, Inference & Platform Internals to design, scale, and optimize the AI systems that power enterprise workloads. You will own the end-to-end stack from GPU-level performance to production reliability, with emphasis on safety, cost efficiency, and observability.

The role calls for senior hands-on leadership across distributed systems, model serving, and complex context engineering, partnering with multiple teams to deliver scalable AI platforms