Senior AI Platform Architect: Inference, GPUs & Reliability

Accellor; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Accellor is an AI-native services firm delivering measurable outcomes through AI, data, and engineering capabilities. We seek a Technical Architect — AI Systems, Inference & Platform Internals to design, scale, and optimize the systems powering advanced AI workloads.

The role focuses on inference runtime, model serving, GPU infrastructure, context engineering, and release safety, requiring a senior hands-on architect able to drive architecture across latency, throughput, and cost.

Qualifications

• 10–12 years in software engineering, systems architecture, or ML infrastructure.
• Strong Python and at least one systems/backend language (C++, Go, Rust, Java, TS).
• Deep understanding of distributed systems, reliability, and scalability.
• Experience with APIs, microservices, orchestration, and monitoring.
• Knowledge of AI/ML systems, model serving, context engineering, evaluation.
• GPU-focused experience: CUDA/Triton, distributed inference, memory optimization.

Responsibilities

• AI Systems Architecture: design large-scale AI systems for ChatGPT/OpenAI APIs and research workloads.
• Inference Runtime & Model Serving: build high-throughput low-latency inference across GPU clusters.
• GPU, Kernel & Distributed Performance: optimize kernels, memory, and interconnects.
• Context Engineering: define prompt structure, retrieval, and context management.
• Cost Optimization: reduce token use, caching, and efficient inference paths.
• Training & Research Infra: support distributed training and experiment velocity.
• Release Safety & Validation: implement gates for safe, reliable platform releases.
• Reliability & Observability: telemetry, dashboards, runbooks, SLOs, post-incident learning.

Skills

Python

C++

Distributed systems

GPU

PyTorch

JAX

Tools

CUDA

Triton

NCCL/RCCL

PyTorch

JAX

Kubernetes

Job description

Accellor is an AI-native services firm delivering measurable outcomes through AI, data, and engineering capabilities. We seek a Technical Architect — AI Systems, Inference & Platform Internals to design, scale, and optimize the systems powering advanced AI workloads.

The role focuses on inference runtime, model serving, GPU infrastructure, context engineering, and release safety, requiring a senior hands-on architect able to drive architecture across latency, throughput, and cost.