Staff AI Platform Architect — Remote

Code Metal, Inc.; Boston (MA); Hybrid; USD 180,000 - 260,000; Full time

Benefits offered by this job

Health care plan

401k matching

PTO

Hybrid/Remote work

Relocation assistance

Job summary

Code Metal, Inc. is seeking a Staff AI Platform Engineer to lead a four-person team and build the AI enablement stack, from GPU inference serving through agent harnesses and observability.

You’ll collaborate with Applied AI Research and product teams to design and operate production systems, enabling reliable, verifiable AI workflows and secure multi-tenant deployments across environments.

Qualifications

• Production-grade Python and distributed systems fundamentals.
• Shipped production agentic systems; reliability-focused.
• Experience with context engineering: retrieval-augmented generation, embeddings, memory.
• Instrumenting services with OpenTelemetry and operating AI services to meet SLOs.
• Strong data science fundamentals: transformers, LLMs; PyTorch/Hugging Face.
• Staff-level technical leadership: architecture across multiple systems; mentoring.

Responsibilities

• Set the technical direction and architecture for Code Metal’s AI platform and lead the team.
• Deploy, benchmark, and tune production inference for open-weight models on vLLM, SGLang, and TensorRT-LLM.
• Own the model gateway with consistent auth, routing, failover, quotas, and cost attribution.
• Design reusable agent harnesses and orchestration primitives for reliable workflows.
• Build context-engineering services for memory, retrieval, and data discovery.
• Instrument the stack with OpenTelemetry traces and metrics; manage experiment tracking and artifacts.
• Design for multi-tenancy, versioned APIs, security, and deployment in diverse environments.

Skills

Python

Distributed systems

Kubernetes

OpenTelemetry

PyTorch

Hugging Face

Architectural design

Team leadership

Tools

vLLM

SGLang

TensorRT-LLM

MLflow

Weights & Biases

Job description

Code Metal, Inc. is seeking a Staff AI Platform Engineer to lead a four-person team and build the AI enablement stack, from GPU inference serving through agent harnesses and observability.

You’ll collaborate with Applied AI Research and product teams to design and operate production systems, enabling reliable, verifiable AI workflows and secure multi-tenant deployments across environments.