Remote Staff AI Platform Engineer - Lead AI Infra

Code Metal, Inc.; Boston, Northern (MA, KY); Hybrid; USD 190,000 - 270,000; Full time

Benefits offered by this job

Health coverage

401k with match

Unlimited PTO

Hybrid/Remote

Relocation assistance

Job summary

Code Metal, Inc. is seeking a Staff AI Platform Engineer to lead the AI platform team building scalable inference, model gateways, and context management.

This role focuses on production systems, collaboration with Applied AI Research, and advancing observability, experimentation, and multi-tenant product readiness. You will mentor engineers and shape architecture through RFCs and design docs.

Qualifications

• Proficient Python and platform engineering fundamentals (APIs, distributed systems, Kubernetes, CI/CD, testing).
• Experience shipping production agentic systems and reliable services.
• Experience with context engineering: RAG, embeddings, vector search, and memory for agents.
• Observability: instrumentation with OpenTelemetry and metrics; operate AI services to SLOs.
• Strong data science/AI fundamentals; working with transformers, PyTorch, Hugging Face; fine-tuning/serving LLMs.
• Staff-level leadership: owned architecture, RFCs, roadmap, mentoring engineers.

Responsibilities

• Set technical direction and architecture for Code Metal's AI platform; mentor the team.
• Deploy, benchmark, and tune production inference for open-weight models on vLLM, SGLang, TensorRT-LLM.
• Own the model gateway with auth, routing, failover, quotas, and cost attribution.
• Design reusable agent harnesses and orchestration primitives for product teams.
• Build context-engineering services for memory, retrieval, and data discovery.
• Instrument stack with OpenTelemetry traces and metrics; build experiment-tracking and artifact layers.
• Design for productization: multi-tenancy, versioned APIs, security; support air-gapped environments.

Skills

Python

API design

Distributed systems

Kubernetes

CI/CD

Testing

Context engineering

Embeddings

Vector search

Memory for agents

Observability

LLM inference

PyTorch

Hugging Face

Fine-tuning

Leadership

Architecture RFCs

Mentoring

Experiment tracking

Multi-tenant APIs

Job description

Code Metal, Inc. is seeking a Staff AI Platform Engineer to lead the AI platform team building scalable inference, model gateways, and context management.

This role focuses on production systems, collaboration with Applied AI Research, and advancing observability, experimentation, and multi-tenant product readiness. You will mentor engineers and shape architecture through RFCs and design docs.