Staff AI Platform Engineer

Technosylva

United States

Remote

USD 160,000 - 230,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Competitive salary
Private health insurance
Annual bonus
Flexible working hours
Remote work options

Job summary

Technosylva seeks a Staff AI Platform Engineer to own the architecture and technical direction of the AI platform, including the AI gateway, MCP registry, skills, plugins, evaluation and observability, and governance.

This is a hands-on role requiring design, code, production service operation, and on-call participation. You will define security controls, drive RFCs, and mentor engineers across the AI community in a fast-moving environment.

Qualifications

  • 8+ years in software, platform, infrastructure or SRE with architecture ownership across teams.
  • Strong understanding of LLM and API fundamentals, including context windows and prompt caching.
  • Hands-on experience with LLM gateway or similar layers and build-vs-buy decisions.
  • Experience with agent orchestration patterns and multi-agent workflows.
  • Security collaboration for threat modelling and production risk.
  • Familiar with Model Context Protocol (MCP) and security scope.
  • Production containerization expertise (Docker/Kubernetes).
  • IaC experience (Terraform or similar) on major cloud (Azure preferred).
  • Observability and reliability engineering for non-deterministic AI systems.
  • Ability to influence through RFCs/Architecture Decisions to non-technical stakeholders.

Responsibilities

  • Own the AI Platform architecture and set technical vision and standards.
  • Lead gateway evolution, provider routing, and policy decisions including failover and caching.
  • Decide orchestration stack and sandboxing for agents and multi-agent patterns.
  • Design identity, access control, and governance for human and non-human identities.
  • Shape MCP ecosystem: skills/plugins repository, versioning, and security model.
  • Standardize tooling for tracing, evals, and drift detection across AI apps.
  • Design cost attribution framework for token and compute spend with defined owners.
  • Be the technical escalation point and unblock designs with engineering teams.
  • Collaborate with Security on AI threat modelling and policy alignment.
  • Mentor engineers, review RFCs/ADRs, and participate in on-call rotations.

Skills

LLM fundamentals
API design
Architecture leadership
Agent orchestration
Security governance
MCP familiarity
Containerization
Observability
Coding proficiency
RFCs & ADRs

Tools

Azure
Terraform
Docker
Kubernetes
OpenRouter

Job description

ABOUT THE ROLE

Technosylva builds software that supports high-stakes, time-sensitive decisions for wildfire and natural-hazard management, and AI is a core part of how we build it. As a Staff AI Platform Engineer, you'll own the architecture and technical direction of our AI platform: the AI gateway, MCP registry, skills and plugins, evaluation and observability, security and governance. This is a hands-on role: you'll design systems, write code, run production services and take part in on-call.

RESPONSIBILITIES
  • Own the AI Platform architecture. Set the technical vision and standards that engineers, product, and other practitioners build against, and keep the architecture documented.
  • Gateway & model routing. Lead the evolution of our AI gateway and the build-vs-buy decisions around it: provider routing, hybrid and on-prem model routing policy, failover, caching, rate limits, and multi-provider compatibility (Anthropic Messages vs. OpenAI-compatible APIs).
  • Agent orchestration & runtime. Decide the orchestration stack (native agent harnesses vs. provider-agnostic runtimes) upon collaboration with pro users, design sandboxing and lifecycle management for agents that run code or touch internal systems, and set the multi-agent and human-in-the-loop patterns.
  • Identity, access & governance. Design identity and access control for both people and non-human agent identities: scoped credentials, least privilege, audit trails and kill-switches. Lead approved model/tool/MCP allowlists and access tiers, and work with Legal and Security on EU AI Act and data-classification requirements.
  • Skills, plugins & MCP ecosystem. Shape how our skills/plugins repository evolves (review gates, version pinning, namespacing, support for more agents) and define the security model for the curated MCP registry.
  • Evaluation & observability. Choose and standardize the tooling for tracing agent decisions, running evals and detecting drift and quality regressions across AI applications, so failures are caught early and can be supported properly on call.
  • FinOps for usage-based AI. Design the cost attribution and allocation framework for token and compute spend (per team, product and use case), with a measurable baseline and a named business owner for each use case so its value can be measured.
  • Escalation point for embedded delivery. Be the technical escalation point when working alongside engineering and other business units on AI use cases. Unblock, review designs, and turn what you learn into golden paths.
  • Security partnership. Work closely with the Security teams to threat-model AI systems and agents, define controls for prompt injection, data exfiltration and over-privileged tools, and align AI platform standards with company security policies.
  • Raise the bar. Mentor engineers on the platform and across the company's AI community, run design reviews and RFCs, and represent AI Platform in engineering-wide technical decisions. Take part in the on-call rotation and incident response for production AI services.
REQUIREMENTS
  • 8+ years in software, platform, infrastructure or SRE engineering, including time in a Staff/Principal-level or tech-lead role where you owned architecture across teams.
  • Strong understanding of LLM and API fundamentals: context windows, tool use, structured outputs, streaming, prompt caching, and the differences between providers' APIs.
  • Hands-on experience designing or running an LLM gateway or similar layer (e.g. Azure API Management / AI Gateway, LiteLLM, Portkey, OpenRouter, Bifrost, TrueFoundry), including making build-vs-buy decisions.
  • Experience with agent orchestration patterns: multi-agent workflows, delegation, human-in-the-loop approval gates, memory and context management, token optimization.
  • Experience working with Security teams on threat modelling, security reviews and risk acceptance for production systems, ideally including AI or agent-based workloads.
  • Working knowledge of the Model Context Protocol (MCP): implementing servers, scoping tools, and reviewing third-party servers for security.
  • Production experience with containers and isolation (Docker, Kubernetes).
  • Infrastructure as code (Terraform or similar) on at least one major cloud, ideally Azure.
  • A solid grasp of observability (OpenTelemetry, Prometheus, Grafana) and reliability engineering, and of how they apply to non-deterministic AI systems.
  • Strong coding skills.
  • A record of influencing without authority: writing clear RFCs and architecture decision records, getting teams to agree, and explaining technical trade-offs to non-technical stakeholders.
NICE TO HAVE
  • Experience with AI evaluation and observability tools (Langfuse, Arize Phoenix, Braintrust, Helicone, Grafana AI Observability).
  • FinOps experience with usage-based cost models: tagging, chargeback/showback, budgets and anomaly detection.
  • Familiarity with AI governance frameworks (EU AI Act, NIST AI RMF, ISO/IEC 42001) and experience partnering with Legal or Compliance.
  • Experience with AI coding agents and their extension models (e.g. Claude Code skills, plugins, hooks, subagents).
  • Hands-on experience designing or evaluating model routing solutions (e.g. LiteLLM, OpenRouter, Portkey, Azure AI Foundry model router, RouteLLM, Not Diamond), including fallback, load balancing and cost-or-quality-based routing.
  • Experience serving self-hosted or on-prem models (vLLM, Triton, KServe or similar).
  • Experience building developer platforms, golden paths or internal developer portals (e.g. Backstage).
BENEFITS
  • Competitive annual salary.
  • Private health insurance and a flexible benefits plan, allowing you to tailor part of your compensation package to your personal needs.
  • Annual bonus based on individual performance and company results.
  • Flexible working hours.
  • Remote work options.
DISCLAIMER

Final salary and benefits will depend on a variety of factors, including location, experience, training, qualifications, and market conditions.

EQUAL OPPORTUNITY STATEMENT

Technosylva is an Equal Opportunity Employer. We are committed to fostering an inclusive environment where diverse perspectives lead to better solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Platform Engineer
Senior AI Platform Engineer

Technosylva • United States

Remote
USD 160,000 - 260,000
Health insurance
Flexible benefits
Annual bonus
+2
AI Platform Engineer
AI Platform Engineer

Valiant Technologies LLC • Boston (MA)

On-site
USD 150,000 - 230,000
AI Platform Engineer
AI Platform Engineer

Quantum Technologies LLC • Boston (MA)

On-site
USD 140,000 - 190,000
Staff AI Engineer - Architecture skills
Staff AI Engineer - Architecture skills

shifttechnology • United States

Hybrid
USD 170,000 - 250,000
Flexible remote and hybrid options
Learning & development sessions
Generous PTO and holidays
+2
Remote AI Engineer for Production LLMs & Agentic Systems
Remote AI Engineer for Production LLMs & Agentic Systems

7Seventy • United States

Hybrid
USD 130,000 - 150,000
Remote work nationwide
Health benefits
Competitive compensation package
+2
AI Platform Engineer
AI Platform Engineer

JustMarkets • United States

Remote
USD 180,000 - 250,000
20 paid vacation days per year
10 paid sick leave days per year
Public holidays as per list
+5
Remote Staff AI Platform Engineer - Lead AI Infra
Remote Staff AI Platform Engineer - Lead AI Infra

Code Metal, Inc. • Boston (MA)

Hybrid
USD 190,000 - 270,000
Health coverage
401k with match
Unlimited PTO
+2
Staff AI Product Engineer
Staff AI Product Engineer

Jobgether • United States

Remote
USD 200,000 - 300,000
Fully remote across continents
Stock options
Paid time off (18 days)
+1
Staff AI Engineer — Agent Platform (all genders)
Staff AI Engineer — Agent Platform (all genders)

Wingmaite GmbH • United States

Remote
USD 180,000 - 260,000
Flexible remote setup
Learning & development opportunities
Generous vacation & workation budget
+1
Remote Lead AI Platform Engineer, MLOps (Production-Grade)
Remote Lead AI Platform Engineer, MLOps (Production-Grade)

TetraScience, Inc. • Cambridge (MA)

Remote
USD 200,000 - 270,000
Employer-paid benefits
Unlimited PTO
401K
+3