LLM Platform Engineer - Production AI Gateway

Mpathic; United States; Remote; USD 140,000 - 200,000; Full time

Job summary

mpathic is building trustworthy AI with a platform that ships LLM-powered features safely and audibly. You will own the LLM gateway and production inference, ensuring cost, latency, and PHI controls are built in from day one.

You will collaborate with AI/ML, security, and research to turn prototypes into scalable, auditable services, with a focus on production reliability and governance.

Qualifications

• 5+ years of backend software engineering in production.
• Production experience with LLM APIs and prompt management.
• Experience designing evaluation pipelines and CI/CD release gates.
• Strong Python; AWS and containers experience.

Responsibilities

• Own the LLM gateway and production inference runtime.
• Build logging, cost attribution, and PHI controls.
• Document model catalog and version pinning.
• Create release paths for model and prompt changes.
• Monitor latency, cost, and quality drift in production.

Skills

Backend engineering

LLM integration

CI/CD

Python

AWS

Observability

Security awareness

Tools

OpenTelemetry

Datadog

Pulumi

Terraform

CDK

ECS

Job description

mpathic is building trustworthy AI with a platform that ships LLM-powered features safely and audibly. You will own the LLM gateway and production inference, ensuring cost, latency, and PHI controls are built in from day one.

You will collaborate with AI/ML, security, and research to turn prototypes into scalable, auditable services, with a focus on production reliability and governance.