Applied AI Engineer

Mpathic

United States

Remote

USD 140,000 - 200,000

Full time

8 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

mpathic is building trustworthy AI with a platform that ships LLM-powered features safely and audibly. You will own the LLM gateway and production inference, ensuring cost, latency, and PHI controls are built in from day one.

You will collaborate with AI/ML, security, and research to turn prototypes into scalable, auditable services, with a focus on production reliability and governance.

Qualifications

  • 5+ years of backend software engineering in production.
  • Production experience with LLM APIs and prompt management.
  • Experience designing evaluation pipelines and CI/CD release gates.
  • Strong Python; AWS and containers experience.

Responsibilities

  • Own the LLM gateway and production inference runtime.
  • Build logging, cost attribution, and PHI controls.
  • Document model catalog and version pinning.
  • Create release paths for model and prompt changes.
  • Monitor latency, cost, and quality drift in production.

Skills

Backend engineering
LLM integration
CI/CD
Python
AWS
Observability
Security awareness

Tools

OpenTelemetry
Datadog
Pulumi
Terraform
CDK
ECS

Job description

About mpathic

mpathic is building the future of trustworthy AI. Grounded in behavioral science and human-centered design, we provide the infrastructure for building AI systems that are safe, aligned, and emotionally intelligent.

Building on our work in areas like RL gyms, red teaming, and benchmarking, we are creating the foundation for training, probing, and measuring advanced AI systems reliably, auditably, and at scale.

We serve frontier AI labs, large enterprises building and deploying AI, and Life Sciences organizations. Our Life Sciences business brings clinician-led AI monitoring, safety detection, and expert clinical review to clinical trials, helping sponsors and CROs protect participants and data quality, with a strong presence in psychedelic and CNS research.

Position Overview

We are looking for an Applied AI Engineer to own how mpathic's production services are built on large language models (LLMs). You will sit on the platform engineering team, reporting to the CTO, and work closely with the AI/ML team, research, and Security and Compliance. The AI/ML team defines how a model should behave: which model, which prompt, which judge, and the evaluation bar it must meet. You make that behavior run safely, reliably, and cost-effectively in production at scale.

Our platform spans red teaming, dataset generation, annotation, and evaluation, much of it for healthcare and life sciences customers where quality and auditability matter. Your job is to build the paved road every team uses to ship LLM-powered features, so that every model and prompt change reaching a customer arrives through the same tested, observable, and auditable path.

This is a hands-on engineering role. You should be as comfortable designing a gateway or a CI release gate as you are sitting with a researcher to turn a working prototype into a production service. You will start as the owner of LLM and inference infrastructure in production, with room to shape how we build with AI as the team grows.

What You'll Accomplish
In your first 60-90 days you'll…
  • Build a deep understanding of the platform, its LLM-backed services, and how a model or prompt change moves from prototype to production today.
  • Inventory every place production code calls an LLM: which providers, models, features, and data each one uses.
  • Publish the first approved list of models, providers, and features, with data retention, BAA coverage, and version pinning documented for each.
  • Stand up the first version of the LLM gateway and move at least one production service onto it.
  • Build trusted relationships across platform engineering, the AI/ML team, research, and Security and Compliance.
In your first year you'll…
  • Route all production LLM traffic through the gateway, with logging, cost attribution, failover, and PHI controls.
  • Ship a release path for model and prompt changes: versioning, evaluation gates in CI, staged rollout, and rollback.
  • Take AI/ML prototypes, with their eval harnesses and acceptance thresholds, into scalable production services.
  • Deliver production monitoring for latency, cost, failures, and quality drift against the evals defined by the AI/ML team.
  • Produce the audit evidence for LLM changes that SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11 require.
You’ll Thrive in This Role If You…
  • Are a backend engineer first, and have carried LLM features through what comes after the demo: evals, regressions, cost, latency, outages, and audits.
  • Enjoy building the platform other people build on, and measure success by how easily and safely the rest of the team ships.
  • Treat a prompt change with the same rigor as a code change, and care deeply about reliability, reproducibility, and auditability.
  • Understand modern AI evaluation concepts (LLM-as-judge, red teaming, regression evals) and how to turn them into automated release gates.
  • Are comfortable with high-stakes, sensitive subject matter such as healthcare, safety, and wellbeing, and the data handling it demands.
  • Can translate between researchers, clinicians, and engineers, turning a working prototype into a production service without losing what made it work.
  • Communicate crisply through decision records, design docs, and diagrams, and can align people around a standard without relying on authority.
  • Have shipped real LLM-powered products in production, shown through what you have built rather than years on a résumé.
What You'll Do
Own the LLM Gateway and Inference
  • Design, build, and run the single path through which production services call model providers.
  • Build in logging, cost attribution, rate limiting, failover, and controls for protected health information (PHI).
  • Remove direct provider API keys from services, so nothing reaches a model except through the gateway.
Own the Approved Model Catalog
  • Maintain the approved list of models, providers, endpoints, and model features, such as structured outputs, tool use, caching, and batch.
  • Document data retention, BAA coverage, and version pinning for each, and record every addition in a short decision record.
  • Keep the team current on changes in the model and provider landscape, and on what they mean for our stack.
Build the Release Path for Model & Prompt Changes
  • Version prompts and model configurations alongside code.
  • Run the AI/ML team's evals as release gates in CI, with staged rollout and rollback.
  • Give engineers and researchers self-service templates, so they move fast in sandbox and staging while production stays gated.
Productize AI/ML Prototypes
  • Take a validated prototype, its eval harness, and its acceptance thresholds, and turn them into a scalable, observable production service.
  • Advise the AI/ML team early on the cost, latency, and availability of the models they choose.
Observe, Operate LLM/Inference Runtime
  • Instrument LLM services end to end with OpenTelemetry: latency, failures, token usage, cost, and output quality.
  • Monitor for drift against the AI/ML team's evals, join on-call, and lead incident response for LLM-backed services.
  • Partner with Security and Compliance to produce audit evidence for SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11.
Experience
Core experience
  • 5+ years of backend software engineering, including multi-tenant SaaS services and APIs in production.
  • Production experience building on commercial LLM APIs (for example Anthropic, OpenAI, AWS Bedrock, or Azure OpenAI), including prompt management, structured outputs, and tool use.
  • Hands-on experience designing evaluation pipelines for LLM output and wiring them into CI/CD as release gates.
  • Strong Python; NodeJS is a plus.
  • AWS, containers (ECS or similar), and infrastructure as code (Pulumi, Terraform, or CDK).
  • Observability tooling such as OpenTelemetry, Datadog, or similar, and the delivery toolchain (version control, feature flags, CI, and an issue tracker such as Linear).
  • Working knowledge of application security: secrets management, access control, and handling of sensitive data.
Preferred
  • Experience in regulated domains such as healthcare or life sciences, including HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11 validated systems.
  • Experience building an internal developer platform or paved road used by other teams.
  • Experience with LLM-as-judge evaluation, red teaming, or conversational analytics.
  • Experience with training or fine-tuning models from scratch.
About the Team

You will work closely with:

  • CTO and platform engineering: to scope, sequence, and ship the work.
  • ML Team, Chief Science Officer and Research Team: to turn validated direction and scientific requirements into shippable product.
  • Domain experts: to ground the work in real practice and quality of findings.
  • Sales and the delivery teams (Post Sales Support and AI Labs Services Delivery): to align what we build with customer needs and deliverables.

We value operational excellence, systems thinking, and high standards for quality, clarity, and auditability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Applied AI Engineer
Applied AI Engineer

Mpathic • Bellevue (WA)

On-site
USD 150,000 - 230,000
AI Engineer (Remote)
AI Engineer (Remote)

7Seventy Recruiting • United States

On-site
USD 150,000 - 175,000
Machine Learning Engineer, Platform
Machine Learning Engineer, Platform

Brain Co. • San Francisco (CA)

On-site
USD 150,000 - 220,000
Machine Learning Engineer, Platform
Machine Learning Engineer, Platform

Brain Co. • New York (NY)

On-site
USD 140,000 - 210,000
Lead Software Platform Engineer, MLOps
Lead Software Platform Engineer, MLOps

TetraScience, Inc. • Cambridge (MA)

Remote
USD 200,000 - 270,000
Employer-paid benefits
Unlimited PTO
401K
+3
Machine Learning Engineer, Platform
Machine Learning Engineer, Platform

Brain Co. • City of Albany (NY)

On-site
USD 120,000 - 170,000
Remote Lead AI Platform Engineer, MLOps (Production-Grade)
Remote Lead AI Platform Engineer, MLOps (Production-Grade)

TetraScience, Inc. • Cambridge (MA)

Remote
USD 200,000 - 270,000
Employer-paid benefits
Unlimited PTO
401K
+3
Staff AI Engineer
Staff AI Engineer

Meeru AI Inc • Pleasanton (CA)

On-site
USD 180,000 - 260,000
ML Ops Engineer — Agentic AI Lab (Founding Team)
ML Ops Engineer — Agentic AI Lab (Founding Team)

Fabrion • San Francisco (CA)

On-site
USD 120,000 - 150,000
Competitive salary
Meaningful equity
Machine Learning Engineer, Applied AI
Machine Learning Engineer, Applied AI

Brain Co. • New York (NY)

On-site
USD 140,000 - 210,000