AI Engineer (Remote)

7Seventy Recruiting

United States

On-site

USD 150,000 - 175,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

7Seventy Recruiting is seeking an Applied AI Engineer to design, build, and operate production infrastructure for LLM-powered services. You will own a centralized LLM gateway, manage model integrations, and automate evaluation and deployment pipelines for scalable, secure AI applications.

Collaborating with platform engineers, AI researchers, and Security, you will drive reliability, cost efficiency, and compliance while delivering observable, production-grade AI solutions in a fast-paced

Qualifications

  • 5+ years backend software engineering with production LLM deployments.
  • Experience building production AI services and gateways.
  • Hands-on with Python, AWS, and CI/CD.
  • Ability to design scalable, observable production infrastructure.
  • Familiarity with healthcare data security preferred.

Responsibilities

  • Design, build, and operate centralized LLM gateway and inference infrastructure.
  • Implement logging, cost attribution, rate limiting, and PHI controls.
  • Document production integrations and maintain approved model catalog.
  • Develop standardized deployment workflows and manage prompt/version control.
  • Enable auditable releases through CI/CD and automated evaluation.
  • Collaborate with researchers and security teams to ensure compliance and reliability.

Skills

Python
LLM API integration
Backend API design
CI/CD pipelines
AWS
Observability

Tools

AWS
Terraform
Pulumi
Docker

Job description

Experience: 5+ years of backend software engineering, with hands-on experience deploying LLM-powered applications and services in production

Core Areas: Large Language Models (LLMs), AI Inference Infrastructure, LLM Gateway Development, Python, LLM Evaluation Pipelines, AWS & Infrastructure as Code, CI/CD & Model Deployment, AI Observability

Compensation: $150,000–$175,000 annually

About the Role

The Applied AI Engineer will design, build, and operate production infrastructure for large language model (LLM) applications. This hands-on engineering position focuses on developing a centralized LLM gateway, managing model integrations, implementing automated evaluation and deployment pipelines, and ensuring AI-powered services operate reliably, securely, and cost-effectively at scale.

Working within platform engineering and collaborating with AI/ML researchers, engineering teams, and Security and Compliance, this role transforms validated AI prototypes into scalable production services. Key technologies include Python, commercial LLM APIs, AWS, containers, infrastructure as code, CI/CD, and observability tooling. The position supports AI evaluation, red teaming, dataset generation, and annotation systems, including applications involving sensitive healthcare and life sciences data.

What You'll Do
LLM Gateway and Inference Infrastructure
  • Design, develop, and operate a centralized LLM gateway that routes production AI services to approved model providers.
  • Implement request logging, cost attribution, rate limiting, provider failover, and protected health information (PHI) controls across LLM inference services.
  • Replace direct model-provider integrations with secure gateway-based access, eliminating exposed provider API keys from individual services.
  • Identify and document existing production LLM integrations, including providers, models, application features, and data usage.
  • Support reliable, scalable, and cost-efficient execution of LLM-powered applications.
Model Catalog and Provider Management
  • Maintain an approved catalog of AI models, providers, API endpoints, and supported capabilities, including structured outputs, tool use, caching, and batch processing.
  • Document model‑specific data retention policies, Business Associate Agreement (BAA) coverage, and version pinning requirements.
  • Maintain decision records for newly approved models, providers, and capabilities.
  • Evaluate changes in the model‑provider ecosystem and assess their implications for production infrastructure.
AI Evaluation and Release Engineering
  • Develop standardized deployment workflows for LLM model configurations and prompt changes.
  • Maintain version‑controlled prompts and model configurations alongside application code.
  • Integrate AI/ML evaluation frameworks and acceptance thresholds into CI/CD pipelines as automated release gates.
  • Implement staged deployments, regression evaluation checks, and rollback mechanisms for model and prompt releases.
  • Provide reusable templates and self‑service workflows that allow engineers and researchers to experiment in sandbox and staging environments while maintaining controlled production releases.
  • Apply consistent testing, reproducibility, and auditability standards to AI application changes.
Production AI Application Development
  • Convert validated AI/ML prototypes, evaluation harnesses, and acceptance criteria into scalable, observable production services.
  • Collaborate with AI/ML researchers to preserve validated model behavior and evaluation requirements throughout production implementation.
  • Advise engineering and research teams on model selection considerations, including inference costs, latency, and availability.
  • Develop production infrastructure supporting LLM‑powered features, AI evaluation, red teaming, dataset generation, and annotation workflows.
  • Work with platform engineering, research, and domain specialists to translate technical and scientific requirements into reliable applications.
AI Observability, Reliability, and Security
  • Implement end‑to‑end observability for LLM‑powered services using OpenTelemetry and related monitoring platforms.
  • Track inference latency, failures, token consumption, operating costs, and model output quality.
  • Monitor model behavior and quality drift against established AI/ML evaluation benchmarks.
  • Participate in on‑call operations and lead incident response for production LLM services.
  • Apply security practices covering secrets management, access control, and sensitive‑data handling.
  • Partner with Security and Compliance to generate audit evidence supporting SOC 2, ISO 27001, HIPAA, and FDA 21 CFR Part 11 requirements.
  • Maintain traceable and auditable release records for model, prompt, and configuration changes.
Implementation Priorities
  • During the first 60–90 days, assess the existing LLM infrastructure, document production model integrations, establish an approved model catalog, and launch an initial LLM gateway with at least one production service migrated to it.
  • During the first year, consolidate production LLM traffic through the gateway, establish automated model and prompt release controls, productionize validated AI/ML prototypes, implement operational monitoring, and deliver compliance‑ready audit evidence.
Qualifications
Required Experience
  • 5+ years of professional backend software engineering experience, including development and operation of production APIs and multi‑tenant SaaS services.
  • Demonstrated experience building and deploying production applications using commercial LLM APIs, such as Anthropic, OpenAI, AWS Bedrock, or Azure OpenAI.
  • Hands‑on experience implementing prompt management, structured outputs, and tool use in LLM‑powered applications.
  • Practical experience designing LLM evaluation pipelines and integrating automated evaluation checks into CI/CD deployment workflows.
  • Proven ability to take LLM‑powered products beyond prototypes and manage production challenges involving evaluation regressions, latency, cost, outages, and audit requirements.
  • Experience translating validated AI/ML research prototypes into reliable production software services.
Required Skills
  • Strong Python programming skills for backend services, API integrations, and production AI infrastructure development.
  • Working knowledge of commercial LLM providers, model APIs, prompt configuration, structured outputs, and tool‑based AI functionality.
  • Ability to design and maintain secure, scalable LLM gateway and inference infrastructure.
  • Understanding of modern AI evaluation methodologies, including LLM‑as‑judge, red teaming, and regression evaluations, and the ability to implement automated release gates.
  • Experience with AWS, containerized applications such as Amazon ECS or similar platforms, and infrastructure as code using Pulumi, Terraform, or AWS CDK.
  • Proficiency with observability systems such as OpenTelemetry, Datadog, or comparable tools for tracking application performance, reliability, and AI inference behavior.
  • Familiarity with software delivery tools and practices, including version control, CI pipelines, feature flags, and issue tracking systems such as Linear.
  • Working knowledge of application security, including secrets management, access controls, and secure handling of sensitive information.
  • Ability to apply reproducibility, reliability, and auditability standards to model configurations, prompt changes, and production releases.
  • Ability to communicate technical decisions through design documentation, decision records, and architectural diagrams.
  • Strong collaboration skills for working with researchers, clinicians, engineers, and security professionals to translate validated prototypes into production systems.
  • Comfort working with sensitive subject matter involving healthcare, safety, and wellbeing.
  • Ability to establish engineering standards and reusable infrastructure that enable other teams to deploy AI‑powered features safely and efficiently.
Preferred Qualifications
  • Experience developing software in regulated healthcare or life sciences environments, including systems subject to HIPAA, SOC 2, ISO 27001, or FDA 21 CFR Part 11 requirements.
  • Experience building internal developer platforms, reusable engineering infrastructure, or standardized deployment workflows used across multiple teams.
  • Experience with LLM‑as‑judge evaluation, AI red teaming, or conversational analytics.
  • Experience training or fine‑tuning machine learning models from scratch.
  • Experience working with Node.js in backend engineering environments.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Engineer - GA
AI Engineer - GA

LawPro.ai • Georgia

On-site
USD 140,000 - 200,000
Applied AI Engineer
Applied AI Engineer

Mpathic • Bellevue (WA)

On-site
USD 150,000 - 230,000
Applied AI Engineer
Applied AI Engineer

Mpathic • United States

Remote
USD 140,000 - 200,000
AI/ML Engineer
AI/ML Engineer

Storm3 • Tennessee

On-site
USD 180,000 - 280,000
AI Engineer - NC
AI Engineer - NC

LawPro.ai • North Carolina

On-site
USD 140,000 - 190,000
AI Engineer - VA
AI Engineer - VA

LawPro.ai • Virginia (MN)

On-site
USD 140,000 - 200,000
AI Engineer - VA
AI Engineer - VA

LawPro.ai • Virginia

On-site
USD 150,000 - 190,000
Senior LLMOps Engineer
Senior LLMOps Engineer

UNAVAILABLE • McLean (VA)

On-site
USD 180,000 - 240,000
AI Engineer
AI Engineer

TechDigital Group • San Jose (CA)

On-site
USD 140,000 - 210,000
AI Product Engineer
AI Product Engineer

7Seventy • United States

Remote
USD 120,000 - 180,000