Applied AI Engineer

OpenLoop Health; United States; Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical, Dental, Vision

Flexible PTO

401(k) Match

Life & Pet Insurance

Job summary

OpenLoop Health is building the future of healthcare infrastructure and seeks engineers to ship production-grade LLM agents serving real users now. We value depth in agent runtime, evaluation, and observability, plus strong security practices for PHI and sensitive data.

This role spans senior to staff levels, calibrated by scope and depth across AI systems and cross-functional collaboration with product, operations, and clinical stakeholders.

Qualifications

• Bachelor's degree in CS, Engineering or related field.
• 5 to 15+ years of software engineering experience.
• Hands-on production LLM experience in real-user systems.
• Depth in at least one core AI area: agent runtime or evaluation.
• Experience measuring AI systems with evaluation data.
• Strong production engineering fundamentals: testing, debugging, APIs, deployment, on-call.
• Data & Security fluency with a security-first mindset.
• Clear communication with technical and non-technical partners.

Responsibilities

• Ship Production Agents: build and run LLM-based agents used by real users.
• Agent Runtime & Orchestration: manage context, tool calls, retries and sandboxing.
• Evaluation: create test sets and AI-graded evals to track progress.
• Model Access & Operations: manage providers and version upgrades.
• Observability & Cost: trace actions and attribute AI spend.
• Grounding & Retrieval: connect agents to real data via retrieval and prompt design.
• Workflow Design: decide which human steps an agent can perform.
• Security, Privacy & Safety: protect PHI and guard against prompt injection.
• Cloud & Data Foundations: collaborate with GCP and data platforms.
• Raise the Bar: contribute to code/design reviews and help engineers.
• Cross-functional Work: explain AI trade-offs to product/clinical teams.
• Other duties as assigned.

Skills

LLM Production

Agent Runtime

Evaluation

Observability

Security-first

Education

Bachelor's in CS/Engineering

Tools

GCP

Job description

About OpenLoop

OpenLoop was co-founded by CEO, Dr. Jon Lensing, and COO, Christian Williams, with the vision to bring care anywhere. Our telehealth support solutions are thoughtfully designed to streamline and simplify go-to-market care delivery for companies offering meaningful virtual support to patients across an expansive array of specialties, in all 50 states.

About the Role

OpenLoop is building the future of healthcare infrastructure. We are hiring for this role at the Senior, Staff and Senior Staff levels. Apply once; we will calibrate level during the process based on the scope you have carried and the depth you bring, not on years alone.

What You'll Do

• Ship Production Agents: Build and run LLM-based agents and assistants that real users depend on, not prototypes.

• Agent Runtime & Orchestration: Build how agents run, call tools, hold context and hand off work, including failure handling, retries, timeouts and sandboxing.

• Evaluation: Build evaluation sets, test harnesses, regression tests and AI-graded evals so the team can show "this got better" with numbers and catch regressions before users do.

• Model Access & Operations: Work across multiple model providers, manage version upgrades and plan fallbacks for when a provider fails.

• Observability & Cost: Trace what agents did and why, monitor behavior in production, and attribute AI spend to each agent and use case.

• Grounding & Retrieval: Connect agents to real, messy company data through retrieval (RAG), context design and prompt engineering, so answers are based on facts.

• Workflow Design: Work with operations teams to decide which steps of a human workflow an agent can do reliably, and when it should elevate to a person.

• Security, Privacy & Safety: Keep patient data (PHI) protected in every AI system, control what agents can access, and defend against misuse such as prompt injection.

• Cloud & Data Foundations: Build on GCP and partner with the Data Platform teams who provide the governed data agents use.

• Raise the Bar: Contribute to code and design review, and help engineers who are newer to AI.

• Cross-functional Work: Explain AI trade-offs in plain language to product, operations and clinical stakeholders, including when the honest answer is "the AI isn't ready for this yet."

• Other duties as assigned.

Required Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related technical field

• Proven Engineering Experience: 5 to 15+ years of software engineering experience.

• Production LLM Experience: Hands-on experience building and running LLM-based systems in production that real users relied on, not only demos, prototypes or personal projects. Senior Staff candidates typically bring 2+ years.

• Core AI Depth: Real hands-on depth in at least one core area: agent runtime, evaluation, retrieval, or observability and cost. At Staff and Senior Staff we look for depth in agent runtime or evaluation, and at Senior Staff a second deep area plus working knowledge across the rest.

• Evaluation Rigor: Experience measuring whether an AI system works with evaluation data, rather than "it seemed to work."

• Engineering Fundamentals: Strong production engineering across testing, debugging, services, APIs, deployment and on-call.

• Data & Security Fluency: Experience handling sensitive data with a security-first mindset.

• Comfort working in a greenfield space, and clear communication with technical and non-technical partners

Preferred Qualifications:

• Regulated-Industry Experience: Healthcare (PHI, HIPAA), finance or insurance.

• GCP Experience: Running production services on Google Cloud.

• Model Operations: Experience migrating a system between model versions or providers.

• Classic ML Background: Machine learning experience beyond LLMs.

• Internal Platforms: Built a platform used by other engineering teams.

• Team Leadership: For Senior Staff, experience leading a small team's technical direction from zero and growing a single pod into multiple teams.

Our Benefits

In addition, for salaried positions you would also be eligible for:

• Medical, Dental, and Vision plans

• Flexible Spending/Health Savings Accounts

• Flexible PTO

• 401(k) + Company Match

• Life Insurance, Pet insurance, and more

Our Company

We have a relatively flat organizational structure here at OpenLoop. Everyone is encouraged to bring ideas to the table and make things happen. This fits in well with our core values of Autonomy, Competence and Belonging, as we want everyone to feel empowered and supported to do their best work.

Sound like a good fit? We’d love to meet you.