Staff AI Engineer

Modernizing Medicine, Inc.; Boca Raton (FL); On-site; USD 170,000 - 250,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits

401(k) with company matching

Paid time off & parental leave

Tuition reimbursement and professional

Job summary

Modernizing Medicine seeks a Staff AI Engineer to define and drive AI/agentic system architectures across multiple product domains. This senior IC role influences high-impact decisions, evolves AI practices, and turns experimental AI into reliable production systems.

You will guide multi-agent orchestration, production RAG, tool and MCP integration, and the observability stack to ensure dependable delivery.

Qualifications

• Master’s or Ph.D. in Computer Science/Software Engineering or related field.
• 10+ years in ML/AI or software engineering with 4+ years in senior roles with production ownership.
• Proven engineering leadership driving strategy and cross-team decisions.
• Expertise in platform and distributed-systems architecture at scale: model serving, APIs, data platforms, and AI/LLM infra.
• Hands-on experience architecting/operating production agentic AI/LLM systems (multi-agent workflows).
• Deep understanding of embeddings and vector DB internals; experience with RAG.
• Strong production debugging, reliability, and incident-response skills.
• Cost-aware for cloud AI/LLM workloads; capacity planning and cost optimization.
• Mentor mid/senior engineers; strong cross-functional communication.

Responsibilities

• Define and drive technical direction for AI/agentic systems and contribute to AI platform roadmap.
• Lead architecture decisions for compute, cloud, and AI infrastructure across teams.
• Design large-scale AI/LLM systems: inference platforms, APIs, distributed architectures.
• Architect production multi-agent systems end-to-end: orchestration, state, tool integration.
• Establish best practices for AI/LLM systems: evaluation, observability, reliability.
• Lead production debugging and incident response; harden fixes into guardrails.
• Mentor engineers; lead architecture reviews and ADRs for AI infra.
• Contribute to capacity planning and cost optimization for AI/LLM workloads.

Skills

Engineering leadership

Platform architecture

Distributed systems

Model serving

APIs

Data platforms

AI/LLM infrastructure

Agentic AI

Production debugging

Cost optimization

Mentorship

Communication

Education

Master’s or Ph.D. degree in CS/SE

Job description

## Staff AI EngineerApplyremote type: Hybridlocations: Boca Raton, FLtime type: Full timeposted on: Posted Todayjob requisition id: R4761**Join the Team Modernizing Medicine**At **ModMed**, we’re not just building software—we’re reimagining the healthcare experience. Founded in 2010 by a practicing physician and a successful tech entrepreneur, we took a radically different approach: **we hired doctors and taught them how to code.** This "for doctors, by doctors" philosophy has allowed us to create an AI-enabled, specialty-specific cloud platform that places patients at the center of care.**A Culture of Excellence**When you join ModMed, you’re joining an award-winning team recognized for innovation and employee satisfaction. From our global headquarters in Boca Raton Florida, and extensive employee base in Hyderabad India, we are a team of 4,500+ passionate problem-solvers on a mission to increase medical practice success and improve patient outcomes:* **Consistently ranked as a Top Place to Work*** **2025 Globee Business Awards:** Gold Globee for “Technology Team of the Year”* **2025 Black Book Awards:** Ranked #1 EHR in 11 Specialties* **Florida Venture Forum:** Venture-Backed Company of the YearWe are growing fast, thinking big, and we are just getting started.**Ready to modernize medicine with us?****Job Description Summary:**As a **Staff AI Engineer,** you define and drive the architecture of AI and agentic systems across multiple teams and product domains. This is a senior individual-contributor leadership role: you influence high-impact architectural decisions, evolve the practices and standards for building agentic AI, and turn experimental AI capabilities into reliable production systems. You set direction for multi-agent orchestration, production RAG (hybrid search, re-ranking, and query routing), tool and MCP integration, and the evaluation and observability stack that keeps them dependable. You mentor senior engineers and represent AI engineering in cross-functional and strategic initiatives. A background in classical ML is an asset; the primary requirement is a proven track record of shipping production agentic AI.****KEY RESPONSIBILITIES***** Define and drive technical direction for AI and agentic systems, and contribute to the AI platform roadmap across teams* Influence architecture decisions for compute, cloud, and AI infrastructure across teams* Lead the design of large-scale AI/LLM systems: inference platforms, APIs, and distributed architectures* Architect production multi-agent systems end-to-end: orchestration, state management, tool integration, and failure handling* Define and drive best practices and standards for AI/LLM systems across teams (agent design, evaluation, observability, reliability)* Lead complex production debugging and incident response across teams, and harden the resulting fixes into platform guardrails* Mentor senior engineers and emerging technical leaders, raising the engineering bar* Lead technical design reviews and architecture decision records (ADRs) for critical AI infrastructure* Contribute to capacity planning and cost optimization strategies for AI/LLM infrastructure****GENAI / AGENTIC AI CAPABILITIES***** Define and drive vector database and RAG architecture decisions across systems and teams: structured RAG, hybrid search (dense + sparse + keyword), re-ranking, and query routing* Lead multi-agent platform architecture decisions: runtime selection, orchestration patterns, and enterprise integration strategy* Set the technical direction for MCP (Model Context Protocol) adoption and agent runtime infrastructure* Shape agent infrastructure adoption: evaluate and standardize frameworks, tooling, and deployment patterns for agentic AI* Architect evaluation infrastructure for non-deterministic LLM systems: synthetic golden-set generation, hierarchical weighted scoring (component, composite, and system-level F1), bootstrap confidence intervals, and paired A/B comparison, treating a change as real only when it is both statistically significant and clears a minimum effect size* Gate deployments on eval results: tiered regression thresholds (hard-gate vs monitor components) wired into CI so a measurable quality regression blocks the release, with observability via tracing across multi-step chains and tool calls and drift detection on LLM inputs and outputs* Drive LLM cost optimization at scale: model routing, caching, batching, token budget management, and provider cost analysis****REQUIRED SKILLS & QUALIFICATIONS***** Master’s or Ph.D. degree in Computer Science, Software Engineering, or a related field.* 10+ years of professional experience in ML/AI or software engineering, including 4+ years in senior or staff-level roles with production system ownership* Demonstrated engineering leadership, including driving technical strategy and influencing cross-team decisions* Expertise in platform and distributed-systems architecture at scale: model serving, APIs, data platforms, and AI/LLM infrastructure* Hands-on experience architecting and operating production agentic AI or LLM systems (multi-agent workflows, production RAG, tool and MCP integration)* Deep understanding of embedding models, retrieval algorithms, and vector database internals* Strong production debugging, reliability, and incident-response skills* Experience building rigorous evaluation for non-deterministic AI systems, including statistical methods (such as bootstrap confidence intervals and minimum effect-size thresholds) to separate genuine quality changes from run-to-run model variance* Cost-awareness for cloud AI/LLM workloads: capacity planning and cost optimization* Proven mentorship of mid-level and senior engineers* Strong communication skills for executive and cross-functional audiences****PREFERRED QUALIFICATIONS (NICE TO HAVE)***** Experience in Healthcare, FinTech, or other regulated industries* Experience building AI/LLM systems or platform components from the ground up* Defined best practices for AI-assisted development (Claude Code): code quality standards, review, and responsible usage across teams* Track record of conference talks, published papers, or significant open-source contributions* Experience with GPU-accelerated inference and model serving optimization* Familiarity with workflow orchestration and streaming architectures for real-time AI***ModMed Benefits Highlight:*** At ModMed, we believe it’s important to offer a competitive benefits package designed to meet the diverse needs of our growing workforce. Eligible Modernizers can enroll in a wide range of benefits:**United States*** Comprehensive medical, dental, and vision benefits, including a company Health Savings Account contribution,* 401(k): ModMed provides a matching contribution each payday of 50% of your contribution deferred on up to 6% of your compensation. After one year of employment with ModMed, 100% of any matching contribution you receive is yours to keep.* Generous Paid Time Off and Paid Parental Leave programs,* Company paid Life and Disability benefits, Flexible Spending Account, and Employee Assistance Programs,* Company-sponsored Business Resource & Special Interest Groups that provide engaged and supportive communities within ModMed,* Professional development opportunities, including tuition reimbursement programs and unlimited access to LinkedIn Learning,* Global presence and in-person collaboration opportunities; dog-friendly HQ (US), Hybrid office-based roles and remote availability for some roles,* Weekly catered breakfast and lunch, treadmill workstations, Zen, and wellness rooms within our BRIC headquarters.