Lead AI Architect: Scalable LLM & Governance

McKesson; Alpharetta (GA); On-site; USD 155,000 - 259,000; Full time

Job summary

McKesson is seeking a Lead AI Engineer to own the architecture of our AI platform and embedded delivery initiatives. You will design, critique, and guide implementations of LLM-powered systems, RAG pipelines, and tooling, while ensuring security, privacy, and PHI compliance in a highly regulated healthcare environment.

You will mentor Senior and Associate AI Engineers, set engineering standards, and collaborate with Legal/Compliance and product teams to align AI architecture with enterprise

Qualifications

• Bachelor’s degree or equivalent; 8+ years in software engineering with AI/ML. Doctorate or Master’s may reduce experience needs.
• Proven architecture leadership beyond a single feature, influencing team and platform decisions.
• Production experience with AI/LLM/NLP systems, including governance and security considerations.

Responsibilities

• Define and own the technical architecture and roadmap for LLM/agentic systems and RAG pipelines.
• Lead architecture reviews ensuring security, privacy, and PHI compliance.
• Drive shared AI platform capabilities including knowledge retrieval and data provenance.
• Architect and build complex AI delivery components with tangible outputs.
• Mentor Senior/Associate AI Engineers; lead design reviews and pair programming.
• Establish secure, observable, and cost-attributed AI operations for production systems.
• Collaborate with product, Legal/Compliance, and engineering leadership to align timelines.
• Evaluate vendor/model options and make build-vs-buy recommendations.
• Ensure architecture decisions meet enterprise governance and risk requirements.
• Represent AI architecture in cross-company forums and enterprise-wide direction.

Skills

AI Architecture Leadership

Production LLM apps

Python/Java

REST APIs

Cloud-native design

Education

Bachelor’s degree or equivalent

Advanced degree (MS/PhD)

Tools

Confluence/Jira governance

MCP protocols

LLM APIs (OpenAI/Anthropic)

Job description

McKesson is seeking a Lead AI Engineer to own the architecture of our AI platform and embedded delivery initiatives. You will design, critique, and guide implementations of LLM-powered systems, RAG pipelines, and tooling, while ensuring security, privacy, and PHI compliance in a highly regulated healthcare environment.

You will mentor Senior and Associate AI Engineers, set engineering standards, and collaborate with Legal/Compliance and product teams to align AI architecture with enterprise