Principal Engineer - AI Platform

iHerb, LLC; United States; Remote; USD 204,000 - 260,000; Full time

Job summary

iHerb, LLC seeks a Principal Engineer to shape the AI Platform and GenAI experiences across its AI initiatives. You will own end‑to‑end technical strategy, collaborating with peer Principals, Engineering Managers, and VP‑level leadership to drive scalable AI systems.

You will lead multi‑team architecture, establish guardrails and SDLC patterns, and contribute hands‑on expertise to production AI pipelines while mentoring engineers and aligning with product goals.

Qualifications

• AI‑driven SDLC demonstrated use of AI‑assisted development tools to ship production systems and guardrail strategies.
• Architecture at scale: design and evolve large‑scale distributed systems across multiple teams and domains.
• Cross‑org technical leadership: drive architectural standards and platform initiatives spanning teams.
• Engineering quality mindset: strong views on code quality, observability, CI/CD, test automation, and velocity.
• Communication and influence: write architecture docs, present to diverse audiences, build consensus.
• Experience working in distributed teams across the US, China, and Latin America.
• Production experience designing and operating AI platform systems: RAG pipelines, embedding infrastructure, etc.
• Hands‑on experience with LLM evaluation: building eval frameworks and metrics.
• MLOps experience: deployment, monitoring, lifecycle management, cost governance.
• Experience building and operating agentic systems (LangChain, LlamaIndex, etc).
• GenAI in production is required; prototype only not sufficient.
• High degree of accuracy and attention to detail
• Excellent organization and multi‑tasking skills.

Responsibilities

• Define and own the AI platform architecture: retrieval infrastructure, model lifecycle, evals framework, guardrails, and GenAI product design.
• Lead the shared AI platform layer consumed by all AI product features and internal teams.
• Hands‑on contributor: build production AI systems and validate architecture with working software.
• Set and enforce technical standards for the AI Platform team; drive reviews of architecture and model quality.
• Coordinate with the Personalization team to define boundaries between GenAI layer and existing ML infrastructure.
• Contribute patterns and lessons to iHerb’s AI‑driven SDLC golden path.
• Drive cross‑team architectural decisions across platform services.
• Establish and evolve iHerb’s AI‑driven SDLC golden path: standards, guardrails, automation.
• Lead multi‑team technical efforts by coordinating reviews and aligning Principals/EMs.
• Mentor and raise technical bar through code reviews and coaching of senior engineers.
• Represent engineering in cross‑functional conversations with product, data science, security, and infrastructure.
• Feed architectural decisions into the shared knowledge base to compound institutional knowledge.

Skills

AI SDLC

Architecture at scale

Cross-org leadership

Code quality

Communication

Distributed teams

MLOps

LLM eval

Agent systems

GenAI in prod

LangChain/LlamaIndex

Production AI

Observability

Education

Bachelor's Degree in Computer Science or related field

Tools

Claude Code

GitHub Copilot

Cursor

LangChain

LlamaIndex

Job description

For A Better You

At iHerb, we believe that living a healthy and balanced life should be easy and accessible to everyone. As a team member, we’ll empower you to live this promise each day as you make a truly global impact in your career.

We constantly strive for innovation, while transforming and improving the online shopping experience for our customers. We believe that individually we are incredible, but when we come together, our growth is infinite. In an industry that is constantly evolving, we are on a mission to make an impact on the global market, and the individual and collaborative efforts of our people are paramount to helping us succeed.

Whether you work in one of our logistics centers, technology hubs, corporate offices or even from home, your role at iHerb will take you beyond what’s expected, turning challenge into change. If you’re ready for it, we want you to join our team.

Job Summary:

The Principal Engineer operates at the intersection of deep technical execution and broad organizational influence. This role will resolve the hardest architectural questions, set the AI-native engineering bar, and ensure that iHerb’s platform decisions compound over time rather than accumulate debt. Reporting to the VP of Engineering this role collaborates closely with peer Principals, Engineering Managers, and peer VPs across the organization.

This role owns the end-to-end technical strategy for iHerb’s AI Platform and GenAI product experiences. This spans the user-facing AI layer (Wellness Agent, LLM-driven recommendations, RAG over catalog and reviews, generative content) and the shared AI infrastructure (RAG pipelines, evals framework, MLOps, guardrails, observability) that makes AI features production-grade and repeatable across the engineering organization.

Job Expectations:

• Define and own the AI platform architecture: retrieval infrastructure, model lifecycle, evals framework, guardrails, and GenAI product feature design.

• Lead the build of the shared AI platform layer consumed by all AI product features and reusable by internal business teams.

• Hands‑on contributor: build production AI systems, write proofs of concept, and validate architecture through working software.

• Set and enforce technical standards for the AI Platform team; drive architecture reviews and model quality reviews.

• Coordinate with the Personalization team to define clear boundaries between the GenAI product layer and existing ML personalization infrastructure.

• Contribute AI platform‑specific patterns and lessons into iHerb’s shared AI‑driven SDLC golden path.

• Drive the hardest cross‑cutting technical decisions across multiple teams and shared platform services.

• Establish and evolve iHerb’s AI‑driven SDLC golden path: shared standards, Claude Code skills, guardrails, and automation patterns.

• Lead complex multi‑team technical efforts by coordinating architecture reviews, aligning peer Principals and EMs, and resolving competing approaches.

• Mentor and raise the technical bar across the engineering organization through code review, architecture review, and direct coaching of senior engineers.

• Represent engineering in cross‑functional conversations with product, data science, security, and infrastructure.

• Feed architectural decisions into the shared knowledge base so institutional knowledge compounds across the organization.

The duties and responsibilities described above may provide only a partial description of this position. This is not an exhaustive list of all aspects of the job. Other duties and responsibilities not outlined in this document may be added as necessary or desirable, with or without notice.

Knowledge, Skills and Abilities:

Required:

• AI‑driven SDLC (required): demonstrated use of AI‑assisted development tools such as Claude Code, GitHub Copilot, or Cursor to ship production systems. Can articulate workflow changes, quality tradeoffs, and guardrail strategies.

• Architecture at scale: experience designing and evolving large‑scale distributed systems across multiple teams and years: APIs, data pipelines, event‑driven architectures, or high‑traffic platforms.

• Cross‑org technical leadership: track record of driving architectural standards, technical roadmaps, or platform initiatives that span multiple teams or organizations.

• Engineering quality mindset: deeply held opinions on code quality, observability, CI/CD, test automation, and maintaining velocity without accumulating hidden debt.

• Communication and influence: able to write clear architecture documents, present to technical and non‑technical audiences, and build consensus without formal authority.

• Experience working in distributed teams across the US, China, and Latin America.

• Production experience designing and operating AI platform systems: RAG pipelines, vector search, embedding infrastructure, or retrieval‑augmented applications at scale.

• Hands‑on experience with LLM evaluation: building eval frameworks, defining quality metrics, and iterating on model and prompt quality using data.

• MLOps experience: model deployment, monitoring, lifecycle management, and cost governance in a production environment.

• Experience building and operating agentic systems using frameworks such as LangChain, LlamaIndex, or equivalent.

• GenAI in production is required; prototype or research‑only experience is not sufficient at this level.

• High degree of accuracy and attention to detail

• Excellent organization skills and ability to multi‑task

Equipment Knowledge:

• Experience with Microsoft Office Suite (Word, Excel, PowerPoint)

• Experience with Google Business Suite (Gmail, Drive, Docs, Sheets, Forms) preferred

Experience Requirements:

Generally requires a minimum of 10+ years of software engineering experience, with a significant portion at senior, staff, or principal IC level.

Education Requirements:

Bachelor’s Degree in Computer Science or related field preferred, or a combination of education and equivalent work experience.

Judgment/Reasoning Ability:

Able to identify, troubleshoot and resolve problems quickly using sound judgment, poise and diplomacy. Ability to use judgment and reasoning skills, and determine when to elevate issues, as required, in a timely manner.

Physical Demands:

The physical demands described here are representative of those that must be met by a Team Member to successfully perform the essential functions of this job. While performing the duties of this job, the Team Member is regularly required to talk and hear. The Team Member is frequently required to sit, walk, climb stairs, use hands and fingers, bend, stoop and reach with hands and arms. Reaching above shoulder heights, below the waist or lifting as required to file documents or store materials throughout the work day. The Team Member may occasionally lift or move office products and supplies up to 25 pounds.Proper lifting techniques required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

Work Environment:

• Hectic, fast‑paced with multi‑level distractions

• Professional, yet casual work environment

• Office / Warehouse environment

• Ability to work extended hours as required

#LI-JC1

Compensation

The expected salary range for this role is $204,000.00 - $260,000.00 USD. The actual base pay offered will be determined by factors such as the candidate’s relevant experience, education, geographic location, and internal equity.

If you like wild growth and working with happy, enthusiastic over‑achievers, you’ll enjoy your career with us!

Staffing Agency Submission Notice

iHerb does not accept unsolicited 3rd party (“Agency”) candidates. If you are an Agency, please send any requests to be considered as a supplier in our Vendor Management System to [e-mail]. Do not contact iHerb employees directly. If requested to work on a role, any Agency candidates would be presented through the internal recruiting organization.

iHerb is an Equal Opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or veteran status. iHerb provides equal employment opportunities to all applicants for employment and prohibits discrimination and harassment.