Principal Machine Learning Engineer

SmartRecruiters, Inc.; Palo Alto (CA); On-site; USD 198,000 - 420,000; Full time

Job summary

SAP is seeking an experienced Senior ML Architect in Palo Alto to design and deploy large-scale ML systems, focusing on LLMs, RAG, and MLOps. You will mentor engineers, shape architectural standards, and ensure compliance with performance and security standards.

You will act as a trusted advisor, bridging customer needs with SAP product capabilities, and contributing to SAP's AI strategy and governance framework.

Qualifications

• 10+ years of software engineering experience with a track record of delivering complex systems
• Bachelor's degree in Computer Science, Mathematics, Physics, or related field
• Hands-on experience with AI/ML in production environments, including large language models (LLM), retrieval-augmented generation (RAG), prompt engineering, vector search, agent workflows, and model evaluation techniques
• Expertise in distributed systems design and large-scale enterprise product architecture
• Strong programming skills, proficiency with ML frameworks, and solid understanding of MLOps practices
• Excellent English communication skills and ability to collaborate effectively in global, multicultural teams

Responsibilities

• Architect ML solutions and deploy at scale.
• Bridge customer requirements to product capabilities and propose solutions.
• Mentor engineers and define architectural standards.
• Ensure production-ready ML systems aligned with product objectives.

Skills

AI/ML in production

Distributed systems design

Large language models

MLOps practices

English communication

Education

Bachelor's degree in Computer Science, Mathematics, Physics, or related field

Tools

ML frameworks

Job description

• Compensation: USD 198,200 - USD 420,000 - yearly

Company Description

Be part of SAP's expert community, influencing global AI initiatives and mentoring talent to deliver impactful innovations.

Job Description

You will architect ML solutions, lead deployment at scale, and ensure compliance with performance and security standards. You'll act as a trusted advisor and contribute to SAP's AI strategy.

What you'll bring

You have deep expertise in advanced ML system architecture, scalable model deployment, and ML platform capabilities. You are expected to have a solid understanding of modern AI application development, including large language models, retrieval-augmented generation (RAG), prompt engineering, vector search, agent workflows, and model evaluation techniques. You are the technical partner who can bridge the gap between customer requirements and product capabilities and are comfortable working directly with stakeholders, gathering requirements, clarifying ambiguous requests, and proposing solutions that maximize customer value.

Your ability to guide teams, optimize workflows, and connect emerging ML advancements to business needs is what this level is built around. Experience with large-scale ML architecture design, advanced MLOps and observability, model governance and explainability, and optimization across performance and cost dimensions is expected. Technical mentorship and the ability to set standards that others follow are also part of what you'll bring.

What you'll build

You'll lead the design and optimization of large-scale ML systems (LLM capabilities and infrastructure), shape cutting-edge infrastructure, mentor engineers, and define architectural and operational standards. Ensuring ML solutions meet business, performance, scalability, and governance requirements, while advancing the adoption of modern ML techniques, is a central part of the role. You'll influence technical direction across products, enable safe experimentation, and ensure systems are production-ready and aligned with product objectives.

Where you belong

You'll shape how ML engineering is practiced across teams and products, not just within your immediate group. The role requires both technical depth and the ability to influence people who don't report to you. Mentoring engineers is an explicit expectation, and the standards you set will affect work well beyond your direct area. If you've reached the point where your value comes as much from what you enable in others as from what you build yourself, this level reflects that.

Skills used in this role include: Design & Architecture, Leadership & Impact, Managing ambiguity, Engineering Excellence, Quality Mindset, Context engineering, AI output quality assurance, Agentic AI day-to-day practice, AI-Assisted Automation and Prototyping, AI adoption capability, Learning Agility, Collaboration, Complex problem solving, Creative thinking, Effective communication

Equal Employment Opportunity (EEO) Statement

Our company does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factor.

Qualifications

10+ years of software engineering experience with a track record of delivering complex systems

Bachelor's degree in Computer Science, Mathematics, Physics, or related field

Hands-on experience with AI/ML in production environments, including large language models (LLM), retrieval-augmented generation (RAG), prompt engineering, vector search, agent workflows, and model evaluation techniques

Expertise in distributed systems design and large-scale enterprise product architecture

Strong programming skills, proficiency with ML frameworks, and solid understanding of MLOps practices

Excellent English communication skills and ability to collaborate effectively in global, multicultural teams

Additional Information

All your information will be kept confidential according to EEO guidelines.

About SAP
With a global network of customers, partners, employees, and thought leaders, SAP helps the world run better and improves people’s lives.
As a leader in enterprise applications and business AI, SAP stands at the nexus of business and technology. For over 50 years, organizations have trusted SAP to bring out their best by uniting business-critical operations spanning finance, procurement, HR, supply chain, and customer experience.

We win with inclusion
SAP’s culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone – regardless of background – feels included and can run at their best. At SAP, we believe we are made stronger by the unique capabilities and qualities that each person brings to our company, and we invest in our employees to inspire confidence and help everyone realize their full potential. We ultimately believe in unleashing all talent and creating a better world.
SAP is committed to the values of Equal Employment Opportunity and provides accessibility accommodations to applicants with physical and/or mental disabilities. If you are interested in applying for employment with SAP and are in need of accommodation or special assistance to navigate our website or to complete your application, please send an e-mail with your request to Recruiting Operations Team: [e-mail].
For SAP employees: Only permanent roles are eligible for the SAP Employee Referral Program, according to the eligibility rules set in the SAP Referral Policy. Specific conditions may apply for roles in Vocational Training.
Qualified applicants will receive consideration for employment without regard to their age, race, religion, national origin, ethnicity, age, gender (including pregnancy, childbirth, et al), sexual orientation, gender identity or expression, protected veteran status, or disability.

Compensation Range Transparency
SAP believes pay transparency is essential to fostering an honest, supportive, and inclusive culture and is an important part of our commitment to pay equity. The salary range for this position is included in the job posting. The actual compensation offered to the successful candidate will fall within that range and will depend on factors such as education, skills, experience, location, and scope of the role, as determined through the selection process. Where applicable, SAP may also provide variable incentive opportunities and additional benefits, subject to the terms of the relevant plans and policies. Any variable incentive amount is target-based, with actual payouts dependent on company and individual performance. Please reference this link for a summary of SAP benefits and eligibility requirements: SAP North America Benefits .

AI Usage in the Recruitment Process
For information on the responsible use of AI in our recruitment process, please refer to our Guidelines for Ethical Usage of AI in the Recruiting Process.
Please note that any violation of these guidelines may result in disqualification from the hiring process.

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply