Principal ML Engineer — AI Platform Architect (Hybrid)

Cisco Systems, Inc.

San Jose (CA)

Hybrid

USD 296,000 - 375,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

401(k) matching
Medical, dental, vision insurance
Paid parental leave

Job summary

Cisco Systems, Inc. in San Jose is seeking a Principal Machine Learning Engineer to design and scale AI foundations powering Cisco IQ and customer deployments. You will lead cross-functional engagements, own foundational AI platform architecture, and guide production-grade multi-tenant systems across cloud, on-premises, and edge environments.

Strong leadership and hands-on coding are essential. The role requires deep expertise in LLM/SLM inference, RAG workflows, and cost-efficient deployments,

Qualifications

  • Bachelors + 14 years of related experience, or Masters + 11 years of related experience, or PhD + 7 years of related experience.
  • Experience designing and building production-grade, multi-tenant distributed systems in GO, Python or similar to include high throughput service APIs, asynchronous/event driven architectures, observability, and reliability engineering at scale.
  • Experience architecting and deploying production LLM and/or SLM inference runtimes optimized with continuous batching, KV-cache management, or speculative decoding.
  • Experience architecting and deploying production RAG systems and agentic workflows using vector databases.
  • Experience designing and implementing automated evaluation frameworks for AI systems, including golden test datasets, LLM-as-a-judge methodologies, drift detection, and/or CI/CD regression gates.

Responsibilities

  • Design and build robust, multi-tenant AI services with scalable APIs, versioning, backward compatibility, and quota management across cloud, on-premises, and air-gapped environments.
  • Select, evaluate, and integrate inference runtimes, orchestration frameworks, and serving technologies. Translate emerging capabilities into production architectures optimized for scalability, latency, memory, and compute across GPU, CPU, and edge environments.
  • Design and implement cost optimization across workloads by improving token efficiency, model selection, inference performance, and compute utilization.
  • Lead architecture and design reviews, resolve complex technical trade-offs, mentor senior engineers, and influence platform direction through hands-on technical leadership and execution.

Skills

Distributed systems
Go/Python
LLM inference
RAG systems
CI/CD
Observability

Education

Bachelor's degree
Master's degree
PhD

Tools

vLLM
SGLang
Triton
LangGraph

Job description

Cisco Systems, Inc. in San Jose is seeking a Principal Machine Learning Engineer to design and scale AI foundations powering Cisco IQ and customer deployments. You will lead cross-functional engagements, own foundational AI platform architecture, and guide production-grade multi-tenant systems across cloud, on-premises, and edge environments.

Strong leadership and hands-on coding are essential. The role requires deep expertise in LLM/SLM inference, RAG workflows, and cost-efficient deployments,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal ML Engineer: AI Platform Architect (Hybrid, San Jose)
Principal ML Engineer: AI Platform Architect (Hybrid, San Jose)

Cisco Systems, Inc. • San Jose (CA)

Hybrid
USD 296,000 - 375,000
Principal AI Platform Engineer - Hybrid (San Jose)
Principal AI Platform Engineer - Hybrid (San Jose)

020 Cisco Systems, Inc. • San Jose (CA)

Hybrid
USD 296,000 - 375,000
Lead ML Engineer — AI Security & Scalable Platforms
Lead ML Engineer — AI Security & Scalable Platforms

Cisco Systems • Milpitas (CA)

Hybrid
USD 198,000 - 323,000
Medical insurance
Dental insurance
Vision insurance
+3
Senior ML Engineer — Agentic AI & Scalable Systems
Senior ML Engineer — Agentic AI & Scalable Systems

Cisco Systems, Inc • San Jose (CA)

Hybrid
USD 230,000 - 360,000
Vision insurance
Senior ML Platform Architect - Foundational AI & Inference
Senior ML Platform Architect - Foundational AI & Inference

Cisco Systems, Inc. • San Jose (CA)

Hybrid
USD 260,000 - 380,000
Medical, Dental, Vision
401(k) matching
Paid parental leave
+1
Senior ML Platform Lead — Enterprise CX AI
Senior ML Platform Lead — Enterprise CX AI

Cisco Systems, Inc • San Jose (CA)

Hybrid
USD 257,000 - 333,000
Medical, dental and vision insurance
401(k) with Cisco matching
Paid parental leave
+2
Lead ML Engineer, AI Defense — Hybrid, Secure AI
Lead ML Engineer, AI Defense — Hybrid, Secure AI

Cisco Systems, Inc • Milpitas (CA)

Hybrid
USD 216,000 - 281,000
Principal ML Architect, Agentic Networking
Principal ML Architect, Agentic Networking

Cisco Systems, Inc. • San Jose (CA)

Hybrid
USD 343,000 - 433,000
Hybrid work model
Competitive Benefits
Career growth
Lead AI Platform Architect for Agentic ML
Lead AI Platform Architect for Agentic ML

Cisco Systems, Inc • San Francisco (CA)

On-site
USD 296,000 - 375,000
401(k) matching
Paid parental leave
Medical, dental, vision
+4
Senior AI Infra Systems Architect & Lead Engineer
Senior AI Infra Systems Architect & Lead Engineer

Cisco Systems, Inc • North Carolina

On-site
USD 150,000 - 230,000