Senior Principal LLM Engineer — AI Platform & Scale

JPMorgan Chase & Co.; Palo Alto (CA); On-site; USD 233,000 - 325,000; Full time

Job summary

JPMorganChase is seeking a Senior Principal Software Engineer in LLM Engineering to lead the design and delivery of trusted AI platforms. You will work across agile teams to build scalable, secure LLM/GNN solutions, driving strategy, architecture, and high‑impact business outcomes.

You will champion best practices in AI governance, observability, and production‑grade ML systems while partnering with data science, SRE, and platform engineering to deliver reliable, cost‑efficient cloud and on‑prem

Qualifications

• Formal training or certification on software engineering concepts with 10+ years of applied experience.
• 8+ years of AI/ML engineering with significant expertise in LLMs, GNNs and other model architectures.
• Architecting and deploying LLM & GNN solutions on AWS at enterprise scale; experience with Azure ML or GCP Vertex AI.
• Hands-on experience building LLM inference engines using Triton and vLLM, including autoscaling and throughput optimization.

Responsibilities

• Advises and leads on strategy, architecture, and development of model serving solutions for LLMs/GNNs across cloud and on‑premises environments.
• Defines and implements MLOps strategies for end‑to‑end model lifecycle management.
• Drives optimization of model inferencing for high throughput and low latency with advanced techniques.
• Sets strategy for agentic AI‑enabled development patterns and governance across teams.
• Leads deployment and observability improvements with SRE for reliable AI services.

Skills

LLM

GNNs

Python

PyTorch

TensorFlow

MLOps

AWS

SageMaker

Kubernetes

Cloud computing

Education

Master’s or PhD in CS or related field

PhD preferred

Tools

Triton Inference Server

vLLM

SageMaker

Bedrock

EKS

Kubernetes

Docker

PyTorch

TensorFlow

MLflow

Job description

JPMorganChase is seeking a Senior Principal Software Engineer in LLM Engineering to lead the design and delivery of trusted AI platforms. You will work across agile teams to build scalable, secure LLM/GNN solutions, driving strategy, architecture, and high‑impact business outcomes.

You will champion best practices in AI governance, observability, and production‑grade ML systems while partnering with data science, SRE, and platform engineering to deliver reliable, cost‑efficient cloud and on‑prem