Senior Principal AI/ML Engineer — LLM & GNN Architect

Fairygodboss; Palo Alto (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

JPMorganChase is seeking a Senior Principal Software Engineer to lead model serving solutions across cloud and on‑prem environments, driving high-throughput AI infrastructure and end-to-end lifecycle management for enterprise-grade platforms.

The role emphasizes collaboration with SRE and data science teams, building reusable ML engineering frameworks, and ensuring secure, observable, and cost-efficient AI deployments at scale.

Qualifications

• Formal training or certification on software engineering concepts and 10+ years of applied experience.
• 8+ years of AI/ML engineering with LLMs, GNNs and other model architectures.
• Experience deploying LLM & GNN solutions on AWS at enterprise scale; Azure ML or Vertex AI experience.
• Hands-on experience building LLM inference engines (Triton, vLLM) with autoscaling and caching.
• Proficiency in Python and DL frameworks (PyTorch, TensorFlow, HuggingFace).
• Deep knowledge of LLMOps/MLOps (MLflow, SageMaker Pipelines, Kubeflow) at scale.
• Experience designing agentic AI-enabled development patterns with governance for safety and auditability.
• Strong understanding of responsible AI and risk-based governance.

Responsibilities

• Advises and leads on strategy, architecture, and development of model serving solutions across cloud and on-prem environments.
• Defines and implements MLOps and LLMOps for end-to-end model lifecycle management.
• Drives optimization of model inference for high throughput and low latency.
• Sets standards for AI-enabled engineering with guardrails for validation, security, and reuse.
• Applies SDLC tooling to improve automation and value at scale.
• Creates durable software and platform frameworks to standardize ML engineering services.
• Establishes CI/CD and infrastructure-as-code best practices using containers and orchestration.
• Partners with data science, platform engineering, and SRE to productionize models on AWS.
• Leads deployment and optimization using Triton and vLLM for high-throughput serving.
• Oversees production operations for AI workloads including observability and incident response.
• Translates trends into strategies and influences senior stakeholders to deliver AI/ML capabilities.

Skills

LLM/GNN expertise

MLOps

Python

AWS SageMaker

GCP/Azure ML

Model serving

CI/CD

Distributed systems

Education

Master's or PhD in CS/Engineering

Tools

Triton Inference Server

vLLM

SageMaker Pipelines

Kubeflow

Job description

JPMorganChase is seeking a Senior Principal Software Engineer to lead model serving solutions across cloud and on‑prem environments, driving high-throughput AI infrastructure and end-to-end lifecycle management for enterprise-grade platforms.

The role emphasizes collaboration with SRE and data science teams, building reusable ML engineering frameworks, and ensuring secure, observable, and cost-efficient AI deployments at scale.