SR Principal Software Engineer - LLM Engineering

JP Morgan Chase

Palo Alto (CA)

On-site

USD 210,000 - 320,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

JPMorganChase is seeking a Senior Principal Software Engineer to lead scalable ML/AI platforms on AWS, guiding model serving for LLMs and graph models. You will drive MLOps, governance, and scalable architectures across agile teams while ensuring security and reliability.

Strong communication with senior leaders is essential. The role emphasizes creating durable ML engineering frameworks, optimizing inference, and aligning technology with business outcomes in a fast-moving financial services

Qualifications

  • 10+ years of software/AI engineering experience.
  • Expertise in LLMs and GNNs.
  • Hands-on experience deploying LLM/GNN solutions on AWS.
  • Proficient in Python and major DL frameworks.
  • Strong knowledge of MLOps and governance.
  • Excellent cross-functional communication.

Responsibilities

  • Advises and leads strategy, architecture, and development of model serving solutions.
  • Defines and implements MLOps and LLMOps for end-to-end model lifecycle management.
  • Drives optimization of model inferencing for high throughput and low latency.
  • Sets strategy and standards for AI-enabled engineering across a portfolio.
  • Applies SDLC tools to improve automation and governance.
  • Creates reusable ML engineering frameworks for scale across teams.
  • Establishes best practices for CI/CD and infrastructure-as-code.
  • Partners with data science and SRE to productionize models on AWS.
  • Leads deployment and optimization using Triton and vLLM for high-scale serving.
  • Oversees AI ops including monitoring, security, and incident response.
  • Translates complex concepts to senior stakeholders to drive business impact.

Skills

Python
LLMs
GNNs
Cloud AWS
MLOps

Education

Formal software engineering training
10+ years AI/ML engineering experience

Tools

SageMaker
Bedrock
EKS
Kubeflow
MLflow
vLLM
Triton

Job description

We're looking for a tech leader ready to take their career to new heights. Join the ranks of top talent at one of the world's most influential companies.

As a Senior Principal Software Engineer at JPMorganChase within the Commercial & Investment Bank Trust & Safety Fraud Prevention team, you provide deep engineering expertise and work across agile teams to enhance, build, and deliver trusted market‑leading technology products in a secure, stable, and scalable way. Leverage your deep expertise to consistently challenge the status quo, innovate for business impact, lead the strategic development behind new and existing products and technology portfolios, and remain at the forefront of industry trends, best practices, and technological advances.

Job responsibilities
  • Advises and leads on the strategy, architecture, and development of model serving solutions for different model architectures (including LLMs & GNNs) across cloud and on‑premises environments, aligning initiatives to business outcomes.
  • Defines and implements MLOps and LLMOps strategies for end‑to‑end model lifecycle management, including training, versioning, deployment, monitoring, and governance.
  • Drives optimization of model inferencing for high throughput and low latency using quantization, model parallelism, intelligent batching, and hardware acceleration for all model architectures.
  • Sets strategy and operating standards for agentic AI-enabled engineering across a portfolio (using enterprise-authorized tools within the work environment) to drive measurable improvements in delivery speed, reliability, and code quality (e.g., AI-orchestrated SDLC/TLM automation, release readiness gating, incident triage/root-cause acceleration, and large‑scale refactoring/test modernization), while defining guardrails for validation, security, resiliency, and reuse across teams and functions.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
  • Creates durable, reusable software and platform frameworks to standardize ML Engineering services, enabling scale across teams and functions.
  • Establishes best practices for automation, CI/CD, and infrastructure‑as‑code using containerization and orchestration technologies.
  • Partners closely with data science, platform engineering, and SRE teams to productionize models on AWS, ensuring observability, reliability, and cost efficiency.
  • Leads deployment and optimization using model inference servers such as Triton Inference Server and vLLM for high‑throughput, low‑latency serving at scale.
  • Oversees production operations for AI workloads, including monitoring, incident response, security, and compliance, with continuous improvement.
  • Translates complex technical concepts and emerging trends into actionable strategies, influencing senior stakeholders and cross‑functional partners to prioritize and deliver AI/ML capabilities that drive measurable business impact while promoting a culture of diversity, opportunity, inclusion, and respect.
Required qualifications, capabilities, and skills
  • Formal training or certification on software engineering concepts and 10+ years of applied experience.
  • 8+ years of AI/ML engineering experience with significant expertise in LLMs, GNNs and other model architectures (e.g., GPT, Llama, Falcon, Mistral).
  • Demonstrated success architecting and deploying LLM & GNN solutions on AWS (e.g., SageMaker, Bedrock, EKS) at enterprise scale; experience with Azure ML or GCP Vertex AI.
  • Experience building LLM and GNN serving platforms in large‑scale environments typical of major tech firms.
  • Hands‑on experience building LLM inference engines using Triton Inference Server and vLLM, including autoscaling, caching, and throughput optimization.
  • Advanced proficiency in Python and optimization techniques applied to deep learning frameworks (PyTorch, TensorFlow, Hugging Face Transformers).
  • Deep understanding of LLMOps/MLOps (e.g., MLflow, SageMaker Pipelines, Kubeflow) with a track record of implementing best practices at scale.
  • Demonstrated experience designing and scaling agentic AI-enabled development patterns (using enterprise-authorized tools within the work environment) across teams/functions, including establishing governance for human-in-the-loop validation, traceability/auditability, and secure handling of sensitive inputs/outputs.
  • Strong understanding of responsible AI use and control expectations at scale, including security/resiliency implications, data sensitivity, and risk-based governance; ability to advise senior leaders on safe adoption, reuse, and measurable outcomes.
  • Expertise in inference optimization and distributed systems for large models focused on high‑throughput, low‑latency applications, including system design, testing, and operational stability for enterprise AI platforms.
  • Excellent communication skills with proven collaboration w
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Engineer - Sr Lead Software Engineer
AI Engineer - Sr Lead Software Engineer

JPMorgan Chase & Co. • Wilmington (DE)

On-site
USD 120,000 - 180,000
SR Principal Software Engineer - LLM Engineering
SR Principal Software Engineer - LLM Engineering

Testing Site - JPMC • Palo Alto (CA)

On-site
USD 250,000 - 330,000
Health insurance
Retirement plan
On-site facilities and amenities
AI Engineer - Sr Lead Software Engineer
AI Engineer - Sr Lead Software Engineer

JPMorgan Chase & Co. • Jersey City (NJ)

On-site
USD 150,000 - 190,000
Sr Lead Software Engineer
Sr Lead Software Engineer

JP Morgan Chase Bank • Jersey City (NJ)

On-site
USD 140,000 - 170,000
Health insurance
Sr Director of Software Engineering - Global Banking Intelligence Services
Sr Director of Software Engineering - Global Banking Intelligence Services

JP Morgan Chase • Houston (TX)

On-site
USD 150,000 - 210,000
Principal Software Engineer - LLM Optimization
Principal Software Engineer - LLM Optimization

JPMorgan Chase & Co. • Jersey City (NJ)

On-site
USD 180,000 - 240,000
Senior Lead Software Engineer - Java/Python/AWS/AI/LLM
Senior Lead Software Engineer - Java/Python/AWS/AI/LLM

JPMorgan Chase & Co. • Plano (TX)

On-site
USD 180,000 - 240,000
Applied AI ML Director for CIB AI4Tech
Applied AI ML Director for CIB AI4Tech

Testing Site - JPMC • Jersey City (NJ)

On-site
USD 240,000 - 360,000
Sr. Lead Software Engineer
Sr. Lead Software Engineer

JP Morgan Chase • Jersey City (NJ)

On-site
USD 140,000 - 190,000
SR Principal Software Engineer - LLM Engineering
SR Principal Software Engineer - LLM Engineering

JP Morgan Chase • Palo Alto (CA)

On-site
USD 180,000 - 240,000