Software Engineer III - AIML

JPMorgan Chase & Co.

McLean (VA)

On-site

USD 120,000 - 190,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

JPMorgan Chase & Co. in McLean, VA seeks a Software Engineer III to help build and scale LLM-enabled backend services. You will design and implement Go-based microservices, leverage gRPC and Protobuf, and integrate with authn/z, caching, and queues to deliver secure, reliable systems.

You will work on end-to-end streaming workloads, contribute to code reviews, and emphasize testing, observability, and secure coding standards in a fast-paced Agile environment.

Qualifications

  • Formal training or certification in software engineering concepts.
  • 3+ years of applied software engineering experience.
  • Proficiency in Go for backend/service development.
  • Experience with gRPC and Protobuf in production.
  • Experience building and operating microservices with a production mindset.
  • Proficiency in automation and CI/CD methods.
  • Secure coding practices and SDLC activities.
  • Hands-on experience with enterprise AI-assisted development tools.
  • Understanding of responsible AI use in engineering workflows.

Responsibilities

  • Execute software solutions for LLM-enabled applications and platforms.
  • Design, build, and maintain backend services in Go.
  • Build gRPC services with Protobuf; API design and compatibility management.
  • Implement bidirectional streaming endpoints for LLM workloads.
  • Integrate LLM services with authn/z, rate limits, caching, queues/streams.
  • Review and debug SDK/service integrations; contribute to standards.
  • Instrument services with observability and runbooks; support alerts/dashboards.
  • Identify issues and drive automation to improve reliability and performance.

Skills

Go backend development
Python
gRPC & Protobuf
Microservices architecture
Security best practices
Testing & CI/CD
Observability & debugging
Enterprise AI tooling usage

Education

Software engineering certification

Tools

Kubernetes
Docker
OpenTelemetry
Kafka/RabbitMQ
Redis
Protobuf tooling

Job description

We have an opportunity to impact your career and provide an adventure where you can push the limits of what’s possible.

As a Software Engineer III at JPMorganChase within Corporate AIML Data Platforms-Chief Data and Analytics Office, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for contributing to critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives—with a focus on Go/Python based backend services, to support LLM-enabled experiences.

Job responsibilities
  • Execute creative software solutions, design, development, and technical troubleshooting for LLM-enabled applications and platforms.
  • Design, build, and maintain backend services in Go, delivering secure, high-quality production code with strong testing discipline.
  • Build gRPC services defined with Protobuf, including API design, schema evolution, and compatibility/versioning practices.
  • Implement bidirectional streaming gRPC endpoints for LLM workloads (e.g., interactive sessions, token/partial-response streaming, incremental results, cancellations, backpressure/flow control).
  • Integrate LLM services with supporting systems (authn/z, rate limits/quotas, caching, queues/streams, persistence) and partner with platform/SRE teams as needed.
  • Review and debug SDK and service integrations; participate in code reviews and contribute to shared engineering standards.
  • Instrument services with strong observability (metrics/logging/tracing), and contribute to runbooks and operational readiness (alerts, dashboards, on-call support as applicable).
  • Identify recurring issues and contribute fixes and automation to improve reliability, latency, and operational stability.
  • Leverages enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity across complex deliverables (e.g., code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standards; contributes learnings and reusable patterns to improve broader team effectiveness.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
Required qualifications, capabilities, and skills
  • Formal training or certification on software engineering concepts and 3+ years applied experience
  • Proficiency in Go for backend/service development, including concurrency patterns (goroutines/channels), context usage, and performance troubleshooting.
  • Working knowledge of Python for supporting services, internal tooling, automation, evaluation harnesses, or data/ETL tasks.
  • Hands-on experience with gRPC and Protobuf in production, including API design and backward/forward compatibility considerations.
  • Experience building and operating microservices/APIs with a production mindset (monitoring, debugging, incident support, and continuous improvement).
  • Proficiency in automation and continuous delivery methods; strong unit/integration testing discipline.
  • Proficient in secure coding practices and core SDLC activities (design, build, test, deploy, operate).
  • Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.
  • Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices.
Preferred qualifications, capabilities, and skills
  • Experience building LLM streaming experiences end-to-end (token/partial-response streaming, tool/function calling orchestration, session state management, safety/guardrail hooks).
  • Familiarity with OpenTelemetry and tracing/metrics patterns for streaming RPCs (correlation IDs, span links, per-message metrics).
  • Experience with Kubernetes/Docker and operating services in containerized environments.
  • Messaging/streaming systems (Kafka/RabbitMQ/SQS), caching (Redis), and patterns for async orchestration alongside streaming.
  • Security best practices for RPC systems: mTLS, authn/z, secrets management, secure-by-default API design, abuse prevention (rate limiting, payload limits).
  • Understanding of cost management and spend alerting for AI/LLM workloads.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Software Engineer III - Full Stack
Software Engineer III - Full Stack

JP Morgan Chase • Jersey City (NJ)

On-site
USD 120,000 - 180,000
Sr Lead Software Engineer
Sr Lead Software Engineer

JP Morgan Chase Bank • Jersey City (NJ)

On-site
USD 140,000 - 170,000
Health insurance
Senior Lead Software Engineer - Java/Python/AWS/AI/LLM
Senior Lead Software Engineer - Java/Python/AWS/AI/LLM

JPMorgan Chase & Co. • Plano (TX)

On-site
USD 180,000 - 240,000
Software Engineer III - AIML
Software Engineer III - AIML

Aumni • Virginia

On-site
USD 130,000 - 180,000
AI Engineer - Sr Lead Software Engineer
AI Engineer - Sr Lead Software Engineer

JPMorgan Chase & Co. • Wilmington (DE)

On-site
USD 120,000 - 180,000
Lead Software Engineer - Python, Observability
Lead Software Engineer - Python, Observability

JPMorgan Chase & Co. • Houston (TX)

On-site
USD 140,000 - 210,000
Lead Software Engineer
Lead Software Engineer

Testing Site - JPMC • Columbus (OH)

On-site
USD 140,000 - 190,000
Lead Software Engineer - Java/Python/AWS/AI/LLM
Lead Software Engineer - Java/Python/AWS/AI/LLM

JPMorgan Chase & Co. • Plano (TX)

On-site
USD 150,000 - 190,000
Senior Lead Software Engineer - Agentic AI/Java/Python
Senior Lead Software Engineer - Agentic AI/Java/Python

JPMorgan Chase & Co. • Plano (TX)

On-site
USD 150,000 - 210,000
Sr. Lead Software Engineer
Sr. Lead Software Engineer

JP Morgan Chase • Jersey City (NJ)

On-site
USD 140,000 - 190,000