Lead Software Engineer - Databricks/AI

JPMorgan Chase & Co.; Wilmington (NC); On-site; USD 61,000 - 101,000; Full time

Job summary

JPMorgan Chase & Co. in the United States seeks a senior, hands-on Lead Software Engineer to build and optimize data pipelines for agentic AI systems. You will design data retrieval layers, APIs, and scalable data services powering enterprise AI workloads.

The role requires expert Python/PySpark knowledge, Databricks/AWS experience, and leadership to mentor engineers while delivering reliable AI-enabled data products on a fast-paced agile team.

Qualifications

• Formal training or certification in software engineering
• At least 5 years of applied software engineering experience
• Expert Python and PySpark programming skills with a production portfolio
• Hands-on Databricks and AWS (Glue, S3, SQS/SNS, Lambda) experience
• Deep Spark and SQL expertise, data warehousing experience
• Experience with CI/CD and automated testing frameworks
• Understanding of agile, DevOps, security, and data governance
• Exposure to agentic AI concepts and API/data service development

Responsibilities

• Build and optimize data pipelines for agentic AI systems
• Develop data retrieval and indexing layers across sources
• Maintain APIs and data services for enterprise systems
• Uphold data quality, security, and governance
• Write secure production code following AWS best practices
• Implement multi-step agent workflows with memory and persistence
• Conduct code reviews and raise technical standards
• Mentor engineers and collaborate with product and data science teams
• Evaluate emerging agentic AI frameworks and data engineering practices

Skills

Python proficiency

PySpark experience

Agile mindset

Mentoring

Team leadership

DevOps practices

Education

Software engineering certification

Tools

Databricks

AWS

Snowflake

Terraform

LangGraph / OpenAI API

Pinecone / FAISS / Chroma

Job description

Salary: $61,000 - 101,000 per year

Requirements

• Formal training or certification in software engineering, with at least 5 years of applied experience
• Expert Python and PySpark programming skills, supported by a portfolio of production-grade code
• Extensive hands‑on experience with Databricks and AWS services, including Glue, S3, SQS/SNS, and Lambda
• Deep expertise in Spark and SQL
• Hands‑on experience with Lakehouse and Delta Lake architecture, application development, testing, and operational stability; experience with Snowflake, Terraform, and LLMs; and knowledge of data observability, data quality, query optimization, and cost optimization
• In‑depth knowledge of enterprise‑scale big data and data warehousing
• Extensive experience with CI/CD processes and automated testing frameworks
• Understanding of agile methodologies, DevOps practices, application resiliency, and security measures
• Understanding of agentic AI concepts, including how autonomous agents plan, reason, use tools, and execute multi‑step workflows, and of the data infrastructure that supports them
• Experience building APIs, data services, and retrieval systems connecting AI agents with enterprise data
• Ability to lead through hands‑on engineering, mentor colleagues, and drive team delivery
• Preferred: Experience with agentic AI frameworks such as LangGraph, AutoGen, CrewAI, or OpenAI Assistants API
• Preferred: Familiarity with tool‑use and function‑calling patterns for LLM‑based agents, including building APIs and data endpoints for autonomous use
• Preferred: Experience with vector databases such as Pinecone, FAISS, or Chroma, and embedding workflows for agent memory, semantic search, and retrieval‑augmented generation
• Preferred: Exposure to agent memory and state‑management patterns, including short‑ and long‑term memory and conversation or task history
• Preferred: Familiarity with guardrails and safety frameworks, including input/output validation, action approvals, and human‑in‑the‑loop controls
• Preferred: Understanding of agentic‑system observability and monitoring, including tracing decisions, logging tool use, and debugging multi‑step workflows
• Preferred: Understanding of responsible AI principles, including autonomous decision‑making, data provenance, and auditability

Responsibilities

• Build and optimize data pipelines and workflows that provide agentic AI systems with reliable, timely access to high‑quality structured and unstructured data
• Develop data retrieval and indexing layers so AI agents can search, query, and synthesize information across multiple sources
• Build and maintain APIs, data services, and function endpoints that agents use to execute tasks, retrieve data, and interact with enterprise systems
• Apply data management best practices and uphold quality, security, compliance, and governance for data used or generated by autonomous agents
• Write secure, high‑quality production code following AWS best practices, and deploy through CI/CD pipelines
• Build orchestration and state‑management layers for multi‑step agent workflows, including memory, context persistence, and task chaining
• Write and review code regularly, conduct thorough code reviews, and raise the teams technical standards
• Mentor and guide junior and mid‑level engineers through pairing, code reviews, and technical coaching
• Work with product owners, data scientists, and business stakeholders to turn business needs into production‑ready agentic AI solutions
• Evaluate and adopt emerging agentic AI frameworks, tools, and data engineering practices to improve the teams capabilities

Technologies

• Agentic AI
• AI
• AI Agents
• API
• AWS
• Lambda
• Big Data
• CI/CD
• Databricks
• DevOps
• Support
• LLM
• Marketing
• Python
• PySpark
• SQL
• Security
• Snowflake
• Spark
• Terraform
• Cloud
• AWS Glue
• Backbone
• RAG
• Windows

More

We are collaborating with hackajob to connect exceptional professionals with this Lead Software Engineer opportunity at JPMorganChase, within our Corporate Sector–Global Finance team. This is a senior, hands‑on technical leadership role on an agile team building data products and infrastructure for agentic AI systems. Our professionals in Corporate Functions work across areas such as finance, risk, human resources, and marketing to support our businesses, clients, customers, and employees. We are one of the worlds oldest financial institutions, serving consumers, small businesses, and corporate, institutional, and government clients across banking, transaction processing, and asset management. We offer competitive total rewards and, for eligible roles, incentive compensation, along with benefits that may include health care, wellness centers, retirement savings, backup childcare, tuition reimbursement, mental health support, and financial coaching. We value diversity and inclusion, are an equal opportunity employer, and provide reasonable accommodations for religious practices and disability‑related needs.

last updated 41 week of 2026