Senior Data Platform Lead – AI, Spark, Databricks

Fairygodboss; Houston (TX); On-site; USD 140,000 - 210,000; Full time

Job summary

JPMorganChase is seeking a Lead Software Engineer to guide an agile data engineering team building a secure, scalable KYC and risk data platform. You will define architecture, engineering practices, and standards across multiple teams and deliver high-impact software that scales.

The role requires hands-on leadership, expert coding in Python/Java, and experience with AI-enabled engineering tools, cloud platforms, and data processing technologies in a regulated financial services environment.

Qualifications

• Formal training or certification on software engineering concepts.
• Hands-on experience delivering system design, application development, testing, and operational stability at enterprise scale.
• Hands-on experience designing and deploying production AI/ML systems, including LLM-based applications and agentic architectures with tool use, memory, and multi-step reasoning in regulated environments.
• Expert in one or more programming languages, particularly Python and/or Java.
• Advanced knowledge of software application development and technical processes, with depth in one or more disciplines (e.g., cloud, AI/ML, data engineering).
• Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
• Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations and secure handling of inputs/outputs.
• Experience in large-scale data processing, microservices, API design, Kafka, Redis, Memcached, observability tools, and orchestration frameworks.
• Practical cloud-native experience (AWS, Azure, or GCP).
• Ability to present and effectively communicate with senior leaders and executives.

Responsibilities

• Develops secure, high-quality production code for data-intensive applications and platforms, and reviews and mentors other engineers
• Creates durable, reusable software frameworks and patterns used across teams and functions
• Designs and governs agentic AI systems, including multi-agent workflows, tool-use integrations, and human-in-the-loop controls in regulated environments
• Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes with measurable validation standards
• Applies knowledge of Software Development Life Cycle toolchain, including enterprise-approved AI-assisted development and automation capabilities, to improve value realized by automation at scale
• Establishes engineering standards for Large Language Model-based applications - RAG pipelines, embedding workflows, vector store integrations, and model serving
• Drives adoption of advanced technical methods and practices aligned with latest industry standards and product development methodologies
• Advises cross-functional teams on technological matters within domain expertise
• Applies knowledge of tools within the SDLC toolchain to improve automation at scale.

Skills

Python

Java

AI/ML systems

Cloud platforms

Data engineering

Leadership / mentoring

Tools

Kafka

Redis

Memcached

Dynatrace

Splunk

Grafana

Airflow

Temporal

Job description

JPMorganChase is seeking a Lead Software Engineer to guide an agile data engineering team building a secure, scalable KYC and risk data platform. You will define architecture, engineering practices, and standards across multiple teams and deliver high-impact software that scales.

The role requires hands-on leadership, expert coding in Python/Java, and experience with AI-enabled engineering tools, cloud platforms, and data processing technologies in a regulated financial services environment.