Lead Data & AI Platform Engineer: Scalable Pipelines

JPMorganChase; Jersey City (NJ); On-site; USD 180,000 - 240,000; Full time

Job summary

JPMorganChase is seeking a Lead Software Engineer for its Payments Technology data platform in Jersey City. You will architect and build data pipelines, oversee governance, and drive AI-enabled automation at scale.

You will collaborate with product, analytics, and engineering teams to deliver production-grade solutions, ensuring security and reliability across complex data ecosystems.

Qualifications

• Formal training or certification on software engineering concepts with 5+ years of applied experience.
• Hands-on experience delivering system design, application development, testing, and operational stability.
• Demonstrated experience in software engineering or data platform development.
• Advanced in one or more programming languages (Python, Java, SQL).
• Hands-on experience with distributed data processing frameworks (Spark, Flink).
• Solid data modeling knowledge (star schema, snowflake) and query optimization.
• Experience building Databricks pipelines using Airflow for orchestration.
• Proficiency with cloud data services (AWS S3, Glue, Redshift, Athena, EMR, Lake Formation).
• Experience deploying data platforms on Kubernetes with open catalog integration (Iceberg/OpenMetadata).
• Advanced understanding of agile methods (CI/CD, resiliency, security).
• Experience developing Agentic AI, LLMs, RAG architectures, and vector databases.

Responsibilities

• Executes creative software solutions and troubleshooting for scalable platforms.
• Designs, builds, and maintains scalable data pipelines and ETL/ELT workflows (batch and real-time).
• Develops data platform components with governance, lineage, and quality controls.
• Implements data modeling strategies to support analytics and reporting.
• Partners with analytics teams and stakeholders to translate data requirements into production solutions.
• Writes secure, production-grade code and reviews others' work.
• Identifies opportunities to automate remediation for stability improvements.
• Leads evaluation sessions with vendors and internal teams on architecture.
• Leads development of Agentic Autonomous Lakehouse capabilities and governance.
• Fosters communities of practice and inclusive team culture.

Skills

Python

Java

SQL

Spark

Flink

Airflow

Databricks

Kubernetes

CI/CD

Security

Agile methodologies

Tools

Kafka

Hive/Data catalog

Iceberg/OpenMetadata

Terraform

Docker

Tableau/dbt Metrics

Job description

JPMorganChase is seeking a Lead Software Engineer for its Payments Technology data platform in Jersey City. You will architect and build data pipelines, oversee governance, and drive AI-enabled automation at scale.

You will collaborate with product, analytics, and engineering teams to deliver production-grade solutions, ensuring security and reliability across complex data ecosystems.