Lead Data Engineer, AI-First Analytics & Data Mesh

JPMorgan Chase & Co.; Jersey City (NJ); On-site; USD 150,000 - 210,000; Full time

Job summary

JPMorgan Chase & Co. in Jersey City is seeking a Lead Software Engineer for Financial Service Data Engineering with PySpark, Java, and Big Data focus. You will drive AI-first analytics and productized data lifecycles, champion a scalable data mesh, and mentor engineers while partnering with risk and product teams.

The role emphasizes autonomous agents for schema drift, governance evidence, and secure, auditable data products across the firm’s analytics stack.

Qualifications

• Formal training or certification in software engineering with 5+ years of applied experience.
• Proven leadership delivering AI-first analytics and data engineering at scale.
• Experience with data ingestion, integration, and analytics-ready datasets.
• Hands-on with Spark/Databricks, Snowflake, Iceberg and robust data pipelines.

Responsibilities

• Collaborate with business and tech teams to develop AI-first analytics and data product solutions.
• Define and enforce architecture for an AI-driven data product lifecycle: semantic extraction/alignment, automated lineage and DQ, pipeline code generation, mesh registration, and self-service consumption.
• Build and operate autonomous agents for data engineering that detect schema drift, propose transformations, reconcile semantics, triage data incidents, and maintain governance evidence under human-in-the-loop controls.
• Design analytics platforms capable of running reporting and other analytics; explore innovative ideas by building real-time and batch analytics solutions.
• Establish appropriate monitoring and alerting of solution events related to performance, scalability, availability, and reliability.
• Provide technical leadership, guidance, and direction to other team members; build prototypes for demonstrations for peer groups, business partners, and senior leaders.
• Drive team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
• Apply knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
• Lead migration and modernization from legacy analytics/reporting stacks to the strategic mesh ecosystem (e.g., Databricks/Iceberg/common services), reducing fragmentation and duplicated data products.
• Industrialize entity resolution and parent identification with ML/LLM solutions and standardize analytical product packaging to enable reuse and monetization.
• Embed governance, lineage, and DQ by design across critical domains and regulatory reporting, improving auditability and control posture

Skills

Python

PySpark

Java

Big Data

Data Lake

AI/ML

CI/CD

Security

Agile

Leadership

Governance

Education

Formal training or certification in software engineering

Tools

Databricks

Iceberg

Snowflake

Job description

JPMorgan Chase & Co. in Jersey City is seeking a Lead Software Engineer for Financial Service Data Engineering with PySpark, Java, and Big Data focus. You will drive AI-first analytics and productized data lifecycles, champion a scalable data mesh, and mentor engineers while partnering with risk and product teams.

The role emphasizes autonomous agents for schema drift, governance evidence, and secure, auditable data products across the firm’s analytics stack.