Lead Data Platform Engineer – Scalable, Secure Pipelines

Next Frontier Capital; Austin (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

JPMorgan Chase is seeking a Lead Software Engineer to join the analytics team within Commercial & Investment Banking. You will be a core technical contributor responsible for building trusted, scalable technology products in a secure environment.

You will design and operate data pipelines with Spark, Airflow, Kafka, and Flink, partner with product managers and stakeholders to translate requirements into production-grade solutions, and advance governance and security across data platforms.

Qualifications

• Formal training or certification in software engineering with 5+ years of applied experience.
• Experience delivering system design, application development, testing, and operational stability.
• Professional experience focused on software engineering or data platform development.
• Advanced in one or more programming languages (Python, Java and SQL).
• Hands-on experience with distributed data processing frameworks such as Apache Spark and Flink.
• Solid understanding of data modeling techniques (star schema, snowflake) and query optimization.
• Experience designing and operating data pipelines on Databricks using orchestration tools such as Apache Airflow.
• Proficiency with cloud data services (AWS S3, Glue, Redshift, Athena, EMR, Lake Formation, or equivalent).
• Proficient in all aspects of the Software Development Life Cycle.
• Advanced understanding of agile methodologies such as CI/CD, Application Resiliency, and Security.
• Demonstrated proficiency in software applications and technical processes within a technical discipline (e.g., cloud, AI/ML, etc.)

Responsibilities

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine approaches.
• Designs, builds, and maintains scalable data pipelines and ETL/ELT workflows for batch and real-time processing using Spark, Airflow, Kafka, and Flink.
• Develops data platform components including data cataloging, data quality frameworks, and semantic/metrics layers with governance, lineage, and compliance standards.
• Implements data modeling strategies (fact and dimensional, wide tables) to support analytics, reporting, and downstream consumption.
• Partners with analytics teams, product managers, and business stakeholders to translate data requirements into production-grade solutions.
• Develops secure high-quality production code, and reviews and debugs code written by others.
• Identifies opportunities to automate remediation of recurring issues to improve stability of software applications and systems.
• Leads evaluation sessions with external vendors, startups, and internal teams to probe architectural designs and applicability for use within existing systems.
• Leads communities of practice across Software Engineering to drive awareness and adoption of new technologies.
• Adds to team culture of diversity, opportunity, inclusion, and respect.

Skills

Python

Java

SQL

Spark

Flink

Airflow

Databricks

AWS

CI/CD

Data Modeling

Education

Formal software engineering training or certification

Tools

Docker

Kubernetes

Terraform

Kafka

Glue

Redshift

Job description

JPMorgan Chase is seeking a Lead Software Engineer to join the analytics team within Commercial & Investment Banking. You will be a core technical contributor responsible for building trusted, scalable technology products in a secure environment.

You will design and operate data pipelines with Spark, Airflow, Kafka, and Flink, partner with product managers and stakeholders to translate requirements into production-grade solutions, and advance governance and security across data platforms.