Lead Data Engineer — Spark & ETL Pipelines

JPMorgan Chase & Co.; Plano (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

JPMorgan Chase & Co. in Plano, TX seeks a Lead Software Engineer to design, automate, and operate scalable ETL/data transformation pipelines in production. You will gather requirements, build Spark-based batch/real-time workflows, and contribute to platform infrastructure while mentoring junior engineers.

The role emphasizes cost and reliability improvements, monitoring, CI/CD, and API delivery via Spring Boot/Docker. Strong Spark, HBase/Cassandra, and cloud tooling experience are preferred.

Qualifications

• Formal training or certification on Software engineering concepts and 5+ years applied experience.

Responsibilities

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.
• Review, understand, code, optimize, and automate existing one-off data transformation pipelines into discrete, scalable tasks.
• Plan, design, and implement data transformation pipelines and monitor operations of the data platform in a production environment.
• Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems.
• Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes.

Skills

Python

Java

Scala

Apache Spark

Databricks

Grafana

Prometheus

Elasticsearch

CloudWatch

Spring Boot API

Docker

Git

Education

5+ years software engineering experience

Tools

HBase

Cassandra

Azure

AWS

CI/CD

Confluence

Job description

JPMorgan Chase & Co. in Plano, TX seeks a Lead Software Engineer to design, automate, and operate scalable ETL/data transformation pipelines in production. You will gather requirements, build Spark-based batch/real-time workflows, and contribute to platform infrastructure while mentoring junior engineers.

The role emphasizes cost and reliability improvements, monitoring, CI/CD, and API delivery via Spring Boot/Docker. Strong Spark, HBase/Cassandra, and cloud tooling experience are preferred.