Lead Data Platform Engineer – Databricks Spark & Terraform

JP Morgan Chase; Chicago (IL); On-site; USD 140,000 - 210,000; Full time

Job summary

JPMorganChase is seeking a Lead Software Engineer to drive the Databricks lakehouse platform within Corporate - Employee Platforms. You will lead data modeling, governance, and performance optimization while championing secure, scalable engineering practices.

You will set standards for CI/CD, code reviews, and testing, mentor engineers, and collaborate with stakeholders to translate business needs into robust data platform capabilities.

Qualifications

• 7+ years in data/ platform engineering with hands-on Databricks in production.
• Proficient in Spark (PySpark/Scala) and SQL, including performance tuning.
• Experience building and operating data platforms: ingestion, transformation, orchestration, curated layers.
• Knowledge of Data Lake design (Delta, partitioning, Z-ORDER) with AWS/Snowflake basics.
• Familiarity with Unity Catalog or comparable governance tools.
• Strong software fundamentals: Git, CI/CD, automated testing, reviews; modular design.
• Experience with observability, data quality checks, and monitoring pipelines/SQL workloads.
• Proven ability to lead technical decisions across teams.
• Experience using approved AI-assisted development tools with validation practices.

Responsibilities

• Platform leadership and architecture: define the technical roadmap for the data lakehouse platform.
• Data modeling and database engineering: design curated datasets with best practices.
• Reliability and operations: establish SLOs, runbooks, alerting, incident response.
• Security, governance, access controls: implement Unity Catalog and least-privilege access.
• Performance and cost management: tune workloads, optimize clusters, enable cost observability.
• Engineering excellence: set standards for code quality, testing, CI/CD, documentation.
• Mentorship and collaboration: coach engineers and lead design reviews.
• Drives team adoption of AI-assisted engineering practices with secure validation standards.
• Applies knowledge of SDLC tools to improve automation and value.

Skills

Databricks

Spark/ PySpark

SQL

Data platforms

Git

CI/CD

Automated testing

Code reviews

Observability

AI-assisted tooling

Security governance

Leadership & collaboration

Tools

Databricks

GitHub Copilot

Terraform

Job description

JPMorganChase is seeking a Lead Software Engineer to drive the Databricks lakehouse platform within Corporate - Employee Platforms. You will lead data modeling, governance, and performance optimization while championing secure, scalable engineering practices.

You will set standards for CI/CD, code reviews, and testing, mentor engineers, and collaborate with stakeholders to translate business needs into robust data platform capabilities.