Lead Databricks ML & Cloud Data Engineer

Worky; Plano (TX); On-site; USD 180,000 - 240,000; Full time

Job summary

JPMorgan Chase is seeking a Lead Software Engineer, Machine Learning and Cloud, to guide the architecture and delivery of high-throughput data pipelines. You will drive lakehouse patterns with Delta Lake and leverage Databricks to enable scalable, secure data processing.

You will champion AI-assisted engineering practices, ensure secure coding and testing, and own cloud and data platform components across multiple business functions.

Qualifications

• Formal training or certification in software engineering concepts.
• 5+ years of applied software/data engineering experience.
• Experience leading AI-assisted software development tools and validating AI outputs.

Responsibilities

• Lead architecture and delivery of high-throughput data pipelines using Databricks and Spark.
• Establish lakehouse patterns with Delta Lake and ensure performance at scale.
• Drive adoption of enterprise AI-assisted engineering practices and secure coding standards.
• Own Databricks cluster strategy, autoscaling, and Spark configurations.
• Design secure data ingestion and transformation frameworks with Databricks services.

Skills

Databricks

Spark

Python/Java

AI-assisted development

CI/CD

Security & compliance

AWS

Airflow

Terraform

Git workflows

Education

Formal training or certification in software engineering

Tools

Databricks

Delta Lake

Unity Catalog

Workflows

Airflow

Terraform

Git-based CI/CD

Job description

JPMorgan Chase is seeking a Lead Software Engineer, Machine Learning and Cloud, to guide the architecture and delivery of high-throughput data pipelines. You will drive lakehouse patterns with Delta Lake and leverage Databricks to enable scalable, secure data processing.

You will champion AI-assisted engineering practices, ensure secure coding and testing, and own cloud and data platform components across multiple business functions.