Platform Engineer Lead: Spark, Databricks on AWS

Next Frontier Capital; Jersey City (NJ); On-site; USD 180,000 - 230,000; Full time

Job summary

JPMorganChase is hiring a Lead Software Engineer for the CDAO AIML Data Platforms Team in Jersey City. You will be a core technical contributor executing cloud data platform solutions across Spark, Databricks, and Kubernetes, while maintaining secure and scalable production systems.

You will lead design, development, and operational stability, collaborating with data engineering, ML, and application teams to advance the firm’s data analytics capabilities.

Qualifications

• Formal training or certification on software engineering concepts.
• 5+ years of applied software engineering experience.
• Hands-on Python and/or Java with automated unit testing.
• Big Data compute engines: Apache Spark Core/SQL, Databricks, Kubernetes.

Responsibilities

• Executes creative software solutions with design, development, and troubleshooting.
• Solves cloud data platform problems by building Spark/Data Lake based solutions.
• Designs and maintains a managed Spark on Kubernetes and Databricks platform.
• Performs platform setup, resource monitoring, and engineering support for data teams.
• Develops secure production code and reviews others' code.
• Identifies and automates recurring issues to improve stability.
• Leads evaluation sessions with vendors and teams on architecture and credentials.
• Leads communities of practice to spread new technologies.
• Contributes to inclusive team culture.

Skills

Python

Java

CI/CD

GitHub/Bitbucket

Docker

Terraform

Apache Spark

Tools

Databricks

Kubernetes

AWS

Terraform

Jenkins

Docker

Artifactory

Job description

JPMorganChase is hiring a Lead Software Engineer for the CDAO AIML Data Platforms Team in Jersey City. You will be a core technical contributor executing cloud data platform solutions across Spark, Databricks, and Kubernetes, while maintaining secure and scalable production systems.

You will lead design, development, and operational stability, collaborating with data engineering, ML, and application teams to advance the firm’s data analytics capabilities.