Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

Aumni

Mumbai

On-site

INR 3,500,000 - 6,000,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

JPMorganChase in Mumbai is seeking a Lead Data Engineer to join an agile team that builds data collection, storage, access, and analytics solutions at scale. You will maintain critical data pipelines and architectures across multiple domains, ensuring security, stability, and performance.

You will design data models, optimize Spark and Databricks workloads, collaborate with data scientists on ML/AI projects, and ensure governance, quality, and reliable production deployment through MLOps

Qualifications

  • Formal training or certification on data engineering concepts.
  • Hands-on experience with Apache Spark at scale.
  • Production ML infrastructure exposure.
  • Databricks experience: notebooks, Delta Lake, job orchestration.
  • Strong SQL skills for large datasets.
  • Cloud data platforms knowledge (AWS/Azure/GCP).
  • Version control with Git.
  • Awareness of responsible AI practices in engineering workflows.

Responsibilities

  • Architect large-scale data pipelines using Apache Spark, ensuring efficient processing of structured and unstructured data.
  • Build and optimize batch and streaming data workloads for analytics and operations.
  • Develop, optimize, and productionize notebooks, jobs, and workflows in Databricks for ML use cases.
  • Design data architecture and modeling strategies balancing performance, scalability, and governance.
  • Write and optimize complex SQL queries across large datasets.
  • Support ML model serving infrastructure in production; monitor and scale.
  • Collaborate with data scientists and ML engineers on MLOps practices (versioning, CI/CD, monitoring).
  • Partner with analytics, risk, and platform teams to translate business needs into data solutions.
  • Maintain data quality, lineage, and security aligned with governance.
  • Leverage AI-assisted development tools within the SDLC to improve quality and speed.
  • Apply secure coding standards and validate outputs via peer review and testing.

Skills

Apache Spark
Databricks
SQL
MLOps
Cloud platforms
Git
Data modeling
Python (notebook usage)

Education

Data engineering certification

Tools

Databricks
Delta Lake
Airflow
Docker
Kubernetes
Kafka
MLflow
Git

Job description

Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.

As a Lead Data Engineer at JPMorganChase within the Consumer and Community Banking , you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives.

Job responsibilities
  • Architects and maintains large-scale data pipelines using Apache Spark and related distributed computing frameworks, ensuring efficient processing of structured and unstructured data.
  • Builds and optimizes both batch and streaming data workloads to support analytical and operational use cases.
  • Develops, optimizes, and productionizes notebooks, jobs, and workflows within the Databricks environment to support analytical and machine learning use cases.
  • Designs and implements data architecture and data modeling strategies, including dimensional models, lakehouse structures, and schema design that balance performance, scalability, and governance requirements.
  • Writes and optimizes complex SQL queries for data transformation, validation, and performance tuning across large datasets.
  • Supports ML model serving infrastructure, ensuring trained models are deployed, monitored, and scaled reliably in production environments.
  • Collaborates closely with data scientists and ML engineers on MLOps practices, including model versioning, CI/CD pipelines for ML artifacts, monitoring, and automated retraining workflows.
  • Partners with cross-functional stakeholders - including analytics, risk, and platform engineering teams - to translate business requirements into technical data solutions.
  • Maintains strong data quality, lineage, and security standards consistent with data governance and compliance expectations.
  • Leverages enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity across complex deliverables (e.g. code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standards; contributes learnings and reusable patterns to improve broader team effectiveness.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation
Required qualifications, capabilities, and skills
  • Formal training or certification on data engineering concepts and 5+ years applied experience
  • Demonstrates strong, hands-on proficiency with Apache Spark or comparable distributed compute frameworks, including experience optimizing jobs for performance and cost at scale.
  • Demonstrates hands-on exposure to distributed computing, cloud data platforms, and production ML infrastructure.
  • Holds direct, production-level experience with Databricks, including notebooks, Delta Lake, job orchestration, and cluster management.
  • Demonstrates solid command of SQL, with the ability to write, debug, and optimize complex queries against large and varied datasets.
  • Brings practical experience supporting ML model serving infrastructure in production environments and shows familiarity with MLOps tooling and practices, such as model monitoring, versioning, and deployment automation.
  • Possesses a strong foundation in data architecture and data modeling, covering both operational and analytical data store design.
  • Has working knowledge of cloud data platforms such as AWS, Azure, or GCP.
  • Maintains proficiency with version control systems such as Git.
  • Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g. for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security.
  • Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices
Preferred qualifications, capabilities, and skills
  • Experience with Airflow or Databricks Workflows, Docker and Kubernetes, and CI/CD pipelines for data and machine-learning workflows.
  • Familiarity with feature stores and model registries such as MLflow, and experience with real-time streaming architectures such as Kafka and Spark Structured Streaming.
  • Prior experience in financial services or another highly regulated industry, with an understanding of data governance and compliance requirements.

Maintain critical data pipelines and architectures across multiple technical areas as an integral part of an agile team

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

JP Morgan Chase • Mumbai

On-site
INR 1,500,000 - 3,500,000
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

Security Capital Research • Mumbai

On-site
INR 3,500,000 - 6,000,000
Lead Data Engineer - Python, Pyspark, AWS, AI
Lead Data Engineer - Python, Pyspark, AWS, AI

J.P. Morgan Chase & Co. • Hyderabad

On-site
INR 4,000,000 - 6,000,000
Senior Lead Software Engineer - Big Data
Senior Lead Software Engineer - Big Data

JP Morgan Chase • Bengaluru

On-site
INR 4,200,000 - 7,000,000
Lead Data Engineer
Lead Data Engineer

JP Morgan Chase • Pune District

On-site
INR 1,500,000 - 2,500,000
Lead Software Engineer - Data Analytics Databricks
Lead Software Engineer - Data Analytics Databricks

JP Morgan Chase • Mumbai

On-site
INR 2,000,000 - 3,600,000
Senior Lead Software Engineer- Data Engineer
Senior Lead Software Engineer- Data Engineer

JP Morgan Chase Bank • Bengaluru

On-site
INR 3,500,000 - 6,500,000
Senior Lead Data Engineer
Senior Lead Data Engineer

JP Morgan Services India Pvt Ltd • Bengaluru

On-site
INR 3,500,000 - 5,500,000
Software Engineer III - Data Analytics Databricks
Software Engineer III - Data Analytics Databricks

JP Morgan Chase • Mumbai

On-site
INR 1,500,000 - 3,000,000
Lead Data Architect- ( Data Architect with Data Modelling/ Data Governance/ Metadata Management, Erwin )
Lead Data Architect- ( Data Architect with Data Modelling/ Data Governance/ Metadata Management, Erwin )

JP Morgan Chase • Hyderabad

On-site
INR 3,600,000 - 6,000,000