Senior Lead Software Engineer - Big Data

JP Morgan Chase

Bengaluru

On-site

INR 4,200,000 - 7,000,000

Full time

48 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

JPMorganChase in Bengaluru seeks a Senior Lead Software Engineer to set technical direction for data platforms, build reusable frameworks, and guide a team of engineers. You will champion scalable, secure data solutions across Databricks and AWS, shaping data ingestion, modeling, and analytics capabilities.

You will mentor engineers, drive architectural decisions, and ensure reliable production systems with robust CI/CD and automation.

Qualifications

  • 5+ years of software/data engineering experience and leadership.
  • Experience designing data warehouses/lakes on Databricks with modern data models.
  • Strong SQL and Python/PySpark skills for large datasets.

Responsibilities

  • Take ownership of data platforms including architecture and reliability.
  • Mentor engineers and drive timely delivery with quality.
  • Architect and build Databricks data warehouse/lake solutions with robust data modeling.
  • Create reusable ETL frameworks for ingestion, transformation, quality, and auditability.
  • Define data ingestion loads from various sources and manage batch/streaming pipelines with Spark, Kafka/Kinesis.
  • Tune performance and cost across Spark/Databricks and AWS services (S3, Glue, EMR, Lambda, Redshift, Athena).
  • Govern data marts behind reporting; ensure accuracy and timeliness of business/regulatory reports.
  • Adopt AI-assisted engineering practices to improve code quality and delivery speed.

Skills

Data engineering leadership
SQL
Python
PySpark
Databricks
AWS
CI/CD
AI-assisted engineering
Data modeling
ETL frameworks
Spark
NoSQL
Kafka/Kinesis

Tools

Spark
Kubernetes
Snowflake
Redshift
Tableau/PowerBI

Job description

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.

As a Senior Lead Software Engineer at JPMorganChase within the Commercial and Investment Bank and C360 Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. You will set the technical direction, build reusable frameworks and guide a team of engineers. You also have a solid understanding of the broader software and application landscape that our data solutions integrate with.

Job responsibilities
  • Take full responsibility for the data platforms, including architecture, roadmap, reliability, cost, operational health and production support.
  • Mentor and guide engineers. Set technical direction, clear blockers quickly, delegate effectively and answer for the team's deliverables and code quality.
  • Architect and build data warehouse and data lake solutions on Databricks. Apply sound data modeling, history tracking and data organization practices so the data is well structured and ready for analytics.
  • Create reusable, configurable ETL and processing frameworks covering ingestion, transformation, data quality, reconciliation and auditability, and drive their adoption across teams.
  • Define and govern standard approaches for bringing data in from databases, files, APIs and event sources, covering both full and incremental loads and ship high-volume batch and streaming pipelines with Spark/PySpark and Kafka/Kinesis. Make sure they're reliable, resilient to failure and meet agreed service levels.
  • Tune performance and cost across Spark, Databricks, SQL engines and NoSQL stores. Design secure, resilient solutions on AWS using services such as S3, Glue, EMR, Lambda, Redshift and Athena.
  • Govern the models, aggregates and data marts behind reporting so that business and regulatory reports are accurate, reconciled, timely and fast.
  • Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
  • Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
  • Enforce modular design, code reviews, CI/CD, automated testing, monitoring and Git-based workflows. Work with application, API and UI teams, using your knowledge of the full software stack to shape integration decisions.
  • Promote reuse-first practices that use enterprise-authorized AI capabilities for design analysis, test generation, control validation and documentation. Validate all outputs, respect data sensitivity, and keep work traceable and auditable.

    Required qualifications, capabilities, and skills
  • Formal training or certification on software engineering concepts and 5+ years applied experience
  • Experience in data engineering with a track record of leading teams, mentoring engineers, removing obstacles and being accountable for delivery, platform stability and incident resolution.
  • Expertise in designing and developing data warehouses and data lakes on Databricks, with strong data modeling skills. Familiarity with Snowflake and comparable platforms such as Redshift, BigQuery or Azure Synapse.
  • Exceptional SQL skills, including complex queries, optimization and analytical problem solving on large datasets with solid grounding in reporting: business-friendly data layers, BI tools (e.g., Tableau, Power BI, QuickSight), dashboard performance, and accuracy controls.
  • Outstanding production-grade Python and PySpark, with clean, efficient, testable and maintainable code.
  • Mastery of data ingestion and processing designs, with a history of building enterprise ETL frameworks and running batch and streaming workloads.
  • Broad knowledge of relational databases (e.g., PostgreSQL, Oracle), NoSQL stores (e.g., DynamoDB, MongoDB, Cassandra) and AWS data services. You can tune performance across these technologies and choose the right one for each use case.
  • Demonstrated experience leading effective use of enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security
  • Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching senior engineers/leads on compliant usage patterns and controls.
  • A well-rounded understanding of software engineering and application development, including the SDLC, design principles, microservices, APIs (e.g., Spring Boot) and front-end concepts. You also troubleshoot well and know CI/CD and GitHub/Bitbucket workflows.
  • Experience using enterprise-authorized AI capabilities in engineering work. You review and validate AI-assisted outputs before using them, escape when unsure, and follow data handling requirements.

    Preferred qualifications, capabilities, and skills
  • Infrastructure provisioning in AWS using Infrastructure as Code (e.g., Terraform, AWS CloudFormation).
  • Databricks and/or AWS certifications.
  • Hands‑on experience with Snowflake, or with migrating data between platforms.
  • Experience with orchestration tools (e.g., Airflow) and with data governance and lineage tools.
  • Experience with containerization (Docker/Kubernetes) and in financial services or other regulated environments.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

JP Morgan Chase • Mumbai

On-site
INR 1,500,000 - 3,500,000
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling
Lead Data Engineer - Data Engineering, Databricks, Model Serving, ML Ops, Data Architecture and modeling

Aumni • Mumbai

On-site
INR 3,500,000 - 6,000,000
Senior Lead Software Engineer- Data Engineer
Senior Lead Software Engineer- Data Engineer

JP Morgan Chase Bank • Bengaluru

On-site
INR 3,500,000 - 6,500,000
Senior Lead Data Engineer
Senior Lead Data Engineer

JP Morgan Services India Pvt Ltd • Bengaluru

On-site
INR 3,500,000 - 5,500,000
Software Engineer III - Data Analytics Databricks
Software Engineer III - Data Analytics Databricks

JP Morgan Chase • Mumbai

On-site
INR 1,500,000 - 3,000,000
Lead Software Engineer - Data Analytics Databricks
Lead Software Engineer - Data Analytics Databricks

JP Morgan Chase • Mumbai

On-site
INR 2,000,000 - 3,600,000
Lead Data Engineer - Python, Pyspark, AWS, AI
Lead Data Engineer - Python, Pyspark, AWS, AI

J.P. Morgan Chase & Co. • Hyderabad

On-site
INR 4,000,000 - 6,000,000
Sr Lead Software Engineer - Data Engineering & Data Platform
Sr Lead Software Engineer - Data Engineering & Data Platform

JP Morgan Chase • Bengaluru

On-site
INR 3,500,000 - 6,500,000
Director of Software Engineering - Data Engineering & Transformation
Director of Software Engineering - Data Engineering & Transformation

J.P. Morgan Chase & Co. • Bengaluru

On-site
INR 3,500,000 - 6,000,000
Lead Software Engineer - AI & Data Strategy
Lead Software Engineer - AI & Data Strategy

JP Morgan Services India Pvt Ltd • Bengaluru

On-site
INR 3,500,000 - 5,500,000