Senior Lead Software Engineer - Java & Databricks

Aumni; Bengaluru; On-site; INR 2,800,000 - 5,600,000; Full time

Job summary

JPMorganChase in Bengaluru seeks a Senior Lead Software Engineer to advance scalable, secure fintech platforms. You will architect and deliver microservices and data pipelines, leveraging Databricks, Spark, and Delta Lake, while guiding AI-assisted practices and governance across teams.

You will collaborate with product and compliance teams to improve alert accuracy and overall platform reliability. Strong leadership and cloud-native experience are required for this senior role.

Qualifications

• 5+ years of hands-on software engineering experience.
• Experience delivering system design, application development, testing, and operations.
• Expertise in building scalable, cloud-native, enterprise-grade products with a focus on compliance.

Responsibilities

• Design and build scalable microservices and APIs for rule-based and ML-based detection pipelines.
• Develop streaming and batch data pipelines for content and alerts.
• Model data in Databricks for surveillance, search, retention, and lifecycle management.
• Develop and optimize large-scale data pipelines with Spark and Delta Lake.
• Implement tests and CI/CD workflows; optimize Databricks jobs, clusters, and pipelines.
• Promote AI-assisted engineering practices to improve code quality and delivery speed.
• Collaborate with product and compliance SMEs to improve alert accuracy and reliability.

Skills

Java/Kotlin

Python

Databricks

Apache Spark

CI/CD

Test-Driven Development

Observability

AI/ML integration

Education

Formal training in software engineering

Tools

Databricks

Apache Spark

Delta Lake

Databricks Workflows

AWS (EC2, S3)

Kafka

PostgreSQL

Prometheus

Grafana

OpenTelemetry

ArgoCD

Terraform

Jenkins

GitHub Actions

Job description

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products.

As a Senior Lead Software Engineer at JPMorganChase within the Digital Communications Compliance team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Drive significant business impact through your capabilities and contributions, and apply deep technical expertise and problem-solving methodologies to tackle a diverse array of challenges that span multiple technologies and applications.

Job responsibilities

• Design and develop scalable, fault-tolerant microservices and APIs that support rule-based and ML-based detection pipelines
  Model and implement supervision and reviewer workflows using state machines
• Build streaming and batch data pipelines that ingest, index, and enrich communications content and alerts
• Design data models on Databricks for surveillance, search, retention, and life cycle management at scale
• Develop and optimize large-scale batch and streaming data pipelines using Databricks, Apache Spark, and Delta Lake
• Build robust unit, integration, and performance tests following Test-Driven Development. Leverage Databricks Workflows, notebooks, and CI/CD pipelines to automate data processing and deployments. Troubleshoot and tune Databricks jobs, clusters, and data pipelines for efficiency and performance
  Drive adoption of Databricks best practices for data engineering, software engineering, observability, and platform operations
• Drives adoption and governance of approved AI-assisted engineering practices across teams to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test acceleration, release readiness, incident/root-cause analysis), while establishing measurable validation standards (secure coding, peer review, automated testing) and promoting reuse of proven patterns and automation within the SDLC/TLM toolchain.
• Applies knowledge of tools within the Software Development Life Cycle toolchain, including approved AI-assisted development and automation capabilities, to improve the value realized by automation at scale.
• Partner with product management and compliance SMEs to monitor and improve alert accuracy and reliability. Proactively identify hidden problems and patterns in communications data to improve detection quality

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on practical experience delivering system design, application development, testing, and operational stability
• Expertise in building resilient, scalable, enterprise-grade cloud-native products, with strong exposure in compliance for the financial industry
• Hands-on Databricks experience including development of production workloads using Spark, Delta Lake, and Databricks Workflows. Expert Java/Kotlin and Python programmer with experience building headless, externally consumable APIs. Experience building cloud-native microservices for streaming and batch architectures using Spark and/or Flink
• Proficient with AWS services including EC2, ECS, EKS, EMR, S3, and Glacier. Hands-on with Semantic search, Kafka, and PostgreSQL. Experience integrating and operationalizing ML/LLM models and pipelines in production. Experience with observability and monitoring tools such as Prometheus, Grafana, and OpenTelemetry
• Experience building CI/CD pipelines using ArgoCD, Helm, Terraform, Jenkins, and GitHub Actions
• Prior experience in Test-Driven Development, delivering products with well-defined SLI/SLO/SLAs
• Strong ownership mentality with excellent communication skills and a collaborative mindset. Experience mentoring engineers and providing technical leadership at a senior level

Preferred qualifications, capabilities, and skills

• Exposure to Unity Catalog, and Databricks Workflows
• Exposure to building and optimizing large-scale batch and streaming data pipelines in cloud-native environments
• Understanding of distributed systems, data engineering best practices, and operationalizing data and ML workloads in production

Drive significant business impact and tackle a diverse array of challenges that span multiple technologies and applications