Delivery Lead – Data Engineering & Databricks - Assistant Vice President

STATE STREET CORPORATION; Hyderabad; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

State Street Corporation in Hyderabad seeks a Databricks Lead to own the GHR Data Platform, defining platform strategy, architecture, and delivery of workforce data, analytics, reporting, AI, and integrations.

You will lead Databricks and AWS cloud architecture, establish governance, security, and platform modernization, and mentor global teams across data, platform, and solution architecture domains.

Qualifications

• 15-20 years of experience in data engineering, analytics platforms, and cloud data solutions.
• Deep expertise with Databricks and modern data platforms.
• Strong experience with Spark, Delta Lake, SQL, Python, and cloud data engineering patterns.
• Experience working with HR, workforce, or enterprise business data preferred.
• Strong leadership, stakeholder management, and product ownership skills.
• Experience leading global teams and delivering large-scale data initiatives.

Responsibilities

• Define and own the AWS Databricks platform architecture roadmap across environments.
• Design scalable, secure Databricks deployments and governance frameworks.
• Architect ingestion and processing for batch, streaming, and real-time data.
• Lead security, compliance, and disaster recovery considerations.
• Provide technical leadership across platform, architecture, and engineering teams.

Skills

Databricks

Spark

Delta Lake

SQL

Python

Cloud architecture

Leadership

Tools

Terraform

AWS CloudFormation

CloudWatch

Secrets Manager

Job description

Role Overview

The Databricks Lead serves as the product owner and technology lead for the GHR Data Platform, responsible for defining platform strategy, architecture, and delivery of workforce data, analytics, reporting, AI, and integration capabilities.

Key Responsibilities

• Databricks Architecture / Platform Management- Define and own the AWS Databricks platform architecture roadmap. Design scalable, secure, and multi-environment Databricks deployments across Development, QA, UAT, and Production. Establish enterprise standards for platform design, governance, monitoring, and operational excellence. Create architecture patterns and reference implementations for engineering teams. Lead platform modernization and cloud transformation initiatives.
• Databricks Platform Management that includes Design and govern of Databricks Workspaces, Unity Catalog, Delta Lake, Databricks SQL, Workflows, Delta Live Tables Develop standards for workspace provisioning, cluster policies, compute optimization, and cost management. Establish multi-tenant architectures and platform governance frameworks. Improve platform performance, reliability, and security posture. Architect ingestion and processing frameworks supporting Batch workloads, Streaming workloads and Real-time integrations that are in-line with Data Lakehouse patterns Establish metadata, data quality, lineage, and observability frameworks.
• AWS Cloud Architecture- Design and manage AWS-native infrastructure supporting Databricks workloads. Architect solutions utilizing AWS S3, EC2, KMS, Secrets Manager, CloudWatch, Lambda Define cloud networking, security zoning, and connectivity models. Ensure high availability, scalability, fault tolerance, and disaster recovery.
• Security & Compliance- Design security controls aligned with enterprise security standards by implementing Unit Catalog Governance, Secrets Management, Role-Based Access Control ect. Support audit, regulatory, and compliance requirements. Partner closely with Cybersecurity and Risk teams.
• DevOps & Automation- Architect Infrastructure-as-Code solutions using Terraform & AWS CloudFormation Implement CI/CD pipelines for Databricks deployments. Automate platform provisioning and configuration management. Define release management and deployment governance processes.
• Leadership & Stakeholder Management- Provide technical leadership across platform, architecture, and engineering teams. Mentor Data Engineers, Platform Engineers, and Solution Architects. Lead architecture reviews and technology evaluation exercises. Collaborate with business, infrastructure, security, and vendor teams.

Qualifications

• 15-20 years of experience in data engineering, analytics platforms, and cloud-based data solutions.
• Deep expertise with Databricks and modern data platforms.
• Strong experience with Spark, Delta Lake, SQL, Python, and cloud data engineering patterns.
• Experience working with HR, workforce, or enterprise business data preferred.
• Strong leadership, stakeholder management, and product ownership skills.
• Experience leading global teams and delivering large-scale data initiatives.

Are you the right candidate? Yes! We truly believe in the power that comes from the diverse backgrounds and experiences our employees bring with them.

Although each vacancy details what we are looking for, we don’t necessarily need you to fulfil all of them when applying. If you like change and innovation, seek to see the bigger picture, make data driven decisions and are a good team player, you could be a great fit.

Why this role is important to us

Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients. We’re driving the company’s digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation. We offer a collaborative environment where technology skills and innovation are valued in a global organization. We’re looking for top technical talent to join our team and deliver creative technology solutions that help us become an end-to-end, next-generation financial services company. Join us if you want to grow your technical skills, solve real problems and make your mark on our industry.

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success. We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential.

As an essential partner in our shared success, you’ll benefit from

• inclusive development opportunities
• flexible work-life support
• paid volunteer days
• vibrant employee networks that keep you connected to what matters most

Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers Read our CEO Statement

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success. We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential.

As an essential partner in our shared success, you’ll benefit from

• inclusive development opportunities
• flexible work-life support
• paid volunteer days
• vibrant employee networks that keep you connected to what matters most

Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers Read our CEO Statement