Data Platform Engineer, Vice President II - State Street Investment Management

State Street; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

State Street is seeking an experienced hands‑on architect/engineer in Bengaluru to build a multi‑cloud data platform that delivers trusted data products to the business functions. This role is 80% individual contributor—hands‑on engineering and POC development—and 20% people management for the legacy platform.

The successful candidate will have deep expertise in large‑scale data platform architecture, distributed computing, data pipelines, and modern cloud architecture with strong operational

Qualifications

• Master’s degree in computer science or a related technical discipline.
• 15+ years of technology experience, preferably in financial services.
• Expert‑level hands‑on programming in Python and SQL, with Java/Scala as a plus.
• Deep hands‑on Apache Spark for batch and streaming workloads.
• Proven experience building and operating enterprise Lakehouse/data platforms using Snowflake and Databricks.
• Strong cloud engineering across AWS, GCP, and Azure.
• Open data architectures and metadata ecosystems experience.
• Experience productionizing data capabilities through DevOps/CI‑CD.

Responsibilities

• Validate target data platform architectures through hands‑on engineering and POCs.
• Translate reference architectures into working prototypes and production‑ready patterns.
• Optimize scalable batch and streaming data pipelines using Python/SQL/Spark.
• Engineer Iceberg‑based data models with partitioning, schema evolution, and metadata management.
• Establish data engineering standards for quality, lineage, observability, and contracts.
• Improve platform performance via partitioning, caching, and cost‑aware practices.
• Lead from POC to production, define non‑functional requirements and SLA readiness.
• Engineer AI‑ready data products with context, embeddings, and RAG patterns.
• Build data services for AI workloads with secure APIs and caching.
• Mentor L2/L3 teams for legacy platform SLAs and issue closure.

Skills

Python

SQL

Apache Spark

Distributed systems

DevOps

Education

MSc in computer science

Tools

Snowflake

Databricks

AWS

GCP

Azure

CI/CD

Job description

Who We Are Looking For

State Street Investment Management’s Data, Analytics & AI Services (DAAIS) team is seeking an experienced hands‑on architect, engineer for building the multi-cloud open architecture‑based data platform that delivers trusted data products to the business functions. This role is an 80% IC role in doing hands‑on engineering and developing POCs to support a resilient, scalable, performance optimized, secure multi‑cloud data platform, and 20% of the capacity is responsible for managing and mentoring the employees, based out of Bangalore, responsible for the production maintenance & support of the legacy data platform. The successful candidate will bring deep hands‑on expertise in large‑scale data platform architecture, distributed computing, data pipeline engineering, and modern cloud architecture and operational experience.

Who We Are Looking For

State Street Investment Management’s Data, Analytics & AI Services (DAAIS) team is seeking an experienced hands‑on architect, engineer for building the multi-cloud open architecture‑based data platform that delivers trusted data products to the business functions. This role is an 80% IC role in doing hands‑on engineering and developing POCs to support a resilient, scalable, performance optimized, secure multi‑cloud data platform, and 20% of the capacity is responsible for managing and mentoring the employees, based out of Bangalore, responsible for the production maintenance & support of the legacy data platform. The successful candidate will bring deep hands‑on expertise in large‑scale data platform architecture, distributed computing, data pipeline engineering, and modern cloud architecture and operational experience.

Why this role is important to us

The team you will be joining is a part of State Street Investment Management, one of the largest asset managers in the world. We partner with many of the world’s largest, most sophisticated investors and financial intermediaries to help them reach their goals through a rigorous, research‑driven investment process. With over four decades of experience and trillions of dollars in assets under management, we offer one of the broadest selections of services across asset classes, risk profiles, regions and styles. As pioneers in index, ETF, and ESG investing, we are always inventing new ways to invest. Join us if making your mark in the asset management industry from day one is a challenge you are up for.

What You Will Be Responsible For

• Validate target data platform architectures and design principles through hands‑on engineering, technical spikes, and POCs for a unified, cloud‑native Lakehouse.
• Partner with architects to translate reference architectures and data strategies into working prototypes, measurable validation criteria, and production‑ready implementation patterns.
• Optimize scalable batch and streaming data pipelines using Python/Scala, SQL, Spark, and modern orchestration frameworks.
• Engineer high‑performance Iceberg‑based data models and tables, including partitioning, compaction, schema evolution, metadata management, and multi‑engine interoperability.
• Establish data engineering standards for data quality, lineage, observability, reliability, versioning, and backward‑compatible data product contracts.
• Optimize platform, pipeline, and query performance through workload testing, partitioning and clustering, caching, concurrency management, and cost‑aware lifecycle practices.
• Lead capabilities from POC to production by defining non‑functional requirements, resolving engineering risks, integrating with platform services, and enabling operational support and SLA readiness.
• Engineer AI‑ready data products with trusted, well‑governed, semantically enriched data, incorporating context engineering, ontologies, embeddings, vector indexing, and RAG patterns.
• Build and productionize data services for Agentic and AI workloads, including MCP‑enabled access, interoperable APIs, LLM gateways, multi‑model routing, caching, observability, security, and cost optimization.
• Mentor and provide technical leadership to the L2/L3 support teams for legacy platform to ensure SLA adherence, escalation discipline, and timely closure of production issues impacting business, clients, or regulators.

Required Qualifications

• Master’s degree in computer science or a related technical discipline, with 15+ years of technology experience, preferably in financial services.
• Expert‑level hands‑on programming in Python and SQL, with Java or Scala as a plus, and proven ability to design, debug, test, and optimize distributed data‑processing solutions.
• Deep hands‑on expertise with Apache Spark for batch and streaming workloads, including performance tuning, troubleshooting, reliability engineering, and cost optimization.
• Proven experience building and operating enterprise Lakehouse and data platforms using Snowflake and Databricks, supporting high‑concurrency analytical, near‑real‑time, and AI‑ready workloads.
• Strong cloud engineering experience across AWS, GCP, and Azure, including cloud‑native data services, security, networking, infrastructure automation, portability, and cost management.
• Working expertise with open data architectures and metadata ecosystems, including Apache Iceberg, schema evolution, table optimization, catalog federation, zero‑copy sharing, data virtualization, and multi‑engine access.
• Extensive experience productionizing data capabilities through DevOps and CI/CD, automated testing, observability, data quality, incident management, and operational support for mission‑critical platforms.

Preferred Qualifications

• Experience with both structured and unstructured data management tools and associated patterns
• Familiarity with modern governance frameworks (Unity Catalog, Collibra, Alation).
• Expertise in AI‑assisted coding using GitHub Copilot or Claude Code Assistant.

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success. We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work‑life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future. As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law. Discover more information on jobs at StateStreet.com/careers Read our CEO Statement Job ID: R-799248