AI Data & Knowledge Engineer - Assistant Vice President

State Street; Hyderabad; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

State Street seeks a senior AI Data & Knowledge Engineer to design and evolve enterprise-grade data foundations for AI systems, ensuring reliable, governed, scalable data platforms. You will lead architecture for ingestion, storage, and retrieval, define scalable pipelines, and own knowledge graphs and ontologies while coaching teams across the pod.

The AVP role requires deep expertise in RAG/GraphRAG, metadata-driven architectures, and governance, with strong cross-functional collaboration and

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related field.
• 8–14 years of experience in data engineering, data architecture, platform engineering, or large-scale data systems.
• Experience building scalable data pipelines with ETL/ELT frameworks.
• Strong experience with distributed data processing: Spark, Databricks, Flink, or equivalent.
• Experience designing enterprise data architectures across data lakes, warehouses, lakehouses, and AI-serving layers.
• Knowledge of orchestration, observability, performance tuning, reliability, and cost-aware design.
• Deep expertise in RAG, GraphRAG, knowledge graphs, ontology design, semantic modeling, and metadata-driven architectures.
• Experience owning data governance, data contracts, lineage, and compliance at scale.
• Cloud certifications in AWS and/or Azure are a plus.
• Proven experience leading technical teams and influencing architecture decisions.
• Strong written and verbal communication to present designs and trade-offs.

Responsibilities

• Lead architecture and design for AI-ready data platforms, including ingestion, transformation, storage, serving, and retrieval layers.
• Define standards for scalable batch and streaming data pipelines, ETL/ELT frameworks, and backend data services supporting AI applications.
• Own data and knowledge architecture across the platform, including knowledge graphs, ontologies, semantic models, and retrieval architectures.
• Establish and enforce standards for data contracts, lineage, observability, governance, and quality controls for AI systems.
• Partner with platform, architecture, and AI engineering teams to design robust integration patterns across enterprise source systems and AI services.
• Drive adoption of modern data engineering practices, including orchestration, CI/CD for data pipelines, reusable framework components, and automated quality checks.
• Lead evaluation and implementation of technologies across big data processing, vector search, graph stores, metadata tooling, and AI data infrastructure.
• Coach and develop Officers and Senior Associates; support hiring and capability building across the pod.
• Represent the Data/Knowledge function in architecture reviews, governance forums, and cross-functional planning discussions.

Skills

Data engineering

Big data architecture

ETL/ELT design

RAG/GraphRAG

Knowledge graphs

Ontology design

Semantic modeling

Data governance

Cloud (AWS/Azure)

Team leadership

Communication

Education

Bachelor's degree

Master's degree

Tools

Spark

Databricks

Flink

Kafka

Airflow

Vector database

Job description

About The Role

We are looking for a senior AI Data & Knowledge Engineer to lead the design and evolution of enterprise-grade data and knowledge foundations for AI systems. This role requires deep expertise in data engineering, big data architecture, ETL/ELT design, and scalable backend pipeline development, alongside strong experience in RAG, GraphRAG, ontologies, and knowledge graph technologies. The role will define standards and architecture patterns that ensure AI applications are powered by reliable, governed, and scalable data platforms.

AI Data & Knowledge Engineer - AVP

About The Role

We are looking for a senior AI Data & Knowledge Engineer to lead the design and evolution of enterprise-grade data and knowledge foundations for AI systems. This role requires deep expertise in data engineering, big data architecture, ETL/ELT design, and scalable backend pipeline development, alongside strong experience in RAG, GraphRAG, ontologies, and knowledge graph technologies. The role will define standards and architecture patterns that ensure AI applications are powered by reliable, governed, and scalable data platforms.

Key Responsibilities

• Lead architecture and technical design for AI-ready data platforms, including ingestion, transformation, storage, serving, and retrieval layers.
• Define standards for scalable batch and streaming data pipelines, ETL/ELT frameworks, and backend data services supporting AI applications.
• Own data and knowledge architecture across the platform, including knowledge graphs, ontologies, semantic models, and retrieval architectures.
• Establish and enforce standards for data contracts, lineage, observability, governance, and quality controls for AI systems.
• Partner with platform, architecture, and AI engineering teams to design robust integration patterns across enterprise source systems and AI services.
• Drive adoption of modern data engineering practices, including orchestration, CI/CD for data pipelines, reusable framework components, and automated quality checks.
• Lead evaluation and implementation of technologies across big data processing, vector search, graph stores, metadata tooling, and AI data infrastructure.
• Coach and develop Officers and Senior Associates; support hiring and capability building across the pod.
• Represent the Data/Knowledge function in architecture reviews, governance forums, and cross-functional planning discussions.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 8–14 years of experience in data engineering, data architecture, platform engineering, or large‑scale data systems.
• Deep expertise in building scalable data pipelines, ETL/ELT frameworks, and backend data platforms.
• Strong experience with distributed data processing technologies such as Spark, Databricks, Flink, or equivalent ecosystems.
• Experience designing enterprise data architectures across data lakes, warehouses, lakehouses, and AI-serving layers.
• Strong understanding of orchestration, observability, performance tuning, reliability engineering, and cost-aware pipeline design.
• Deep expertise in RAG, GraphRAG, knowledge graphs, ontology design, semantic modeling, and metadata-driven architectures.
• Experience owning data governance frameworks, data contracts, lineage, and compliance controls at scale.
• Cloud certifications in AWS and/or Azure are a plus.
• Proven experience leading technical teams and influencing architecture decisions across multiple domains.
• Strong written and verbal communication skills, including the ability to present technical designs and trade-offs clearly.

Work Schedule

On‑premise

Keywords

Data Engineering, Big Data Architecture, ETL, ELT, Data Pipelines, Spark, Databricks, Kafka, Airflow, Lakehouse, Data Governance, Data Contracts, Metadata, Lineage, RAG, GraphRAG, Knowledge Graphs, Ontology, Vector Database, AI Data Platform

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement

Job ID: R-795830