AI Data & Knowledge Engineer - Assistant Vice President

State Street; Hyderabad; On-site; INR 4,000,000 - 6,500,000; Full time

Job summary

State Street in Hyderabad is seeking a senior AI Data & Knowledge Engineer to lead the design and evolution of enterprise-grade data foundations for AI systems. The role requires deep expertise in data engineering, ETL/ELT, and scalable backend pipelines, with strong focus on RAG, GraphRAG, ontologies and knowledge graphs.

You will own data governance, semantics, and retrieval architectures, collaborating across multiple teams to define standards and ensure reliable AI platforms.

Qualifications

• Bachelor’s or Master’s degree in Computer Science or related field.
• 8–14 years of experience in data engineering, data architecture, platform engineering, or large-scale data systems.
• Deep expertise in building scalable data pipelines, ETL/ELT frameworks, and backend data platforms.
• Strong experience with distributed data processing technologies such as Spark, Databricks, Flink, or equivalent ecosystems.
• Experience designing enterprise data architectures across data lakes, warehouses, lakehouses, and AI-serving layers.
• Strong understanding of orchestration, observability, performance tuning, reliability engineering, and cost-aware pipeline design.
• Deep expertise in RAG, GraphRAG, knowledge graphs, ontology design, semantic modeling, and metadata-driven architectures.
• Experience owning data governance frameworks, data contracts, lineage, and compliance controls at scale.
• Cloud certifications in AWS and/or Azure are a plus.
• Proven experience leading technical teams and influencing architecture decisions across multiple domains.
• Strong written and verbal communication skills, including the ability to present technical designs and trade-offs clearly.

Responsibilities

• Lead architecture and technical design for AI-ready data platforms, including ingestion, transformation, storage, serving, and retrieval layers.
• Define standards for scalable batch and streaming data pipelines, ETL/ELT frameworks, and backend data services supporting AI applications.
• Own data and knowledge architecture across the platform, including knowledge graphs, ontologies, semantic models, and retrieval architectures.
• Establish and enforce standards for data contracts, lineage, observability, governance, and quality controls for AI systems.
• Partner with platform, architecture, and AI engineering teams to design robust integration patterns across enterprise source systems and AI services.
• Drive adoption of modern data engineering practices, including orchestration, CI/CD for data pipelines, reusable framework components, and automated quality checks.
• Lead evaluation and implementation of technologies across big data processing, vector search, graph stores, metadata tooling, and AI data infrastructure.
• Coach and develop Officers and Senior Associates; support hiring and capability building across the pod.
• Represent the Data/Knowledge function in architecture reviews, governance forums, and cross-functional planning discussions.

Skills

Data engineering

Data architecture

ETL/ELT design

Big data platforms

RAG

GraphRAG

Knowledge graphs

Ontology design

Semantic modeling

Metadata & lineage

Orchestration

CI/CD data pipelines

Communication skills

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Spark

Databricks

Flink

Kafka

Airflow

Lakehouse

Job description

About The Role

We are looking for a senior AI Data & Knowledge Engineer to lead the design and evolution of enterprise-grade data and knowledge foundations for AI systems. This role requires deep expertise in data engineering, big data architecture, ETL/ELT design, and scalable backend pipeline development, alongside strong experience in RAG, GraphRAG, ontologies, and knowledge graph technologies. The role will define standards and architecture patterns that ensure AI applications are powered by reliable, governed, and scalable data platforms.

AI Data & Knowledge Engineer - AVP

About The Role

We are looking for a senior AI Data & Knowledge Engineer to lead the design and evolution of enterprise-grade data and knowledge foundations for AI systems. This role requires deep expertise in data engineering, big data architecture, ETL/ELT design, and scalable backend pipeline development, alongside strong experience in RAG, GraphRAG, ontologies, and knowledge graph technologies. The role will define standards and architecture patterns that ensure AI applications are powered by reliable, governed, and scalable data platforms.

Key Responsibilities

• Lead architecture and technical design for AI-ready data platforms, including ingestion, transformation, storage, serving, and retrieval layers.
• Define standards for scalable batch and streaming data pipelines, ETL/ELT frameworks, and backend data services supporting AI applications.
• Own data and knowledge architecture across the platform, including knowledge graphs, ontologies, semantic models, and retrieval architectures.
• Establish and enforce standards for data contracts, lineage, observability, governance, and quality controls for AI systems.
• Partner with platform, architecture, and AI engineering teams to design robust integration patterns across enterprise source systems and AI services.
• Drive adoption of modern data engineering practices, including orchestration, CI/CD for data pipelines, reusable framework components, and automated quality checks.
• Lead evaluation and implementation of technologies across big data processing, vector search, graph stores, metadata tooling, and AI data infrastructure.
• Coach and develop Officers and Senior Associates; support hiring and capability building across the pod.
• Represent the Data/Knowledge function in architecture reviews, governance forums, and cross-functional planning discussions.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 8-14 years of experience in data engineering, data architecture, platform engineering, or large-scale data systems.
• Deep expertise in building scalable data pipelines, ETL/ELT frameworks, and backend data platforms.
• Strong experience with distributed data processing technologies such as Spark, Databricks, Flink, or equivalent ecosystems.
• Experience designing enterprise data architectures across data lakes, warehouses, lakehouses, and AI-serving layers.
• Strong understanding of orchestration, observability, performance tuning, reliability engineering, and cost-aware pipeline design.
• Deep expertise in RAG, GraphRAG, knowledge graphs, ontology design, semantic modeling, and metadata-driven architectures.
• Experience owning data governance frameworks, data contracts, lineage, and compliance controls at scale.
• Cloud certifications in AWS and/or Azure are a plus.
• Proven experience leading technical teams and influencing architecture decisions across multiple domains.
• Strong written and verbal communication skills, including the ability to present technical designs and trade-offs clearly.

Work Schedule

On-premise

Keywords

Data Engineering, Big Data Architecture, ETL, ELT, Data Pipelines, Spark, Databricks, Kafka, Airflow, Lakehouse, Data Governance, Data Contracts, Metadata, Lineage, RAG, GraphRAG, Knowledge Graphs, Ontology, Vector Database, AI Data Platform

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you'll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.