Databricks AI & Data Engineer

NTT DATA, Inc.; Hyderabad; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

NTT DATA, Inc. is seeking a senior Data & AI governance architect to design and deploy Agentic AI solutions that automate data governance, metadata management, and lineage analysis across enterprise platforms.

You will build scalable AI agents using LLMs and orchestration frameworks, integrate with data catalogs and governance tools, and work with stakeholders to accelerate adoption of governance controls with an emphasis on security and compliance.

Qualifications

• 8+ years in Data Management or related architecture.
• 5+ years in enterprise data governance or MDM.
• 3+ years AI/ML or Agentic AI with LLMs and orchestration.
• 3+ years Python with LangChain/related frameworks.
• 3+ years cloud platform experience (Azure/AWS/GCP).
• 2+ years governance platform integration.
• 2+ years designing APIs and enterprise integrations.
• Experience with Agile delivery and backlog-driven development.
• Strong client-facing communication and workshop leadership.

Responsibilities

• Design, develop, and deploy Agentic AI solutions for data governance and stewardship workflows.
• Build scalable AI agents leveraging LLMs and orchestration frameworks.
• Collaborate with governance engineers, data engineers, and stakeholders to define capabilities.
• Develop autonomous and human-in-the-loop agents for cataloging, policy validation, and data ownership.
• Integrate AI agents with data catalogs, MDM, and governance tools across platforms.
• Create pilot implementations to validate governance automation opportunities.
• Refine governance backlogs and onboard governance controls incrementally.
• Implement monitoring, guardrails, and responsible AI governance practices.
• Support proposal development and architecture discussions related to Data & AI governance.

Skills

Data governance

Data engineering

Metadata management

Enterprise data architecture

AI/ML / Generative AI

Python

Cloud platforms

APIs & integration

Agile delivery

Client-facing communication

Tools

LangChain

LangGraph

Semantic Kernel

AutoGen

CrewAI

OpenAI

Azure AI Foundry

Amazon Bedrock

Databricks Unity Catalog

Collibra

Ataccama

Microsoft Purview

Job description

Day to Day Job Duties: (what this person will do on a daily/weekly basis)

Design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows across enterprise data platforms.

Build scalable AI agents leveraging Large Language Models (LLMs), Retrieval Augmented Generation (RAG), orchestration frameworks, and enterprise governance tools to support data management and regulatory compliance initiatives.

Partner with Data Governance Engineers, Data Engineers, Architects, and business stakeholders to rapidly discover governance use cases, define agent capabilities, and align solution architecture with enterprise data strategy.

Develop and maintain autonomous and human‑in‑the‑loop agents that assist with data cataloging, business glossary management, policy compliance validation, data ownership assignment, issue remediation, and governance workflow orchestration.

Integrate AI agents with enterprise platforms including data catalogs, metadata repositories, data quality tools, MDM platforms, cloud data platforms, and governance technologies such as Ataccama, Collibra, Informatica, Microsoft Purview, Databricks, Snowflake, AWS, and Azure.

Create proof‑of‑value demonstrations and pilot implementations to validate governance automation opportunities and accelerate stakeholder adoption.

Collaborate with internal delivery teams and client stakeholders to continuously refine governance backlogs, prioritize agent development initiatives, and incrementally onboard governance controls across the enterprise.

Implement monitoring, observability, guardrails, and governance controls for Agentic AI solutions to ensure security, compliance, transparency, and responsible AI practices.

Support proposal development, solution architecture discussions, effort estimation, and client presentations related to Data & AI governance modernization initiatives.

Contribute to reusable frameworks, accelerators, and best practices for enterprise‑scale Agentic AI implementations focused on data governance and management.

Basic Qualifications: (what are the skills required to this job with minimum years of experience on each)

Minimum 8+ years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture.

Minimum 5+ years of experience designing and implementing enterprise data governance, master data management (MDM), metadata management, data quality, or regulatory compliance solutions.

Minimum 3+ years of experience developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks.

Minimum 3+ years of hands‑on experience with Python and modern AI development frameworks such as LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI, Azure AI Foundry, Amazon Bedrock, or equivalent platforms.

Minimum 3+ years of experience working with cloud platforms including Azure, AWS, or Google Cloud Platform.

Minimum 2+ years of experience integrating governance platforms such as Collibra, Databricks Unity Catalog, or related data management technologies.

Minimum 2+ years of experience designing APIs, workflow orchestration, event‑driven architectures, or enterprise system integrations.

Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities.

Experience working in Agile delivery environments with backlog‑driven development, iterative releases, and stakeholder feedback cycles.

Strong client‑facing communication skills with the ability to lead workshops, requirements gathering sessions, architecture reviews, and executive demonstrations.