Databricks AI & Data Engineer

NTT DATA North America; Hyderabad; On-site; INR 1,200,000 - 1,900,000; Full time

Job summary

NTT DATA North America is hiring for a senior Data & AI Governance Architect to design and deploy Agentic AI solutions that automate governance, quality, and stewardship across enterprise data platforms.

You will build scalable agents using LLMs, RAG, and orchestration frameworks, and collaborate with governance engineers, data engineers, and business stakeholders to align the architecture with enterprise strategy.

Qualifications

• 8+ years in Data Management, Data Engineering, Governance or Enterprise Data Architecture.
• 5+ years implementing data governance, MDM, metadata management, data quality or regulatory solutions.
• 3+ years developing AI/ML or Agentic AI with LLMs and orchestration tools.
• 3+ years Python with modern AI frameworks (LangChain, OpenAI, Azure AI Foundry, etc.).
• 3+ years cloud experience (Azure, AWS, GCP).
• 2+ years integrating governance platforms (Collibra, Databricks Unity Catalog, etc.).
• 2+ years API design, workflow orchestration, event-driven architectures.
• Experience defining governance policies, lineage, quality controls, compliance monitoring.
• Agile delivery experience; stakeholder workshops and executive demos.

Responsibilities

• Design, develop, and deploy Agentic AI solutions for data governance and stewardship workflows.
• Build scalable AI agents using LLMs, RAG, and orchestration frameworks.
• Collaborate with engineers and business stakeholders to identify governance use cases.
• Develop autonomous/human-in-the-loop agents for cataloging, policy validation, and ownership.
• Integrate AI agents with data catalogs, MDM, and governance tech like Purview, Collibra, Databricks.
• Create pilot demos to validate governance automation opportunities.
• Refine governance backlogs and onboard controls across the enterprise.
• Implement monitoring, guardrails, and responsible AI practices.
• Support solution architecture discussions, proposals, and client presentations.
• Contribute to reusable frameworks and accelerators for data governance implementations.

Skills

Data governance

Data engineering

Enterprise data architecture

Metadata management

AI/ML development

Python

Cloud platforms

APIs & integrations

Agile delivery

Stakeholder communication

Tools

LangChain

LangGraph

Semantic Kernel

AutoGen

CrewAI

OpenAI

Azure AI Foundry

Amazon Bedrock

Job description

Day to Day Job Duties: (what this person will do on a daily/weekly basis)

• Design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows across enterprise data platforms.
• Build scalable AI agents leveraging Large Language Models (LLMs), Retrieval Augmented Generation (RAG), orchestration frameworks, and enterprise governance tools to support data management and regulatory compliance initiatives.
• Partner with Data Governance Engineers, Data Engineers, Architects, and business stakeholders to rapidly discover governance use cases, define agent capabilities, and align solution architecture with enterprise data strategy.
• Develop and maintain autonomous and human-in-the-loop agents that assist with data cataloging, business glossary management, policy compliance validation, data ownership assignment, issue remediation, and governance workflow orchestration.
• Integrate AI agents with enterprise platforms including data catalogs, metadata repositories, data quality tools, MDM platforms, cloud data platforms, and governance technologies such as Ataccama, Collibra, Informatica, Microsoft Purview, Databricks, Snowflake, AWS, and Azure.
• Create proof-of-value demonstrations and pilot implementations to validate governance automation opportunities and accelerate stakeholder adoption.
• Collaborate with internal delivery teams and client stakeholders to continuously refine governance backlogs, prioritize agent development initiatives, and incrementally onboard governance controls across the enterprise.
• Implement monitoring, observability, guardrails, and governance controls for Agentic AI solutions to ensure security, compliance, transparency, and responsible AI practices.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to Data & AI governance modernization initiatives.
• Contribute to reusable frameworks, accelerators, and best practices for enterprise-scale Agentic AI implementations focused on data governance and management.

Basic Qualifications: (what are the skills required to this job with minimum years of experience on each)

• Minimum 8+ years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture.
• Minimum 5+ years of experience designing and implementing enterprise data governance, master data management (MDM), metadata management, data quality, or regulatory compliance solutions.
• Minimum 3+ years of experience developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks.
• Minimum 3+ years of hands‑on experience with Python and modern AI development frameworks such as LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI, Azure AI Foundry, Amazon Bedrock, or equivalent platforms.
• Minimum 3+ years of experience working with cloud platforms including Azure, AWS, or Google Cloud Platform.
• Minimum 2+ years of experience integrating governance platforms such as Collibra, Databricks Unity Catalog, or related data management technologies.
• Minimum 2+ years of experience designing APIs, workflow orchestration, event‑driven architectures, or enterprise system integrations.
• Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities.
• Experience working in Agile delivery environments with backlog-driven development, iterative releases, and stakeholder feedback cycles.
• Strong client‑facing communication skills with the ability to lead workshops, requirements gathering sessions, architecture reviews, and executive demonstrations.