Databricks AI & Data Engineer

NTT DATA, Inc.; Indianapolis (IN); On-site; USD 140,000 - 190,000; Full time

Job summary

NTT DATA, Inc. is seeking an experienced data governance and AI solutions architect to design, develop, and deploy Agentic AI workflows across enterprise data platforms.

You will build scalable AI agents leveraging LLMs, RAG, and orchestration frameworks to automate governance tasks and ensure regulatory compliance. Collaborating with governance engineers, data engineers, architects, and business stakeholders, you will prioritize use cases, refine backlogs, and deliver pilot implementations.

Qualifications

• 8+ years in data management or governance

Responsibilities

• Design, develop, and deploy Agentic AI solutions for data governance and stewardship workflows.

Skills

Data governance

Data management

Enterprise data architecture

Stakeholder communication

Agile delivery

AI/ML development

Python programming

Tools

LangChain

LangGraph

Semantic Kernel

AutoGen

CrewAI

OpenAI

Azure AI Foundry

Amazon Bedrock

Azure/AWS/GCP

Job description

Day to Day Job Duties:

• Design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows across enterprise data platforms.
• Build scalable AI agents leveraging Large Language Models (LLMs), Retrieval Augmented Generation (RAG), orchestration frameworks, and enterprise governance tools to support data management and regulatory compliance initiatives.
• Partner with Data Governance Engineers, Data Engineers, Architects, and business stakeholders to rapidly discover governance use cases, define agent capabilities, and align solution architecture with enterprise data strategy.
• Develop and maintain autonomous and human-in-the-loop agents that assist with data cataloging, business glossary management, policy compliance validation, data ownership assignment, issue remediation, and governance workflow orchestration.
• Integrate AI agents with enterprise platforms including data catalogs, metadata repositories, data quality tools, MDM platforms, cloud data platforms, and governance technologies such as Ataccama, Collibra, Informatica, Microsoft Purview, Databricks, Snowflake, AWS, and Azure.
• Create proof-of-value demonstrations and pilot implementations to validate governance automation opportunities and accelerate stakeholder adoption.
• Collaborate with internal delivery teams and client stakeholders to continuously refine governance backlogs, prioritize agent development initiatives, and incrementally onboard governance controls across the enterprise.
• Implement monitoring, observability, guardrails, and governance controls for Agentic AI solutions to ensure security, compliance, transparency, and responsible AI practices.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to Data & AI governance modernization initiatives.
• Contribute to reusable frameworks, accelerators, and best practices for enterprise-scale Agentic AI implementations focused on data governance and management.

Basic Qualifications:

• Minimum 8+ years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture.
• Minimum 5+ years of experience designing and implementing enterprise data governance, master data management (MDM), metadata management, data quality, or regulatory compliance solutions.
• Minimum 3+ years of experience developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks.
• Minimum 3+ years of hands-on experience with Python and modern AI development frameworks such as LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI, Azure AI Foundry, Amazon Bedrock, or equivalent platforms.
• Minimum 3+ years of experience working with cloud platforms including Azure, AWS, or Google Cloud Platform.
• Minimum 2+ years of experience integrating governance platforms such as Collibra, Databricks Unity Catalog, or related data management technologies.
• Minimum 2+ years of experience designing APIs, workflow orchestration, event-driven architectures, or enterprise system integrations.
• Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities.
• Experience working in Agile delivery environments with backlog-driven development, iterative releases, and stakeholder feedback cycles.
• Strong client-facing communication skills with the ability to lead workshops, requirements gathering sessions, architecture reviews, and executive demonstrations.