Databricks AI & Governance

Compunnel, Inc.; Plano (TX); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Compunnel, Inc. is seeking a senior Data Governance/AI Governance engineer to design, develop, and deploy Agentic AI solutions for data governance, quality, lineage, and policy enforcement.

The role collaborates with governance engineers, data engineers, architects, and client teams across enterprise platforms. The position requires expertise in Python, Generative AI, LLMs, and cloud platforms (Azure, AWS, GCP), with experience integrating governance tools like Collibra and Purview.

Qualifications

• 8+ years in Data Management/Enterprise Data Architecture.
• 5+ years in enterprise data governance and related solutions.
• 3+ years in AI/ML/Generative AI using LLMs and orchestration.
• 3+ years Python and modern AI development frameworks.
• 3+ years cloud platforms (Azure/AWS/GC).
• 2+ years integrating governance platforms (Collibra, Databricks Unity Catalog).
• 2+ years API design, workflow orchestration, and integrations.
• Experience defining governance policies, lineage, data quality, and compliance monitoring.
• Experience in Agile with backlog-driven development and stakeholder feedback.
• Strong client-facing communication for workshops and executive demos.

Responsibilities

• Design, develop, and deploy Agentic AI solutions for governance and stewardship workflows.
• Build scalable AI agents using LLMs, RAG, and orchestration frameworks.
• Identify governance use cases with stakeholders and define agent capabilities.
• Develop autonomous and human-in-the-loop agents for data cataloging and policy validation.
• Integrate AI agents with data catalogs, metadata repositories, and governance tools.
• Create demos and pilots to validate governance automation opportunities.
• Collaborate to refine backlogs and onboard governance controls incrementally.
• Implement monitoring, guardrails, and governance controls for compliance and transparency.
• Support proposal development, solution architecture discussions, and client presentations.
• Contribute to reusable frameworks and accelerators for enterprise governance.

Skills

Client-facing communication

Agile delivery

Requirements gathering

Tools

Python

LangChain

LangGraph

Semantic Kernel

AutoGen

OpenAI

Azure AI Foundry

Amazon Bedrock

Databricks Unity Catalog

Collibra

Informatica

Purview

AWS

GCP

Job description

The Databricks AI & Governance role will design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows across enterprise data platforms. The role requires strong expertise in data management, governance, AI/ML, Generative AI, Agentic AI, Python, modern AI frameworks, cloud platforms, and governance technologies. The engineer will collaborate with Data Governance Engineers, Data Engineers, Architects, business stakeholders, and client teams to identify governance use cases, develop autonomous and human-in-the-loop agents, integrate enterprise data and governance platforms, and deliver secure, compliant, transparent, and responsible AI solutions. This is a hybrid role with work locations in Dallas, Chicago, Atlanta, Charlotte, Minnesota, Cupertino, or San Francisco.

Key Responsibilities

• Design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows.
• Build scalable AI agents using Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), orchestration frameworks, and enterprise governance tools.
• Partner with Data Governance Engineers, Data Engineers, Architects, and business stakeholders to identify governance use cases and define agent capabilities aligned with enterprise data strategy.
• Develop and maintain autonomous and human-in-the-loop agents supporting data cataloging, business glossary management, policy compliance validation, data ownership assignment, issue remediation, and governance workflow orchestration.
• Integrate AI agents with data catalogs, metadata repositories, data quality tools, MDM platforms, cloud data platforms, and governance technologies including Ataccama, Collibra, Informatica, Microsoft Purview, Databricks, Snowflake, AWS, and Azure.
• Create proof-of-value demonstrations and pilot implementations to validate governance automation opportunities and accelerate stakeholder adoption.
• Collaborate with internal delivery teams and client stakeholders to refine governance backlogs, prioritize agent development initiatives, and incrementally onboard governance controls.
• Implement monitoring, observability, guardrails, and governance controls for Agentic AI solutions to support security, compliance, transparency, and responsible AI practices.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to Data and AI governance modernization.
• Contribute to reusable frameworks, accelerators, and best practices for enterprise-scale Agentic AI implementations focused on data governance and management.

Required Qualifications

• 8+ years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture.
• 5+ years of experience designing and implementing enterprise data governance, Master Data Management (MDM), metadata management, data quality, or regulatory compliance solutions.
• 3+ years of experience developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks.
• 3+ years of hands-on experience with Python and modern AI development frameworks such as LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI, Azure AI Foundry, Amazon Bedrock, or equivalent platforms.
• 3+ years of experience working with cloud platforms including Azure, AWS, or Google Cloud Platform.
• 2+ years of experience integrating governance platforms such as Collibra, Databricks Unity Catalog, or related data management technologies.
• 2+ years of experience designing APIs, workflow orchestration, event-driven architectures, or enterprise system integrations.
• Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities.
• Experience working in Agile delivery environments with backlog-driven development, iterative releases, and stakeholder feedback cycles.
• Strong client-facing communication skills with the ability to lead workshops, requirements-gathering sessions, architecture reviews, and executive demonstrations.