Senior Data Architect

Talentola; Chicago (IL); On-site; USD 160,000 - 210,000; Full time

Job summary

Talentola seeks an Azure Databricks & Agentic AI Architect to design and implement AI-powered data platforms blending Lakehouse architecture with autonomous AI workflows.

You will lead data engineering teams, define HITL governance, and drive enterprise GenAI adoption, ROI, and transformation across AI-DLC and platform architecture.

Qualifications

• Design and implement AI-powered Data Engineering platforms using Azure Databricks and Lakehouse.
• Define autonomous workflows using AI Agents for data quality remediation and root cause analysis.
• Lead architecture for AI-enabled Data Development Lifecycle across requirements analysis and automated testing.
• Design agent orchestration frameworks using Semantic Kernel and MCP-enabled architectures.
• Establish governance models for autonomous AI agents.

Responsibilities

• Design and implement AI-powered Data Engineering platforms with Azure Databricks and Lakehouse.
• Define autonomous workflows with AI Agents for data quality remediation and root cause analysis.
• Lead architecture for AI-enabled Data Development Lifecycle across requirements to automated testing.
• Build scalable Lakehouse platforms using Unity Catalog and RAG-based architectures.
• Establish guardrails for enterprise GenAI adoption and governance.
• Mentor engineers and lead executive conversations on AI ROI and transformation.

Skills

Azure Databricks

Lakehouse Architecture

Data Engineering

Generative AI

Agentic AI

AI Governance

Autonomous AI

Executive AI ROI

Education

Databricks Certified Data Engineer Professional

Azure Data Engineer (DP-203)

Generative AI / Agentic AI Certifications

Tools

Unity Catalog

Semantic Kernel

MCP-enabled architectures

GitHub Actions

Job description

We are seeking a visionary Azure Databricks & Agentic AI Architect to design and implement next-generation AI-powered data platforms. This role combines deep expertise in Azure Databricks, Lakehouse Architecture, Data Engineering, and Generative AI to build intelligent, autonomous, and self-optimizing data ecosystems.

The ideal candidate will lead the adoption of Agentic AI within Data Engineering and AI-DLC, enabling autonomous data ingestion, transformation, quality management, lineage discovery, observability, optimization, testing, and governance.

Key Responsibilities

Agentic Data Engineering Leadership

• Design and implement AI-powered Data Engineering platforms leveraging Azure Databricks and Lakehouse architecture.
• Define autonomous workflows using AI Agents for:
• Data quality remediation
• Root cause analysis
• Establish frameworks for Human-in-the-Loop (HITL) decision-making and governance.

AI-Driven Data Lifecycle (AI-DLC)

• Lead architecture for AI-enabled Data Development Lifecycle across:
• Requirement analysis
• Automated testing
• Implement AI copilots to accelerate developer productivity.
• Enable automated lineage creation and intelligent impact analysis.

Lakehouse & Data Platform Architecture

• Design scalable Lakehouse platforms using:
• Unity Catalog
• Architect RAG-based solutions using enterprise data assets.
• Design agent orchestration frameworks using:
• Semantic Kernel
• MCP-enabled architectures
• Build domain-specific AI agents supporting Data Engineering and Analytics teams.

AI Governance & Responsible AI

• Define guardrails for enterprise GenAI adoption.
• Prompt governance
• Observability
• Auditability
• Explainability
• Security controls
• Establish governance models for autonomous AI agents.

AI-Powered Platform Optimization

• Incident triage
• Capacity planning
• Enable intelligent workload placement and model routing.

Technical Skills

• Unity Catalog

AI & Agentic Frameworks

• RAG Architecture
• DevOps & AI-DLC
• GitHub Actions
• Drive AI-First Data Engineering transformation.
• Define enterprise patterns, accelerators, and reusable AI agents.
• Mentor architects, data engineers, and AI engineers.
• Lead executive conversations on AI adoption, ROI, and transformation roadmaps.

Preferred Certifications

• Databricks Certified Data Engineer Professional
• Azure Data Engineer (DP-203)
• Generative AI / Agentic AI Certifications