Senior AI & Data Engineer — Azure Databricks Architect

Accenture; Irving (TX); Hybrid; USD 140,000 - 210,000; Full time

Benefits offered by this job

Medical coverage

401(k) plan

Bonus opportunities

Paid holidays

Paid time off

Job summary

Accenture in Irving, TX seeks a Sr. AI & Data Engineer for a hybrid role, with three days in the office weekly.

You will design, build, and optimize scalable data pipelines using Azure Databricks, Spark, and Delta Lake, while developing autonomous AI agents and retrieval/orchestration capabilities for AI initiatives. You will collaborate with data scientists and analysts to deliver curated, validated datasets, monitor pipeline health, and ensure low-latency performance across the Databricks

Qualifications

• Minimum 3 years as a Data & AI Engineer or related role.
• Minimum 3 years working with Microsoft Azure cloud data services.
• Minimum 2 years with Databricks (PySpark, SQL, Delta Lake, workflows).
• Bachelor's degree or equivalent (minimum 12 years work experience). If Associate Degree, minimum 6 years work experience.

Responsibilities

• Architect, develop, and maintain scalable, low-latency data pipelines using Azure Databricks workflows and Apache Spark.
• Design and optimize Lakehouse architecture with Delta Lake tables across Bronze, Silver, and Gold layers.
• Collaborate with data scientists and business analysts to deliver clean, validated datasets.
• Design and implement autonomous AI agents using Databricks Mosaic AI Agent Framework with LangChain or AutoGen.
• Build and optimize vector search indexes with Databricks Vector Search for accurate context retrieval.
• Process large-scale datasets on the Databricks Intelligence Platform with Apache Spark.
• Design data models addressing business requirements and data usability.
• Continuously improve data architecture processes for efficiency and effectiveness.
• Design AI agents with retrieval, orchestration, tool invocation, evaluation harnesses, and lifecycle observability.
• Monitor pipeline health with automated alerting and observability for fast troubleshooting.

Skills

Data & AI Engineering

Azure cloud services

Databricks

PySpark

SQL

Delta Lake

Lakehouse architecture

AI agent design

LangChain

AutoGen

Vector search

Observability

Education

Bachelor's degree or equivalent

12+ years work experience

Tools

Azure Databricks

Git-based version control

CI/CD pipelines

Azure DevOps

GitHub Actions

Auto Loader

MLOps / LLMOps

Databricks Mosaic AI Agent Framework

Job description

Accenture in Irving, TX seeks a Sr. AI & Data Engineer for a hybrid role, with three days in the office weekly.

You will design, build, and optimize scalable data pipelines using Azure Databricks, Spark, and Delta Lake, while developing autonomous AI agents and retrieval/orchestration capabilities for AI initiatives. You will collaborate with data scientists and analysts to deliver curated, validated datasets, monitor pipeline health, and ensure low-latency performance across the Databricks