Sr. AI & Data Engineer

Accenture; Irving (TX); Hybrid; USD 140,000 - 210,000; Full time

Benefits offered by this job

Medical coverage

401(k) plan

Bonus opportunities

Paid holidays

Paid time off

Job summary

Accenture in Irving, TX seeks a Sr. AI & Data Engineer for a hybrid role, with three days in the office weekly.

You will design, build, and optimize scalable data pipelines using Azure Databricks, Spark, and Delta Lake, while developing autonomous AI agents and retrieval/orchestration capabilities for AI initiatives. You will collaborate with data scientists and analysts to deliver curated, validated datasets, monitor pipeline health, and ensure low-latency performance across the Databricks

Qualifications

• Minimum 3 years as a Data & AI Engineer or related role.
• Minimum 3 years working with Microsoft Azure cloud data services.
• Minimum 2 years with Databricks (PySpark, SQL, Delta Lake, workflows).
• Bachelor's degree or equivalent (minimum 12 years work experience). If Associate Degree, minimum 6 years work experience.

Responsibilities

• Architect, develop, and maintain scalable, low-latency data pipelines using Azure Databricks workflows and Apache Spark.
• Design and optimize Lakehouse architecture with Delta Lake tables across Bronze, Silver, and Gold layers.
• Collaborate with data scientists and business analysts to deliver clean, validated datasets.
• Design and implement autonomous AI agents using Databricks Mosaic AI Agent Framework with LangChain or AutoGen.
• Build and optimize vector search indexes with Databricks Vector Search for accurate context retrieval.
• Process large-scale datasets on the Databricks Intelligence Platform with Apache Spark.
• Design data models addressing business requirements and data usability.
• Continuously improve data architecture processes for efficiency and effectiveness.
• Design AI agents with retrieval, orchestration, tool invocation, evaluation harnesses, and lifecycle observability.
• Monitor pipeline health with automated alerting and observability for fast troubleshooting.

Skills

Data & AI Engineering

Azure cloud services

Databricks

PySpark

SQL

Delta Lake

Lakehouse architecture

AI agent design

LangChain

AutoGen

Vector search

Observability

Education

Bachelor's degree or equivalent

12+ years work experience

Tools

Azure Databricks

Git-based version control

CI/CD pipelines

Azure DevOps

GitHub Actions

Auto Loader

MLOps / LLMOps

Databricks Mosaic AI Agent Framework

Job description

Accenture is hiring a Sr. AI & Data Engineer in Irving, TX for a hybrid role with three days per week in the office at specified hiring locations. This position focuses on hands-on engineering across Azure data services and Databricks, with a strong emphasis on scalable data pipelines and autonomous agent capabilities.

In this role, you will design and optimize end-to-end data and AI workflows, partnering with data scientists and business analysts to deliver curated, validated datasets for analytics and AI initiatives. You will also be responsible for building retrieval and orchestration-driven AI agents and monitoring data pipeline health to support reliable, low-latency performance.

Responsibilities

• Architect, develop, and maintain scalable, low-latency data pipelines for ingesting and transforming diverse datasets using Azure Databricks workflows and Apache Spark.
• Design and optimize Lakehouse architecture with Delta Lake tables using Bronze, Silver, and Gold layers for efficient, reliable, and governed storage.
• Collaborate with data scientists and business analysts to produce clean, validated, well-modeled datasets for advanced analytics and AI initiatives.
• Design and implement autonomous agents using Databricks Mosaic AI Agent Framework along with frameworks such as LangChain or AutoGen.
• Build and optimize vector search indexes using Databricks Vector Search to support accurate context retrieval.
• Process large-scale distributed datasets on the Databricks Intelligence Platform with Apache Spark.
• Design and implement data models that address business requirements and improve data usability.
• Continuously evaluate and improve data architecture processes to enhance efficiency and effectiveness.
• Design AI agents with retrieval, orchestration, tool invocation, evaluation harnesses, and lifecycle observability.
• Continuously monitor pipeline health and performance, implementing automated alerting and observability frameworks for rapid troubleshooting and proactive optimization.

Requirements

• Minimum of 3 years of professional experience as a Data & AI Engineer or in a related role.
• Minimum of 3 years working with Microsoft Azure cloud data services.
• Minimum of 2 years with Databricks (PySpark, SQL, Delta Lake, workflows).
• Bachelor's degree or equivalent (minimum 12 years work experience). If an Associate Degree is held, a minimum of 6 years work experience is required.

Technologies

• Databricks; Azure data services; Azure Databricks; Spark; Apache Spark
• Databricks Mosaic AI Agent Framework; LangChain; AutoGen
• Delta Lake; Bronze, Silver, and Gold layers (Lakehouse Medallion Architecture)
• Databricks Vector Search
• Databricks Intelligence Platform; PySpark; SQL
• Git-based version control; CI/CD pipelines; Azure DevOps; GitHub Actions
• Auto Loader; MLOps / LLMOps
• Genie product family
• Databricks Certified Data Engineer Professional (2025); Microsoft Certified: Azure Data Engineer Associate

Benefits

• Medical, dental, vision, life, and long-term disability coverage
• 401(k) plan
• Bonus opportunities
• Paid holidays
• Paid time off