AI Engineer: Azure Databricks, LLMs & Automation

IT America Inc; Austin (TX); Hybrid; USD 150,000 - 190,000; Full time

Job summary

IT America Inc. seeks a Databricks AI Engineer in Austin, TX, with hybrid onsite (3 days/week) for a long-term contract. You will design and deploy AI agents to automate data engineering workflows in Azure Databricks, leveraging PySpark, Python, SQL, and LLM-based automation.

Responsibilities include building AI-powered automation for data quality, CI/CD pipelines, and governance, plus collaboration with data teams to scale AI-driven data platforms.

Qualifications

• Bachelor's degree or higher in a technical field.
• Experience with Azure Databricks and cloud data platforms.
• Proven AI/LLM-powered automation experience.

Responsibilities

• Design, develop, and deploy intelligent AI agents to automate data engineering within Azure Databricks.
• Build LLM-powered agents for code generation, review, refactoring, and optimization of PySpark/SQL workloads.
• Develop AI-driven automation for data quality validation, testing, reconciliation, and QA processes.
• Create agents for metadata management, data lineage, governance, and compliance automation.
• Implement CI/CD for AI workflows, including tests, deployment validation, and rollback.
• Monitor and diagnose data pipelines with AI-powered observability and root-cause analysis.
• Develop reusable prompt engineering frameworks and agent patterns for data engineering use cases.
• Integrate LLMs/agents with Databricks, Azure services, APIs, and enterprise workflows.

Skills

Azure Databricks

Python

PySpark

LLMs

Generative AI

AI Agents

Prompt Engineering

CI/CD

Data Automation

Data Engineering

Education

Bachelor's degree in CS/SE/Data Science

Master's degree (plus)

Tools

Databricks notebooks

Delta Lake

Databricks Asset Bundles

GitHub Actions

Azure DevOps Pipelines

LangChain

OpenAI API

Azure OpenAI

Job description

IT America Inc. seeks a Databricks AI Engineer in Austin, TX, with hybrid onsite (3 days/week) for a long-term contract. You will design and deploy AI agents to automate data engineering workflows in Azure Databricks, leveraging PySpark, Python, SQL, and LLM-based automation.

Responsibilities include building AI-powered automation for data quality, CI/CD pipelines, and governance, plus collaboration with data teams to scale AI-driven data platforms.