Databricks AI Engineer - AI Agents for Data Automation

Ven Soft LLC; Austin (TX); On-site; USD 96,000 - 207,000; Part time

Job summary

Ven Soft LLC seeks a senior data/AI engineer to architect and deploy AI agents that automate Databricks workflows. You will build LL-powered tools for code generation, testing, and data quality validation, integrating with Azure services and enterprise systems.

The role combines data engineering, AI agent development, and automation with emphasis on governance, observability, and security. Austin onsite with long-term contract terms.

Qualifications

• Bachelor’s degree in CS/Engineering, data science, or related field.
• 10+ years in data or software engineering or cloud data platforms.
• Hands-on with Azure Databricks and modern cloud data platforms.
• Proven AI/LLM-powered automation and intelligent agents experience.
• Strong Python and SQL, production-grade apps.

Responsibilities

• Design and deploy AI agents to automate data engineering workflows in Databricks.
• Build LL-powered agents for code generation, review, and refactoring of Python/SQL workloads.
• Develop AI-driven data quality validation, testing, and QA automation.
• Create governance, metadata management, and lineage automation in Databricks.
• Develop CI/CD automation, test generation, deployment validation, and rollback checks.
• Implement AI-powered monitoring for data pipelines and root-cause analysis.
• Document architectures, prompts, and orchestration patterns; ensure logging and observability.

Skills

AI agent design

Azure Databricks

Python

PySpark

Generative AI

LLMs

Automation

AI governance

CI/CD

Monitoring & observability

Education

Bachelor's degree in CS/ENG

Master's degree (preferred)

Tools

Databricks

Delta Lake

Databricks Asset Bundles

LangChain

Azure OpenAI

GitHub Actions

Azure DevOps

Job description

Ven Soft LLC seeks a senior data/AI engineer to architect and deploy AI agents that automate Databricks workflows. You will build LL-powered tools for code generation, testing, and data quality validation, integrating with Azure services and enterprise systems.

The role combines data engineering, AI agent development, and automation with emphasis on governance, observability, and security. Austin onsite with long-term contract terms.