Data Platform Architect (Databricks & AI Pipelines)

CLEAResult; Austin (TX); On-site; USD 142,000 - 213,000; Full time

Job summary

CLEAResult in Austin, TX is seeking a data engineer to build and run a Databricks-based data platform, covering source ingestion, governed silver and gold layers, analytics data products, and AI-assisted pipelines.

You will work with Python, SQL and PySpark, Delta Lake, and Unity Catalog, implement CI/CD via GitHub Actions, and collaborate with stakeholders to deliver scalable data solutions for business KPIs.

Qualifications

• 3+ years in data engineering with production Databricks experience.
• Strong Python, SQL and PySpark; experience with Delta Lake and Delta Live Tables.
• Experience with Unity Catalog governance: grants, tagging, masking and row filters.
• Experience deploying data infrastructure as code (Databricks Asset Bundles, Terraform or similar) with Git-based CI/CD.
• Familiar with Salesforce data models and CRM ingestion patterns.
• Experience building or integrating LLM workflows (prompting, retrieval/RAG, model evaluation) is a strong plus.

Responsibilities

• Build and run Databricks Asset Bundles for ingestion, transform, product and orchestration jobs.
• Ingest across client orgs (Lakeflow Connect) with secure handling of PII-tagged data.
• Enforce data governance with Unity Catalog and ABAC masking.
• Develop document-processing pipelines and retrieval pipelines using foundation-model endpoints.
• Create reporting marts and data products to support KPIs and stakeholder reporting.

Skills

Databricks

Python

SQL

PySpark

Delta Lake

Unity Catalog

Terraform

CI/CD

Education

Bachelor’s degree in CS/Engineering

Tools

GitHub Actions

Service Principals

Azure

Job description

CLEAResult in Austin, TX is seeking a data engineer to build and run a Databricks-based data platform, covering source ingestion, governed silver and gold layers, analytics data products, and AI-assisted pipelines.

You will work with Python, SQL and PySpark, Delta Lake, and Unity Catalog, implement CI/CD via GitHub Actions, and collaborate with stakeholders to deliver scalable data solutions for business KPIs.