Principal Databricks & Data Platform Architect

CLEAResult Consulting; United States; On-site; USD 142,000 - 213,000; Full time

Job summary

CLEAResult builds and runs its Databricks data platform, covering source ingestion, governed silver and gold layers, analytics data products, and AI-assisted pipelines. The role includes managing bundle standards, CI/CD, and secure access controls to protect sensitive data.

You will work across ingestion, transformation, and orchestration tasks, and help shape reporting marts and KPIs for clients. A strong background in data engineering and governance is essential.

Qualifications

• 3+ years in data engineering with production Databricks experience.
• Strong Python, SQL and PySpark skills; Delta Lake and Delta Live Tables/Lakeflow experience.
• Experience with Unity Catalog governance: grants, tagging, masking, and row filters.
• Infrastructure as code for data platforms (Databricks Asset Bundles, Terraform or similar) with Git‑based CI/CD.
• Familiarity with Salesforce data models and CRM ingestion patterns.
• Experience building/integrating LLM workflows (prompting, retrieval/RAG, model evaluation).
• Working knowledge of Azure (identity, service principals, storage).
• Bachelor's degree in computer science, engineering or related field, or equivalent experience.
• Clear communicator who documents well with technical and business stakeholders.

Responsibilities

• Build and maintain Databricks Asset Bundles (ingestion, transform, product and orchestration jobs).
• Run Salesforce data ingestion across client orgs, handling re-auth and field-level issues without breaking PII-tagged tables.
• Enforce Unity Catalog governance: grants, tagging, masking, and access controls for sensitive data.
• Develop document-processing and retrieval pipelines using foundation-model endpoints for classification and inference.
• Create reporting marts and data products to support KPIs and stakeholder reporting.
• Evaluate LLM serving endpoints and perform model bake-offs or migrations.
• Debug failing pipelines, document defects, and maintain runbooks for the team.

Skills

Databricks

Python

SQL

PySpark

Delta Lake

Unity Catalog

Terraform

GitHub Actions

Salesforce data

LLM workflows

Education

Bachelor's degree in CS/Engineering

Tools

Terraform

GitHub Actions

Azure

Job description

CLEAResult builds and runs its Databricks data platform, covering source ingestion, governed silver and gold layers, analytics data products, and AI-assisted pipelines. The role includes managing bundle standards, CI/CD, and secure access controls to protect sensitive data.

You will work across ingestion, transformation, and orchestration tasks, and help shape reporting marts and KPIs for clients. A strong background in data engineering and governance is essential.