Databricks Lakehouse Architect for Enterprise Data

CSCI Consulting, Inc.; Indianapolis, Northern (IN, KY); Hybrid; USD 140,000 - 195,000; Full time

Benefits offered by this job

Generous PTO

Health benefits

Team building events

Job summary

CSCI Consulting, Inc. is seeking a Databricks Subject Matter Expert (SME) to lead data modernization for a DoW client, providing architecture, engineering, governance, and analytics to enable mission-critical data, reporting, AI/ML, and decision-support capabilities.

You will design scalable Databricks Lakehouse architectures, implement Delta Lake and Unity Catalog, develop ETL/ELT pipelines, and guide DevSecOps and IaC practices across Azure and multi-cloud environments while mentoring

Qualifications

• 7+ years of enterprise data engineering or data platform experience.
• 5+ years hands-on Databricks and Apache Spark experience.
• Experience with lakehouse architectures, Delta Lake, and Unity Catalog.
• Proficient in SQL, Python, PySpark, and distributed processing.
• Experience with Azure, AWS, or other cloud data platforms.

Responsibilities

• Serve as the primary Databricks SME for enterprise data and analytics initiatives.
• Design, develop, and optimize scalable data platforms using Lakehouse architecture.
• Lead data engineering, integration, and transformation solutions with Databricks and Spark.
• Develop and maintain ETL/ELT pipelines across structured and unstructured data.
• Configure Delta Lake, Unity Catalog, Workflows, and Delta Live Tables.
• Support governance, metadata, data quality, and MDM initiatives.
• Mentor engineers and participate in Agile ceremonies.

Skills

Databricks

Apache Spark

SQL

Python

Cloud platforms

Data governance

CI/CD

Tools

Delta Lake

Unity Catalog

Databricks Workflows

Delta Live Tables

Job description

CSCI Consulting, Inc. is seeking a Databricks Subject Matter Expert (SME) to lead data modernization for a DoW client, providing architecture, engineering, governance, and analytics to enable mission-critical data, reporting, AI/ML, and decision-support capabilities.

You will design scalable Databricks Lakehouse architectures, implement Delta Lake and Unity Catalog, develop ETL/ELT pipelines, and guide DevSecOps and IaC practices across Azure and multi-cloud environments while mentoring