Databricks Lakehouse Architect & Lead Data Engineer

BuzzClan; Dallas (TX); On-site; USD 150,000 - 230,000; Full time

Job summary

BuzzClan seeks a Lead/Principal Databricks Subject Matter Expert to lead enterprise data engineering, cloud architecture, and data transformation initiatives using the Databricks Lakehouse Platform. The role requires expert Apache Spark (PySpark/Scala), Delta Lake, Unity Catalog, MLflow, and MLOps practice, with strong leadership, stakeholder advisory, and production deployment governance.

This onsite role in Dallas, TX, five days per week, offers high-impact work across cloud environments and a

Qualifications

• 10+ years of professional experience in data engineering, analytics, or data platform architecture.
• 5+ years of hands-on experience with Databricks, including enterprise-scale production implementations.
• Expert-level proficiency in Apache Spark, PySpark and/or Scala, Delta Lake, and SQL.
• Proven experience designing and implementing cloud-based Lakehouse architectures using Azure, AWS, or GCP.
• Strong experience with Unity Catalog for data governance, access control, and data lineage management.
• Demonstrated experience leading technical engagements and communicating complex architectural concepts to executive stakeholders.
• Experience working in consulting, professional services, or enterprise technology environments.
• Ability to manage multiple projects or engagements simultaneously.
• Bachelor's degree in Computer Science, Engineering, Data Science, or a related technical discipline, or equivalent practical experience.
• Strong analytical, problem-solving, communication, collaboration, and technical leadership skills.
• Ability to work onsite in Dallas, TX, five days per week.

Responsibilities

• Lead end-to-end design and implementation of Databricks Lakehouse architectures, including Bronze/Silver/Gold layers.
• Define enterprise data architecture standards, reference architectures, and reusable accelerators.
• Evaluate deployment strategies across Azure, AWS, and GCP.
• Develop roadmaps and technical strategies for complex data transformations.
• Establish best practices for platform design, data integration, and governance.
• Architect, develop, and oversee large-scale ETL/ELT pipelines with Spark, PySpark, Scala, and Delta Live Tables.
• Design batch and real-time data ingestion frameworks.
• Ensure data quality, lineage, monitoring, and observability.
• Guide end-to-end ML workflows using MLflow and Databricks tools.
• Provide technical leadership and mentoring across teams.
• Engage with senior stakeholders on data platform strategy and Databricks adoption.

Skills

Leadership

Stakeholder management

Strategic thinking

Communication

Problem-solving

Education

Bachelor's degree in Computer Science, Engineering, Data Science, or related field

Tools

Databricks

Apache Spark

PySpark

Scala

Delta Lake

Unity Catalog

Cloud platforms (Azure/AWS/GCP)

MLflow

Feature Store

Databricks Workflows

Photon engine

Job description

BuzzClan seeks a Lead/Principal Databricks Subject Matter Expert to lead enterprise data engineering, cloud architecture, and data transformation initiatives using the Databricks Lakehouse Platform. The role requires expert Apache Spark (PySpark/Scala), Delta Lake, Unity Catalog, MLflow, and MLOps practice, with strong leadership, stakeholder advisory, and production deployment governance.

This onsite role in Dallas, TX, five days per week, offers high-impact work across cloud environments and a