Databricks Data Lead — Lakehouse Architect (Hybrid Bay Area)

Incedo Inc.; San Rafael (CA); Hybrid; USD 130,000 - 150,000; Full time

Benefits offered by this job

Medical insurance

Vision insurance

401(k)

Job summary

A leading technology consulting firm is seeking a Databricks Data Lead in San Rafael, CA. The ideal candidate will have 8-12 years of experience in data engineering and strong hands-on expertise with the Databricks Lakehouse Platform, Apache Spark, and PySpark. This role requires designing and optimizing cloud-native data platforms and involves direct onsite collaboration with client stakeholders. The position offers opportunities for growth from data engineering into architecture responsibilities.

Qualifications

• 8-12 years in data engineering or analytics engineering.
• Strong hands-on experience with the Databricks Lakehouse Platform.
• Ability to work onsite 3 days/week in San Rafael, CA.

Responsibilities

• Design, develop, and maintain data pipelines using Databricks.
• Implement medallion lakehouse architectures and data quality.
• Collaborate onsite to gather technical requirements.

Skills

Databricks Lakehouse Platform

Apache Spark internals

Advanced Python (PySpark)

SQL development

Data warehousing concepts

Cloud data platforms (AWS, Azure, GCP)

Delta Lake

Git-based version control

Verbal and written communication

Education

Bachelor’s degree in Computer Science, Engineering, or related field

Tools

Databricks

Apache Airflow

Azure Data Factory

Job description

A leading technology consulting firm is seeking a Databricks Data Lead in San Rafael, CA. The ideal candidate will have 8-12 years of experience in data engineering and strong hands-on expertise with the Databricks Lakehouse Platform, Apache Spark, and PySpark. This role requires designing and optimizing cloud-native data platforms and involves direct onsite collaboration with client stakeholders. The position offers opportunities for growth from data engineering into architecture responsibilities.