Principal Data Engineer: Azure & Databricks Architect

BayRock Labs; Milpitas (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

BayRock Labs is seeking a Senior Data Engineer / Data Architect to build and scale enterprise data platforms in Azure and Databricks. You will design Lakehouse architectures, implement Delta Lake and data governance, and lead cross-functional teams to deliver high-quality data solutions.

Ideal candidates have 5–10+ years of hands-on experience in data engineering or architecture, with strong PySpark, Python, SQL, and EDW design skills.

Qualifications

• 5–10+ years of hands-on data engineering or data architecture experience.
• Strong expertise in Azure + Databricks ecosystems and data platforms.
• Proven ability to design scalable, secure enterprise data solutions.

Responsibilities

• Design and lead enterprise-scale data platforms and lakehouse architectures.
• Drive governance, metadata, and data lineage across pipelines.
• Mentor teams and collaborate with stakeholders on architecture decisions.
• Implement CI/CD and IaC for data workloads (Terraform, ARM).
• Migrate legacy EDWs to modern Lakehouse/EDW hybrid models.

Skills

Azure ecosystem

Databricks

PySpark

Python

SQL

Data warehousing

Data modeling

Data governance

EDW architecture

Tools

Terraform

ARM templates

Azure DevOps

GitHub Actions

Bitbucket

Job description

BayRock Labs is seeking a Senior Data Engineer / Data Architect to build and scale enterprise data platforms in Azure and Databricks. You will design Lakehouse architectures, implement Delta Lake and data governance, and lead cross-functional teams to deliver high-quality data solutions.

Ideal candidates have 5–10+ years of hands-on experience in data engineering or architecture, with strong PySpark, Python, SQL, and EDW design skills.