Senior Databricks Data Engineer — Lakehouse Architect (Remote)

Compass

United States

Hybrid

USD 140,000 - 200,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Meal allowance
Health insurance
Pet health insurance
Home office allowance
Welcome kit
Life insurance
Wellness support (Zenklub)
TotalPass
Continuous Learning program with top•s

Job summary

Compass is seeking a senior data engineer to design and evolve brand-new data architectures and scalable pipelines. You will optimize complex data workflows, implement batch and streaming processing, and work across Spark, Airflow, Kafka, and dbt in a cloud-centric environment.

Responsibilities include building Lakehouse models, leveraging Unity Catalog for governance, and delivering robust data ingestion and integrations using ADF and Lakeflow.

Qualifications

  • Experience defining and evolving data architectures using modern tools.
  • Proven ability to build scalable, high-performance data pipelines.
  • Strong knowledge of Databricks, Spark, and Delta Lake.
  • Hands-on with Azure Data Factory and data orchestration.
  • Familiarity with CI/CD, Git, and deployment automation.

Responsibilities

  • Define and evolve data architectures using Databricks, ensuring pipeline optimization and processing efficiency.
  • Build and maintain scalable data pipelines for high-quality data ingestion, transformation, and delivery.
  • Work with Spark, Airflow, Kafka, dbt, SQL/NoSQL databases, and cloud platforms (Databricks, AWS, BigQuery).
  • Design and implement Lakehouse models and Medallion Architecture (Bronze/Silver/Gold).
  • Implement batch and streaming data processing, including CDC and Structured Streaming.
  • Develop data ingestion/integration using ADF, Auto Loader, and Lakeflow.
  • Implement Unity Catalog governance, security, and access control.

Skills

Databricks
PySpark
Spark SQL
Delta Lake
Airflow
dbt
SQL/NoSQL
Python
REST APIs
Git & CI/CD
Unity Catalog
Azure Data Factory
Lakehouse
Delta table optimization

Tools

Databricks
Airflow
Kafka
dbt
PostgreSQL
MongoDB
Cassandra
AWS
BigQuery
Auto Loader
Lakeflow
Unity Catalog
Git

Job description

Compass is seeking a senior data engineer to design and evolve brand-new data architectures and scalable pipelines. You will optimize complex data workflows, implement batch and streaming processing, and work across Spark, Airflow, Kafka, and dbt in a cloud-centric environment.

Responsibilities include building Lakehouse models, leveraging Unity Catalog for governance, and delivering robust data ingestion and integrations using ADF and Lakeflow.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Lakehouse Data Engineer | Databricks
Remote Lakehouse Data Engineer | Databricks

BairesDev • United States

Remote
USD 120,000 - 180,000
100% remote work
Competitive USD salary
Home office setup provided
+3
Senior Databricks Data Architect & Lakehouse Lead
Senior Databricks Data Architect & Lakehouse Lead

Celebal Technologies • United States

On-site
USD 180,000 - 240,000
Senior Databricks Data Engineer - Lakehouse & Streaming
Senior Databricks Data Engineer - Lakehouse & Streaming

Valiant Technologies LLC • Columbus (OH)

On-site
USD 120,000 - 190,000
Senior Databricks Data Engineer: Lakehouse on Azure (Hybrid)
Senior Databricks Data Engineer: Lakehouse on Azure (Hybrid)

Sapient Corporation • Houston (TX)

Hybrid
USD 104,000 - 120,000
Inclusive workplace and collaboration
Learning and development opportunities
Competitive compensation and benefits
+4
Senior Databricks Data Engineer – Lakehouse Real-Time
Senior Databricks Data Engineer – Lakehouse Real-Time

Capgemini • New York (NY)

On-site
USD 89,000 - 166,000
Paid time off
Health benefits
401(k) plan
+1
Remote Senior Data Engineer — Lakehouse & DataOps Specialist
Remote Senior Data Engineer — Lakehouse & DataOps Specialist

FCamara Consulting & Training • United States

Remote
USD 150,000 - 210,000
Fully remote
Senior Databricks Architect - AWS Lakehouse & Streaming
Senior Databricks Architect - AWS Lakehouse & Streaming

Quantum Technologies. LLC • Columbus (OH)

On-site
USD 120,000 - 180,000
Senior Databricks Architect - AWS Lakehouse Leader
Senior Databricks Architect - AWS Lakehouse Leader

Valiant Technologies LLC • Atlanta (GA)

On-site
USD 170,000 - 210,000
Remote Databricks Data Engineer — AI & Lakehouse Architect
Remote Databricks Data Engineer — AI & Lakehouse Architect

Livefront, Inc. • Peru (IL)

Remote
USD 90,000 - 140,000
Senior Databricks Data Engineer - Remote, Lakehouse/Delta
Senior Databricks Data Engineer - Remote, Lakehouse/Delta

Entrada • United States

Remote
USD 120,000 - 180,000
MacBook provided
Referral bonus
Databricks cert coverage
+3