Databricks Data Engineer: Lakehouse & Streaming Specialist

CloudBigD

Houston (TX)

On-site

USD 120,000 - 180,000

Full time

3 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

CloudBigD is seeking a skilled Databricks engineer in Houston to design and optimize large-scale Spark jobs, data pipelines, and streaming workflows in Azure Databricks. The role emphasizes data lakehouse architectures, Unity Catalog, and serverless Databricks features.

Candidates should be proficient with PySpark/Scala, SQL performance tuning, and containerized deployments using Docker and Kubernetes. Prior experience with AKS and DevOps pipelines is a plus.

Qualifications

  • Proficiency in Apache Spark, Spark DataFrames, Spark SQL and Spark Structured Streaming.
  • Experience with Azure Databricks environment and Databricks tooling.
  • Knowledge of data lakehouse architectures and related Databricks components.

Responsibilities

  • Develop and optimize Spark jobs, data transformations, and processing workflows.
  • Implement complex data streaming and transformation solutions using Azure Databricks.
  • Design and implement data warehouse/lakehouse solutions.
  • Monitor, debug, and resolve issues in live Databricks jobs.
  • Troubleshoot DevOps pipelines, Azure, and AKS services.
  • Build data ingestion, streaming, and transformation solutions.

Skills

Apache Spark
Spark DataFrames
Spark SQL
ETL/ELT
Azure Databricks
Databricks
Unity Catalog
Auto Loader
Lakehouse
Medallion Architecture
Databricks Serverless
Dataset
Spark Structured Streaming
Docker
Kubernetes
Scala
PySpark

Tools

Docker
Kubernetes

Job description

CloudBigD is seeking a skilled Databricks engineer in Houston to design and optimize large-scale Spark jobs, data pipelines, and streaming workflows in Azure Databricks. The role emphasizes data lakehouse architectures, Unity Catalog, and serverless Databricks features.

Candidates should be proficient with PySpark/Scala, SQL performance tuning, and containerized deployments using Docker and Kubernetes. Prior experience with AKS and DevOps pipelines is a plus.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks Data Engineer: Lakehouse & Spark Expert
Databricks Data Engineer: Lakehouse & Spark Expert

CloudBigD • Louisville (KY)

On-site
USD 120,000 - 180,000
Databricks Data Engineer - Spark, Lakehouse & Streaming
Databricks Data Engineer - Spark, Lakehouse & Streaming

Robotic Process Automation LLC • Houston (TX)

On-site
USD 120,000 - 160,000
Data Engineer: Spark, Databricks & Lakehouse Architect
Data Engineer: Spark, Databricks & Lakehouse Architect

Quantum Technologies LLC • Houston (TX)

On-site
USD 110,000 - 170,000
Azure Databricks Data Engineer — Lakehouse & Spark Expert
Azure Databricks Data Engineer — Lakehouse & Spark Expert

Quantum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 180,000
Senior Azure Databricks Engineer - Remote, Lakehouse Expert
Senior Azure Databricks Engineer - Remote, Lakehouse Expert

Dataflix • United States

Remote
USD 130,000 - 170,000
Senior Databricks Data Engineer: Lakehouse on Azure (Hybrid)
Senior Databricks Data Engineer: Lakehouse on Azure (Hybrid)

Sapient Corporation • Houston (TX)

Hybrid
USD 104,000 - 120,000
Inclusive workplace and collaboration
Learning and development opportunities
Competitive compensation and benefits
+4
Databricks Data Engineer: Spark, Lakehouse & Streaming
Databricks Data Engineer: Spark, Lakehouse & Streaming

Valiant Technologies LLC • Quincy (MA)

On-site
USD 77,000 - 110,000
Azure Databricks Data Engineer - Lakehouse & Pipelines
Azure Databricks Data Engineer - Lakehouse & Pipelines

Ethereum Technologies LLC • Dallas (TX)

On-site
USD 120,000 - 160,000
Azure Data Engineer: Build Scalable Pipelines & Lakehouse
Azure Data Engineer: Build Scalable Pipelines & Lakehouse

CloudBigD • Raleigh (NC)

On-site
USD 110,000 - 160,000
Databricks Engineer - Lakehouse Data Pipelines & Spark
Databricks Engineer - Lakehouse Data Pipelines & Spark

Quantum Technologies LLC • Austin (TX)

On-site
USD 120,000 - 190,000