Databricks Data Engineer - Spark, Lakehouse & Streaming

Robotic Technologies Automation; Houston (TX); Presencial; USD 120 000 - 160 000; Tempo integral

Resumo da oferta

Robotic Technologies Automation seeks a skilled Databricks/Big Data Engineer to develop and optimize Spark jobs and data pipelines on Azure Databricks. You will design lakehouse solutions, implement streaming transformations, and ensure reliable operation of live Databricks workloads.

Ideal candidates have expertise in PySpark/Scala, SQL performance tuning, and container orchestration with Docker/Kubernetes. Prior DevOps experience is a plus.

Qualificações

• Experience with Apache Spark and Databricks platforms.
• Develop and optimize Spark jobs and data pipelines.
• Design data warehouse/lakehouse solutions using Databricks.
• Ability to monitor and debug live Databricks jobs.
• Knowledge of Azure Databricks and data processing at scale.

Responsabilidades

• Develop and optimize Spark jobs and data transformations.
• Implement streaming and batch data pipelines with Databricks.
• Design and implement data warehouse/lakehouse solutions.
• Monitor, debug, and resolve issues in live Databricks jobs.
• Troubleshoot DevOps pipelines, Azure, and AKS services.
• Build ingestion, streaming, and transformation solutions.

Conhecimentos

Databricks

Apache Spark

Spark DataFrames

Spark SQL

Azure Databricks

Databricks Job Orchestration

Spark Structured Streaming

Data pipelines

Dataset and DataFrame

Cluster sizing

Serverless Databricks

DevOps for Databricks

Python/Scala

Docker

Kubernetes

Ferramentas

Docker

Kubernetes

Azure

Descrição da oferta de emprego

Robotic Technologies Automation seeks a skilled Databricks/Big Data Engineer to develop and optimize Spark jobs and data pipelines on Azure Databricks. You will design lakehouse solutions, implement streaming transformations, and ensure reliable operation of live Databricks workloads.

Ideal candidates have expertise in PySpark/Scala, SQL performance tuning, and container orchestration with Docker/Kubernetes. Prior DevOps experience is a plus.