Remote Databricks Engineer - Scalable Data Pipelines

SFE; United States; A distancia; USD 96.000 - 165.000; Jornada completa

Descripción de la vacante

SFE is seeking a Databricks Engineer to design and optimize data pipelines on the Databricks platform, enabling scalable ingestion and processing of enterprise data. This is a 100% remote, long-term contract role.

The ideal candidate has 6+ years in data engineering, including 3+ years on Databricks, with strong Python, PySpark, SQL and Delta Lake experience, and familiarity with Unity Catalog, Auto Loader and Azure DevOps.

Formación

• 6+ years of experience in data engineering with Databricks.
• 6+ years of data engineering, including 3+ years with Databricks.
• Develop scalable Databricks pipelines for ingestion, transformation and processing of enterprise data.
• Implement Delta Lake architecture with data-quality controls and error handling.

Responsabilidades

• Develop scalable Databricks pipelines for ingestion, transformation and processing of enterprise data.
• Build reusable PySpark/SQL frameworks for high-volume and metadata-driven ingestion patterns.
• Implement Delta Lake architecture with data-quality controls and error handling.

Conocimientos

Databricks

PySpark

SQL

Delta Lake

Data Engineering

Batch Ingestion

Streaming Ingestion

Unity Catalog

Auto Loader

Python

Performance Optimization

Herramientas

Azure DevOps

Databricks Platform

Python

Descripción del empleo

SFE is seeking a Databricks Engineer to design and optimize data pipelines on the Databricks platform, enabling scalable ingestion and processing of enterprise data. This is a 100% remote, long-term contract role.

The ideal candidate has 6+ years in data engineering, including 3+ years on Databricks, with strong Python, PySpark, SQL and Delta Lake experience, and familiarity with Unity Catalog, Auto Loader and Azure DevOps.