Databricks Engineer

SFE; United States; Remote; USD 96.000 - 165.000; Vollzeit

Zusammenfassung

SFE is seeking a Databricks Engineer to design and optimize data pipelines on the Databricks platform, enabling scalable ingestion and processing of enterprise data. This is a 100% remote, long-term contract role.

The ideal candidate has 6+ years in data engineering, including 3+ years on Databricks, with strong Python, PySpark, SQL and Delta Lake experience, and familiarity with Unity Catalog, Auto Loader and Azure DevOps.

Qualifikationen

• 6+ years of experience in data engineering with Databricks.
• 6+ years of data engineering, including 3+ years with Databricks.
• Develop scalable Databricks pipelines for ingestion, transformation and processing of enterprise data.
• Implement Delta Lake architecture with data-quality controls and error handling.

Aufgaben

• Develop scalable Databricks pipelines for ingestion, transformation and processing of enterprise data.
• Build reusable PySpark/SQL frameworks for high-volume and metadata-driven ingestion patterns.
• Implement Delta Lake architecture with data-quality controls and error handling.

Kenntnisse

Databricks

PySpark

SQL

Delta Lake

Data Engineering

Batch Ingestion

Streaming Ingestion

Unity Catalog

Auto Loader

Python

Performance Optimization

Tools

Azure DevOps

Databricks Platform

Python

Jobbeschreibung

Title: RM - Databricks Engineer

Location: 100% Remote

Duration: Long term contract

Description:

• Databricks, PySpark, SQL, Delta Lake, Data Engineering, Batch/Streaming ingestion Unity Catalog, Auto Loader, DLT/Lakeflow, Azure DevOps, Asset Bundles, Python, Performance Optimization
• 6+ years, including 3+ years Databricks
• Develop scalable Databricks pipelines for ingestion, transformation and processing of enterprise data.
• Build reusable PySpark/SQL frameworks capable of supporting high-volume and metadata-driven ingestion patterns.
• Implement Delta Lake architecture, data-quality controls, error handling