Azure Databricks Data Engineer: Build Scalable Pipelines

BlackCube Labs; California (MO); On-site; USD 120,000 - 140,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

Paid holidays

Parental Leave

Job summary

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. The candidate will collaborate with architects, data scientists, and business teams to deliver analytics-ready data platforms.

The ideal candidate has 6–10 years in data engineering, strong PySpark, SQL, and cloud-native data warehousing skills, plus experience with Azure data services and governance frameworks.

Qualifications

• Bachelor's or Master's degree in CS/IT/Engineering or related field.
• 6–10 years of experience in Data Engineering and Analytics.
• Minimum 3+ years hands-on experience with Azure Databricks.
• Strong experience with PySpark, SQL, and cloud-native data platforms.

Responsibilities

• Design, build, and optimize scalable data pipelines using Azure Databricks.
• Develop ETL/ELT workflows for large-scale datasets (structured and unstructured).
• Implement batch and real-time data processing with PySpark and Spark SQL.
• Build and maintain data ingestion frameworks from multiple sources.
• Develop and manage data lake solutions using ADLS Gen2.
• Integrate data from cloud and on-premise systems into enterprise platforms.
• Implement data quality, validation, and monitoring.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Data Modeling

Data Governance

Data Quality

Python

SQL

Scala

Education

Bachelor's or Master's in Computer Science/IT/Engineering

Tools

Azure DevOps

Terraform

Azure Synapse

ADLS Gen2

Job description

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. The candidate will collaborate with architects, data scientists, and business teams to deliver analytics-ready data platforms.

The ideal candidate has 6–10 years in data engineering, strong PySpark, SQL, and cloud-native data warehousing skills, plus experience with Azure data services and governance frameworks.