Azure Databricks Data Engineer - Build Data Pipelines

BlackCube Labs; California (MO); On-site; USD 120,000 - 140,000; Full time

Benefits offered by this job

Health insurance

Dental & Vision

Life Insurance

Disability

401(k)

Paid holidays

Parental Leave

Paid time off

Job summary

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. You will collaborate with data scientists, architects, and business teams to deliver analytics-ready platforms.

The role requires strong Azure Data Engineering skills, Databricks, data pipelines, and data warehousing expertise to support AI initiatives in a fast-paced environment.

Qualifications

• Bachelor's or Master's degree in CS/IT/Engineering or related field.
• 6–10 years in Data Engineering/Analytics with Azure cloud experience.
• 3+ years hands-on with Azure Databricks and PySpark.
• Experience building data lakes and data warehouses at scale.
• Strong Spark performance tuning and Agile delivery experience.

Responsibilities

• Design, build, and optimize scalable data pipelines using Azure Databricks.
• Develop ETL/ELT workflows for large datasets and BI readiness.
• Implement batch and real-time processing with PySpark and Spark SQL.
• Create data lake solutions using ADLS Gen2 and integrate cloud/on‑prem data.
• Establish data quality, validation, and monitoring frameworks.
• Optimize Spark jobs for cost, scale, and performance.
• Model dimensional data and support data warehousing initiatives.
• Collaborate with stakeholders to define analytics needs and reports.
• Automate deployment via CI/CD and contribute to architecture discussions.
• Ensure security, governance, and compliance in data platforms.

Skills

PySpark

SQL

Azure Databricks

Spark SQL

Delta Lake

Data Modeling

Data Governance

CI/CD

Terraform

Python

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Azure Data Factory

ADLS Gen2

Azure Synapse

Unity Catalog

Databricks Workflows

Snowflake

Power BI

Microsoft Fabric

Job description

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. You will collaborate with data scientists, architects, and business teams to deliver analytics-ready platforms.

The role requires strong Azure Data Engineering skills, Databricks, data pipelines, and data warehousing expertise to support AI initiatives in a fast-paced environment.