Azure Databricks Data Engineer — Data Pipelines & Lakehouse

Fractal Analytics; California (MO); On-site; USD 120,000 - 140,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

11 paid holidays

Parental Leave

PTO policy

Job summary

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. The role focuses on Azure Databricks, PySpark, and data pipelines to support analytics and AI initiatives.

The ideal candidate has 6–10 years in data engineering, hands-on Azure Databricks experience, and a strong background in data lakes and data warehouses. Collaboration with stakeholders and CI/CD deployment are essential aspects of the position.

Qualifications

• Bachelor's or Master's degree in Computer Science, IT, Engineering, or related field.
• 6–10 years of experience in Data Engineering and Analytics.
• Minimum 3+ years of hands-on experience with Azure Databricks.
• Strong experience with PySpark, SQL, and cloud-native data platforms.
• Experience building enterprise-scale data lakes and data warehouses.

Responsibilities

• Design, build, and optimize scalable data pipelines using Azure Databricks.
• Develop ETL/ELT workflows for large-scale structured and unstructured datasets.
• Implement batch and real-time data processing using PySpark and Spark SQL.
• Build and manage data ingestion frameworks from multiple sources.
• Develop data lake solutions using ADLS Gen2.
• Integrate data from cloud and on-premises systems into enterprise platforms.
• Implement data quality, validation, and monitoring frameworks.
• Optimize Spark jobs for cost, scalability, and performance.
• Collaborate with business users to support reporting and analytics needs.
• Automate deployments with CI/CD pipelines and provide technical leadership.

Skills

Azure Databricks

PySpark

SQL

Delta Lake

Data Warehousing

ETL/ELT

Azure Data Lake

Azure Synapse

CI/CD

Terraform

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Azure DevOps

Git

Terraform

Spark

Job description

Fractal is seeking an experienced Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions. The role focuses on Azure Databricks, PySpark, and data pipelines to support analytics and AI initiatives.

The ideal candidate has 6–10 years in data engineering, hands-on Azure Databricks experience, and a strong background in data lakes and data warehouses. Collaboration with stakeholders and CI/CD deployment are essential aspects of the position.