Lead Azure Data Engineer: Scalable Pipelines & Data Lakes

Fractal; Cincinnati (OH); On-site; USD 120,000 - 165,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Life insurance

Disability plans

401(k) plan

Parental Leave (12 weeks)

11 paid holidays

Paid time off

Job summary

Fractal is hiring a Lead Data Engineer to integrate and harmonize data from multiple sources and build scalable, enterprise-grade pipelines on Azure. You will collaborate with clients, vendors, and cross-functional teams to understand data sources, define business metrics, and develop trusted datasets that support analytics, reporting, and business initiatives.

A strong candidate will have 7+ years in data engineering, deep experience with Azure Data Factory and Databricks, Python and SQL, plus

Qualifications

• 7+ years in data engineering or related fields.
• 5+ years hands-on with Azure Data Factory building enterprise data pipelines.
• 5+ years with Databricks, including cluster management, jobs, and performance tuning.
• Advanced Python proficiency and strong SQL across relational and distributed data systems.
• Strong experience with Azure Storage (Blob, Data Lake, Table) and hands‑on experience with Azure Functions and/or Logic Apps.
• Proficiency in ETL/ELT design; experience with Git and CI/CD for data engineering.
• Agile/Scrum experience with strong problem‑solving, communication, and documentation skills.

Responsibilities

• Translate business requirements into technical specifications; evaluate and select optimal tools, technologies, and architectural patterns based on scalability and cost.
• Lead and coordinate with multiple teams to understand data sources, design the data model and build in the pipelines.
• Design and develop production-optimized pipelines using Azure Data Factory, Databricks, and other Azure services.
• Build reusable, modular Python scripts and configure Azure Storage (Blob, Data Lake) with appropriate security and partitioning.
• Develop serverless, event-driven workflows using Azure Functions and Logic Apps.
• Implement error handling, logging, monitoring, and testing (unit, integration, performance) to ensure pipeline reliability.
• Optimize performance by tuning Databricks clusters, ADF activities, and Python code.
• Document solutions, enforce engineering standards and code reviews, mentor junior engineers, and troubleshoot production pipelines.

Skills

Agile/Scrum

Problem-solving

Communication

Documentation

Tools

Azure Data Factory

Databricks

Azure Storage

Azure Functions

Logic Apps

Python

SQL

Git

CI/CD

Blob Storage

Data Lake

Docker

Kubernetes

Azure Event Hubs

Kafka

Job description

Fractal is hiring a Lead Data Engineer to integrate and harmonize data from multiple sources and build scalable, enterprise-grade pipelines on Azure. You will collaborate with clients, vendors, and cross-functional teams to understand data sources, define business metrics, and develop trusted datasets that support analytics, reporting, and business initiatives.

A strong candidate will have 7+ years in data engineering, deep experience with Azure Data Factory and Databricks, Python and SQL, plus