Azure Data Solutions Engineer — ELT Pipelines & Delta Lake

Southern Company; Durham, Northern (NC, KY); Hybrid; USD 90,000 - 130,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k) Matching

PTO

Job summary

PowerSecure, a Southern Company subsidiary, designs, builds, and operates reliable data pipelines on Azure to enable trusted analytics and self‑service BI. This role partners with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high‑quality data using Azure Data Factory, Databricks, and Delta Lake.

Responsibilities include batch and streaming ELT, data quality enforcement, and operational support, with collaboration on governance, security, and cost

Qualifications

• 2+ years of hands‑on data engineering or equivalent experience.
• 2+ years building pipelines on Azure and Databricks or equivalent.
• Proficiency with SQL and Python (PySpark) including performance tuning.
• Bachelor’s degree in MIS, Computer Science, Engineering, or equivalent experience.
• Experience with Azure Databricks, Delta Lake, Delta Live Tables (DLT), Azure Data Factory or Fabric Data Pipelines.
• Knowledge of Unity Catalog and/or Microsoft Purview for governance, lineage, and security.
• Understanding of dimensional modeling for analytics‑ready datasets.

Responsibilities

• Design, implement, and support end‑to‑end ELT pipelines in Databricks/ADF.
• Implement data quality checks with alerting and remediation runbooks.
• Build curated, analytics‑ready Delta tables using dimensional modeling for BI consumption.
• Implement CDC and deletion patterns; manage schema drift and partitioning strategies.
• Operate jobs with monitoring, logging, and on‑call rotation as needed.
• Collaborate with Data Architect to align designs with governance and cost standards.
• Document pipelines, data contracts, and SLAs; continuously improve performance.

Skills

Azure Data Factory

Databricks PySpark

SQL

Python PySpark

Delta Lake

Data modeling

CI/CD

Education

Bachelor’s degree in MIS/CS/Engineering

Tools

Azure Databricks

Delta Live Tables

ADLS Gen2

Git

Job description

PowerSecure, a Southern Company subsidiary, designs, builds, and operates reliable data pipelines on Azure to enable trusted analytics and self‑service BI. This role partners with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high‑quality data using Azure Data Factory, Databricks, and Delta Lake.

Responsibilities include batch and streaming ELT, data quality enforcement, and operational support, with collaboration on governance, security, and cost