Azure Data Engineer – Pipelines, Delta Lake & Spark (Hybrid)

Southern Company; Durham, Northern (NC, KY); Hybrid; USD 90,000 - 130,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) plan with company match

Paid time off and holidays

Flexible spending accounts (FSA/HSA)

Job summary

PowerSecure is seeking a Data Engineer to design, build, and operate reliable data pipelines on our Azure data platform. You will collaborate with BI developers, the Data Architect, and stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake.

Responsibilities include batch/stream ingestion, data transformation, orchestration, and data quality enforcement.

Qualifications

• 2+ years hands-on data engineering or equivalent software work.
• 2+ years building pipelines on Azure and Databricks.
• Strong SQL, PySpark, and data modeling fundamentals.
• Bachelor’s degree in MIS, CS, Engineering, or equivalent experience.
• Proficiency with SQL and Python (PySpark) and performance tuning.
• Experience with governance, lineage, and security concepts in cloud data platforms.

Responsibilities

• Design, implement, and support end-to-end ELT pipelines in Databricks/ADF.
• Implement data quality checks with alerting and runbooks.
• Build analytics-ready Delta tables using dimensional modeling.
• Apply CDC and deletion-flag patterns; manage schema drift and partitioning.
• Operate jobs with monitoring and on-call rotation as needed.
• Document pipelines and SLAs; improve performance and reliability.

Skills

SQL

PySpark

Python

Azure Databricks

Delta Lake

Data Modeling

ETL/ELT

CI/CD

Governance & RBAC

Communication

Education

Bachelor’s degree in MIS/CS/Engineering

Tools

Azure Data Factory

Databricks

Delta Live Tables

ADLS Gen2

Azure DevOps/Git

Job description

PowerSecure is seeking a Data Engineer to design, build, and operate reliable data pipelines on our Azure data platform. You will collaborate with BI developers, the Data Architect, and stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake.

Responsibilities include batch/stream ingestion, data transformation, orchestration, and data quality enforcement.