Azure Data Engineer: ELT Pipelines & Data Quality

Southern Company; Raleigh (NC); Hybrid; USD 110,000 - 160,000; Full time

Benefits offered by this job

Medical/Dental/Vision/Life

401(k) matched

Vacation & Holidays

FSA/HSA

Wellness programs

Employee Referral

Tuition Reimbursement

Job summary

PowerSecure, a Southern Company subsidiary, seeks a Data Solutions Engineer to design, build, and operate reliable data pipelines on our Azure data platform to enable trusted analytics and self-service BI.

You will partner with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks (PySpark), and Delta Lake, while implementing data quality checks and scalable ELT processes.

Qualifications

• 2+ years hands-on data engineering with pipelines on Azure and Spark.
• Proficiency in SQL and PySpark, with data modeling fundamentals.
• Bachelor’s degree in MIS/CS/Engineering or equivalent.
• Experience with Azure Databricks, Delta Lake, and DLT for governance and CI/CD.
• Familiarity with data ingestion patterns and schema evolution.
• Excellent communication and collaboration skills.

Responsibilities

• Design, implement, and support end-to-end ELT pipelines (ingest → transform → publish).
• Implement data quality checks with alerting and remediation runbooks.
• Build analytics-ready Delta tables using dimensional modeling.
• Implement CDC and partitioning strategies; manage schema drift and Z-ordering.
• Operate jobs with monitoring, logging, and on-call rotations.
• Collaborate with the Data Architect to ensure governance and cost efficiency.
• Document pipelines, SLAs, and data contracts; optimize performance.

Skills

SQL

Python (PySpark)

Data modeling

Azure Databricks

Azure Data Factory

Delta Lake / DLT

ETL/ELT pipelines

CI/CD for data assets

Communication

Education

Bachelor's degree in MIS, Computer Science, Engineering

Tools

Azure Data Factory

Delta Lake

Delta Live Tables

ADLS Gen2

Azure DevOps/Git

Unity Catalog / Purview

Job description

PowerSecure, a Southern Company subsidiary, seeks a Data Solutions Engineer to design, build, and operate reliable data pipelines on our Azure data platform to enable trusted analytics and self-service BI.

You will partner with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks (PySpark), and Delta Lake, while implementing data quality checks and scalable ELT processes.