Azure Data Engineer — ELT Pipelines & Delta Lake

Mississippi Power Co; Durham, Northern (NC, KY); Hybrid; USD 90,000 - 140,000; Full time

Benefits offered by this job

Medical/Dental/Vision

401(k) Matching

PTO & Holidays

Health Savings Account

Wellness Programs

Employee Referral

Tuition Reimbursement

Job summary

PowerSecure, a Southern Company subsidiary, is seeking a Data Engineer to design, build, and operate reliable data pipelines on the Azure platform to enable trusted analytics and self-service BI.

This role partners with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake. 2+ years of experience required; hybrid work setup.

Qualifications

• 2+ years hands-on data engineering or equivalent data-focused software work.
• 2+ years building pipelines on Azure and Databricks or equivalent cloud + Spark.
• Strong SQL, PySpark, and data modeling fundamentals.
• Bachelor’s degree in MIS, CS, or Engineering or equivalent experience.
• Proficiency with SQL and Python (PySpark), including performance tuning on large datasets.
• Experience with Azure Databricks, Delta Lake, Delta Live Tables, Azure Data Factory (or Fabric Data Pipelines), ADLS Gen2, and Azure DevOps/Git for CI/CD.
• Knowledge of governance, lineage, and RBAC in cloud data platforms.
• Ingestion patterns, schema evolution, CDC, and deletion detection.
• Understanding of dimensional modeling for analytics-ready BI datasets.
• Experience with orchestration/monitoring, cost optimization, and runbook-driven operations.
• Data pipeline design (batch & streaming) with DLT data quality and robust error handling.
• Source control and CI/CD for data assets.

Responsibilities

• Design, implement, and support end-to-end ELT pipelines in Databricks/ADF.
• Implement data quality checks with alerting and remediation runbooks.
• Build curated, analytics-ready Delta tables using dimensional modeling for BI consumption.
• Implement CDC and deletion-flag patterns; manage schema drift and partitioning/Z-Ordering.
• Operationalize jobs with monitoring, logging, alerting; participate in on-call rotation.
• Collaborate with the Data Architect to align designs with governance, security, and cost standards.
• Document pipelines, data contracts, and SLAs; continuously improve performance and reliability.

Skills

SQL

PySpark

Data modeling

Python

Data integration

Education

Bachelor’s degree in MIS, CS, or Engineering

Tools

Azure Data Factory

Databricks

Delta Lake

Delta Live Tables

ADLS Gen2

Azure DevOps

Job description

PowerSecure, a Southern Company subsidiary, is seeking a Data Engineer to design, build, and operate reliable data pipelines on the Azure platform to enable trusted analytics and self-service BI.

This role partners with BI Developers, the Data Architect, and business stakeholders to ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake. 2+ years of experience required; hybrid work setup.