Azure Data Pipelines Engineer

Socket.dev; Durham (NC); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Medical, dental, vision insurance

401(k) matching plan

Paid time off (PTO)

Tuition reimbursement

Wellness incentive programs

Employee referral program

Job summary

PowerSecure, a Southern Company subsidiary, seeks a Data Solutions Engineer to design, build, and operate scalable data pipelines on the Azure data platform. You will ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake, partnering with BI developers and stakeholders to enable trusted analytics and self-service BI.

Responsibilities include batch/stream ingestion, ELT transformation, data quality enforcement, and operational support, with a hybrid

Qualifications

• 2+ years of hands-on data engineering or equivalent
• 2+ years building pipelines on Azure and Databricks
• Strong SQL and PySpark, plus data modeling fundamentals
• Bachelor’s degree as noted in education
• Proficiency with SQL and Python (PySpark) including tuning
• Experience with governance tools like Unity Catalog or Purview
• Familiarity with data ingestion patterns and schema evolution
• Understanding of dimensional modeling for analytics-ready datasets
• Experience with CI/CD for data assets
• Knowledge of privacy, security, and RBAC in cloud data platforms
• Excellent communication and cross-functional collaboration skills
• Eligibility to work in the United States without sponsorship

Responsibilities

• Design, build, and operate end-to-end ELT pipelines (ingest → transform → publish) in Databricks/ADF
• Implement data quality checks with alerting and remediation runbooks
• Create analytics-ready Delta tables using dimensional modeling for BI consumption
• Apply CDC and deletion-flag patterns; manage schema drift and partitioning
• Operationalize jobs with monitoring, logging, and on-call readiness
• Collaborate with Data Architect to align designs with governance and cost standards
• Document pipelines, data contracts, and SLAs; continuously improve performance

Skills

PySpark / Spark SQL

SQL proficiency

Data modeling fundamentals

Azure Data Factory / Fabric Pipelines

Delta Lake / Delta Live Tables

CI/CD for data assets

Data governance

Communication & collaboration

Education

Bachelor’s degree in MIS, CS, Eng or equivalent

Tools

Azure Data Factory

Databricks

Delta Lake

Delta Live Tables

ADLS Gen2

Azure DevOps / Git

Job description

PowerSecure, a Southern Company subsidiary, seeks a Data Solutions Engineer to design, build, and operate scalable data pipelines on the Azure data platform. You will ingest, model, and deliver high-quality data using Azure Data Factory, Databricks, and Delta Lake, partnering with BI developers and stakeholders to enable trusted analytics and self-service BI.

Responsibilities include batch/stream ingestion, ELT transformation, data quality enforcement, and operational support, with a hybrid