Hybrid Data Engineer: Databricks & Azure Lead

AbilityLinks; Atlanta (GA); Híbrido; USD 120.000 - 180.000; Tempo integral

Resumo da vaga

Southern Company Services in Atlanta is seeking a senior Data Engineer to design, build, and operate data pipelines across hybrid and multi-cloud environments, with a strong focus on Databricks and Azure. You will translate complex datasets into reliable, consumable formats to support analytics and AI use cases, collaborating with cross‑functional teams in a large energy company.

The role emphasizes data standardization, taxonomy, governance, and scalable solution delivery.

Qualificações

• Bachelor's degree in CS/Engineering or related field.
• 9+ years of data engineering experience in batch and real-time processing.
• Experience with big data technologies (Hadoop, Hive, Spark).
• Python-based AI/ML experience preferred.
• 4+ years with on-prem tools (MSBI, Informatica, Oracle GoldenGate, SQL Server).
• 2+ years with Microsoft Azure tools (Data Lake, Data Factory, Databricks, Synapse, Power BI).
• Experience with containerization (Docker, OpenShift).

Responsabilidades

• Design and implement data pipelines for structured and unstructured data.
• Develop release pipelines for automation and deployment tasks.
• Build reusable templates and semantic models.
• Establish data promotion gatekeeping and guard rails for production data.
• Collaborate with cross-functional data, business, and technical teams.
• Promote data reusability and modular data product delivery.
• Define and enforce data quality and integrity controls.
• Provide technical guidance to junior data engineers.
• Optimize performance with parallel processing and minimal data movement.
• Stay current with data tech and share insights across the org.
• Standardize and develop Databricks apps, data integrations, and automated processes.

Conhecimentos

Databricks

Python

SQL

Apache Spark

Azure

Data modeling

CI/CD

Communication

Mentoring

Formação

Bachelor's degree in CS/Engineering

Machine Learning

Ferramentas

Hadoop

Hive

Informatica

Oracle GoldenGate

SQL Server

Data Lake

Azure Data Factory

Databricks

Synapse

Power BI

Docker

OpenShift

Descrição da vaga

Southern Company Services in Atlanta is seeking a senior Data Engineer to design, build, and operate data pipelines across hybrid and multi-cloud environments, with a strong focus on Databricks and Azure. You will translate complex datasets into reliable, consumable formats to support analytics and AI use cases, collaborating with cross‑functional teams in a large energy company.

The role emphasizes data standardization, taxonomy, governance, and scalable solution delivery.