Lead Data Engineer – Lakehouse & Streaming (Hybrid)

Norfolk Southern Corp; Atlanta (GA); Hybrid; USD 150,000 - 230,000; Full time

Job summary

Norfolk Southern Corp seeks an experienced Lead Enterprise Data Engineer in Atlanta, GA. You will set technical direction for a Databricks-based lakehouse and streaming analytics platform, modernize ingestion and transformation patterns, and guide operations while collaborating with cross-functional partners to deliver production-grade data pipelines.

The role emphasizes Databricks Lakehouse expertise, streaming, data modeling, governance, and technical leadership to deliver scalable data

Qualifications

• Minimum of 10+ years in data engineering with Databricks Lakehouse experience.
• Strong hands-on with Spark/PySpark, Delta Lake, and Delta Live Tables; governance with Unity Catalog.
• Leadership in delivering end-to-end ETL/ELT pipelines in lakehouse environments.
• Experience with cloud analytics (AWS) and data security/privacy basics.
• Excellent SQL skills and data modeling capabilities across conceptual/logical/physical models.

Responsibilities

• Define data architecture direction for lakehouse, ingestion, and transformation patterns.
• Remove blockers via hands-on problem solving and architectural decisions across teams.
• Design enterprise data models across domains aligned with pipeline architecture.
• Develop and enforce data modeling standards and governance using Erwin and related tools.
• Ingest, wrangle, and validate large-scale structured and unstructured data for analytics.

Skills

Technical leadership

Cross-functional collaboration

Problem solving

Data modeling

SQL proficiency

Education

Bachelor’s degree in Information Systems/Computer Science/Computer Information Systems or related field

Tools

Databricks Lakehouse

Spark/PySpark

Delta Lake

Delta Live Tables

Unity Catalog

Genie

LakeFlow Designer

Kafka

Confluent

AWS analytics services (S3, IAM, Glue, Lambda, MSK)

SQL

Erwin

Python

Scala

Snowflake

BigQuery

HBASE

Cassandra

Azure

Kinesis

TIBCO EMS

IBM MQ Series

MSK

Git

REST API

Web Services

NoSQL

Job description

Norfolk Southern Corp seeks an experienced Lead Enterprise Data Engineer in Atlanta, GA. You will set technical direction for a Databricks-based lakehouse and streaming analytics platform, modernize ingestion and transformation patterns, and guide operations while collaborating with cross-functional partners to deliver production-grade data pipelines.

The role emphasizes Databricks Lakehouse expertise, streaming, data modeling, governance, and technical leadership to deliver scalable data