Principal Data Engineer

The Intersect Group; Atlanta (GA); Hybrid; USD 140,000 - 200,000; Full time

Job summary

The Intersect Group is seeking a Principal Data Engineer in Atlanta, GA, with a strong focus on building scalable data platforms and products for analytics, ML/AI, and reporting. Hybrid work is available and you will partner with multiple teams to shape data architecture and governance.

You will lead the design and delivery of lakehouse, data warehouse, and semantic layers, while driving CI/CD and modern DevOps practices across the data stack.

Qualifications

• Deep expertise in Python, SQL, PySpark and/or Scala and distributed data processing frameworks.

Responsibilities

• Define and evolve enterprise data engineering architectures, design patterns, standards, and best practices.
• Create reusable engineering frameworks, templates, automation standards, and playbooks.

Skills

Python

SQL

PySpark

Scala

Azure

Databricks

Delta Lake

CI/CD

Docker

Kubernetes

Infrastructure as Code

DevOps

Observability

Data Governance

Tools

Azure Databricks

Delta Lake

Docker

Kubernetes

Terraform

Job description

## Principal Data Engineer**Atlanta,GA30328**Posted: 11/10/2026Employment Type:Contract to HireCategory: IT - Development - Developer/ArchitectJob Number: 251463Work Location: Hybrid## Job Description**JOB TITLE** Principal Data Engineer **LOCATION** Atlanta, Georgia | Hybrid **ROLE SUMMARY** Design and deliver enterprise data platforms and products that support reporting, analytics, machine learning, AI, and digital business capabilities. Define architectural direction, reusable engineering standards, and scalable data solutions across multiple teams and business domains. Partner with technical and business stakeholders to turn complex problems into reliable, governed data products. **KEY RESPONSIBILITIES** * Define and evolve enterprise data engineering architectures, design patterns, standards, and best practices.* Create reusable engineering frameworks, templates, automation standards, and playbooks.* Design and deliver scalable data platforms and products across lake, warehouse, lakehouse, data mart, and semantic layers.* Build and modernize batch, streaming, event-driven, and API-based data integration pipelines.* Establish production practices for monitoring, logging, incident response, runbooks, platform support, and continuous improvement.* Address systemic gaps in data quality, reliability, observability, performance, cost, resiliency, and operational readiness.* Ensure data platforms and products meet standards for security, governance, privacy, data quality, and responsible data use.* Provide technical mentorship and lead cross-functional alignment without relying on formal reporting authority. **KEY REQUIREMENTS** * Deep expertise in Python, SQL, PySpark and/or Scala, and distributed data processing frameworks.* Strong experience with Azure cloud platforms and Azure Databricks, including Delta Lake and platform-scale data processing.* Experience designing and operating data lakehouse, warehouse, data mart, semantic layer, and enterprise analytical data products.* Experience with CI/CD, workflow orchestration, Git-based development, automated testing, and production release practices.* Experience with Docker, Kubernetes, Infrastructure as Code, cloud-native deployment patterns, and modern DevOps or DataOps practices.* Strong understanding of observability, monitoring, logging, performance optimization, reliability engineering, and cost management.* Knowledge of data governance, data quality, data security, access controls, metadata management, and compliance-sensitive environments.* Preferred experience with streaming technologies, event-driven architectures, message queues, real-time data integration, BI tools, generative AI, or enterprise AI platforms. Apply if you are ready to define enterprise data engineering direction and deliver platforms that support analytics, AI, and business decision-making. The role offers the opportunity to influence standards, architecture, and engineering practices across multiple teams and domains.