Senior Databricks Data Engineer

Georgia Pacific Corp.; Atlanta (GA); Hybrid; USD 140,000 - 170,000; Full time

Benefits offered by this job

Medical, dental, vision

Flexible spending accounts

Retirement

Paid vacation

Job summary

Georgia-Pacific is hiring a Senior Databricks Data Engineer to join the Data Platform team in Atlanta, GA (hybrid). The role focuses on building and improving Databricks-native data engineering solutions that are governed, scalable, and aligned with enterprise standards for data quality, security, and reliability.

This role is based at Georgia-Pacific headquarters on Peachtree St in Atlanta, Georgia, with a hybrid schedule that expects three (3) days in the office per week.

Qualifications

• Production-grade Databricks, Delta Lake, and Apache Spark pipelines experience for batch and streaming workloads.
• Experience building Databricks Jobs, Lakeflow Declarative Pipelines, and Structured Streaming, with scheduling and monitoring.
• IaC deployment automation using Terraform or Databricks Asset Bundles, plus Git and CI/CD.
• Data governance via Unity Catalog with permissions, lineage, and metadata controls.
• Proficiency in Python and Spark diagnostic tools to optimize workloads.
• Legal authorization to work permanently in the United States.

Responsibilities

• Design, build, deploy, and support Databricks-native data solutions.
• Develop governed data platforms to improve accuracy, consistency, accessibility, and trust.
• Create automated tools and reusable templates for data development and ingestion.
• Operate and improve platform with monitoring, migrations, and cost optimization.
• Partner with product owners and teams to identify use cases and drive adoption.
• Integrate Databricks pipelines with governance capabilities (metadata, lineage, classification, ownership, data-quality).
• Develop governed data assets for AI and analytics (semantic models, APIs, Genie Spaces, AI agents).
• Implement validation, monitoring, remediation for critical data assets.

Skills

Databricks

Delta Lake

Apache Spark

Python

CI/CD

Terraform

Git

Structured Streaming

Unity Catalog

Lakeflow

Tools

Databricks Jobs

Auto Loader

Databricks Asset Bundles

Job description

Georgia-Pacific is hiring a Senior Databricks Data Engineer to join the Data Platform team in Atlanta, GA (hybrid). The role focuses on building and improving Databricks-native data engineering solutions that are governed, scalable, and aligned with enterprise standards for data quality, security, and reliability.

Role Overview

In this position, you will develop long-term value across multiple business units by expanding access to trusted, high-quality enterprise data across hundreds of systems and thousands of data assets. You will enhance the Databricks platform, deliver reliable and scalable data pipelines, and help establish reusable engineering patterns that match enterprise architecture and governance requirements.

This role is based at Georgia-Pacific headquarters on Peachtree St in Atlanta, Georgia, with a hybrid schedule that expects three (3) days in the office per week.

Responsibilities

• Design, build, deploy, and support Databricks-native data engineering solutions, including Delta Lake pipelines, Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader ingestion pipelines, Unity Catalog resources, and CI/CD automation
• Develop reliable, governed data platforms and pipelines to improve the accuracy, consistency, accessibility, and trustworthiness of enterprise data assets
• Create automated tools, reusable templates, and self-service patterns that accelerate data development, ingestion, feature engineering, and consumption
• Operate and improve the platform through performance monitoring, incident resolution, migrations, upgrades, observability, and cost optimization aligned with long-term scalability and resilience goals
• Partner with product owners, operations teams, architects, and delivery teams to identify common use cases, prioritize platform enhancements, and expand adoption
• Integrate Databricks pipelines with enterprise governance and capabilities, including metadata, lineage, classification, certification, ownership, and data-quality controls
• Develop and curate governed data assets for AI and advanced analytics, including semantic models, APIs, Genie Spaces, and AI agents
• Implement validation rules, monitoring, and remediation processes to improve the fitness and reliability of critical data assets

Required Qualifications

• Experience designing, building, and supporting production-grade Databricks, Delta Lake, and Apache Spark pipelines for batch and streaming workloads
• Experience developing Databricks Jobs, including Lakeflow Declarative Pipelines and Structured Streaming, plus multi-task workflows with scheduling, parameters, dependencies, retries, notifications, and operational monitoring
• Experience implementing Databricks infrastructure and environment-aware deployment automation using Terraform, Databricks Asset Bundles, or other Infrastructure as Code practices, along with Git and CI/CD
• Experience implementing data governance and access controls with Unity Catalog, including catalogs, schemas, tables, views, external locations, storage credentials, lineage, and fine-grained permissions
• Experience using Python and Spark diagnostic tools, including job logs, Spark UI, query plans, partitioning, join optimization, and Adaptive Query Execution, to troubleshoot and optimize production workloads
• Legal authorization to work permanently in the United States for any employer without requiring a visa transfer or visa sponsorship

Technologies

• Databricks, Delta Lake, Apache Spark
• Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader
• Unity Catalog, CI/CD, Terraform, Databricks Asset Bundles, Git
• Python, Structured Streaming, Adaptive Query Execution
• Apache Hudi, Apache Iceberg
• AWS, Microsoft Azure, Google Cloud, Amazon S3, Azure Data Lake Storage Gen2, OneLake, Google Cloud Storage
• Semantic models, Genie Spaces, AI agents, API

Team and Collaboration

The Georgia-Pacific Data Platform team partners with Data Governance, Data Delivery, Data Product Owners, Solution Architects, and other platform and product teams. The group develops governed, reusable capabilities that improve data accessibility, reliability, lineage, and quality while supporting analytics, advanced analytics, and AI use cases across the enterprise.

Benefits

• Medical, dental, vision
• Flexible spending and health savings accounts
• Life insurance, AD&D, disability
• Retirement
• Paid vacation/time off
• Educational assistance
• May also include infertility assistance, paid parental leave, and adoption assistance

About Georgia-Pacific

Georgia-Pacific and its subsidiaries are among the world’s leading manufacturers and marketers of bath tissue, paper towels and napkins, tableware, paper-based packaging, cellulose, specialty fibers, and building products. Our products are part of everyday life for people around the world.

Hiring Philosophy

• All Koch companies value diversity of thought, perspectives, aptitudes, experiences, and backgrounds
• We are Military Ready and Second Chance employers