Databricks Lead, Data Platform & AI Engineering Manager

Publicis Sapient; Atlanta (GA); On-site; USD 130,000 - 180,000; Full time

Benefits offered by this job

Inclusive workplace benefits

Learning opportunities

Competitive compensation

Flexible work-life balance

Health benefits

Paid leave

Job summary

Publicis Sapient is seeking a Manager, Data Engineering with deep Databricks expertise to lead the design, delivery, and modernization of enterprise-scale data platforms. You will guide architecture, develop modern lakehouse solutions, and mentor engineering teams across client engagements.

You will drive batch and streaming pipelines, partner with stakeholders on roadmaps, governance, and delivery approaches, and apply best practices for quality, observability, and security in cloud-native

Qualifications

• 10+ years of experience leading the implementation of enterprise-scale data platforms and end-to-end data engineering solutions in production environments.
• Hands-on experience with Databricks as a primary data engineering platform, including Delta Lake, Databricks Workflows, Databricks SQL, notebooks, jobs, and modern Lakehouse architecture patterns.
• Strong experience designing and implementing scalable data platforms on one or more public cloud platforms including AWS, Azure, or GCP.
• Advanced proficiency in Python and practical experience using Python-based frameworks for data engineering, platform automation, and AI-enabled engineering workflows.
• Strong expertise in Apache Spark, PySpark, Spark SQL, and distributed data processing technologies.
• Experience implementing both batch and real-time data pipelines using technologies such as Spark Streaming, Glue ETL, Lambda, Dataflow, Azure Data Factory, Databricks, or similar frameworks.
• Experience with data modeling, dimensional modeling, data warehousing, and modern architectural patterns including Lakehouse and data mesh approaches.
• Experience with columnar data platforms such as Snowflake, BigQuery, Redshift, Vertica, or similar technologies.
• Experience with NoSQL technologies such as DynamoDB, Bigtable, Cosmos DB, or equivalent distributed databases.
• Experience implementing software engineering best practices including source control, CI/CD, automated testing, release management, infrastructure automation, and production support processes.
• Familiarity with MLOps concepts and supporting data engineering responsibilities related to model deployment, validation, monitoring, rollback, governance, and operational reliability.
• Experience leading engineering teams, managing delivery workstreams, and collaborating effectively across cross-functional and client-facing environments.
• Strong communication, stakeholder management, and consulting skills.

Responsibilities

• Lead the architecture, design, and delivery of enterprise-scale data engineering solutions built on Databricks and cloud-native data platforms.

Skills

Databricks leadership

Delta Lake

Python

Cloud data platforms

Apache Spark / PySpark

Data modeling

Lakehouse architecture

AI engineering basics

Tools

Databricks

Spark

AWS

Azure

GCP

Snowflake

Airflow

Job description

Publicis Sapient is seeking a Manager, Data Engineering with deep Databricks expertise to lead the design, delivery, and modernization of enterprise-scale data platforms. You will guide architecture, develop modern lakehouse solutions, and mentor engineering teams across client engagements.

You will drive batch and streaming pipelines, partner with stakeholders on roadmaps, governance, and delivery approaches, and apply best practices for quality, observability, and security in cloud-native