Senior Data Engineer - Hybrid Data Pipelines

Sapient Corporation; Arlington (VA); Hybrid; USD 109,000 - 130,000; Full time

Benefits offered by this job

Health benefits

Paid leave

Wellness program

Learning opportunities

Job summary

Publicis Sapient in Arlington, VA seeks a Senior Data Engineer to build scalable data pipelines, orchestrate with Azure Data Factory, and load trusted data into the Enterprise Data Lake, Delta Lake, and Azure Synapse Analytics dedicated SQL pools.

You will own testing, data quality configuration, performance tuning, and production support across enterprise data solutions, collaborating with data architects, analysts, and business stakeholders. Hybrid work arrangement offered.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
• 4+ years of data engineering experience.
• 2+ years of Azure Databricks experience.
• Strong experience with Databricks, Delta Live Tables (DLT), Delta Lake, PySpark, Python, SQL, and Spark SQL.
• Strong experience with Azure Data Factory.
• Experience loading data via Enterprise Data Lake, Delta Lake, and Synapse.
• Experience configuring Databricks compute, data quality rules, monitoring, and optimization.
• Experience unit testing and coordinating integration testing/UAT.
• Experience creating HLDs/DDs and data pipeline documentation.
• Experience with Azure DevOps, Git, CI/CD and production deployments.
• Experience supporting production data platforms and incident resolution.

Responsibilities

• Create new batch and streaming data pipelines by leveraging existing ingestion frameworks and tools.
• Orchestrate, schedule, and monitor data pipelines using Azure Data Factory.
• Develop and enhance data transformations using PySpark, Python, and SQL.
• Parse, transform, and load data into Enterprise Data Lake, Delta Lake, and Azure Synapse dedicated SQL pools.
• Build and maintain Databricks data engineering solutions, including Delta Live Tables (DLT) and pipeline maintenance.
• Configure and maintain data quality rules, monitoring, and automated alerts.
• Perform unit testing and coordinate integration testing and UAT with business/tech partners.
• Create and maintain high-level designs (HLDs), detailed designs (DDs), data mappings, and runbooks.
• Tune and optimize Databricks, Spark, SQL, and Synapse workloads for performance/cost.
• Provide production support including incident troubleshooting and root-cause analysis.
• Use Azure DevOps and Git for source control, CI/CD, and controlled deployments.
• Collaborate with data architects, analysts, and stakeholders to deliver reliable data products.

Skills

Data engineering

Azure Databricks

Delta Live Tables

Delta Lake

PySpark

Python

SQL

Spark SQL

Azure Data Factory

Synapse

CI/CD

DevOps

Education

Bachelor's degree (CS/Engineering/IS)

Tools

Azure DevOps

Git

Job description

Publicis Sapient in Arlington, VA seeks a Senior Data Engineer to build scalable data pipelines, orchestrate with Azure Data Factory, and load trusted data into the Enterprise Data Lake, Delta Lake, and Azure Synapse Analytics dedicated SQL pools.

You will own testing, data quality configuration, performance tuning, and production support across enterprise data solutions, collaborating with data architects, analysts, and business stakeholders. Hybrid work arrangement offered.