Senior Data Engineer - Azure Data & Pipelines (Remote)

Remote; San Francisco (CA); Remote; USD 64,000 - 86,000; Full time

Benefits offered by this job

Life insurance

Wellness programs

Significant paid time off

Employee discounts

Job summary

WPP Media is seeking a Senior Data Engineer to design, build, and optimize enterprise ETL/ELT pipelines using Azure Data Factory and Data Lake, with a focus on scalable, secure, and reliable data processing. You will craft T-SQL solutions, enable real-time analytics, and automate data lifecycles through Python APIs and modern DevOps practices.

The role collaborates with US onshore leadership and analytics teams to deliver high-performance solutions, while mentoring engineers and enforcing coding

Qualifications

• 3–4 years of progressive experience in data engineering.
• Expert-level hands-on experience with Azure Data Factory, advanced Azure T-SQL/SQL, and cloud data platforms such as Azure and/or GCP.
• Proven experience designing, optimizing, and managing large-scale ETL/ELT pipelines, including ADF orchestration.
• Advanced SQL performance tuning, including query optimization, partitioning, indexing, and high-volume data processing.
• Strong Python and API orchestration skills for automation.
• Advanced experience with production operations, troubleshooting, root-cause analysis, high-availability processing, monitoring, SLA management, and CI/CD automation.
• Experience with one or more of Adverity, Dataflow, BigQuery, Databricks, dbt, Terraform, Wrike, Jira or Azure DevOps is a plus.
• DP-203 and/or GCP Professional Data Engineer certification is a plus.

Responsibilities

• Design, develop, and optimize enterprise ETL/ELT pipelines using Azure Data Factory, Azure Data Lake Storage, and serverless technologies, ensuring scalability, security, and reliability.
• Build and optimize complex Azure T-SQL solutions, including query tuning, partitioning, and indexing, to support advanced analytics models and real-time data processing.
• Develop automation frameworks with Python and API orchestration to streamline the data lifecycle and improve operational efficiency.
• Own production pipeline reliability by performing root-cause analysis, resolving critical incidents, and implementing long-term stability measures.
• Define and enforce coding and architectural standards through code reviews and technical audits, while mentoring engineers and raising delivery quality.
• Maintain documentation, data lineage, governance, and automated CI/CD deployment practices across data assets.
• Partner with US onshore leadership and analytics and media teams to translate business needs into scalable, high-performance technical solutions.
• Lead end-to-end pipeline architecture and modernization initiatives involving Databricks, dbt, and advanced DevOps practices; monitor SLAs, mitigate risk, and support compliance.

Skills

Azure Data Factory

Azure Data Lake Storage

Python

API orchestration

Databricks

dbt

Terraform

CI/CD automation

SQL performance tuning

DevOps practices

Tools

Azure Data Factory

Azure Data Lake Storage

Azure SQL / T-SQL

Databricks

dbt

Terraform

Wrike

Jira

Azure DevOps

Job description

WPP Media is seeking a Senior Data Engineer to design, build, and optimize enterprise ETL/ELT pipelines using Azure Data Factory and Data Lake, with a focus on scalable, secure, and reliable data processing. You will craft T-SQL solutions, enable real-time analytics, and automate data lifecycles through Python APIs and modern DevOps practices.

The role collaborates with US onshore leadership and analytics teams to deliver high-performance solutions, while mentoring engineers and enforcing coding