Remote Senior Data Engineer – Azure & BI Pipelines Leader

wppmedia; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

wppmedia seeks a Senior Data Engineer to design, build, and maintain large-scale data pipelines supporting analytics and real-time processing for stakeholders across regions. You will blend hands-on engineering with architectural leadership, enforcing standards and contributing to cross-regional initiatives.

In this role, you will leverage Azure Data Factory, Azure Data Lake Storage, and Databricks, with a focus on scalable, secure pipelines, SQL optimization, and automation through Python.

Qualifications

• 3–4 years of professional data engineering experience.
• Expert-level hands-on experience with Microsoft Azure and Azure data services in production environments.
• Advanced experience designing, building, orchestrating, monitoring, and troubleshooting production‑grade pipelines using Azure Data Factory.
• Relevant hands-on experience developing or managing data workloads using Azure Databricks.
• Proven experience designing and optimizing large-scale ETL/ELT pipelines, including ADF orchestration and Azure Data Lake Storage.
• Strong SQL fundamentals, including joins, aggregations, subqueries, CTEs, window functions, data validation, and query troubleshooting.
• Advanced Azure T-SQL and SQL performance‑tuning skills, including partitioning, indexing, and high‑volume data processing.
• Strong Python and API orchestration skills for developing automation solutions.
• Experience with production operations, troubleshooting, root‑cause analysis, high‑availability processing, monitoring, SLA management, and CI/CD automation.

Responsibilities

• Design, develop, and optimize enterprise ETL/ELT pipelines using Azure Data Factory, Azure Data Lake Storage, and serverless technologies.
• Build and tune complex Azure T-SQL solutions to support analytics models and real-time data processing.
• Develop automation frameworks with Python and API orchestration to streamline the data lifecycle.
• Own production pipeline reliability through root-cause analysis and incident resolution.
• Define and enforce coding and architectural standards through code reviews and audits; mentor engineers.
• Maintain documentation, data lineage, governance, and CI/CD deployment practices across data assets.
• Partner with regional leadership and analytics teams to translate business requirements into scalable solutions.
• Lead end-to-end pipeline architecture and modernization initiatives involving Databricks, dbt, and DevOps practices.
• Monitor SLAs, mitigate technical risks, and support data security and compliance.

Skills

Azure Data Factory

Azure Data Lake Storage

Azure Databricks

SQL

Python

CI/CD

ETL pipelines

API orchestration

Monitoring & troubleshooting

Tools

Adverity

Dataflow

BigQuery

dbt

Terraform

Wrike

Jira

Azure DevOps

Job description

wppmedia seeks a Senior Data Engineer to design, build, and maintain large-scale data pipelines supporting analytics and real-time processing for stakeholders across regions. You will blend hands-on engineering with architectural leadership, enforcing standards and contributing to cross-regional initiatives.

In this role, you will leverage Azure Data Factory, Azure Data Lake Storage, and Databricks, with a focus on scalable, secure pipelines, SQL optimization, and automation through Python.