Senior Data Engineer

Advancing Analytics; United States; Remote; USD 134,000 - 171,000; Full time

Job summary

Advancing Analytics is seeking a data engineer for a three-month contract to roll out and operate a large, multi-service data and AI platform. The role is remote for UK-based staff with occasional customer and team visits.

You will build data solutions using Azure, Spark, Python and SQL, develop data pipelines with orchestration tools and data lakes, and apply governance practices across the data lifecycle. Collaboration and Agile delivery are essential.

Qualifications

• Practical experience with Microsoft Azure and cloud computing concepts.
• Experience with Apache Spark, using Databricks, Microsoft Fabric or another Spark engine.
• Strong Python skills and ability to write complex, performant SQL.
• Familiarity with Azure Data Factory or another cloud orchestration tool, plus Azure Data Lake and Delta Lake.
• Experience with data governance tools such as Unity Catalog or Purview, and infrastructure as code such as Terraform.
• Understanding of object-oriented development, CI/CD and agile project delivery.

Responsibilities

• Build and maintain data solutions using cloud services, Spark, Python and SQL.
• Develop and operate data pipelines with orchestration tools, data lake storage and Delta Lake.
• Apply data governance practices and tools to support reliable, well-managed data assets.
• Contribute to infrastructure as code, deployment workflows and the wider software development lifecycle.
• Gather requirements, consult with stakeholders and work within agile delivery teams.
• Improve data quality, resilience and observability across ETL processes.

Skills

Cloud computing concepts (Azure)

Python development

SQL proficiency

CI/CD & Agile delivery

Tools

Apache Spark / Databricks

Azure Data Factory

Azure Data Lake / Delta Lake

Terraform

Data governance tools (Purview / Unity Catalog)

Job description

Role overview

Support the rollout and ongoing operation of a large, multi-service data and AI platform. This three-month contract calls for an experienced data engineer who can work across the platform lifecycle, from gathering requirements and shaping solutions to building and maintaining data pipelines. The role is remote for people based in the UK, with occasional customer and team visits.

Responsibilities

• Build and maintain data solutions using cloud services, Spark, Python and SQL.
• Develop and operate data pipelines with orchestration tools, data lake storage and Delta Lake.
• Apply data governance practices and tools to support reliable, well-managed data assets.
• Contribute to infrastructure as code, deployment workflows and the wider software development lifecycle.
• Gather requirements, consult with stakeholders and work within agile delivery teams.
• Improve data quality, resilience and observability across ETL processes.

Requirements

• Practical experience with Microsoft Azure and cloud computing concepts.
• Experience with Apache Spark, using Databricks, Microsoft Fabric or another Spark engine.
• Strong Python skills and ability to write complex, performant SQL.
• Familiarity with Azure Data Factory or another cloud orchestration tool, plus Azure Data Lake and Delta Lake.
• Experience with data governance tools such as Unity Catalog or Purview, and infrastructure as code such as Terraform.
• Understanding of object-oriented development, CI/CD and agile project delivery.

Nice to have

• Experience with Power BI or Tableau, streaming services such as Event Hubs, or distributed compute systems.
• Familiarity with Azure Functions using Python or C#.

Benefits and work setup

• Remote within the UK, with occasional visits to customers and the team.
• Three-month contract.
• The team describes its approach as collaborative and inclusive, with attention to sustainable workloads and learning.