Lead Cloud-Scale Data Engineering & AI Pipelines

Accenture UK & Ireland; Glasgow; On-site; GBP 110,000 - 140,000; Full time

Benefits offered by this job

30 days vacation

Private medical insurance

Volunteer leave

Job summary

Accenture UK & Ireland is seeking a Data Engineering Manager to lead cloud-scale data engineering programmes across platforms such as Azure, Databricks, Snowflake, and AWS. You will drive AI-assisted pipeline development, implement data quality and governance, and own end-to-end data product design while mentoring teams.

You will shape modern ETL/ELT migrations, architect observability and resiliency, and champion agentic AI patterns to accelerate delivery and value realization for complex

Qualifications

• Design and deliver cloud-scale data engineering solutions across cloud platforms.
• Experience with AI-assisted pipeline development and GenAI tooling.
• Strong data quality, observability, and governance capabilities.
• Proficiency in dimensional or Kimball data modelling and medallion architecture.
• Experience in ETL/ELT delivery and legacy-to-cloud migrations at scale.
• Ability to think in terms of data products and consumption-first models.
• Familiarity with agentic AI patterns in data pipelines.
• Proficiency in CI/CD and testing for data platforms.
• Proven track record of leading and mentoring engineering teams.

Responsibilities

• Lead delivery of cloud-scale data engineering programmes and platform migrations across multiple cloud environments.
• Drive AI-assisted pipeline development and governance of AI-generated outputs.
• Architect data quality, schema drift detection and lineage with AI-driven automation.
• Own data product design end-to-end from requirements to governance.
• Lead ETL/ELT modernisation and cloud migration initiatives.
• Define observability, resiliency and self-healing pipeline capabilities.
• Promote autonomous and human-in-the-loop AI operations within workflows.
• Translate data patterns into reusable agent skills and automation frameworks.
• Set CI/CD standards and testing practices for data platforms.

Skills

Cloud-scale data engineering

AI-assisted pipelines

Data quality & observability

Data modelling

ETL/ELT delivery

Data product thinking

Agentic AI patterns

CI/CD for data platforms

Leadership & mentoring

Tools

Azure

Databricks

Snowflake

GCP

Job description

Accenture UK & Ireland is seeking a Data Engineering Manager to lead cloud-scale data engineering programmes across platforms such as Azure, Databricks, Snowflake, and AWS. You will drive AI-assisted pipeline development, implement data quality and governance, and own end-to-end data product design while mentoring teams.

You will shape modern ETL/ELT migrations, architect observability and resiliency, and champion agentic AI patterns to accelerate delivery and value realization for complex