Senior Azure Data Engineer

ENGINEERINGUK; United Kingdom; On-site; GBP 125,000 - 150,000; Full time

Job description

You will need to login before you can apply for a job.

Job Description Senior Azure Data Engineer

Position Description
Challenge yourself and change lives, unlocking complex system solutions as part of a highly motivated and supportive team. Our Data Engineers are trusted to work closely with a wide range of clients on exciting projects with real-world purpose and impact.

CGI was recognised in the Sunday Times Best Places to Work List 2023 and has been named one of the 'World's Best Employers' by Forbes magazine. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a Partner not just an employee.

CGI are on the hunt for a dynamic Senior Data Engineer to join our consulting team and drive innovative, high-impact data transformation solutions for our diverse client base. In this role, you will design, build, and optimize robust ETL pipelines and staging systems for complex data migration projects using Azure Data Factory and Azure Databricks, while also integrating key Google Cloud Platform services such as BigQuery, Cloud Dataflow, and Cloud Dataproc. Your extensive experience in processing large-scale data, managing millions of customer records, and developing Spark-based transformations will be crucial in converting legacy data models into structured, actionable insights.

Your future duties and responsibilities
In addition to your technical expertise, you will work closely with data architects and client stakeholders to ensure alignment with target data models and drive seamless data migrations. Your role will involve automating cleansing workflows, implementing data quality frameworks, and troubleshooting issues to maintain data integrity. The ideal candidate will have a strong multi-cloud background, deep technical skills, and proven consulting experience, enabling you to deliver tailored, strategic solutions that propel business transformation and client success.

Required qualifications to be successful in this role

1. Strong experience in ETL development, data pipeline automation, and large-scale data processing.
2. Proficiency in Azure Data Factory (ADF) for orchestrating data workflows and extensive hands-on experience with Azure Databricks-including cluster management, notebook development, and performance optimization-for scalable data transformations.
3. Familiarity with Google Cloud Platform services-such as BigQuery, Cloud Dataflow, and Cloud Dataproc-in multi-cloud environments is highly advantageous.
4. Proven track record in developing, utilizing, and optimizing staging systems to manage complex data cleansing, transformation, and migration requirements.
5. Expertise in processing millions of customer records using SQL-based transformations and Spark-based frameworks (Python, Scala, or SQL) across platforms like Azure Databricks, Azure Synapse, or equivalent environments.
6. Strong knowledge of data quality frameworks, validation techniques, and error handling to ensure high-integrity data processes.
7. Experience working with structured and semi-structured data (JSON, Parquet, Avro, CSV, etc.) in migration contexts, and a deep understanding of legacy versus target data models.
8. Ability to optimize data cleansing and transformation processes for improved efficiency and performance.

Skills

• English
• Python
• SQL
• Google Cloud Platform

Reference 1199218

Company

Learn more about this company

Visit this company’s hub to learn about their values, culture, and latest jobs.