Senior Data Engineer

South 11; Manchester; On-site; GBP 55,000 - 80,000; Full time

Benefits offered by this job

Generous discretionary bonus schemes

Access to online GP

Employee wellbeing support

Enhanced Maternity & Paternity Pay

Job summary

South 11 is looking for a Senior Data Engineer to enhance their data platform. This role involves migrating from SQL Server to AWS, developing data solutions using AWS services, and automating data processes. The successful candidate will work with data scientists, mentor junior engineers, and implement data quality checks. The role offers a salary between £55,000 and £80,000, alongside various employee benefits including bonuses, enhanced parental pay, and wellness support.

Qualifications

• Proficiency in designing and maintaining ETL processes.
• Experience with cloud data migration from on-premises systems.
• Ability to communicate effectively with diverse stakeholders.

Responsibilities

• Develop and maintain data pipelines using AWS services.
• Support migration from SQL Server to AWS.
• Investigate and resolve data quality issues.

Skills

AWS Glue

AWS Lambda

AWS S3

AWS Redshift

Python

SQL

Pipeline Orchestration

Data Warehousing

Agile methodologies

Data security best practices

Job description

Job Purpose

This Senior Data Engineer will play a key role in the evolution of our data platform, driving innovation and delivering high-quality data solutions. This role will focus on both migration from our current Data Warehouse to an AWS ecosystem, and developing cutting edge real-time data solutions including automation, business intelligence, and enabling analytics.

Job Duties

• Design, develop, and maintain data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda and AWS S3.
• Support the migration of the existing Data Warehouse from SQL Server to AWS through S3 and Redshift.
• Develop and implement data quality checks and validation procedures.
• Design and implement best practice data lakehouse architectures and data warehousing solutions.
• Collaborate with data scientists and analysts to support the deployment of machine learning and advanced analytical solutions.
• Develop and maintain data documentation and operational procedures.
• Investigate and resolve data quality issues and performance bottlenecks.
• Stay abreast of the latest data technologies and industry best practices.
• Mentor junior data engineers and provide technical guidance to other team members where applicable.
• Contribute to the development and improvement of data platform best practices.

Knowledge, Skills and Experience

• Good understanding of AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR.
• Proficiency in Python, SQL, Pipeline Orchestration, and data warehousing concepts. Ability to diagnose and resolve complex data issues.
• Ability to effectively communicate with both technical and non-technical stakeholders.
• Experience with data lakehouse architectures and data warehousing solutions.
• Experience in migrating data from on-premises systems to cloud environments would be desirable.
• Experience with Agile development methodologies.
• Experience with data security and privacy best practices.

What is in it for me?

• A competitive rate of pay and pension contribution (£55,000 - £80,000)
• Generous discretionary bonus schemes, incentives and competitions
• An annual leave entitlement that increases with length of service
• Access to an online GP 24/7, 365 days a year for you and your immediate family.
• Employee wellbeing support through our Employee Assistance Programme
• Enhanced Maternity & Paternity Pay
• Long Service Recognition
• Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream.