Senior Data Engineer

CMSPI; Manchester; On-site; GBP 50,000 - 80,000; Full time

Job summary

A leading company in the payments industry seeks a Senior Data Engineer to join their agile development team. The successful candidate will architect innovative data solutions on a premier analytics platform and collaborate closely with engineers and stakeholders. With a focus on Python and Microsoft Azure, you'll contribute to scalable data architectures while mentoring junior staff and driving continuous improvement. This role offers excellent career growth opportunities, comprehensive training, and a competitive salary.

Benefits

Comprehensive, payments industry training

Performance-based earning opportunity

Future equity opportunity for high performers

Personal and professional learning opportunities

Quarterly values award with financial prize

Monthly reimbursement of commuting costs

Regular company socials and team building events

22 vacation days + UK public holidays

Competitive pension plan and healthcare cover

Qualifications

• 3 years' experience as a Data Engineer/DevOps Engineer.
• Advanced proficiency in Python for data tasks.
• Expertise in Microsoft Azure for analytics.

Responsibilities

• Design scalable data architectures for data retrieval.
• Mentor junior engineers and promote best practices.
• Collaborate to deliver data solutions aligned with business goals.

Skills

Python

Microsoft Azure

SQL/NoSQL databases

DevOps CI/CD

Infrastructure as Code (Terraform)

Problem-solving

Communication

Collaboration

Education

Degree in Computer Science, Mathematics or related subject

Tools

Azure DevOps

Terraform

Docker

Kubernetes

Job description

As a Senior Data Engineer, you will be an integral part of a cross-functional feature development squad, dedicated to delivering innovative data solutions on our leading analytics platform. You will utilize your diverse skill set to contribute to all aspects of data engineering, with a core focus in designing scalable and secure data architectures for data retrieval. Collaborating closely with other engineers, a scrum master, and business stakeholders, you will ensure a continuous delivery of value in an agile environment.

• Collaborate within the squad to define and deliver scalable, secure data features aligned with business goals.
• Architect high-performance data retrieval solutions (APIs, web scrapers, SFTP) with a focus on security, efficiency, and best practices.
• Oversee client data onboarding, ensuring swift, compliant, and secure integration processes.
• Provide expert support for complex issues, acting as a technical authority and ensuring secure coding standards through code reviews.
• Mentor junior engineers, promoting best practices in clean code, testing, and CI/CD.
• Drive continuous improvement in data feature development and stay updated on industry trends, implementing innovative tools and solutions.
• Proficient in Microsoft Azure (DevOps, Data Factory, Data Lake, Functions, Azure Container Instances, RBAC and Entra) and DevOps CI/CD.
• Proficient in Infrastructure as Code (Terraform), secure FTP configurations (SFTP/FTPS), and remediation of security vulnerabilities (DAST, Azure Defender).
• Expertise in Python for writing efficient code and maintaining reusable libraries.
• Experienced with microservice design patterns, and Databricks/Spark for big data processing.
• Strong knowledge of SQL/NoSQL databases corresponding ELT workflows.
• Excellent problem-solving, communication, and collaboration skills in fast-paced environments.

Requirements

• 3 years' professional experience as a Data Engineer/ DevOps Engineer/ Systems Engineer.
• Advanced proficiency in Python for data retrieval, processing, automation, and integration tasks.
• Expertise on Microsoft Azure platform for data analytics, including design and deployment of infrastructure.
• Expertise in creating CI/CD pipelines.
• Experience in creating FTP (SFTP/FTPS) configurations.
• Experience in working with ETL/ELT workflows for data analytics.
• Degree in Computer Science, Mathematics or related subject.

Highly desirable skills & exposure:

• Working collaboratively as part of an Agile development squad.
• Experience and knowledge of the payments industry.
• Experience using Azure Databricks.
• Experience using containerised services such as Docker/ Kubernetes.
• Experience using IaC tools such as Terraform/Bicep.
• Experience using programming languages; Scala, Powershell, YAML.

Benefits

• Comprehensive, payments industry training by in-house and industry experts.
• Excellent performance-based earning opportunity, including OKR-driven bonuses.
• Future opportunity for equity, rewarded to high performers.
• Personal and professional learning opportunities and growth experiences aligned with your career aspirations.
• Quarterly values award for all employees - with a financial prize.
• Monthly Reimbursement of commuting costs
• Regular companywide socials and team building events.
• 22 vacation days + U.K. public holidays and discretionary office closure during Christmas.
• Competitive Pension plan, Vitality healthcare cover (after 6-months) and Cycle to work scheme.