Senior Data Engineer, Data Platform

Macquarie Bank Limited; City Of London; Hybrid; GBP 65,000 - 85,000; Full time

Job summary

A global financial services group is seeking a Data Platform Engineer to design and optimize a petabyte-scale data platform. The role involves managing a Data Lake and leveraging AWS tools for high-performance workloads. Ideal candidates possess a strong Data and Engineering background, expertise in Python, and knowledge of modern data technologies. The company offers competitive benefits and a diverse working environment.

Benefits

25 days annual leave

Parental leave benefits

Paid volunteer leave

Employee Assistance Program

Qualifications

• Strong Data background with problem-solving skills.
• Experience building automation and CI/CD Pipelines.
• Good understanding of Data Lakes and Query Engines.

Responsibilities

• Design and build a cutting-edge, petabyte-scale data platform.
• Manage and enhance a petabyte-scale Data Lake.
• Ensure seamless execution of high-performance data workloads.

Skills

Data, DevOps, and Engineering background

Modern Data Platforms

SQL/DDLs

Python coding/testing

Linux/Unix knowledge

Tools

AWS

EKS

Airflow

Trino

Parquet

Job description

At Macquarie, our Data Services Team drives innovation by delivering a diverse suite of data products that empower data-driven decision-making and enhance operational efficiency. With a strong focus on engineering, data science, and analytics, the team delivers innovative solutions that empower business users primarily trading teams to tackle complex challenges and unlock new opportunities.

At Macquarie, our advantage is bringing together diverse people and empowering them to shape all kinds of possibilities. We are a global financial services group operating in 31 markets and with 56 years of unbroken profitability. You’ll be part of a friendly and supportive team where everyone - no matter what role - contributes ideas and drives outcomes.

What role will you play?

As a Data Platform Engineer, you will play a pivotal role in designing, building, and optimising a cutting-edge, petabyte-scale data platform powered by modern cloud technologies.

In this role, you will manage and enhance a petabyte-scale Data Lake and will create secure, efficient, and scalable environments for our data platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive queries, including 600+ billion-row queries in Redshift, by designing and maintaining robust, scalable infrastructure. Our flat structure means that you will directly contribute to our strategy while taking ownership of a diverse range of projects utilising the latest technologies.

What you offer

• A dynamic individual with a strong Data, DevOps and Engineering background who loves to solve data problems and build automation and CI/CD Pipelines in the data space and cloud infrastructure.
• Good understanding of modern Data Platforms, Databases, Data Lakes, Data Warehouses and Query Engines, SQL/DDLs
• Experience with Python coding/testing or any Cloud-based technology (AWS preferred)
• Good understanding of Hosting Platform Linux/Unix (EKS and Container experience is a plus)

What we offer

At Macquarie, you’re empowered to shape a career that’s rewarding in all the ways that matter most to you. Macquarie employees can access a wide range of benefits which, depending on eligibility criteria, include:

• 1 wellbeing leave day per year and a minimum of 25 days of annual leave.
• 26 weeks’ paid parental leave for primary caregivers along with 12 days of paid transition leave upon return to work and 6 weeks’ paid leave for secondary caregivers
• Paid fertility leave for those undergoing or supporting fertility treatment
• 2 days of paid volunteer leave and donation matching
• Access to a wide range of salary sacrificing options
• Benefits and initiatives to support your physical, mental and financial wellbeing including, comprehensive medical and life insurance cover
• Access to our Employee Assistance Program, a robust behavioural health network with counselling and coaching services
• Access to a wide range of learning and development opportunities, including reimbursement for professional membership or subscription
• Access to company funded emergency and backup dependent care services
• Recognition and service awards
• Hybrid and flexible working arrangements, dependent on role
• Reimbursement for work from home equipment

About Technology

Technology enables every aspect of Macquarie, for our people, our customers and our communities. We’re a global team that is passionate about accelerating the digital enterprise, connecting people and data, building platforms and applications and designing tomorrow’s technology solutions.

Our commitment to diversity, equity and inclusion

We are committed to providing a working environment that embraces diversity, equity, and inclusion. We encourage people from all backgrounds to apply regardless of their identity, including age, disability, neurodiversity, gender (including gender identity or expression), sexual orientation, marriage or civil partnership, pregnancy, parental status, race (including ethnic or national origin), religion or belief, or socio-economic background. We welcome further discussions on how you can feel included and belong at Macquarie as you progress through our recruitment process.

Our aim is to provide reasonable adjustments to individuals as required during the recruitment process and in the course of employment. If you require additional assistance, please let us know during the application process.